mirror of
https://github.com/binaricat/Netcatty.git
synced 2026-11-16 12:01:13 +00:00
99 lines
5.4 KiB
Markdown
99 lines
5.4 KiB
Markdown
# Test the selected Note assistant on an isolated desktop
|
|
|
|
This recipe tests a sandbox plugin against a deterministic loopback provider.
|
|
The provider does not run a model or contact a paid service. The profile contains
|
|
only synthetic data. The parent owns native screenshots and video on the working
|
|
cloud GUI machine. The local executor's sandbox-helper failure remains a separate
|
|
failed launch record.
|
|
|
|
## Prepare the integrated head
|
|
|
|
Use a head containing the market source API, #3985's malformed-context repair,
|
|
#3986, and the transport task's canonical AI/context registration. Record the exact
|
|
commit and artifact hashes. Keep all PRs Draft. Do not merge or release them.
|
|
|
|
Run the build and tests with the repository's supported Node version:
|
|
|
|
```sh
|
|
npm run build:plugin-runtime-deps
|
|
node examples/plugins/assistant/build.mjs
|
|
node packages/plugin-cli/dist/cli.js pack examples/plugins/assistant/.package --out /tmp/assistant-example.ncpkg
|
|
node packages/plugin-cli/dist/cli.js validate /tmp/assistant-example.ncpkg
|
|
sha256sum /tmp/assistant-example.ncpkg
|
|
```
|
|
|
|
The package contains six files: its manifest, README, three panel files, and one
|
|
browser runtime entry. It has no Node entry, networking, filesystem, secrets,
|
|
storage, terminal, or command-execution permission. Its `commands` permission
|
|
registers its own view callbacks. It cannot execute terminal commands.
|
|
|
|
## Start the deterministic fixture
|
|
|
|
Start `node scripts/plugin-assistant-provider-fixture.mjs` from the repository.
|
|
The server prints its loopback URL. It listens only on `127.0.0.1`; use a fresh
|
|
unused port. Its response is fixed synthetic text, split into delayed SSE chunks.
|
|
The response includes fixed usage metadata. It accepts no request tools or system
|
|
messages. It keeps only request counts and prints no request content or headers.
|
|
|
|
Launch the exact integrated app on the working desktop with a fresh disposable
|
|
user-data directory. Preserve Chromium sandboxing. Do not use a real app profile,
|
|
cloud account, credential, host, Note, or chat history.
|
|
|
|
In AI Settings, create a custom OpenAI-compatible provider with the fixture's
|
|
`/v1` URL, model `deterministic-fixture`, and the literal synthetic credential marker
|
|
`fixture-only`. This marker exercises the existing credential injection path
|
|
without a real API credential. Select that
|
|
provider and model through the host UI. Do not enable an external Agent or tools.
|
|
|
|
Create one Note named `Synthetic retry plan` containing:
|
|
|
|
```text
|
|
Synthetic retry plan: add a retry button and show a connection status label.
|
|
```
|
|
|
|
## Record the installed-plugin path
|
|
|
|
1. Import the package. Record its ID `com.netcatty.assistant`, version `0.1.0`,
|
|
source, SHA-256, browser runtime, required permissions, and optional permissions.
|
|
Installation must not grant optional access. Source labels do not imply that
|
|
an author or package is officially maintained.
|
|
2. Enable the plugin and open its `Selected Note Assistant` modal from the plugin
|
|
view catalog. Allow only its required view and command registration permissions.
|
|
3. Grant `ai.invoke` and `context.selected.list` when the host prompts. Record the
|
|
host-selected model label. The host's Note chooser must show app-owned chrome.
|
|
Select only `Synthetic retry plan` and click **Share selected summaries**.
|
|
4. Verify that the plugin reference dropdown starts at **No reference**. Select
|
|
`Synthetic retry plan` explicitly, then click **Send**.
|
|
5. Deny `context.selected.read`. No fixture request may be sent. The optional
|
|
model permission must not authorize Note access.
|
|
6. Retry. Allow the single selected-context read. The separate host text dialog
|
|
must show the exact synthetic text, scope, and version, with **Cancel** selected
|
|
by default. Cancel once and confirm that no fixture request is sent.
|
|
7. Retry and click **Share this text**. Record streamed text, completion, and token
|
|
usage. Capture the full host window, not only the plugin iframe.
|
|
8. Click **Send** again, share the same synthetic text, and click **Stop** while
|
|
the delayed fixture is streaming. The text must stop changing; the fixture's
|
|
aborted-connection counter must increase. Record this sequence as video.
|
|
9. Change the Note during a pending review and attempt to share the old dialog.
|
|
The expected-version check must reject the result. Do not send stale content.
|
|
10. Revoke the plugin's AI/context permission while a turn is active. Disable,
|
|
re-enable, and close its view. Old handles and late results must not resume.
|
|
Lock/unlock the app and navigate or reload the main window in the isolated
|
|
profile. Record a fresh successful run after reauthorization.
|
|
11. Keep the fixture request counter and aborted-connection counter in the
|
|
evidence. Do not include prompt bodies, headers, credentials, or Note content
|
|
other than the synthetic text above in logs.
|
|
|
|
The automated suites cover source substitution, runtime identity, two-handle
|
|
revocation after malformed input, late replies, byte/token/time bounds, and
|
|
no-key serialization. Native execution must still verify the real prompts,
|
|
installed runtime, configured provider transport, and UI lifecycle.
|
|
|
|
## Record remaining limits
|
|
|
|
Independent review found the malformed-context revocation bug. The regression
|
|
reproduced and the repair passed focused tests. Further independent review was
|
|
blocked by the review tool/model restriction. That review remains incomplete and
|
|
has not been retried through another route. CI and native evidence do not replace
|
|
that missing review.
|