Commit Graph
4 Commits
Author SHA1 Message Date
陈大猫andClaude Opus 4.6 4e26852482 feat: support multimodal attachments in AI chat (#397)
* feat: support multimodal attachments (images, PDFs, files) in AI chat

Previously uploaded images were displayed in the UI but never sent to
the AI model, and non-image files (PDF, text) were silently rejected.

- Rename useImageUpload → useFileUpload; accept image/*, PDF, and text/*
- Rename ChatMessageImage → ChatMessageAttachment with filePath support
- Build multimodal SDK messages (ImagePart/FilePart) for Catty Agent
- Fix ACP agent path: images inline, non-image files via local path hint
  so ACP agents (Claude Code, etc.) read them with native file access
- Use Electron webUtils.getPathForFile() for reliable file path capture
- Compact user message bubble padding

Closes #294 (AI file upload issues)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: show real tool names in AI chat instead of ACP wrapper names

- Unwrap ACP dynamic tool calls in serializeStreamChunk to extract
  real tool name, args, and toolCallId from chunk.input
- Simplify MCP tool name prefixes (mcp__server__tool → tool)
- Pass toolCallId from ACP tool-call events to match tool results
- Prevent onToolResult from overwriting correct names with wrapper name
- Build toolCallNames map in ChatMessageList for tool result display

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: backward-compatible fallback for legacy `images` field in chat messages

Persisted sessions may still have `images` instead of `attachments`.
Add `?? m.images` fallback in SDK message builder and renderer so
historical image attachments are not silently dropped after upgrade.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: broaden file type support and handle pasted files without path

- Accept all file types except video/audio (instead of allowlist)
  so .json, .yaml, .sh, etc. are not silently rejected
- For ACP agents, save pasted/virtual files (no filePath) to temp
  directory so the agent can still read them

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use managed temp dir for pasted ACP attachments

Use tempDirBridge.getTempFilePath() instead of manual os.tmpdir() path
so pasted file attachments are tracked by the app's cleanup system.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 11:45:50 +08:00
bincxzandClaude Opus 4.6 8949394756 feat: add Claude Agent SDK streaming and Codex OAuth integration
Integrate Claude Agent SDK for direct streaming chat, add Codex login/logout
flow with OAuth support in settings, improve AI chat panel UI and agent
discovery, and update build config for new dependencies.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-14 21:27:41 +08:00
bincxzandClaude Opus 4.6 7f3214e088 feat: integrate external AI agents via ACP protocol
- Add auto-discovery of CLI agents (Claude Code, Codex, Gemini) from system PATH
- Integrate ACP (Agent Client Protocol) for real-time streaming with codex-acp
- Bundle @zed-industries/codex-acp binary for reliable agent spawning
- Add ThinkingBlock component with shimmer animation and auto-collapse
- Refactor chat UI: no avatars, bordered user bubbles, plain assistant text
- Support {prompt} placeholder in agent args for flexible invocation
- Add persistent ACP sessions with proper cleanup on app exit
- Detect auth errors and show actionable messages to users
- Fallback to raw process spawn for agents without ACP support

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-14 17:21:28 +08:00
bincxzandClaude Opus 4.6 72887c35b5 feat: add AI chat panel with Vercel AI SDK integration
Add a comprehensive AI assistant feature to netcatty, enabling AI-powered
terminal automation and multi-host orchestration.

Core features:
- AI chat side panel (Zed-style) with agent selector, session history,
  conversation export (Markdown/JSON/TXT), and streaming responses
- Catty Agent: built-in terminal assistant with 9 tools (terminal exec,
  SFTP read/write, multi-host orchestration) using zod schemas
- BYOK provider support: OpenAI, Anthropic, Google, Ollama, OpenRouter,
  and custom OpenAI-compatible endpoints
- Three-tier permission model: Observer / Confirm / Autonomous
- Command safety: blocklist patterns, doom loop detection, abort support
- External agent support: ACP protocol (JSON-RPC over stdio) for
  Claude Agent, Codex CLI, Gemini CLI, and custom agents

Tech stack:
- Vercel AI SDK (ai, @ai-sdk/openai, @ai-sdk/anthropic, @ai-sdk/google)
  with streamText + fullStream for streaming tool-call loops
- AI Elements: adapted conversation (use-stick-to-bottom), message
  (streamdown markdown), prompt-input (InputGroup) components
- Custom bridge fetch adapter routing all API calls through Electron
  main process IPC to avoid CORS
- zod for tool parameter schemas

UI components:
- AIChatSidePanel, AgentSelector, ChatInput, ChatMessageList
- PermissionDialog, ExecutionPlan, ConversationExport
- AI Elements: conversation, message, tool-call, prompt-input
- New UI primitives: InputGroup, Spinner
- Settings AI tab for BYOK configuration and safety settings

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-14 14:22:28 +08:00