test: add Vitest and Playwright testing infrastructure (#512)
* test: add Vitest and Playwright testing infrastructure
- Add Vitest for unit tests (39 tests)
- cached-responses.test.ts
- ai-providers.test.ts
- chat-helpers.test.ts
- utils.test.ts
- Add Playwright for E2E tests (3 smoke tests)
- Homepage load
- Japanese locale
- Settings dialog
- Add CI workflow (.github/workflows/test.yml)
- Add vitest.config.mts and playwright.config.ts
- Update .gitignore for test artifacts
* test: add more E2E tests for UI components
- Chat panel tests (interactive elements, iframe)
- Settings tests (dark mode, language, draw.io theme)
- Save dialog tests (buttons exist)
- History dialog tests
- Model config tests
- Keyboard interaction tests
- Upload area tests
Total: 15 E2E tests, all passing
* test: fix E2E test issues from review
Fixes based on Gemini and Codex review:
- Remove brittle nth(1) selector in keyboard tests
- Remove waitForTimeout(500) race condition
- Remove if(isVisible) silent skip patterns
- Add proper assertions instead of no-op checks
- Remove expect(count >= 0) that always passes
- Remove unused hasProviderUI variable
All 14 E2E tests and 39 unit tests pass.
* style: auto-format with Biome
* fix: resolve lint errors for CI
* test(e2e): add diagram generation tests with mocked AI responses
- Add tests for generate, edit, and append diagram operations
- Use SSE mocked responses matching AI SDK UI message stream format
- Generate mxCell XML directly in tests for deterministic assertions
- Tests verify tool card rendering and 'Complete' badge state
* test: add comprehensive E2E tests for all major features
- Error handling tests (API errors, rate limits, network timeout, truncated XML)
- Multi-turn conversation tests (sequential requests, history preservation)
- File upload tests (upload button, file preview, sending with message)
- Theme switching tests (dark mode toggle, persistence, system preference)
- Language switching tests (EN/JA/ZH, persistence, locale URLs)
- Iframe interaction tests (draw.io loading, toolbar, diagram rendering)
- Copy/paste tests (chat input, XML input, special characters)
- History restore tests (new chat, persistence, browser navigation)
* refactor: extract shared test helpers and improve error assertions
- Create tests/e2e/lib/helpers.ts with shared SSE mock functions
- Add proper error UI assertions to error-handling.spec.ts
- Remove waitForTimeout calls in favor of real assertions
- Update 6 test files to use shared helpers
* docs: add testing section to CONTRIBUTING.md
* fix: improve test infrastructure based on PR review
- Fix double build in CI: remove redundant build from playwright webServer
- Export chat helpers from shared module for proper unit testing
- Replace waitForTimeout with explicit waits in E2E tests
- Add data-testid attributes to settings and new chat buttons
- Add list reporter for CI to show failures in logs
- Add Playwright browser caching to speed up CI
- Add vitest coverage configuration
- Fix conditional test assertions to use test.skip() instead of silent pass
- Remove unused variables flagged by linter
* fix: improve E2E test assertions and remove silent skips
- Replace silent test.skip() with explicit conditional skips
- Add actual persistence assertion after page reload
- Use data-testid selector for new chat button test
* refactor: add shared fixtures and test.step() patterns
- Add tests/e2e/lib/fixtures.ts with shared test helpers
- Add tests/e2e/fixtures/diagrams.ts with XML test data
- Add expectBeforeAndAfterReload() helper for persistence tests
- Add test.step() for better test reporting in complex tests
- Consolidate mock helpers into fixtures module
- Reduce code duplication across 17 test files
* fix: make persistence tests more reliable
- Remove expectBeforeAndAfterReload from mocked API tests
- Add explicit test.step() for before/after reload checks
- Add retry config for flaky clipboard tests
- Add sleep after reload for language persistence test
* test: remove flaky XML paste test
* docs: run both unit and e2e tests before PR
* chore: add type check and unit test git hooks
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-01-05 01:37:32 +09:00
|
|
|
/**
|
|
|
|
|
* Shared test helpers for E2E tests
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Creates a mock SSE response for the chat API
|
|
|
|
|
* Format matches AI SDK UI message stream protocol
|
|
|
|
|
*/
|
|
|
|
|
export function createMockSSEResponse(
|
|
|
|
|
xml: string,
|
|
|
|
|
text: string,
|
|
|
|
|
toolName = "display_diagram",
|
|
|
|
|
) {
|
|
|
|
|
const messageId = `msg_${Date.now()}`
|
|
|
|
|
const toolCallId = `call_${Date.now()}`
|
|
|
|
|
const textId = `text_${Date.now()}`
|
|
|
|
|
|
|
|
|
|
const events = [
|
|
|
|
|
{ type: "start", messageId },
|
|
|
|
|
{ type: "text-start", id: textId },
|
|
|
|
|
{ type: "text-delta", id: textId, delta: text },
|
|
|
|
|
{ type: "text-end", id: textId },
|
|
|
|
|
{ type: "tool-input-start", toolCallId, toolName },
|
|
|
|
|
{ type: "tool-input-available", toolCallId, toolName, input: { xml } },
|
|
|
|
|
{
|
|
|
|
|
type: "tool-output-available",
|
|
|
|
|
toolCallId,
|
|
|
|
|
output: "Successfully displayed the diagram",
|
|
|
|
|
},
|
|
|
|
|
{ type: "finish" },
|
|
|
|
|
]
|
|
|
|
|
|
|
|
|
|
return (
|
|
|
|
|
events.map((e) => `data: ${JSON.stringify(e)}\n\n`).join("") +
|
|
|
|
|
"data: [DONE]\n\n"
|
|
|
|
|
)
|
|
|
|
|
}
|
|
|
|
|
|
feat(diagram-engine): wire up restructure_diagram + stencil catalog
Closes the loop: the model can now build and edit AWS architecture diagrams by
declaring structure, and never writes an mxCell again.
catalog.ts — 983 AWS icon and 19 group stencils as a name→style map, generated from
drawio-ai-kit's catalog (itself generated from jgraph's draw.io shape index). Styles
are verbatim, so the official category colours, connection points and aspect=fixed
come along for free and nothing is hand-assembled. An invented name is rejected with
suggestions instead of rendering as a blank square, which is what draw.io does with
an unknown resIcon today.
operations.ts — what the model actually sends: add_icon / add_container / move /
link / set_dir and so on, applied in order against the tree. Guards the things that
break a diagram quietly: duplicate ids (draw.io drops one of the two cells), edges
left pointing at a removed node, and moving a container inside itself.
index.ts — the entry point. current XML → parse → apply ops → check names → layout →
render → new XML. The tree is not stored between calls; it is re-derived from the
canvas every time, so a user's manual edits are input to the next layout rather than
state to reconcile.
Token cost, measured with Claude's tokenizer rather than estimated:
- build a VPC diagram: 515 tok as operations vs 3180 as XML (6.2x)
- add one icon: 27 tok as an operation vs 3823 re-emitting (142x)
- read current state: 216 tok as an outline vs 3180 as XML (14.7x)
The 142x is the one that matters day to day: "add a Redis" is one operation, not a
rewrite of the whole diagram.
Routing in the system prompt sends AWS architecture through this path and leaves
flowcharts, BPMN, sequence diagrams, mind maps and Azure/GCP on display_diagram —
the layout engine's primitives (nested rows, columns, grids) do not model a sequence
diagram's lifelines or a mind map's radial spread, and pretending otherwise would
make those worse rather than better.
Also: added a NOTICE recording the MIT port and the AWS Architecture Icons terms,
and a narrow .gitignore exception so the generated catalog is tracked while the
root data/ directory (admin settings, contains secrets) stays ignored.
403 unit tests + 3 new e2e. Verified in the real app: a structural tool call renders
with real stencils and container markers; a second call adds one node and keeps
everything from the first; an invented name is refused and nothing is drawn. The 13
existing diagram e2e tests still pass.
2026-08-09 12:13:54 +09:00
|
|
|
/**
|
|
|
|
|
* Creates a mock SSE response for a tool whose input is not `{ xml }`.
|
|
|
|
|
*
|
|
|
|
|
* createMockSSEResponse hardcodes the display_diagram shape; this takes the input
|
|
|
|
|
* object verbatim, so tools like restructure_diagram can be driven through the same
|
|
|
|
|
* path the model uses.
|
|
|
|
|
*/
|
|
|
|
|
export function createMockToolResponse(
|
|
|
|
|
toolName: string,
|
|
|
|
|
input: unknown,
|
|
|
|
|
text: string,
|
|
|
|
|
) {
|
|
|
|
|
const messageId = `msg_${Date.now()}`
|
|
|
|
|
const toolCallId = `call_${Date.now()}`
|
|
|
|
|
const textId = `text_${Date.now()}`
|
|
|
|
|
|
|
|
|
|
const events = [
|
|
|
|
|
{ type: "start", messageId },
|
|
|
|
|
{ type: "text-start", id: textId },
|
|
|
|
|
{ type: "text-delta", id: textId, delta: text },
|
|
|
|
|
{ type: "text-end", id: textId },
|
|
|
|
|
{ type: "tool-input-start", toolCallId, toolName },
|
|
|
|
|
{ type: "tool-input-available", toolCallId, toolName, input },
|
|
|
|
|
{ type: "finish" },
|
|
|
|
|
]
|
|
|
|
|
|
|
|
|
|
return (
|
|
|
|
|
events.map((e) => `data: ${JSON.stringify(e)}\n\n`).join("") +
|
|
|
|
|
"data: [DONE]\n\n"
|
|
|
|
|
)
|
|
|
|
|
}
|
|
|
|
|
|
test: add Vitest and Playwright testing infrastructure (#512)
* test: add Vitest and Playwright testing infrastructure
- Add Vitest for unit tests (39 tests)
- cached-responses.test.ts
- ai-providers.test.ts
- chat-helpers.test.ts
- utils.test.ts
- Add Playwright for E2E tests (3 smoke tests)
- Homepage load
- Japanese locale
- Settings dialog
- Add CI workflow (.github/workflows/test.yml)
- Add vitest.config.mts and playwright.config.ts
- Update .gitignore for test artifacts
* test: add more E2E tests for UI components
- Chat panel tests (interactive elements, iframe)
- Settings tests (dark mode, language, draw.io theme)
- Save dialog tests (buttons exist)
- History dialog tests
- Model config tests
- Keyboard interaction tests
- Upload area tests
Total: 15 E2E tests, all passing
* test: fix E2E test issues from review
Fixes based on Gemini and Codex review:
- Remove brittle nth(1) selector in keyboard tests
- Remove waitForTimeout(500) race condition
- Remove if(isVisible) silent skip patterns
- Add proper assertions instead of no-op checks
- Remove expect(count >= 0) that always passes
- Remove unused hasProviderUI variable
All 14 E2E tests and 39 unit tests pass.
* style: auto-format with Biome
* fix: resolve lint errors for CI
* test(e2e): add diagram generation tests with mocked AI responses
- Add tests for generate, edit, and append diagram operations
- Use SSE mocked responses matching AI SDK UI message stream format
- Generate mxCell XML directly in tests for deterministic assertions
- Tests verify tool card rendering and 'Complete' badge state
* test: add comprehensive E2E tests for all major features
- Error handling tests (API errors, rate limits, network timeout, truncated XML)
- Multi-turn conversation tests (sequential requests, history preservation)
- File upload tests (upload button, file preview, sending with message)
- Theme switching tests (dark mode toggle, persistence, system preference)
- Language switching tests (EN/JA/ZH, persistence, locale URLs)
- Iframe interaction tests (draw.io loading, toolbar, diagram rendering)
- Copy/paste tests (chat input, XML input, special characters)
- History restore tests (new chat, persistence, browser navigation)
* refactor: extract shared test helpers and improve error assertions
- Create tests/e2e/lib/helpers.ts with shared SSE mock functions
- Add proper error UI assertions to error-handling.spec.ts
- Remove waitForTimeout calls in favor of real assertions
- Update 6 test files to use shared helpers
* docs: add testing section to CONTRIBUTING.md
* fix: improve test infrastructure based on PR review
- Fix double build in CI: remove redundant build from playwright webServer
- Export chat helpers from shared module for proper unit testing
- Replace waitForTimeout with explicit waits in E2E tests
- Add data-testid attributes to settings and new chat buttons
- Add list reporter for CI to show failures in logs
- Add Playwright browser caching to speed up CI
- Add vitest coverage configuration
- Fix conditional test assertions to use test.skip() instead of silent pass
- Remove unused variables flagged by linter
* fix: improve E2E test assertions and remove silent skips
- Replace silent test.skip() with explicit conditional skips
- Add actual persistence assertion after page reload
- Use data-testid selector for new chat button test
* refactor: add shared fixtures and test.step() patterns
- Add tests/e2e/lib/fixtures.ts with shared test helpers
- Add tests/e2e/fixtures/diagrams.ts with XML test data
- Add expectBeforeAndAfterReload() helper for persistence tests
- Add test.step() for better test reporting in complex tests
- Consolidate mock helpers into fixtures module
- Reduce code duplication across 17 test files
* fix: make persistence tests more reliable
- Remove expectBeforeAndAfterReload from mocked API tests
- Add explicit test.step() for before/after reload checks
- Add retry config for flaky clipboard tests
- Add sleep after reload for language persistence test
* test: remove flaky XML paste test
* docs: run both unit and e2e tests before PR
* chore: add type check and unit test git hooks
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-01-05 01:37:32 +09:00
|
|
|
/**
|
|
|
|
|
* Creates a text-only SSE response (no tool call)
|
|
|
|
|
*/
|
|
|
|
|
export function createTextOnlyResponse(text: string) {
|
|
|
|
|
const messageId = `msg_${Date.now()}`
|
|
|
|
|
const textId = `text_${Date.now()}`
|
|
|
|
|
|
|
|
|
|
const events = [
|
|
|
|
|
{ type: "start", messageId },
|
|
|
|
|
{ type: "text-start", id: textId },
|
|
|
|
|
{ type: "text-delta", id: textId, delta: text },
|
|
|
|
|
{ type: "text-end", id: textId },
|
|
|
|
|
{ type: "finish" },
|
|
|
|
|
]
|
|
|
|
|
|
|
|
|
|
return (
|
|
|
|
|
events.map((e) => `data: ${JSON.stringify(e)}\n\n`).join("") +
|
|
|
|
|
"data: [DONE]\n\n"
|
|
|
|
|
)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Creates a mock SSE response with a tool error
|
|
|
|
|
*/
|
|
|
|
|
export function createToolErrorResponse(text: string, errorMessage: string) {
|
|
|
|
|
const messageId = `msg_${Date.now()}`
|
|
|
|
|
const toolCallId = `call_${Date.now()}`
|
|
|
|
|
const textId = `text_${Date.now()}`
|
|
|
|
|
|
|
|
|
|
const events = [
|
|
|
|
|
{ type: "start", messageId },
|
|
|
|
|
{ type: "text-start", id: textId },
|
|
|
|
|
{ type: "text-delta", id: textId, delta: text },
|
|
|
|
|
{ type: "text-end", id: textId },
|
|
|
|
|
{ type: "tool-input-start", toolCallId, toolName: "display_diagram" },
|
|
|
|
|
{
|
|
|
|
|
type: "tool-input-available",
|
|
|
|
|
toolCallId,
|
|
|
|
|
toolName: "display_diagram",
|
|
|
|
|
input: { xml: "<invalid>" },
|
|
|
|
|
},
|
|
|
|
|
{ type: "tool-output-error", toolCallId, error: errorMessage },
|
|
|
|
|
{ type: "finish" },
|
|
|
|
|
]
|
|
|
|
|
|
|
|
|
|
return (
|
|
|
|
|
events.map((e) => `data: ${JSON.stringify(e)}\n\n`).join("") +
|
|
|
|
|
"data: [DONE]\n\n"
|
|
|
|
|
)
|
|
|
|
|
}
|