Files
next-ai-draw-io/tests/e2e/error-handling.spec.ts

137 lines
4.7 KiB
TypeScript
Raw Normal View History

test: add Vitest and Playwright testing infrastructure (#512) * test: add Vitest and Playwright testing infrastructure - Add Vitest for unit tests (39 tests) - cached-responses.test.ts - ai-providers.test.ts - chat-helpers.test.ts - utils.test.ts - Add Playwright for E2E tests (3 smoke tests) - Homepage load - Japanese locale - Settings dialog - Add CI workflow (.github/workflows/test.yml) - Add vitest.config.mts and playwright.config.ts - Update .gitignore for test artifacts * test: add more E2E tests for UI components - Chat panel tests (interactive elements, iframe) - Settings tests (dark mode, language, draw.io theme) - Save dialog tests (buttons exist) - History dialog tests - Model config tests - Keyboard interaction tests - Upload area tests Total: 15 E2E tests, all passing * test: fix E2E test issues from review Fixes based on Gemini and Codex review: - Remove brittle nth(1) selector in keyboard tests - Remove waitForTimeout(500) race condition - Remove if(isVisible) silent skip patterns - Add proper assertions instead of no-op checks - Remove expect(count >= 0) that always passes - Remove unused hasProviderUI variable All 14 E2E tests and 39 unit tests pass. * style: auto-format with Biome * fix: resolve lint errors for CI * test(e2e): add diagram generation tests with mocked AI responses - Add tests for generate, edit, and append diagram operations - Use SSE mocked responses matching AI SDK UI message stream format - Generate mxCell XML directly in tests for deterministic assertions - Tests verify tool card rendering and 'Complete' badge state * test: add comprehensive E2E tests for all major features - Error handling tests (API errors, rate limits, network timeout, truncated XML) - Multi-turn conversation tests (sequential requests, history preservation) - File upload tests (upload button, file preview, sending with message) - Theme switching tests (dark mode toggle, persistence, system preference) - Language switching tests (EN/JA/ZH, persistence, locale URLs) - Iframe interaction tests (draw.io loading, toolbar, diagram rendering) - Copy/paste tests (chat input, XML input, special characters) - History restore tests (new chat, persistence, browser navigation) * refactor: extract shared test helpers and improve error assertions - Create tests/e2e/lib/helpers.ts with shared SSE mock functions - Add proper error UI assertions to error-handling.spec.ts - Remove waitForTimeout calls in favor of real assertions - Update 6 test files to use shared helpers * docs: add testing section to CONTRIBUTING.md * fix: improve test infrastructure based on PR review - Fix double build in CI: remove redundant build from playwright webServer - Export chat helpers from shared module for proper unit testing - Replace waitForTimeout with explicit waits in E2E tests - Add data-testid attributes to settings and new chat buttons - Add list reporter for CI to show failures in logs - Add Playwright browser caching to speed up CI - Add vitest coverage configuration - Fix conditional test assertions to use test.skip() instead of silent pass - Remove unused variables flagged by linter * fix: improve E2E test assertions and remove silent skips - Replace silent test.skip() with explicit conditional skips - Add actual persistence assertion after page reload - Use data-testid selector for new chat button test * refactor: add shared fixtures and test.step() patterns - Add tests/e2e/lib/fixtures.ts with shared test helpers - Add tests/e2e/fixtures/diagrams.ts with XML test data - Add expectBeforeAndAfterReload() helper for persistence tests - Add test.step() for better test reporting in complex tests - Consolidate mock helpers into fixtures module - Reduce code duplication across 17 test files * fix: make persistence tests more reliable - Remove expectBeforeAndAfterReload from mocked API tests - Add explicit test.step() for before/after reload checks - Add retry config for flaky clipboard tests - Add sleep after reload for language persistence test * test: remove flaky XML paste test * docs: run both unit and e2e tests before PR * chore: add type check and unit test git hooks --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-01-05 01:37:32 +09:00
import { TRUNCATED_XML } from "./fixtures/diagrams"
import {
createErrorMock,
expect,
getChatInput,
getIframe,
sendMessage,
test,
} from "./lib/fixtures"
test.describe("Error Handling", () => {
test("displays error message when API returns 500", async ({ page }) => {
await page.route(
"**/api/chat",
createErrorMock(500, "Internal server error"),
)
await page.goto("/", { waitUntil: "networkidle" })
await getIframe(page).waitFor({ state: "visible", timeout: 30000 })
await sendMessage(page, "Draw a cat")
// Should show error indication
const errorIndicator = page
.locator('[role="alert"]')
.or(page.locator("[data-sonner-toast]"))
.or(page.locator("text=/error|failed|something went wrong/i"))
await expect(errorIndicator.first()).toBeVisible({ timeout: 10000 })
// User should be able to type again
const chatInput = getChatInput(page)
await chatInput.fill("Retry message")
await expect(chatInput).toHaveValue("Retry message")
})
test("displays error message when API returns 429 rate limit", async ({
page,
}) => {
await page.route(
"**/api/chat",
createErrorMock(429, "Rate limit exceeded"),
)
await page.goto("/", { waitUntil: "networkidle" })
await getIframe(page).waitFor({ state: "visible", timeout: 30000 })
await sendMessage(page, "Draw a cat")
// Should show error indication for rate limit
const errorIndicator = page
.locator('[role="alert"]')
.or(page.locator("[data-sonner-toast]"))
.or(page.locator("text=/rate limit|too many|try again/i"))
await expect(errorIndicator.first()).toBeVisible({ timeout: 10000 })
// User should be able to type again
const chatInput = getChatInput(page)
await chatInput.fill("Retry after rate limit")
await expect(chatInput).toHaveValue("Retry after rate limit")
})
test("handles network timeout gracefully", async ({ page }) => {
await page.route("**/api/chat", async (route) => {
await new Promise((resolve) => setTimeout(resolve, 2000))
await route.abort("timedout")
})
await page.goto("/", { waitUntil: "networkidle" })
await getIframe(page).waitFor({ state: "visible", timeout: 30000 })
await sendMessage(page, "Draw a cat")
// Should show error indication for network failure
const errorIndicator = page
.locator('[role="alert"]')
.or(page.locator("[data-sonner-toast]"))
.or(page.locator("text=/error|failed|network|timeout/i"))
await expect(errorIndicator.first()).toBeVisible({ timeout: 10000 })
// After timeout, user should be able to type again
const chatInput = getChatInput(page)
await chatInput.fill("Try again after timeout")
await expect(chatInput).toHaveValue("Try again after timeout")
})
test("shows truncated badge for incomplete XML", async ({ page }) => {
const toolCallId = `call_${Date.now()}`
const textId = `text_${Date.now()}`
const messageId = `msg_${Date.now()}`
const events = [
{ type: "start", messageId },
{ type: "text-start", id: textId },
{ type: "text-delta", id: textId, delta: "Creating diagram..." },
{ type: "text-end", id: textId },
{
type: "tool-input-start",
toolCallId,
toolName: "display_diagram",
},
{
type: "tool-input-available",
toolCallId,
toolName: "display_diagram",
input: { xml: TRUNCATED_XML },
},
{
type: "tool-output-error",
toolCallId,
error: "XML validation failed",
},
{ type: "finish" },
]
await page.route("**/api/chat", async (route) => {
await route.fulfill({
status: 200,
contentType: "text/event-stream",
body:
events
.map((e) => `data: ${JSON.stringify(e)}\n\n`)
.join("") + "data: [DONE]\n\n",
})
})
await page.goto("/", { waitUntil: "networkidle" })
await getIframe(page).waitFor({ state: "visible", timeout: 30000 })
await sendMessage(page, "Draw something")
// Should show truncated badge
await expect(page.locator('text="Truncated"')).toBeVisible({
timeout: 15000,
})
})
})