Automated testing is valuable when broken behavior produces useful evidence, not merely when generated files run. This guide uses a fictional community repair-class booking feature: a user must not occupy the same session twice, cancellation releases a place, and network failure must not display unsupported success. Test design, failure diagnosis, and framework/CI maintenance form one reviewable chain. No tests or scans were executed against the reader's project for this article.
Define behavior independently of implementation
Specify users, sessions, capacity, states, and idempotency. A second request for the same user and session might return the existing booking or a defined conflict; the product contract must choose. The agent should not infer correctness from whatever the current implementation happens to do. Clarify rebooking, deadlines, administrator operations, and concurrency before encoding assumptions as tests.
Provide acceptance criteria, interfaces, existing framework, fixtures, permitted environment, forbidden dependencies, and the success command. Retain code and contract versions so design, execution, and diagnosis share expectations. Unit, integration, API, and E2E layers address different risks. Coverage percentages provide clues rather than proof of useful behavior.
We searched GitHub for mature open-source approaches and consulted Microsoft Playwright's browser, CI, and trace guidance. It fits critical user journeys and failure evidence, subject to version, license, browser dependencies, environment, and data checks. It need not wrap every unit rule in a browser. Documentation examples do not establish that the reader's project works. No dependencies were installed or tests run for this guide.
Scenario one: map risk to the cheapest useful layer
Use unit tests for deadline calculations, state transitions, and input validation with fixed time and timezone. Use an isolated database for persistence, uniqueness, and transaction integration. API tests check responses, identity, and contract; browser tests cover selected critical journeys. A lower-cost layer that exposes the defect is often more maintainable than clicking every rule through the UI.
Build a matrix of input, expectation, failure meaning, and layer. Ordinary booking checks storage and feedback. Repeated requests check that occupancy does not increase twice. Capacity cases include one remaining place and a full session. Cancellation checks state and availability. Identity cases include unauthenticated access and another user's booking. Timeout cases require honest unknown or failure feedback. Last-place concurrency needs actual constraint testing; two sequential clicks are not simultaneous contention.
Fixtures use fictional users, independent sessions, and cleanable state with separate databases and notification doubles. Do not inherit production email, payment, API, or personal credentials. Necessary external integration checks require explicit destination and budget scope. Label doubles so mock success is not reported as real-service success.
Derive tests from contracts and design, using implementation to locate behavior rather than mirror every line. Assert meaningful results such as stored identifiers, occupancy, state, and visible feedback. A called function or the word success alone rarely proves completion. Review snapshot changes; bulk regeneration can convert a defect into a new expected image.
Test for false green through a controlled mutation: break duplicate protection and confirm the targeted case fails, then restore and rerun. Use an isolated reversible copy, preserving evidence without damaging real data. Confirm failure is caused by the intended rule rather than a broken test startup. Only after a case turns red for the right reason should it be considered for the long-lived suite.
Scenario two: stabilize execution and classify failures
Record command, working directory, revision, locked dependencies, environment-variable names, initialization, browser, and system information. Keep secret values out of reports. Observe service readiness instead of relying on a fixed sleep. Browser automation should inspect actual DOM and choose stable controls rather than fragile positional selectors. Each test constructs and cleans its own state without depending on another test's order.
Retain exit status, executed count, passes, failures, skips, duration, logs, and necessary screenshots or traces. Reconcile totals; zero cases or all skipped is not success. After disconnection, truncation, or an unknown timeout, inspect processes and artifacts before repeating side-effecting work. Separate planned, executed, and verified states.
Classify red runs as product defects, test defects, unstable dependencies, or environment failures. Compare stored state with the contract for product issues. Inspect selectors, fixtures, assumptions, and assertions for test issues. Investigate timing, shared state, randomness, clocks, and external services for instability. Check startup, dependencies, browsers, and configuration for environment problems. Classification may remain unresolved rather than justify changing business rules for a green result.
Keep first-failure evidence before focused retries. A passing retry is still an instability observation, not erasure of the original failure. Record attempts and frequency; unlimited retries conceal defects. Temporary quarantine needs an owner, reason, mitigation, and expiry, with alternative critical-path checks. It is not permanent silence.
After repair, run the relevant set and broaden only where change impact, new failures, or remaining uncertainty justify it. Do not remove valid assertions, inflate timeouts beyond usefulness, or swallow errors to obtain success. A maintainer should reproduce the issue and understand expected versus actual behavior from the report.
Use a verified Skill at the local browser step
Webapp Testing fits local booking UI interaction, screenshots, logs, and reproducible findings. Its public detail, pinned Anthropic source, and Apache-2.0 license record were checked. Prepare Python, Playwright, suitable browser binaries, startup instructions, a port, and approved fictional data.
Use it after observing the rendered DOM to exercise booking, duplicate submission, cancellation, and failure feedback with explicit assertions. It does not certify database concurrency, complete API authorization, security, or every device. Source verification is not a claim that tests ran for the reader. The original eight package names are not automatically verified SkillPI records.
Scenario three: maintain frameworks and CI as a system
Reuse the project's existing frameworks where adequate, with clear unit, API, and browser responsibilities. Before migration, compare fixture construction, mocking, assertions, features, and reports. Port a small known set and check equivalence before claiming old coverage has moved. A green new framework does not establish complete migration.
CI defines dependency setup, data, service readiness, commands, timeouts, concurrency, and artifact retention. Match browser and library versions, control external dependencies, and preserve useful failures. Begin with a stable low-concurrency baseline before sharding. Local parameters may be unsuitable for a resource-constrained runner.
Give parallel jobs independent data and ports so tests do not compete for the same fictional session. Distinguish unexecuted paths from tested behavior in coverage. Repeating similar tests at one layer can inflate numbers without new information. Arrange fast valuable checks for early feedback and slower work by risk, while explaining any release-gate omission.
Logs, HTML reports, and traces can contain accounts, tokens, and page data. Use approved artifact storage with access and retention controls, not automatic public hosting. Disabling one sharing option does not prove every dependency stays offline; inspect actual configuration. Notifications and report recipients also need clear authorization.
Assign owners to cases and document fixtures, external dependencies, retry policy, known gaps, and retirement criteria. Retest after contract, rule, framework, or browser changes. Rewrite or remove obsolete tests because the rule or risk changed, not merely because they often fail. Cleanup also needs a traceable data and artifact scope.
Discover and compare more testing methods
Search with stack, executable acceptance, layer, existing framework, and forbidden production or external dependencies. Verify public detail, pinned source, license, prerequisites, and purpose. Authoring, diagnosis, and CI upkeep are different jobs; combinations should share the same criteria and fixtures instead of inventing expectations independently.
Trial one candidate against the baseline for assertion strength, diagnostics, repeatability, duration, and maintenance. Discovery recommendations do not grant execution authority or guarantee the best match. Popularity and blanket security claims are not evidence. Refresh associations as verified catalog entries appear, retaining old rationale and new trials.
Worked acceptance request
A useful request is: “For the repair-class booking contract, map duplicate, capacity, cancellation, and timeout risks to test layers. Reuse the current framework and fictional fixtures. Propose cases before implementation, run only in the named test environment, and retain the first failure. Explain which results use doubles. Do not alter production data or weaken assertions.”
The review should ask whether each case would catch its intended defect, whether results can be reproduced independently, and whether evidence survives a CI failure. Include setup failure and missing-data cases so the system cannot silently skip meaningful coverage. Document test-data cleanup separately from product behavior; a suite that passes while contaminating the next run is not ready for CI.
FAQ and final acceptance
Unit or E2E first? Choose by risk and feedback speed, commonly testing rules below the browser and reserving UI work for critical journeys. What materials are needed? Behavior, inputs, expected results, boundaries, framework, fixtures, limits, and a success command. Can several Skills cooperate? Yes, with shared data, responsibilities, and acceptance rather than a generator changing its own standard and declaring success.
How is false green detected? A controlled broken behavior should fail, restoration should pass for the right reason, and execution count should be nonzero. What about flaky CI? Retain evidence, inspect timing and isolation, and bound retries and quarantine with ownership. Is high coverage sufficient? No: assertions, boundaries, diagnostics, and maintenance determine value.
Deliver the risk matrix, cases, fixtures, environment and commands, first-failure and repair evidence, actual results, CI configuration, and maintenance notes. State where local, mock, integration, and target-environment evidence ends. One layer's success cannot be enlarged into full-system acceptance. This article provides a method, not generated, executed, or submitted tests for the reader.
Sources
This original booking example covers the bilingual sources' design/coverage, execution/diagnosis, and framework/CI scenarios. Browser and CI references are Playwright's official CI documentation source and trace guidance. The linked Skill is limited to its pinned-source scope, without adopting install figures or all-catalog security guarantees.