Fix

What Is Fix?

The "Fix" skill is a specialized Claude Code automation designed to streamline the process of diagnosing and resolving failing or flaky Playwright tests. Integrated into development workflows, Fix provides a systematic approach for identifying, categorizing, and addressing the root causes of intermittent or persistent test failures. By leveraging reproducible steps and an established taxonomy, Fix empowers engineering teams to maintain reliable end-to-end test suites, improving software quality and deployment confidence.

Why Use Fix?

Automated testing is vital for modern software delivery, but test failures—especially those that occur sporadically—can erode trust in the test suite, slow down development, and mask genuine defects. Flaky tests, which pass sometimes but fail under certain conditions (such as in CI environments), are particularly notorious for creating noise and false alarms.

The Fix skill addresses these challenges by:

• Reducing Debugging Time: It encapsulates proven debugging strategies and tools, saving engineers from reinventing troubleshooting workflows.
• Promoting Consistency: By following a defined taxonomy and methodology, Fix ensures that all team members use the same approach to diagnose and fix test issues.
• Supporting Automation: Fix can be integrated into CI/CD pipelines and developer tools, making test diagnostics and remediation repeatable and scalable.
• Enabling Better Collaboration: Consistent categorization and trace capture facilitate easier communication among developers, QA engineers, and DevOps teams.

How to Get Started

To start using the Fix skill, ensure you have a Playwright test suite and access to the skill source at github.com/alirezarezvani/claude-skills.

Typical usage scenario:

Suppose you encounter a test that fails in your CI environment but passes locally. You can invoke the skill by describing your problem:

"The checkout test fails in CI but passes locally."

The Fix skill expects the following inputs:

• Test file path (e.g., e2e/login.spec.ts)
• Test name (e.g., "should redirect after login")
• Description of the failure scenario

Step-by-Step Example

1. Reproduce the Failure

   Start by running the test in question:

   bash

   npx playwright test e2e/login.spec.ts --reporter=list

   If it passes, the test might be flaky. To confirm, repeat the test multiple times:

   bash

   npx playwright test e2e/login.spec.ts --repeat-each=10 --reporter=list

   For further stress testing, use parallel execution:

   bash

   npx playwright test --fully-parallel --workers=4 --repeat-each=5

2. Capture Trace

   Enable Playwright's tracing to collect detailed information:

   bash

   npx playwright test e2e/login.spec.ts --trace=on --retries=0

   Analyze the resulting trace file for insights into failure points. Optionally, use /debug endpoints or tools to inspect trace files.

3. Categorize the Failure

   Refer to the flaky-taxonomy.md in the skill directory, which helps classify the failure into one of four categories (explained in Key Features).

Key Features

The Fix skill stands out through its structured, repeatable approach:

• Reproducibility Checks: By running tests multiple times and in parallel, Fix determines whether a failure is truly flaky or deterministic.

• Trace Acquisition: Built-in guidance for capturing Playwright trace files allows for deep inspection of test execution, including network activity, DOM snapshots, and console logs.

• Failure Taxonomy: Every failing test is classified into one of four categories, based on symptoms and diagnostic clues:

  Category	Symptom	Diagnosis Example
  Timing/Async	Fails intermittently everywhere	--repeat-each=20 reproduces locally
  Test Isolation	Fails in suite, passes when run alone	--workers=1 --grep "test name" passes
  Environment	Passes locally, fails in CI	Only fails in specific environments (check for env/config issues)
  Non-deterministic	No clear pattern, often data/state race	Multiple runs show random failure patterns

• Guided Remediation: Each category includes targeted recommendations, such as adding explicit waits, improving test isolation, or adjusting environment configurations.

• Easy Integration: The skill can be adopted as a standalone CLI workflow or embedded within existing automation pipelines.

Best Practices

To maximize the effectiveness of the Fix skill:

• Always Capture Context: Include as much detail as possible when describing test failures, such as error messages, affected file paths, and relevant environment variables.
• Automate Burn-in: Integrate repeat testing (--repeat-each) into your CI process to proactively detect flaky tests.
• Review Trace Files Regularly: Make trace analysis a standard part of test failure triage, not just for intermittent issues.
• Update the Taxonomy: As your team encounters new failure patterns, extend the taxonomy for future reference.
• Document Fixes: Record the root cause and resolution for each fixed test to build institutional knowledge and avoid regressions.

Important Notes

• The Fix skill is optimized for Playwright test suites but its methodology can inspire similar troubleshooting workflows for other test frameworks.
• Trace files can contain sensitive data; handle them securely, especially in shared or public CI environments.
• Some failures may be caused by external dependencies or infrastructure instability; always check for external factors before investing deeply in test code fixes.
• The taxonomy in flaky-taxonomy.md is a living document—review and refine it as your project evolves.
• Effective use of Fix requires a basic understanding of Playwright and familiarity with your application’s test architecture. Training the team on these practices ensures consistent and successful adoption.