Skip to content
Testing

Systematic Debugging

Investigate a reproducible failure before choosing a code change.

Install with Skills CLI
npx skills add obra/superpowers --skill systematic-debugging

What it does

Repeated speculative fixes can obscure the original problem and introduce new failures. This skill gives the agent a staged debugging process that gathers evidence, compares working behavior, and tests a focused explanation.

What changes after install

  • Starts with errors, reproduction steps, and recent changes.
  • Traces data across component boundaries to locate the failing layer.
  • Tests one hypothesis before applying a minimal root-cause fix.

Install

Run this command with Node.js 22.20 or later and select your agent in the Skills CLI.

Install with Skills CLI
npx skills add obra/superpowers --skill systematic-debugging

Before using Systematic Debugging

  • Provide a reproducible failure, relevant logs and access to the affected code and environment. Be able to run a focused reproduction to test the suspected cause.

Read the reviewed setup instructions for the selected workflow.

When to use it

  • Diagnosing a failing test, build, or reproducible application defect.
  • Investigating a problem that crosses several services or processing stages.

When not to use it

  • Applying repeated unrelated patches without checking what each one proves.

A worked example

Give the agent a failing CI run, the build script from that commit and the instructions for the same build on a developer machine. Suppose the failing job stops while opening config/build.json, yet the file exists in the checkout and the local build succeeds. Ask: “Find where the two builds first disagree. Show how to reproduce the failure and which change addresses it.” This example applies the upstream investigation steps to the local-versus-CI case.

  1. Record the exact command, the first file-open error, the revision and the directory where the command started. Read the preceding lines of the log as well as the final job summary. Try the same command on that revision. A green local run is useful evidence if you can say what was different about it, including the runtime version, dependency installation and launch directory.
  2. Follow the path through the job. At the workflow step, record the working directory and the path handed to the build script. At the file reader, record the resolved path and whether the file is present there. Compare those observations with the passing run. Use only the values needed to distinguish the two cases. A report about a missing file has no use for unrelated environment variables or credentials.
  3. Find a working job that calls the same script and compare the calls in full. Suppose both pass the argument config/build.json, but one runs from the repository root and the other from a package subdirectory. The argument looks identical in the logs. The two processes can still be asking the filesystem for different files. Check the resolved paths before deciding the checkout is incomplete.
  4. State one explanation and one experiment: “If this relative path depends on the caller’s directory, launching the same revision from the package directory should reproduce the error.” Keep the input, runtime and dependencies fixed while changing that one condition. If both invocations succeed, record that result and revisit the next observed difference. Do not make the path change on the strength of the error message alone.
  5. If the experiment reproduces the failure, capture the smaller failing invocation as a regression case. Check the build script’s intended contract with its caller. If the caller is responsible for launching from the root, correct that invocation. If callers are allowed to launch from either directory, address the path resolution at its source. Apply the one change supported by the contract, rerun the failing case, then run the previously passing one.

The suggested investigation follows the four phases in SKILL.md. The method of following a bad path back to the caller is described in root-cause-tracing.md.

Troubleshooting

  • The failed job passes on a retry. Save the failing invocation and compare its inputs and timing with the passing one. Record how often the symptom occurs and which evidence the two logs contain. If the decisive value was never recorded, add a focused observation at that boundary for the next run. Leave the cause open until the evidence distinguishes a timing difference from a change in input.
  • Three settings changed and CI turned green. Return to a disposable reproduction of the failing configuration. Carry over just the change predicted by the hypothesis. If it has no effect alone, compare what the bundled run actually changed before deciding which part belongs in the fix. A simultaneous dependency update and directory change cannot tell you which one restored the missing file.
  • Each attempted fix reveals a different failure elsewhere. Record the symptom and assumption behind each attempt. The upstream guide calls for discussing the architecture after three unsuccessful fixes. Bring the conflicting assumptions to the maintainer, such as two callers that require incompatible starting directories, before attempting a fourth patch.

How it chooses an approach

The upstream workflow divides investigation into evidence gathering, comparison with working examples, hypothesis testing and implementation. For a multi-component flow, it recommends examining what enters and leaves each boundary. If an input already has the wrong value on entry to a component, move one caller back. If the input arrives intact and the output changes there, examine that transformation. In the CI example, the place that reports “file missing” may be several calls away from the place that selected the directory.

Ask the agent to state one explanation that fits the evidence and one check that would support or contradict it. When the symptom is reliable, a short command or a focused test gives the investigation a repeatable starting point. When it only happens in CI, collect the missing observation there first. Diagnostic work needs access to relevant logs and a safe reproduction environment. If the only available record says “job failed,” identify the log or permission needed to answer the next question.

When to use another skill

Use Test-Driven Development for the regression cycle once the intended behavior is clear. Hand over the smallest failing invocation, its expected outcome and the evidence that connects it to the original job. If the team has yet to decide whether the script should accept launches from a subdirectory, settle that expectation first. A regression test that encodes the wrong contract can preserve the very behavior the caller needs changed.

What to check after it runs

  • Repeat the original failing command from its recorded starting directory and revision with the proposed change applied. Inspect the resolved config path and the build step that uses it. A job can exit successfully after skipping a step, so look for the output that previously failed to be produced.
  • Run the regression case against the unmodified code and again with the focused fix. Confirm that it fails for the reported reason on the former and reaches the expected result on the latter. Save the command and expected result alongside the change so another maintainer can repeat the comparison.
  • Run the formerly passing invocation from the repository root and compare its input and output with the earlier passing run. Check related callers of the same build script for a change in which configuration file they load. Two files can have the same name and different settings.
  • For an intermittent report, list which conditions were actually reproduced and which remain unobserved. Remove temporary diagnostic output that no longer serves the investigation. Save the failing command, the directory it ran in and the path the file reader tried to open.

FAQ sources: Saving evidence before a rerun · #2232, matching the reported input · #2449.

More in Testing

FAQ

Does this skill need a test suite?
A test suite helps, but the initial requirement is a reliable reproduction and evidence. A targeted script or manual reproduction may be the starting point.
What if the failure is intermittent?
Gather enough context and diagnostic evidence to understand when it occurs instead of assuming a successful rerun proves it is fixed.
What should I save before rerunning a flaky test?
Copy the failed run’s log, screenshot or diff to a separate location first. In issue #2232 the runner cleared its output directory at the start of the next run. Keep the command and input with the saved evidence so a passing retry cannot erase the comparison.
Can a passing regression test confirm that I fixed the user's reported bug?
Check the input that triggered the report. In issue #2449, a constructed all-day event reproduced a two-day display, but the reported event was a timed event ending at midnight. Compare the original record or a redacted equivalent with the regression input before closing the report.