/pstack:create-formal-verification-skill
Call when neededIn one lineFor cases tests can't reach (like concurrency or retries), it checks that a rule holds every time.
- When to use
- Places where a mistake is costly and tests can hardly trigger it.
- Install with
- maintain-formal-verification-skill, statechart
- Full how-to
- Bugs tests can't reach: formal verification
/pstack:create-formal-verification-skill Job leases and retry logic. The same job must never run twice at once.Just asking vs. using it
The same request, typed plainly or with /pstack:create-formal-verification-skill. Here's what changes.
With /pstack:create-formal-verification-skillWhat the document says
6 sections, about 900 words, 4 min to read. Click a section to read it.
OpeningIt contrasts tests with checking every case in a scope, then says the skill verifies one property and leaves a verify skill for the next agent.
- 1Choose the target from the codeFinds a promise in the code whose break hurts users and that tests rarely reach, starting where you point.
- 2Let the question choose the toolMatches the tool to the question: TLA+ for bad step orders, a code-level verifier for every-input questions.
- 3What makes it goodSeven standards: true to the code, able to fail, findings kept, honest scope, readable, cheap to rerun, kept in sync.
- 4Generate the verify skillWrites a project skill with frontmatter, the coverage record and sync commit, and sends TLA+ results to
/pstack:statechart. - 5Prove it before you hand offBreaks the property with a scratch change in real code. The check must catch it, then return to its old result.
- 6Hand offTells you what was verified, the tool and why, the bounds, the findings, how to run it, and what's left out.
Wording worth borrowing
A few sentences from the original: what each one makes the AI do, why it works, and how you can borrow it.
Make the check fail once
A check that you have never seen fail proves nothing.
- Makes the AI
- It breaks the code on purpose to see the check catch it before telling you the check passes.
- Why it works
- One short, absolute sentence says what makes a pass count, so a check that never fails can't be passed off as proof.
- Borrow it
- After setting up an alert, add: "Create one record that should trigger it, and show me it fires."
Name the easy way out
Choose the strongest such target in the repo, not the one that is easiest to verify.
- Makes the AI
- It goes after the code that matters most, not the part that's easiest to finish.
- Why it works
- AIs drift toward the easy option; naming that pull out loud keeps it from taking it.
- Borrow it
- When you ask for analysis: "Pick the issue most likely to lose customers, not the easiest one to analyze."
Forbid moving the goalposts
Do not weaken the property or the model until it passes
- Makes the AI
- When the code is really wrong, it reports the problem with an example instead of quietly loosening the check.
- Why it works
- Naming the most common way to cheat works far better than a general "be honest".
- Borrow it
- When you ask for a fix, add: "If a test fails, tell me what's wrong; don't edit the test to make it pass."
Say how far a pass reaches
A pass covers the model or the bound, not the whole system.
- Makes the AI
- The report says how much was checked, so you don't mistake it for a clean bill for the whole system.
- Why it works
- Putting a boundary on the conclusion stops a partial pass from reading as a full guarantee.
- Borrow it
- When you ask for test results: "Tell me which browsers this covered, and list the ones it didn't."
Full text
The SKILL.md the AI reads, as written.
descriptionGenerate a project-local skill that formally verifies the code where a bug costs the most and tests reach the least, in any language, with whichever verification tool fits. Use for /pstack:create-formal-verification-skill, "model-check this race", "prove this invariant", or when concurrency, retries, state machines, parsers, or unsafe code need more than tests.
Create a formal verification skill
A test checks the cases that someone thought to write. Formal verification checks every case in a stated scope: every order of steps in a model, every input up to a bound, or every input. This skill finds the part of the repo where that matters most, verifies one property of it, and leaves a project-local skill that lets the next agent check a change against that property. Write that skill for an agent who reads it cold, mid-change, and has never seen your work. The user must be able to trust the result and act on it without learning the verification language.
Choose the target from the code
Answer from the codebase, and ask the user only what you cannot observe. If the user named an area or a bug, start there.
A good target is a promise the code makes, where a break harms users and tests rarely reach the case that breaks it. Choose the strongest such target in the repo, not the one that is easiest to verify.
Let the question choose the tool
Pick the tool that answers the question at the lowest cost: a model checker, a proof assistant, an SMT-backed verifier, or another tool. Whether events can happen in a bad order usually suits a model checker over a small model of the design, and TLA+ is the default for that in any language. Whether the code itself behaves for every input, as with memory safety, overflow, or a data-structure invariant, usually suits a verifier that runs on the code. If no verifier runs on this language, say so, and test the real code with generated inputs, not a hand model of the function. A property that must hold at every size, where a bound is not enough, can need a proof. Prefer a tool that the repo's developers can install and run, and tell the user why you chose it.
What makes it good
- True to the code. Every part of the model or proof points to code that a reviewer can open. What the verification treats as one indivisible step is one indivisible step in the code. Names come from this repo.
- Able to fail. A check that you have never seen fail proves nothing. Show each property failing on a broken variant of the code it guards, such as a removed check or a reordered step.
- Findings stay findings. When the current code breaks the property, report it with the counterexample in the code's terms. Do not weaken the property or the model until it passes, and do not change product code unless the user asks.
- Honest about scope. Say what is verified, under which bounds and assumptions, and what is left out. A pass covers the model or the bound, not the whole system.
- Readable without the tool. State the property and the harm in one plain sentence each. Tell a counterexample as a short sequence of steps in the code's words.
- Cheap to rerun. One command, a reproducible tool version, and a run short enough that the next agent will use it on every relevant change.
- Kept in step with the code. Record which files and functions each check covers, and the commit at which you last confirmed that the verification describes them. When the covered code changes after that commit, the next agent can see that the verification needs another look.
Generate the verify skill
Write a project-local skill with frontmatter that names the area and says when to use it. Without frontmatter, the skill never registers. From the skill alone, the next agent must be able to tell whether a change touches the verified code, run the check, tell a real bug from a modeling error, and update the verification when a change is intended. Put the coverage record and the sync commit in the skill, and tell the next agent to move the sync commit forward only after it confirms that the verification describes the code at the new commit. Use this repo's real commands, paths, and nouns, and leave no placeholders.
If the check is a TLA+ model, tell the next agent to read every result with the pstack /pstack:statechart skill. It shows each invariant violation the checker finds as its shortest path in plain words. When the check passes, /pstack:statechart draws a chart of how the system moves between states. Refer to /pstack:statechart by name. Its commands, options, and file paths change between releases and machines, so copy none of them into the generated skill or its scripts.
Prove it before you hand off
Follow the generated skill once, as the next agent would, on a scratch change that breaks the property in the real code. It must catch the break. Then drop the scratch change and confirm that the check is back where it started: passing, or failing only on the findings you will report. A verify skill that was never run is a draft. If the skill misses the break or a step does not run, fix the skill or the verification, not the scratch change.
Hand off
Tell the user what you verified, which tool you used and why, the bounds, what the check found in plain words, how to run it, and what it does not cover. Suggest a second target only if you found a close one. Point the user at /pstack:maintain-formal-verification-skill for bringing the verification back in line after changes that did not recheck it.
Original by Lauren Tan (poteto), MIT license. Checked against the pstack mirror at 97de40d. View on GitHub