Search the docs, videos, blog, apps, teams, skills and release notes.
Open source. macOS, Windows and Linux.
At a glance
What it is
Decide whether a test worked: what counts as validated, what counts as failed, and when to stop.
A skill is a written procedure a bot loads when a job calls for it. This one is a single file, SKILL.md, and the whole file is on this page.
The whole file, exactly as it downloads. Read it before you rely on it.
SKILL.md
66 lines · 849 words
A test finished. The team is staring at numbers and reaching for a story. Load whenever someone asks "did it work," "is that enough," "what does this number mean," or — most importantly — "should we keep going."
Every test ends in one of three places. Name it plainly.
Decide whether a test worked: what counts as validated, what counts as failed, and when to stop.
Nothing. It ships with Murage, which is open source. Your bots run on the AI plan you already have.
Yes. Open it in Settings → Skills to read, edit or duplicate it. Skill Guard scans it again after every edit.
Download the app, connect the AI you already pay for, and tell your Chief of Staff what needs doing. Plan on about ten minutes from install to a working team.
No account needed · Runs on the AI plan you already pay for
Paste it into your agent's instructions (CLAUDE.md, AGENTS.md or a custom GPT), or add it to a bot in Murage.
Ask Ember for the bot, then switch the skill on in its settings.
Step 1
Send your Chief of Staff this message. It proposes the bot and its role first, and creates it when you say go.
Send this to Ember
“Create a bot that uses the Validation rubric: when has it validated, when has it failed, and when to kill skill, and show me the plan first.”
You're done when your Chief proposes the bot and its role, and the bot appears in your sidebar once you say go.
Step 2
You're done when you see the bot’s skills and the library.
Step 3
You're done when the skill shows as on for that bot.
Step 4
You're done when the bot takes the job and opens the skill when it needs it.
Prefer to do it yourself? Choose + in the sidebar, then New Bot.Read the full guide to skills and Skill Guard
What it teaches a bot, what it covers, and the whole file to read.
What it covers
When to load this mode
The three honest verdicts
The rubric
Decision rules
Anti-patterns
Before / after
Each of these library teams gives its bots this skill. Import one and look inside first.
Go (validated). Result met or exceeded the pre-registered threshold within the sample and window. Hypothesis survived. Move to the next leap of faith.
Kill (falsified). Result fell below the kill criterion. Hypothesis is dead. Don't redesign the test to save the idea — that's confirmation bias wearing a lab coat. Write the kill memo.
Pivot (ambiguous, but learning). Result missed threshold, but a specific signal inside the data points to a neighboring hypothesis worth testing next. You're using the corpse to find a better idea.
"Directionally positive," "trending toward," and "just need more data" aren't verdicts. They're excuses dressed as analysis. If you reach for them, the kill criterion wasn't tight enough or the team isn't ready to hear it.
Score every completed test against six criteria. All six must be satisfied for the read to be trustworthy.
Pre-registered threshold. Was the kill criterion written down before the test ran? If not, the result is unfalsifiable theater. Re-run with a pre-registered number.
Adequate sample size. Did the test reach the minimum sample defined upfront? Under-sampled, the result is rumor no matter how good it looks.
Clean stimulus. Was the stimulus held constant across the window? Mid-test edits invalidate the read.
Behavioral measurement. Did you measure an action — click, sign-up, payment, return visit — or did you measure an opinion? Opinions don't count.
Matched audience. Did the traffic match the audience the hypothesis named? Wrong audience answers a different question.
Honest threshold comparison. Is the result being compared to the pre-registered number, or to a number invented after the fact to make the result look better? Only the original number counts.
A test that passes all six is a test you can decide on. A test that fails any of them needs to be re-run, not narrated around.
Validated, all six satisfied: ship to the next leap of faith. Update TEAM_MEMORY.md with the dated entry. Hand customer-language insight to Scout, pricing implications to Offer, message learnings to Copy.
Validated, but one of the six failed: the result is provisional. Re-run with the fix. Don't act on shaky validation; it costs more downstream than a re-run costs now.
Falsified, all six satisfied: write the kill memo. Name what was tested, what threshold failed, what the team learned that's worth keeping, and what won't be re-tested without new information. The kill is a deliverable, not a defeat.
Falsified, but one of the six failed: the kill is unsafe. Re-run cleanly. A bad test that says "no" is no more useful than a bad test that says "yes."
Pivot territory: the test missed threshold, but inside the data lives a specific, falsifiable next hypothesis. State the new hypothesis in the same template. Run a new test. Don't pivot more than twice on the same idea without an external sanity check.
Don't move the threshold after the result is in. Renegotiating the kill criterion to match the number you got is the most common failure mode. Guarantees you'll never kill anything.
Don't aggregate failed tests into a "promising trend." Three failed tests are three failed tests, not a heat-check.
Don't validate on vanity metrics. Impressions, reach, engagement-without-action — none of these tell you whether the value hypothesis holds.
Don't re-run hoping for a better day. If the test was clean, the answer is the answer. Re-running for a better result is gambling.
Don't skip the memo. Unwritten learnings evaporate within a week. Written ones survive into the next milestone.
Before (narrated rescue of a failed test):
"We hit 1.8% sign-up against a 3% target, but engagement on the page was strong and we think with better copy we could get there — let's keep iterating."
After (honest read):
"Pre-registered threshold: 3%. Actual: 1.8%. Sample 412, window 7 days, all six rubric criteria satisfied. Hypothesis falsified. Kill memo filed. Specific pivot candidate: the page got 4× the time-on-page for parents of 5–8-year-olds vs. 1–4. New hypothesis worth testing: same offer, older child segment. New test designed, kill criterion 3%, runs next week."
Same data, two responses. One traps the team on the wrong hill. The other walks them to the right one.
SKILL.md
66 lines · 849 words
The text of SKILL.md is shown as it downloads, under the Apache-2.0 license, by Ferrox Labs. The license is the one in the file's own frontmatter.
Your new bot
Validation rubric: when has it validated, when has it failed, and when to kill bot
Waiting in your sidebar once you say go.
No account needed · Runs on the AI plan you already pay for