Search the docs, videos, blog, apps, teams, skills and release notes.
At a glance
What it is
Choose the one metric your team steers by and break it into inputs you can move. Then set up A/B tests with a hypothesis, sample size and stopping rule.
A skill is a written procedure a bot loads when a job calls for it. This one is a single file, SKILL.md, and the whole file is on this page.
The whole file, exactly as it downloads. Read it before you rely on it.
SKILL.md
76 lines · 841 words
As of: 2026-05-16
Mode skill. Default-enabled on the Analyst specialist.
Use any time a user asks "what should we be measuring?", "what's our north star?", "should we A/B test this?", "is this experiment significant?", or proposes a test without a hypothesis. Use before shipping any experiment, and use to write or critique the metric tree the whole team optimizes toward.
Trigger phrases:
Choose the one metric your team steers by and break it into inputs you can move. Then set up A/B tests with a hypothesis, sample size and stopping rule.
Nothing. It ships with Murage, which is open source. Your bots run on the AI plan you already have.
Yes. Open it in Settings → Skills to read, edit or duplicate it. Skill Guard scans it again after every edit.
Download the app, connect the AI you already pay for, and tell your Chief of Staff what needs doing. Plan on about ten minutes from install to a working team.
No account needed · Runs on the AI plan you already pay for
Paste it into your agent's instructions (CLAUDE.md, AGENTS.md or a custom GPT), or add it to a bot in Murage.
Ask Ember for the bot, then switch the skill on in its settings.
Step 1
Send your Chief of Staff this message. It proposes the bot and its role first, and creates it when you say go.
Send this to Ember
“Create a bot that uses the North Star and Experiments skill, and show me the plan first.”
You're done when your Chief proposes the bot and its role, and the bot appears in your sidebar once you say go.
Step 2
You're done when you see the bot’s skills and the library.
Step 3
You're done when the skill shows as on for that bot.
Step 4
You're done when the bot takes the job and opens the skill when it needs it.
Prefer to do it yourself? Choose + in the sidebar, then New Bot.Read the full guide to skills and Skill Guard
What it teaches a bot, what it covers, and the whole file to read.
The principle
**Mode skill.** Default-enabled on the Analyst specialist.
What it covers
When to use
Procedure
Decision rules
Anti-patterns
Before / after
Each of these library teams gives its bots this skill. Import one and look inside first.
"What's our north-star metric?"
"Let's A/B test this."
"Is the variant winning?"
"How long do we run this?"
If NSM + inputs are already locked in TEAM_MEMORY.md under ## Analyst, skip to step 4.
1. Test the candidate against three criteria. A north-star metric (NSM) must: (a) represent customer-perceived value, not company activity; (b) correlate with long-term revenue, not short-term signups; (c) move when the team does the right work, not when seasonality shifts. Revenue itself usually fails (a). Signups fail (b). Vanity counts fail (c). Reject any candidate that fails one.
2. Decompose into input metrics. The NSM sits on top of a small set of inputs whose product or sum approximates it. Example: NSM = (active users) × (actions per active user) × (value per action). Each input is a metric a team can move. If you cannot decompose, the NSM is too abstract, pick a closer one.
3. Stamp NSM + inputs to TEAM_MEMORY. Channels, Smith, whoever owns pricing and margin, and Copy all need the same north star and the same input tree, or they pull in different directions.
4. Write the hypothesis in three parts. "If we change X, then Y will move by at least Z, because [mechanism]." X is a specific change. Y is a single primary metric. Z is the minimum detectable effect, the smallest move that would justify shipping. The mechanism is the why. No mechanism, no test.
5. Compute sample size before launch. From baseline Y and MDE Z, compute required sample per arm (two-proportion or two-mean; 80% power, α = 0.05). State sample and time-to-accrue at current traffic. If that exceeds the decision window, the test is underpowered, propose a larger MDE, sharper change, or smaller scope before launching.
6. Set the stopping rule in advance. Name (a) the sample size at which you check, (b) the threshold at which you call a winner, (c) the maximum runtime past which you stop regardless. No peeking before the planned check; sequential testing inflates false positives without an explicit sequential-design correction.
7. Pre-register guardrails. At least two metrics you do not want to harm even if the primary moves, latency, downstream conversion, revenue per session, support-ticket rate. A primary win with a guardrail loss is a trade-off requiring explicit decision, not an auto-ship.
8. Report with intervals, not point estimates. Report lift, confidence interval, primary p-value or Bayesian probability, guardrail movements, and segment cut (mobile vs. desktop, new vs. returning). A 3% lift with a CI spanning −2% to +8% is not a winner; say so plainly.
One primary metric per test. Multiple primaries inflate false-positive rates and turn experiments into fishing expeditions.
No early stopping without a sequential design. Peeking and stopping at the first significant moment is how false winners ship.
Underpowered tests do not ship as conclusions. They can run as directional learnings, the report must say so.
Guardrails are non-negotiable. A primary win with a guardrail loss returns for trade-off, not auto-ship.
Segment analysis is post-hoc unless pre-registered. Finding the segment where the test "worked" is pattern-matching, not analysis.
A/B testing without a hypothesis. That is a button push, not a test.
Choosing the NSM by what's easy to measure. Choose by what represents value; build the measurement to match.
Reporting only the winner without confidence interval and guardrail movements.
Running 12 simultaneous tests on overlapping traffic with no isolation. The results are uninterpretable.
Letting the highest-paid opinion override the stopping rule. The rule was set in advance for a reason.
Brief: "Let's test a new pricing page."
Before (no hypothesis, no sample plan):
Run the new page for two weeks and see if conversions go up.
After (hypothesis + sample + stop rule + guardrails):
Hypothesis: moving the annual toggle above the fold lifts annual-share by ≥4 pp (22% → ≥26%), because earlier anchor exposure shifts default selection. Primary: annual-share. MDE: 4 pp. Sample/arm at α=0.05, 80% power: ~2,900 paid conversions; at 180/week that is 16 weeks, underpowered. Tighten scope to direct + organic, accept MDE 6 pp, target 8 weeks. Guardrails: paid-conversion rate, day-14 refund rate. Stop rule: check week 4 only if sample reached. Stamping to TEAM_MEMORY.
SKILL.md
76 lines · 841 words
The text of SKILL.md is shown as it downloads, under the Apache-2.0 license, by Ferrox Labs. The license is the one in the file's own frontmatter.
Your new bot
North Star and Experiments bot
Waiting in your sidebar once you say go.
No account needed · Runs on the AI plan you already pay for