Skills, Standards and Context: What My Agents Know, and How Fresh It Is

When an agent gets something wrong, the first instinct is to blame the model. In my experience the cause is usually quieter: the agent was handed the wrong instructions, a stale picture of the code, or the right facts in the wrong place.

So a large share of the work on Starship has gone into three things that sit in front of the model:

• skills, so it knows how to do the job
• standards, so it knows what good looks like
• context, so it knows what's true right now

A skill is a playbook, not an agent#

I keep four words separate, because blurring them causes most of the confusion:

• An agent is a worker with a role.
• A tool is something it can call.
• A method is a typed job with inputs, outputs and a retry policy.
• A skill is the written procedure a method loads when deciding how to do that job.

A skill does one thing. Its description says when to use it, not just what it is. It also:

• names its tools explicitly
• states the approval gates for risky actions
• says what to do when things fail and how to escalate
• has a clear license and provenance
• maps to at least one department and one method

That's the promotion checklist, and a skill that can't meet it doesn't belong in the library.

Harvested, not hand-waved#

I didn't write a thousand skills by hand. The library is harvested from open catalogs whose authors already curate them, and today it holds 1,489 skills in the shared tier.

Harvesting is deliberately boring. A script shallow-clones each upstream catalog and copies every skill into a review queue. Next to each one it writes the upstream license and a provenance record:

# Provenance

- source: <upstream repo>
- commit: <sha>
- upstream_path: <path in the catalog>
- imported_at: <timestamp>
- status: candidate

Candidates are visible in the dashboard, but they're never loaded into production runs. A second script promotes them, with two rules I care about. First, every promoted skill carries a source prefix, so you can always tell where it came from. Second, an existing shared skill always wins a name collision. Promotion is idempotent and never overwrites an edit I've made.

For vetted upstream catalogs, promotion is direct. Anything authored from my own run lessons, or contributed from an untrusted source, goes through the full lifecycle: candidate, reviewed, promoted, then eventually deprecated. Deprecated skills stay on disk for auditability, so nothing silently disappears.

At runtime the lookup is simple. A per-product override comes first, then the shared skill, then the upstream reference, and the first match wins. Bundles per role stay intentionally small, three to six skills, so the default path fits inside the context budget. A big library is only useful if each agent loads very little of it.

Standards are the constitution#

Skills tell an agent how to do something. Standards tell it what it must never do.

A standards agent reviews every change against the target repo's own coding standards file. If the repo doesn't have one, it falls back to a short default:

• no new dependencies without a reason
• no lint suppressions without a justification
• no console logging in production code
• no undocumented global state
• no deleting tests without replacing them

When I mapped the system against a simple four-layer model (constitution, skills, plan and gates), the honest result was uneven:

• Skills and gates were strong.
• The constitution was partial. Most target repos didn't ship their own standards file, so everything fell back to the generic one.
• The plan layer was partial too. The coder submits a structured plan, but there's no durable, versioned plan document living in the repo. That matters more for small local models than for large ones. A smaller model loses the thread, and a plan in the repo is its external memory.

Context: live, but was it fresh?#

In June I audited how context actually reached the models. The verdict fit in one line: live per-turn grounding, but no event-driven freshness loop.

Every chat turn assembled its picture live from the repository, the product database and the local repo-intelligence on the coordinator, so the obvious things stayed current. The audit also found real gaps:

• Nothing refreshed context when a pull request merged.
• The project wiki and security policy were never ingested.
• The refresh queue accepted jobs that no worker ever drained. It was a dead letter.
• The cross-run learning store that the design described hadn't been built.

The fix was to treat freshness as a contract rather than a hope.

Every source got a time-to-live. Product docs get 14 days, system docs 90, packed repository snapshots 7, symbol indexes 60, project rules 180, and prompts a single day.

Rehydration became deterministic. A pull request's diff maps to a rehydration scope: the same diff always produces the same scope. If the diff touches an anchor file, such as dependencies, schema, routes or build config, or if it touches more than a set number of files, everything gets re-packed.

Each stage got a gate. The gates cover jobs reaching a terminal state, determinism replay, freshness assertions, provenance, drift, redaction and a retention watermark.

When the verification wave ran, the code was complete and the new scope and webhook tests passed 16 of 16. Once drift was actually computed, 58 of 133 context artifacts were flagged as expired. Full sign-off stayed blocked on live gates, including ingestion that hadn't happened yet and approved lessons that didn't exist yet. I'd rather publish that than round it up.

MCP, and the danger of quiet degradation#

Shared context is served through a small MCP server with 18 tools, 5 resources and 2 prompts. It runs on a managed container platform behind identity-only access, with no public unauthenticated ingress. Heavier repository intelligence, like symbol search, code search and packed snapshots, stays on the coordinator, next to the code. The bulk of that "brain" lives on a large local disk, and the cloud database holds only pointers to it.

One finding stuck with me. If the context endpoint wasn't configured, the client quietly fell back to a local address. It didn't fail. It just answered with thinner context, and nobody would know. The fix is small: say so, loudly. Thin context that announces itself is a warning. Thin context that stays silent is a bug you'll spend a week chasing.

Context that learns, carefully#

By mid-June the learning loop closed. When a run resolves, a synopsis of the fix becomes a lesson: the rule, the file, the guidance, the merged PR. Relevant approved lessons then flow into the next coding task's brief. Three rules keep it honest:

• A success needs proof. The task must be done, or the PR merged. A run that only claims success writes nothing.
• Failures are recorded as risks, never as fixes. They're always human-gated, so a failed run is never recorded as a success.
• Old failures expire once they're fixed. The knowledge base then reflects what's true now, not what was once broken.

New lessons wait for review by default. Only non-sensitive machine-verified fixes can skip that step.

Context also has limits, and I measured some of them. A finding-focused context pack was 21,657 tokens against a 12,000-token budget, padded with generic docs and missing the actual target. Rebuilt around a window of the exact lines in question, it came to 11,962 tokens. Better context fixed the localization problem, but it couldn't fix every capability problem. Some tasks are simply hard for the model in front of them.

The takeaway#

I think of skills, standards and context as the operating environment I build for the agents, the same way I'd think about onboarding for a person. Give them a short list of good habits, a clear sense of what's off-limits, and an honest, current picture of the work. Then make sure the system notices when that picture goes stale, rather than trusting it to stay fresh on its own.