The System

Three services talk over frozen HTTP contracts, in four public repos. An audit found that each side's tests asserted against its own copy of the shape.

Case study summary

Result and evidence: Notes are classified and tagged automatically and the system is browsable end to end. The audit also exposed a false-green contract seam. I closed it under SYS-018 on 2026-07-18: each provider publishes one generated schema, and every consumer asserts against that schema in its own CI.
Problem: Make four independently deployable repositories operate as one system without hiding coupling in shared code.
Ownership: I directed the solo build, defined the cross-repo decisions and contracts, and audited whether their controls worked.
Constraints: Note creation had to stay responsive when the classifier was unavailable; reprocessing had to be safe; and each repository kept its own release boundary.
Actions: I designed an asynchronous, idempotent enrichment loop, recorded system-level ADRs, added contract checks, and built a portal linking decisions to the source.

Plate 01

Two unit tests that happen to agree are not a contract test. Each side was green. The seam was not. One published schema now holds the shape, and a consumer that drifts from it fails its own build.

I direct Claude Code to build it. The operating layer is public, and it is agent-ops. It holds the PreToolUse guards that enforce the security posture mechanically. It holds the working agreements that parallel sessions run under. It also holds an incident log at a deliberate severity bar, with the failures left in.

The repos

notes-apiFastAPI REST service. The knowledge base and data layer.
defense-news-classifierLLM classifier with a real eval harness. Labels the content.
kb-agentRAG and tool-use agent. The hub that reasons over the system.
learning-notesPlain-language notes on the techniques behind it all.

All four repos are public. Three of them are services. learning-notes is a notes site, and it holds no contract. The ADRs and contract tests below live in those repos.

Contents

The map

Click a box to see what each part does. The list below carries the same flow.

Map 02

Click a box above to see what each part does.

Contents

How it flows

POST /notes creates a note in notes-api. The endpoint enqueues a FastAPI BackgroundTask to classify and tag it. The 201 returns immediately. The task runs after notes-api sends the response.
The background task POSTs the note text to the classifier's /classify endpoint.
The classifier classifies the text into a category, an operational domain, and a region. It returns them. One Sonnet call does the work, and tool-use forces the structured output.
The task writes the labels back as namespaced tags (PUT /notes/{id}/tags, replace semantics). It preserves the user's own tags. It maps all three fields. It mapped only the first two until 2026-07-18, because the region field was newer than this consumer. An unset CLASSIFIER_URL makes the whole step a no-op.
The kb-agent reads notes (GET /notes) to ground its RAG answers, and can also call the classifier synchronously.

That is the loop in words. The run page draws the field this consumer dropped, and the check that closed the gap. Watch one note go through it, with the real schemas →

Contents

The problem

The four repos should act as one system and stay independently deployable. That lives in the contracts between them, not in shared code.

Contents

The decisions

A doubled rule marks a decision that was reversed.

Decision · SYS-004 / SYS-006 / SYS-018

Decouple with contracts, not shared code

The repos talk over frozen HTTP contracts with explicit versioning rules. The provider on each seam publishes the shape as a committed schema, and every consumer asserts against that one file.

Why: a renamed field breaks the consumer silently at runtime. Tradeoff: freeze the wire shape and publish it, so drift fails a build instead of production.

I audited my own decision log in July 2026 and found this seam broken. The classifier shipped a third field. The frozen contract required a coordinated consumer update. That update never happened, and both repos' CI stayed green. Each side's “contract test” asserted against its own copy of the shape.

Map 03

The provider added a field and updated its own fixture in the same commit. Nobody updated the consumer stub. Each copy of the shape then moved on its own, and neither side could see the other. The map does not show the fix. One shared artifact replaced the two copies, and that mechanism stays in the prose below. The map holds no verdict either: the CI on both repositories stayed green, and that sentence stays above it.

A real contract test needs a single shared artifact that both sides assert against. A change to one copy then fails the other's build. I amended SYS-004 in the open, and I struck the falsified consequences through rather than delete them.

Then I built it. SYS-018 replaced the claim with a mechanism, and it landed on 2026-07-18. The classifier generates contracts/classify-response.schema.json from its live response model and its label constants. It is never hand-edited, so it cannot describe a shape the service does not return. The CI step Contract artifact is current (SYS-004) fails on a stale file. Both consumers, kb-agent and notes-api, fetch that published file in their own CI and fail on divergence. notes-api publishes the second seam the same way, and kb-agent asserts against it in the step Notes read contract matches the provider (SYS-006).

The two contracts are deliberately different shapes. The /classify schema is closed (additionalProperties: false). That is what turns an added field into a detectable break rather than a silent one, and its absence is what let the third field through. The GET /notes schema is open, because kb-agent reads a subset of the note and an added provider field must not redden its build. There, the failure worth catching is a field kb-agent reads that disappears upstream. That one is quiet: the consumer parses with .get(), so it keeps answering with empty titles instead of raising an error.

What is still not guarded. A consumer that cannot fetch the published schema prints a loud warning and exits 0. A GitHub outage must not redden an unrelated build, and the price is that a genuine provider outage reads as a pass. The check scripts record that trade in their own docstrings rather than hide it. False Green collects six more checks that reported confidence they never earned. Context Architecture is the same split one layer up. There I promote the rules an agent reads from prose into hooks.

Reversed. Decision · SYS-005

Make the enrichment loop async and idempotent by design

A new note triggers a FastAPI BackgroundTask in notes-api. The task calls the classifier. It then writes the returned labels back as namespaced tags (category:, domain:, region:) with replace semantics. notes-api owns the writeback, and the classifier stays a pure provider. I first built this loop on Kafka. I cut it, because one user and an idempotent workload do not need a broker.

Why: classification inside a BackgroundTask keeps note creation fast, and it decouples note creation from classifier latency. Replace semantics make reprocessing converge, so it never accumulates duplicate tags. Tradeoff: each task needs its own retry path and its own error handling.

Decision · SYS-002

Gate model choice on evidence

Default to Sonnet across the system. Escalate to Opus only where an eval shows the quality gain pays for itself. The same gate ran the July 2026 migration to Sonnet 5: eval before and after on the same gold set.

Why: I built the escalation and measured it three separate times. I declined it all three times. The verdicts sit on the classifier and the judge pages, where I measured them. Tradeoff: you have to run the eval to justify the tier. You do not default to the biggest model.

Decision · SYS-001 / ADR-001

Record decisions, then make the system legible

Cross-repo decisions live as two-tier ADRs. A repo-local choice stays local, and a system-level one is SYS-NNN in one place. A generated portal pulls every repo's docs into one browsable view that deep-links back to the code. It links to each doc where it already lives. It does not copy the doc, so there is a single source of truth.

Why: I re-argue a decision that I did not write down.

Contents

The outcome

Create a note, and the system classifies and tags it automatically. The work runs asynchronously, with no coupling between note creation and classifier availability.
The four repos stay independently deployable. I claimed a red build found cross-repo drift. It did not. An audit proved that, and SYS-018 closed it. Each provider now publishes one schema generated from its live response model, and every consumer compares its own field set against that schema in CI. One limit stays: a consumer that cannot fetch the schema prints a warning and exits 0, so a real provider outage reads as a pass.
The whole system is browsable in one portal, with links straight to the real code.
A skill map (SYS-007) spells out what the system exercises: evals, context engineering, agents, and observability. It also names where I am weak, which is security on the agent's tool seam.

Contents

Where it goes next

The roadmap is an autonomy ladder: the same classifier, more self-direction one level at a time. The ladder, level by level →