MnB Research Agent

MnB Security is a small commercial-security installer. The work it wants more of comes from bigger firms — integrators and rollout companies that win a contract for forty sites and subcontract the installation. Finding those firms means reading their websites, and every one says "we serve the GTA" and none says which person hands out the work.

An LLM can read all of that in minutes, but nothing it writes back can be trusted unread. It will invent a contact, quote a sentence that isn't on the page, or describe a manufacturer as an installer. So the question was not "can a model research a company" but "what would it take for a brief to be worth a person's five minutes". The answer turned into three repositories.

Three repos, one direction

harness is a generic agent runtime in Go. patchright-cli is a browser tool for agents, in TypeScript on Bun. mnb-leads is the product: prompts, policy, SQLite, the review UI. The rule in the workspace AGENTS.md is that the product may depend on both tools and neither tool may know the product exists. Both tools started inside AlchemizeCV, another product of mine, in April and May 2026 and became their own repos in September; AlchemizeCV, mnb-leads and a code-audit tool all consume the same harness.

1. request
2. →host browser session
3. →no-network container
4. →bash-only agent
5. →capture journal
6. →CheckFinal
7. →verifier
8. →human review

The sandbox

A run starts when request new writes a row to requests; a partial unique index, requests_company_active, stops two active runs targeting the same company. worker serve --concurrency 3 claims rows, holds a per-company lock file, and heartbeats to worker_state every five seconds so interrupted runs are recovered.

For each claimed request the host creates a fresh, recorded patchright-cli session on a Unix socket in the run's scratch directory, then starts a rootless Podman container with --network=none, --cap-drop=ALL, no-new-privileges, pid and memory limits, from an image pinned by digest (deploy/agent-shell.Containerfile). The agent loop runs with its shell inside that container and the prompt in internal/session/research.md. The model's only tool is bash; its only way to the web is the patchright-cli binary baked into the image, attached to the host's browser through that socket.

The session's capabilities are fixed at creation: navigate,extract,consent (internal/session/attach.go), enforced by the daemon rather than the prompt. The agent can search, open and read public pages, and make one bounded attempt to reject optional cookies. It cannot click, type, screenshot, download or run script, whoever asks; it never logs in, never accepts cookies or terms, never fills a form. A watchdog checks every five seconds: a dead browser is restarted at the same socket; a socket replaced by anything else ends the run as tampered.

Every read is appended to captures.jsonl with a capture ID and a content hash, failures included; those IDs are the only thing the model can cite.

The final-answer gate

The harness has a RunIO.CheckFinal hook: the caller inspects the model's final answer and either accepts it or sends problems back. In mnb-leads that hook is checkFinal in internal/session/agent.go, which ingests the capture journal and hands the brief to checkBrief in research.go.

The brief is one strictly decoded JSON block: pursue, park or reject; a category; at most three named contacts; a main line; a draft opener; findings with citations. leads.ValidateBrief then checks it against the captures: every quote must appear verbatim in the cited capture's text, and so must every contact's name. Business rules apply on top: a pursue needs supported geography and services findings, an entry finding, a named contact and an opener shaped for its channel (a 120-word email, a 200-character LinkedIn note, or a three-line call script); a manufacturer or distributor is a reject. Every failure is collected and returned in one reply, so the model fixes all of it at once. Nothing is saved until it passes.

An independent verifier, measured

Verbatim matching stops fabricated quotes. It does not stop a real quote being used for something it doesn't say — a parent company's coverage claimed for a branch, a "we plan to" read as "we do". For that, internal/verify sends each cited passage, the quote plus about 1,500 characters either side, to a second model through OpenRouter's decisions API with a rubric I named support@2: does the passage support the claim, is it about this entity, is it past tense, is it more qualified than the claim. Answers are cached by SHA-256 so a resubmitted brief only asks about what changed; four calls run at once, with retries on 429 and 5xx.

I measured it before trusting it. On 110 hand-built claim pairs, the check cost $0.0075 in total with a 263 ms median. The first rubric kept 40 of 71 true claims. Redesigning it as the four questions above, with thresholds in policy.go (supports at 0.9 or more, same entity at 0.8 or more, past and qualified at 0.5 or less), kept 65 of 71 with zero false claims accepted. Those thresholds were chosen on the same set, so that is a fit, not a held-out result. A refusal goes back to the model with the reading behind it; anything weaker is flagged for the reviewer, not blocked.

The runtime

harness (github.com/mattboback/harness) owns the loop: model calls, shell execution local or in Podman, JSON Schema validation with bounded in-context repair, and append-only JSONL transcripts that rebuild session state for resume. The public API in pkg/harness is Run, Complete, ResumeSession and a portable CheckpointV1, with a live event stream, the CheckFinal hook and typed stop reasons. Ten providers sit behind three adapters (OpenAI-compatible, OpenAI Responses, Google GenAI).

Two of the stop reasons came from watching real runs. internal/engine/spiral.go fingerprints command output and, when the same failing command returns the same output twice, skips it and tells the model to change strategy. StoppedReasonFabricatedOutput fires when a reply contains the reserved control character the harness uses to frame real command output — the model was writing the result it wanted rather than running the command.

It is about 38,000 lines of Go plus 20,000 of tests. An eval suite runs 20 tasks in isolated containers with a network allowlist and reports turns, tokens, cost and redundant actions per model.

The browser tool

patchright-cli is a client and a per-session daemon over a Unix socket, compiled to one binary, driving a real, consistent Chrome through Patchright, a Playwright fork. One command registry (src/protocol/commands.ts) generates the help, the JSON schema and the agent guide. open prints a numbered action map from the accessibility tree, and later commands act on those numbers:

LANDMARKS
- banner
- navigation "Main"
- main
- contentinfo

HEADINGS
h1 Example Domain
h2 Services

ACTIONS
# banner
[1] link "Example Domain" -> / @e1
# navigation "Main"
[2] link "Services" -> /services @e2
[3] link "Projects" -> /projects @e3
[4] link "Contact" -> /contact @e4
# main
[5] link "See our work" -> /projects @e5
[6] textbox "Search" @e6
[7] button "Search" @e7

### Page
- Page URL: https://example.com/
- Page Title: Example Domain

A ref from an earlier page generation is rejected as stale_ref instead of clicking whatever now sits at that position. The research session never sees this map: without interact, open returns page state only, and the agent reads with open --read and extract page.

With --json, every result is an envelope with contract_version: 1, and a blocked page is a typed error, never empty text: captcha_required, anti_bot_challenge or authentication_required, none of which the tool ever tries to get past:

{
  "contract_version": 1,
  "ok": false,
  "command": "open",
  "capture_id": "5b1e0d2c9a7f4e3b8c6d1f0a2b3c4d5e",
  "error": {
    "code": "captcha_required",
    "message": "CAPTCHA",
    "retriable": false,
    "retry_after_ms": null,
    "hint": "Keep this session open. Ask a person to clear the challenge in the browser, then run snapshot in the same session before retrying the read."
  },
  "changes": [],
  "artifacts": [],
  "notices": []
}

The hint appears only for sessions that can take a snapshot; the research agent's cannot. Searches take turns across every session on the host (src/daemon/searchPace.ts), and any block pauses searching for 90 seconds. Requests to private addresses are refused, and redirects are checked hop by hop before the next request exists (networkPolicy.ts, redirectPolicy.ts). About 440 test cases run in CI, the integration half against real Chrome.

Running it

The worker takes a fallback chain of models. When the harness surfaces ErrProviderExhausted, the worker passes over that provider for an hour and moves to the next. Generated briefs are never rewritten; the reviewer's decision and notes go into their own tables (reviews, feedback), so a failed retry cannot replace an accepted brief.

The admin UI is Go html/template with no JavaScript, a default-src 'none' CSP, X-Frame-Options: DENY, SameSite=Strict cookies with a same-origin check on every POST, and bcrypt password hashes. It ships as a FROM scratch image run by a Podman quadlet, with the worker as a systemd user service. Deploys are a content-fingerprinted bundle: artifacts are hash-verified and run from an immutable versioned path, SQLite is backed up before migrate, rollback commands are printed if anything after that fails, and the newest three versions are kept.

The newest slice is internal/knowledge: a relationship graph in its own knowledge.db, where a person can have several affiliations, an assertion is separate from its evidence, and a human's acceptance or rejection is independent of any verifier's opinion. The ADRs in docs/adr/ and the glossary in CONTEXT.md exist because I kept catching myself using "contact" for both a person and a way to reach them.

What I'd revisit

The verifier thresholds are fitted to 110 pairs. That is not a benchmark, and I'd want a held-out set from a different kind of company before trusting the 65-of-71 figure to generalise.

The prompt in research.md is 177 lines of rules that the gate also enforces in code. Some of that duplication is deliberate — the model does better when told the rules it will be checked against — but every rule that lives in two places has drifted at least once.

CheckpointV1 is a lossy projection of the transcript: it carries text but not native tool-call IDs, so a run that used the native exec tool can only resume from its durable session directory. I'd like one resume path, not two.