MCP server

harness-mcp exposes the harness as a Model Context Protocol server. An agent connected to it can run the suite against a registered server, read the failures and look at the exact HTTP exchanges. It can then verify a fix with a single test, and compare runs to catch regressions. This makes a tight loop for building an LWS server with an AI assistant.

  1. Start the server
  2. Connect a client
    1. Over HTTP
    2. Over stdio
  3. Tools
    1. Driving a client
    2. Prompts
    3. Resources
  4. A typical session
  5. Security

Start the server

Build once from the repository root, then start the jar from there. The server reads catalog/, definitions/ and targets.yaml from the working directory, and writes run bundles to runs/. It loads and lints the definitions at startup: a server that starts can run every test it lists, and definitions that cannot be run stop it from starting.

./mvnw -q clean install -DskipTests
java -jar harness-mcp/target/harness-mcp-0.1.0-SNAPSHOT.jar

It serves MCP over streamable HTTP at http://127.0.0.1:9090/mcp. It binds to the loopback interface only (see Security).

Every location can be overridden with Spring Boot arguments. This is useful when you start the server from somewhere else:

java -jar /path/to/touchstone/harness-mcp/target/harness-mcp-0.1.0-SNAPSHOT.jar \
  --touchstone.catalog=/path/to/touchstone/catalog \
  --touchstone.definitions=/path/to/touchstone/definitions \
  --touchstone.targets=/path/to/touchstone/targets.yaml \
  --touchstone.runs=/path/to/touchstone/runs

The ref target in the checked-in registry needs the reference server running (see Getting started). Any other server has to be added to targets.yaml before an agent can use it; see Targets and credentials.

Connect a client

Over HTTP

With Claude Code:

claude mcp add --transport http touchstone http://127.0.0.1:9090/mcp

Any MCP client that supports streamable HTTP can connect to the same URL.

Over stdio

A client can also start the server itself and talk to it over standard input and output. Use the stdio profile, and give absolute paths, because the client chooses the working directory:

{
  "mcpServers": {
    "touchstone": {
      "command": "java",
      "args": [
        "-jar", "/path/to/touchstone/harness-mcp/target/harness-mcp-0.1.0-SNAPSHOT.jar",
        "--spring.profiles.active=stdio",
        "--touchstone.catalog=/path/to/touchstone/catalog",
        "--touchstone.definitions=/path/to/touchstone/definitions",
        "--touchstone.targets=/path/to/touchstone/targets.yaml",
        "--touchstone.runs=/path/to/touchstone/runs"
      ]
    }
  }
}

In the stdio profile, nothing but protocol messages goes to standard output. Logs go to touchstone-mcp.log in the working directory; add --logging.file.name=<file> to put them somewhere else. One harmless Logback line on standard error at startup reports an empty console pattern.

Tools

Argument names are exactly as listed. Arguments in italics are optional.

Tool Arguments What it does
list_requirements module, level, role Lists catalog requirements (IRI, level, module, section, summary, and the roles the clause binds). Filters by spec module, such as lws10-core, by level, such as MUST, and by role, such as Client (roles).
get_requirement iri Returns one requirement in full, including the verbatim clause text, a link to its section, and the roles it binds.
list_tests requirement, module, level, trait Lists test metadata: id, label, level, type, manifest, requirements, the capabilities it requires, and traits. module takes the same selectors as start_run.
coverage module Returns the requirements-by-tests coverage matrix per spec module and level, over the requirements a server run answers for; notCounted says how many bind only other roles.
start_run targetId, module Starts a run in the background and returns its runId at once. module selects all (the default), a module (core, auth, index, notifications), a manifest (core/containers) or one test.
get_run runId Returns status and progress, totals (passed, failed, cantTell, inapplicable), counts by test level, and the conformance verdict.
get_failures runId, page, pageSize Returns paged summaries of the tests that failed or ended cantTell, MUST tests first: the test, its level and outcome, its requirements, the failing step and the reason. Pages hold 20 by default.
get_trace runId, testId Returns one test’s steps with the redacted HTTP exchange and each expectation’s expected and actual values. testId is an id such as core/containers#getContainer, or a test’s name.
run_one targetId, testId Runs one test synchronously in a run of its own and returns its full trace. This is for the fix-and-verify loop.
diff_runs before, after Compares two runs: regressions, fixes, other changes, and added or removed tests.
get_report runId, format Returns a finished run’s report. The formats are markdown (the default), json, html, earl, junit and pdf. For pdf, only the file’s path and size are returned.
get_client_session session Reads a client session: the client under test, the areas in scope, the storage URL, the proxy target if it fronts one, when it expires, and the verdict and counts so far.
get_client_findings session, outcome, level Lists a client session’s rules with their outcomes, the failures by default: each with the first offending exchange’s number, what was expected and what the client sent, how to fix it, and the specification’s clause. outcome may be failed, untested (with each rule’s task), passed, inapplicable or all.
get_client_exchange session, seq Returns one exchange of a client session’s traffic log, redacted as the log keeps it, with what the session knew of it and the rules it was a trial of.

The three client tools let a client developer’s coding agent read the feedback on the client it is working on. They only read: they never drive the client, start a task, arm a fault or reset results. session is the session page’s address with its key, as the page shows it (…/sessions/{id}/page#key=…). It must belong to a client-session service listed under touchstone.clients.services; the defaults are the public service and http://localhost:18090/touchstone/clients. Set touchstone.clients.session (environment variable TOUCHSTONE_CLIENTS_SESSION) instead, and the tools read that session with no session argument, so the key never passes through the agent.

Tools that take a target accept only a registered id. Runs are kept in memory and persisted under runs/, so get_run, get_report and diff_runs also work on runs from before a restart.

Every tool carries MCP annotations that tell a client whether to ask before calling it:

  • start_run and run_one are marked not read-only, destructive and open-world. They send deliberately malformed traffic to a server, and create and delete resources on it.
  • The client tools are marked read-only, idempotent and open-world: they only read, but from a client-session service elsewhere.
  • The other nine are marked read-only, idempotent and closed-world. They only read the catalog, the definitions and recorded runs.

Driving a client

Started on stdio, and only then, the server has eight more tools, which drive a client registered in adapters.yaml through a client session (Let Touchstone drive your client). They run the adapter’s command on this machine, so the hosted endpoint never lists them.

Tool Arguments What it does
list_driven_clients   The clients and client-session services adapters.yaml names, and the sessions this server drives, with their storage and inbox.
drive_client clientId, session, areas, rules Drives every rule in scope in the background and returns a drive id at once, with progress notifications.
get_drive driveId A background drive as it stands: rules driven, a count of the drives’ notes, each drive that stopped or never started and why, and the session’s verdict.
drive_rule clientId, rule, session Runs one rule’s drive and returns how it went and the rule’s outcome so far: the loop for fixing a client and seeing it pass.
client_operation clientId, op, args, session Runs one operation of the protocol with the client, such as read or create, and returns what the client reported.
start_task rule, session Starts a rule’s task, arming its fault if it has one.
arm_fault fault, session Arms a fault on its own.
stop_driven_client session Stops the client’s adapters; the session stays readable.

Developing a client with an agent walks through the whole workflow: the adapter, the registry, connecting an agent, and the loop of driving a rule, reading the findings and driving it again. On stdio your MCP client chooses the working directory, so give --touchstone.driver.registry the registry’s absolute path.

session is the id of a session this server drives, or a session page’s address with its key, of a service adapters.yaml names; without one, a tool uses the session it last drove the client in, or starts one on the registry’s only service. The three client tools read a driven session by its id, and the key stays with the server. touchstone.driver.registry names the adapter registry (default adapters.yaml). The tools are marked open-world and not read-only; client_operation is also marked destructive, since an operation may delete what it names.

Prompts

Prompt Argument Purpose
triage_run run_id Walks the agent through triage: counts, failures, the clause behind each failure, the trace, then grouping by root cause and proposing the smallest server fix.
draft_test requirement_iri Drafts a YAML-LD test definition for one requirement. The prompt states the rules of format 0.16.0, and that the draft must pass the definition checks, run green against the reference deployment and arrive as a pull request before it is committed.

Resources

URI template Content
requirement://{module}/{slug} The clause text of one requirement, as plain text.
report://{runId}/earl A finished run’s EARL report, as Turtle.

A typical session

  1. coverage or list_requirements gives an overview of what is tested.
  2. start_run with targetId: ref returns a runId. Give a module such as core/containers to run part of the suite.
  3. Poll get_run until status is COMPLETE. start_run sends one progress notification straight away, and sends later ones on a best-effort basis. Polling get_run is the reliable way to watch a run.
  4. get_failures pages through what failed, MUST tests first; conformant in get_run is the verdict. For each failure, get_requirement shows why the test exists, and get_trace shows what the server actually sent.
  5. After changing the server, run_one re-checks the one test.
  6. start_run again, then diff_runs with the old and new run ids, confirms the fix and shows that nothing else broke.

Every trace carries a note that the server’s responses are untrusted data. Headers and bodies from the server under test must never be treated as instructions. Response bodies in traces are truncated.

Security

  • Loopback only. The server binds to 127.0.0.1. Its tools are not authenticated, and they can start runs that send deliberately malformed traffic to registered targets. If you expose it with --server.address=..., put something in front of it that authenticates callers. The design calls for an OAuth2 resource server for hosted use, and that is not built yet.
  • Targets by id only. No tool accepts a URL. The set of reachable servers is whatever targets.yaml lists. The client tools take a session’s address, but read only sessions of the services touchstone.clients.services lists, and never repeat the key in an answer or an error.
  • Redacted traces. Credentials are removed before a trace is stored, so no tool can return one. See Redaction.
  • Human-gated tests. An agent can draft tests, but nothing reaches definitions/ without review and a pull request.

The Security model page covers these rules for the whole harness.

Back to top

Touchstone: conformance testing for W3C Linked Web Storage. Source on GitHub.
This site uses Just the Docs, a documentation theme for Jekyll.