A general AI assistant with your EA tool's MCP server isn't an EA agent

A general assistant connected to your EA tool's MCP server answers questions about one application, one fact sheet or one decision well. Estate questions are different. They need thousands of records from several systems, the same application recognised under three names, and every figure traced to a record. That is a different job, and it takes a harness around the model, not a better prompt.

What a connected assistant does well

Since most large EA vendors shipped MCP servers, any MCP-capable assistant can read an EA repository. Ask "who owns the payments gateway, and when was it last reviewed?" and it finds the fact sheet and answers. Ask it to draft an architecture principle or summarise a decision record, and it does that well too.

For a question that fits in one turn and one system, a connected assistant is the cheaper choice. Use it.

Where do estate questions break it?

Scale

A 1,200-application estate is about 8,000 records in the EA repository, plus a service and a business application for each in the CMDB, plus the epics in the delivery tracker. An assistant reads through its context window. It pages through lists a few dozen records at a time, and the tool definitions alone can take tens of thousands of tokens before the first question. On a portfolio question it either spends dozens of turns paging, or reads part of the estate and infers the rest. Either way, the answer reads well, and nothing tells you which one happened.

Identity across systems

"Where do our EA repository and our CMDB disagree on owner, criticality and cost?" needs every application recognised in both systems, under names that differ: Payments Hub, PAY-HUB-PRD, payments-hub. Matching them is a job that needs evidence and a confidence for each pair, not a guess the model makes in passing.

Traceability

A head of EA can't take a figure to a steering committee without knowing where it came from. A general assistant writes the figure, and nothing checks it against the record it read.

Finishing

Estate questions take dozens of steps. A run that stops at its output limit, or waits on its own subagents until a time cap, produces nothing. Finishing is a property of the loop around the model, not of the model.

Writes

When the fix is a change in a source system, someone should see what will be sent and approve it first. In a general assistant, whether a write waits for a person depends on that client's settings, not on a policy your organization sets once for every chat.

How we're measuring it

EA-Bench is our study of the gap. It asks AI systems 17 analysis questions an EA team gets, in five areas: portfolio and strategy, technology and transformation, risk and regulation, AI governance, and repository trust. Each runs at two scopes, one domain of about 200 applications and the whole estate.

• The estate is generated: 1,200 applications in a retail bank, with an EA repository, a CMDB, a delivery tracker and three AI agent platforms, messy where real estates are messy.
• The answers are planted, with decoys: records that disagree across systems, a cheap duplicate that is the only system of record for card data, unregistered and duplicate agents.
• Code computes the answer keys from the records and scores every answer: a complete and correct list scores, and a trap or an invented record fails it.
• Three systems, the same models and limits: an EA agent harness; a general assistant connected through the same MCP servers; and that assistant with a sandbox to run code over what it reads.

The harness tested is ours, so the questions and the scoring rules are published in full, and the results will be too.

What does the harness do that the model doesn't?

• Reads the estate in place. It searches, filters, joins and totals thousands of records without pasting them into the conversation. The model sees a summary and a handle.
• Matches records across systems with evidence. Each cross-system link carries its reason and a confidence, and ambiguous pairs go to a decision model that refuses rather than guesses.
• Checks every figure. Each answer shows how many of its figures were found in the records the run read. A recommendation should rest only on verified evidence.
• Keeps long runs going. Goals, subagents that share the chat's data, a steer when a turn ends without an answer, and a session you can reopen and redirect.
• Asks before it writes. A change to a source system shows what it will send and waits for approval, unless your admin allows that connection to write.

When is a general assistant enough?

When the question is about one record or one system, or the answer is a paragraph rather than a table someone will act on, use the assistant you have. Use a harness when the question spans systems, the estate is large, or the answer goes to a decision forum.

Test your own

We published the 17 questions as a kit you can run on the assistant you use today, with what a good answer contains and the traps that should fail it: Test your AI assistant on EA work.