01 / Knowledge
Answers from the
firm’s own knowledge.
Proposals, project files and live business data. Tested with questions about the firm.
Every question passed.
10 questions · 3 runs each
Benchmark
Before bringing Agnes to its clients as Strata, Cuesta Partners tested it inside its own firm against the AI tools its people already used. Then it rolled it out to everyone in 17 days.
Cuesta’s benchmark
Share of graded answers that passed
Same 10 questions, 3 runs each,
graded by Cuesta.
The story
That is the promise Cuesta Partners makes to the mid-market companies and private equity sponsors it serves. As a partner bringing Agnes to clients, it decided to hold itself to the same standard first: put a company brain on its own documents, numbers and people, measure it first, and only then take it to clients.
“Instead of looking at slides, we’re looking at an inventory of agents and Agnes. It’s going to totally change the game for us.”
Riley ScottCo-Founder and Partner, Cuesta PartnersThe benchmark scope
01 / Knowledge
Proposals, project files and live business data. Tested with questions about the firm.
10 questions · 3 runs each
02 / Consistency
Each question asked three times. Because a client needs an answer they can rely on.
03 / Permissions
Agents use the permissions of the person who starts them. Checked on every request.
Firm numbers
What Agnes sits on
Agnes connects what Cuesta knows in three forms: what it wrote, what it measured, and how it defines things.
Your knowledge
The firm’s knowledge
Agnes
Company context
One connected context
Benchmark results
Cuesta fixed the questions and the scoring rubric before any system saw its data. Every setup received the questions word for word. The four setups with data ran them three times each and every answer was graded; plain Claude, with no data, is the baseline.
10
questions, each testing one capability
5
setups: 4 graded, plus a no-data baseline
3
runs per graded setup
7
weighted scoring dimensions
Finding the closest past project
“A prospect is a building-products manufacturer, PE-owned, and wants to know if we've done this kind of rapid AI roadmap work before. What's our closest precedent and why is it close?”
Hard because: The right precedent is filed under a different service name.
Counting across many folders
“Which PE sponsor has driven the most of our AI assessment engagements this year, and across all of those engagements, what industries have we touched?”
Hard because: The answer spans dozens of folders and a CRM field that is often blank.
Spotting a question with two meanings
“Show me our manufacturing work.”
Hard because: It can fairly mean two different things. A good answer says so.
Other questions asked for live business metrics, where an answer came from, how two companies are related, and one that has no right answer.
| 30graded answers for Agnes10 questions × 3 runsPassPartialFailNo accessOne dot per run | Connected company data and knowledgeAgnes30 / 3010 of 10 passed all three runs | Document connectorChatGPT21 / 276 of 9 passed all three runs | Document connectorClaude19 / 274 of 9 passed all three runs | Documents and curated definitionsClaude + context16 / 275 of 9 passed all three runs |
|---|---|---|---|---|
| Structured metrics (live data) | Agnes: pass, pass, pass | ChatGPT + SharePoint: no access, no access, no access | Claude + SharePoint: no access, no access, no access | Claude + SharePoint + Cuesta's context pack: no access, no access, no access |
| Precedent recall | Agnes: pass, pass, pass | ChatGPT + SharePoint: pass, partial, pass | Claude + SharePoint: pass, partial, partial | Claude + SharePoint + Cuesta's context pack: partial, partial, partial |
| Closest match | Agnes: pass, pass, pass | ChatGPT + SharePoint: pass, pass, pass | Claude + SharePoint: pass, pass, partial | Claude + SharePoint + Cuesta's context pack: fail, fail, pass |
| Ambiguity | Agnes: pass, pass, pass | ChatGPT + SharePoint: partial, partial, partial | Claude + SharePoint: fail, pass, partial | Claude + SharePoint + Cuesta's context pack: partial, partial, partial |
| Counting | Agnes: pass, pass, pass | ChatGPT + SharePoint: pass, pass, pass | Claude + SharePoint: pass, fail, pass | Claude + SharePoint + Cuesta's context pack: partial, partial, partial |
| Entity resolution | Agnes: pass, pass, pass | ChatGPT + SharePoint: fail, fail, pass | Claude + SharePoint: fail, fail, pass | Claude + SharePoint + Cuesta's context pack: pass, pass, pass |
| Every setup passed every run | ||||
| Synthesis | Agnes: pass, pass, pass | ChatGPT + SharePoint: pass, pass, pass | Claude + SharePoint: pass, pass, pass | Claude + SharePoint + Cuesta's context pack: pass, pass, pass |
| Specific facts | Agnes: pass, pass, pass | ChatGPT + SharePoint: pass, pass, pass | Claude + SharePoint: pass, pass, pass | Claude + SharePoint + Cuesta's context pack: pass, pass, pass |
| Provenance | Agnes: pass, pass, pass | ChatGPT + SharePoint: pass, pass, pass | Claude + SharePoint: pass, pass, pass | Claude + SharePoint + Cuesta's context pack: pass, pass, pass |
| Negative case | Agnes: pass, pass, pass | ChatGPT + SharePoint: pass, pass, pass | Claude + SharePoint: pass, pass, pass | Claude + SharePoint + Cuesta's context pack: pass, pass, pass |
“We were able to get everything for the last two years in our customer folders. And we measure that in days, not months.”
Zach PapinAI Transformation Lead, Cuesta PartnersReal impact
Proposals
A brief and discovery notes become a draft deck and SOW, each claim tied to a past engagement.
Two Cuesta leaders asked for every opportunity across a segment.
“We never would’ve been able to do that just with Claude and SharePoint alone.”Zach PapinAI Transformation Lead, Cuesta Partners
Definitions
Two definitions the firm had never pinned down now live in the semantic layer.
Shared across the firm instead of rebuilt by each person.