Skip to content
Agnes

Benchmark

Cuesta benchmarked Agnes, then took it firm-wide

Before bringing Agnes to its clients as Strata, Cuesta Partners tested it inside its own firm against the AI tools its people already used. Then it rolled it out to everyone in 17 days.

Cuesta Partners

Cuesta’s benchmark

Share of graded answers that passed

Same 10 questions, 3 runs each,
graded by Cuesta.

SharePoint documents indexed
Unstructured270,000+
business tables, refreshed daily
Structured28
defined once in the semantic layer
Semantic26 metrics · 36 terms
from kickoff to firm-wide rollout
Time to value17 days

The story

Beyond AI experimentation, to measurable impact.

That is the promise Cuesta Partners makes to the mid-market companies and private equity sponsors it serves. As a partner bringing Agnes to clients, it decided to hold itself to the same standard first: put a company brain on its own documents, numbers and people, measure it first, and only then take it to clients.

“Instead of looking at slides, we’re looking at an inventory of agents and Agnes. It’s going to totally change the game for us.”
Riley ScottCo-Founder and Partner, Cuesta Partners

The benchmark scope

Three things a company brain had to prove.

01 / Knowledge

Answers from the
firm’s own knowledge.

Proposals, project files and live business data. Tested with questions about the firm.

100%

Every question passed.

02 / Consistency

The same answer.
Every time.

Each question asked three times. Because a client needs an answer they can rely on.

10/10

Correct answers.
Every single run.

03 / Permissions

Access that follows
the role.

Agents use the permissions of the person who starts them. Checked on every request.

0 leaks

Restricted data
stayed protected.

What Agnes sits on

One brain over the firm’s documents, data and definitions.

Agnes connects what Cuesta knows in three forms: what it wrote, what it measured, and how it defines things.

Your knowledgeAgnesCompany context THE FIRM’S KNOWLEDGE 270k+ documents Two years of client work Roadmap readouts Statements of work Diligence reports Value backlogs 28 business tablesRefreshed daily by Keboola time_entryopportunityaccountengagementbilling ONE CONNECTED CONTEXT 26 metrics · 36 termsA shared semantic layer Revenue = billingsMargin includes contractorsUtilization · Win rate 10 entity types12 relationships worked onfor clientin industrypart ofevidenced by

Your knowledge

The firm’s knowledge

270k+ documentsTwo years of client work
  • Roadmap readouts
  • Statements of work
  • Diligence reports
  • Value backlogs
28 business tablesRefreshed daily by Keboola
  • time_entry
  • opportunity
  • account
  • engagement
  • billing

Agnes

Company context

One connected context

26 metrics · 36 termsA shared semantic layer
  • Revenue = billings
  • Margin includes contractors
  • Utilization · Win rate
10 entity types12 relationships
  • worked on
  • for client
  • in industry
  • part of
  • evidenced by

Benchmark results

Same questions, five setups.

Cuesta fixed the questions and the scoring rubric before any system saw its data. Every setup received the questions word for word. The four setups with data ran them three times each and every answer was graded; plain Claude, with no data, is the baseline.

  1. 10questions, each testing one capability
  2. 5setups: 4 graded, plus a no-data baseline
  3. 3runs per graded setup
  4. 7weighted scoring dimensions

10

questions, each testing one capability

5

setups: 4 graded, plus a no-data baseline

3

runs per graded setup

7

weighted scoring dimensions

  • Claude, no data (baseline)
  • Claude + SharePoint
  • ChatGPT + SharePoint
  • Claude + SharePoint + context pack
  • Agnes

Example questions

Finding the closest past project

“A prospect is a building-products manufacturer, PE-owned, and wants to know if we've done this kind of rapid AI roadmap work before. What's our closest precedent and why is it close?”

Hard because: The right precedent is filed under a different service name.

Counting across many folders

“Which PE sponsor has driven the most of our AI assessment engagements this year, and across all of those engagements, what industries have we touched?”

Hard because: The answer spans dozens of folders and a CRM field that is often blank.

Spotting a question with two meanings

“Show me our manufacturing work.”

Hard because: It can fairly mean two different things. A good answer says so.

Other questions asked for live business metrics, where an answer came from, how two companies are related, and one that has no right answer.

Pass, partial or fail for each of three runs, per setup and question
30graded answers
for Agnes
10 questions × 3 runsPassPartialFailNo accessOne dot per run
Connected company data and knowledgeAgnes30 / 3010 of 10 passed all three runsDocument connectorChatGPT21 / 276 of 9 passed all three runsDocument connectorClaude19 / 274 of 9 passed all three runsDocuments and curated definitionsClaude + context16 / 275 of 9 passed all three runs
Structured metrics (live data)Agnes: pass, pass, passChatGPT + SharePoint: no access, no access, no accessClaude + SharePoint: no access, no access, no accessClaude + SharePoint + Cuesta's context pack: no access, no access, no access
Precedent recallAgnes: pass, pass, passChatGPT + SharePoint: pass, partial, passClaude + SharePoint: pass, partial, partialClaude + SharePoint + Cuesta's context pack: partial, partial, partial
Closest matchAgnes: pass, pass, passChatGPT + SharePoint: pass, pass, passClaude + SharePoint: pass, pass, partialClaude + SharePoint + Cuesta's context pack: fail, fail, pass
AmbiguityAgnes: pass, pass, passChatGPT + SharePoint: partial, partial, partialClaude + SharePoint: fail, pass, partialClaude + SharePoint + Cuesta's context pack: partial, partial, partial
CountingAgnes: pass, pass, passChatGPT + SharePoint: pass, pass, passClaude + SharePoint: pass, fail, passClaude + SharePoint + Cuesta's context pack: partial, partial, partial
Entity resolutionAgnes: pass, pass, passChatGPT + SharePoint: fail, fail, passClaude + SharePoint: fail, fail, passClaude + SharePoint + Cuesta's context pack: pass, pass, pass
Every setup passed every run
SynthesisAgnes: pass, pass, passChatGPT + SharePoint: pass, pass, passClaude + SharePoint: pass, pass, passClaude + SharePoint + Cuesta's context pack: pass, pass, pass
Specific factsAgnes: pass, pass, passChatGPT + SharePoint: pass, pass, passClaude + SharePoint: pass, pass, passClaude + SharePoint + Cuesta's context pack: pass, pass, pass
ProvenanceAgnes: pass, pass, passChatGPT + SharePoint: pass, pass, passClaude + SharePoint: pass, pass, passClaude + SharePoint + Cuesta's context pack: pass, pass, pass
Negative caseAgnes: pass, pass, passChatGPT + SharePoint: pass, pass, passClaude + SharePoint: pass, pass, passClaude + SharePoint + Cuesta's context pack: pass, pass, pass
“We were able to get everything for the last two years in our customer folders. And we measure that in days, not months.”
Zach PapinAI Transformation Lead, Cuesta Partners

Real impact

What changed in the work.

Proposals

Proposals start
from precedent.

A brief and discovery notes become a draft deck and SOW, each claim tied to a past engagement.

~20minutes

A full cross-sell list
for a client segment.

Two Cuesta leaders asked for every opportunity across a segment.

“We never would’ve been able to do that just with Claude and SharePoint alone.”Zach PapinZach PapinAI Transformation Lead, Cuesta Partners

Definitions

One number for
revenue and margin.

Two definitions the firm had never pinned down now live in the semantic layer.

18

18 reusable skills.

Shared across the firm instead of rebuilt by each person.

Agnes

See Agnes on your own data.

Book a meeting with the Agnes team.