We achieved state-of-the-art results on Databricks' OfficeQA Pro V2 benchmark. Navi answered 73.3% of the questions correctly, 10 points ahead of the next best result. It cost $2.24 per question, less than Claude Code working from the raw PDFs.
This builds on our state-of-the-art result on the first version of OfficeQA. In this post, we go through what made the difference this time and how we approach grounded reasoning at scale (see also Navi at scale).
One note before we start: we ran this evaluation with Navi alone, without Parsewise agents, which search every page of the corpus. We'll cover a run with agents in a follow-up post.

Harder questions, rougher scans
OfficeQA Pro V2 covers more than two centuries of US federal accounts, close to 120,000 pages in all. Much of it is old, scanned and packed with dense financial tables, like the pages below. As in the first version, the same figure often shows up in several documents with different values, and gets revised over time. That's very close to what we see in our customers' data, and it's why we keep coming back to this benchmark as it evolves.
What makes V2 harder than its predecessor is mostly the questions. In the first version, a question usually needed a couple of documents. Now a typical question pulls from five or six, and the hardest one needs more than twenty. Only a small part of the corpus is ever relevant, so before you can reason about anything, you have to find the right handful of documents in a corpus full of annual reports that look much alike. The scans also look worse overall than in the first version, so even reading the numbers correctly turned out to be a challenge.
Some of the hardest pages to read (yes, this is the actual resolution)

Baseline: What if we just used a frontier model?
Before running Navi, we wanted to know what the current frontier looked like. Models have improved a lot since Databricks published the benchmark, and Navi uses Opus 5.5 High as its reasoning model, so the obvious question was how far Opus gets on its own. We ran it in Claude Code on the raw PDFs, and it scored 62%, close to the best published result. That didn't surprise us. Opus handled the raw PDFs well and was good at finding the right source documents, and the jump in model capability did the rest.
What did surprise us was how long some questions took. Opus spent a lot of time reading scanned pages and checking them, and the longest question took over an hour, which pushed costs up too. We've long believed that giving a model a parsed text layer makes a big difference to speed and cost, and this was a good chance to test it.
We ran the same setup again, but this time gave Claude Code the corpus as parsed by Parsewise, the same text Navi works from. The typical question took a third of the time and cost less than half as much. Accuracy barely moved, from 62% to 63%. We think that's because Opus was already good at finding the right documents, so the text layer mostly saved it work.
Same model, raw PDFs against parsed text
| Raw PDFs | Parsed text | ||
|---|---|---|---|
| Accuracy | 62% | 63% | About the same |
| Typical time per question | 7 min | 2.3 min | 3× faster |
| Longest question | 65 min | 10.5 min | 6× faster |
| Cost per question | $2.63 | $1.12 | 57% less |
Claude Opus 5.5 High in Claude Code on all 90 OfficeQA Pro V2 questions, November 2026.
Use a strong PDF parsing model
As you have seen above, pre-parsing the text makes processing a lot more efficient for your harness. When asked to find the "Total Patent Office receipts for March 1908", your AI can now quickly look through the extracted text. The alternative is finding the scanned document for 1908 and then parsing its pages.
You might already see how a revised figure published five years later can easily be skipped. Having the corpus as searchable text makes your agents more efficient, and it also makes those later revisions findable. Whether the agent actually uses them is down to the harness, which we cover below.
An 1873 scan and the text we extracted from it
Scan

Extracted text
RECEIPTS AND EXPENDITURES. 469
| Custom-house, [Portland, Oreg.] (1874) | 41 | 78,500 00 | ........... | ........... | 78,500 00 | 28,225 00 | ........... | 50,275 00 |
| Custom-house, Machias, Me. | 38 | 2,000 00 | 3,501 65 | ........... | 5,501 65 | 5,501 65 | ........... | ........... |
| Custom-house, Portland, Me. | ........... | 4,851 31 | 1,251 12 | 6,102 43 | 2,267 05 | ........... | 3,835 38 | |
| Custom-house, Chicago, Ill. | ........... | 1,997,547 47 | 66,365 71 | 2,063,913 18 | 1,556,619 60 | ........... | 507,293 58 | |
| Custom-house, Chicago, Ill. (1874) | 41 | 800,000 00 | ........... | ........... | 800,000 00 | ........... | ........... | 800,000 00 |
| Custom-house, Newport, R. I. | ........... | 2,141 75 | ........... | 2,141 75 | 2,141 75 | ........... | ........... | |
| Appraiser's stores, Philadelphia, Pa. | ........... | 4,877 90 | ........... | 4,877 90 | 4,866 70 | ........... | 11 20 | |
| Appraiser's stores, San Francisco, Cal. | 41 | 408,000 00 | ........... | ........... | 408,000 00 | 20,000 00 | ........... | 388,000 00 |
| Custom-house, &c., Fall River, Mass. | 39 | 200,000 00 | ........... | ........... | 200,000 00 | ........... | 200,000 00 | ........... |
| Custom-house, &c., Fall River, Mass. (1874) | 41 | 200,000 00 | ........... | ........... | 200,000 00 | 103,200 00 | ........... | 96,800 00 |
| Custom-house, &c., Boston, Mass. (1874) | 41 | 30,000 00 | ........... | ........... | 30,000 00 | ........... | ........... | 30,000 00 |
| Custom-house, &c., Hartford, Conn. | 38, 39 | 400,000 00 | ........... | ........... | 400,000 00 | 1,129 80 | 300,000 00 | 98,870 20 |
| Custom-house, &c., Cincinnati, Ohio | 38, 39 | 1,000,000 00 | ........... | ........... | 1,000,000 00 | 417 30 | 370,000 00 | 629,582 70 |
| Custom-house, &c., Cincinnati, Ohio (1874) | 41 | 750,000 00 | ........... | ........... | 750,000 00 | ........... | ........... | 750,000 00 |
| Custom-house, &c., Charleston, S. C. | ........... | 90,000 00 | ........... | 90,000 00 | 90,000 00 | ........... | ........... | |
| Custom-house, &c., Charleston, S. C. (1874) | 41 | 172,081 52 | ........... | ........... | 172,081 52 | 79,841 58 | ........... | 92,239 94 |
| Custom-house, &c., Memphis, Tenn. | 41 | 25,000 00 | ........... | ........... | 25,000 00 | ........... | ........... | 25,000 00 |
| Custom-house, &c., Nashville, Tenn. (1874) | 41 | 150,000 00 | ........... | ........... | 150,000 00 | ........... | ........... | 150,000 00 |
| Custom-house, &c., Saint Louis, Mo. | 39 | 500,000 00 | 300,000 00 | ........... | 800,000 00 | 309,098 42 | ........... | 490,901 58 |
| Custom-house, &c., Saint Louis, Mo. (1874) | 41 | 1,000,000 00 | ........... | ........... | 1,000,000 00 | 9 13 | ........... | 999,990 87 |
| Custom-house, &c., Albany, N. Y. | 38, 39 | 450,000 00 | ........... | ........... | 450,000 00 | 25 30 | 350,000 00 | 99,974 70 |
| Custom-house, &c., Albany, N. Y. (1874) | 41 | 150,000 00 | ........... | ........... | 150,000 00 | ........... | ........... | 150,000 00 |
| Custom-house, &c., Ogdensburg, N. Y. | 38 | 3,000 00 | ........... | ........... | 3,000 00 | ........... | ........... | 3,000 00 |
| Custom-house, &c., New Orleans, La. | ........... | 265,000 00 | ........... | 265,000 00 | 265,000 00 | ........... | ........... | |
| Custom-house, &c., New Orleans, La. (1874) | 41 | 170,000 00 | ........... | 16,942 75 | 186,942 75 | 185,938 75 | ........... | 1,004 00 |
| Custom-house, &c., Bath, Me. | 45 | 5,000 00 | ........... | ........... | 5,000 00 | ........... | ........... | 5,000 00 |
| Custom-house, &c., Rockland, Me. | 39 | 25,000 00 | ........... | ........... | 25,000 00 | 12,000 00 | ........... | 13,000 00 |
| Custom-house, &c., Rockland, Me. (1874) | 41 | 75,000 00 | ........... | ........... | 75,000 00 | ........... | ........... | 75,000 00 |
| Custom-house, &c., Baltimore, Md. | 38 | 50,000 00 | ........... | ........... | 50,000 00 | 50,000 00 | ........... | ........... |
| Custom-house, &c., Port Huron, Mich. | 39 | 200,000 00 | ........... | ........... | 200,000 00 | 5,000 00 | ........... | 195,000 00 |
| Custom-house, &c., Port Huron, Mich. (1874) | 41 | 100,000 00 | ........... | ........... | 100,000 00 | ........... | ........... | 100,000 00 |
| Custom-house, &c., Evansville, Ind. | 41 | 100,000 00 | ........... | ........... | 100,000 00 | 45,356 30 | ........... | 54,643 70 |
| Marine hospital, Chicago, Ill. (1874) | 41 | 23,825 44 | ........... | ........... | 23,825 44 | 23,825 44 | ........... | ........... |
| Marine hospital, San Francisco, Cal. | 45 | 58,789 56 | ........... | ........... | 58,789 56 | ........... | ........... | 58,789 56 |
| Repayments to importers for unascertained duties | 47 | 3,120,192 10 | 373 65 | 63,784 60 | 3,184,350 35 | 3,183,394 37 | ........... | 955 98 |
| Payment of debentures, drawbacks, bounties, or allowances | 47 | 1,176,055 18 | ........... | 24,694 00 | 1,200,749 18 | 1,200,749 18 | ........... | ........... |
| Distributive shares of fines, penalties, and forfeitures | 47 | 626,176 74 | 2,242 68 | 6,050 51 | 634,469 93 | 634,389 93 | ........... | 80 00 |
| Refunding duties, sec. 4, act March 3, 1871 | 47 | 134,552 06 | ........... | ........... | 134,552 06 | 134,312 66 | ........... | 239 40 |
| Drawbacks on certain articles imported into Chicago | 47 | 192,155 95 | ........... | 11,199 77 | 203,355 72 | 203,355 72 | ........... | ........... |
| Unclaimed merchandise | 47 | 1,440 80 | 79 86 | ........... | 1,520 66 | 1,520 66 | ........... | ........... |
| Refunding duties to extend the warehousing system | 47 | 935 24 | ........... | ........... | 935 24 | 935 24 | ........... | ........... |
| Debentures and other charges | 47 | 35,655 81 | 109 19 | 782 10 | 36,547 10 | 36,546 03 | ........... | 1 07 |
| Compensation of persons employed in insurrectionary States | 47 | 710 02 | ........... | ........... | 710 02 | 710 02 | ........... | ........... |
| Additional compensation to collectors | 47 | 319 53 | ........... | ........... | 319 53 | 319 53 | ........... | ........... |
| Refunding duties on tea and coffee | 47 | 257,231 82 | ........... | ........... | 257,231 82 | 257,231 82 | ........... | ........... |
| Relief of James F. Joy | 48 | 47,776 40 | ........... | ........... | 47,776 40 | 47,776 40 | ........... | ........... |
| Refunding to James O. P. Burnside proceeds of cotton seized | 46 | 5,284 00 | ........... | ........... | 5,284 00 | 5,284 00 | ........... | ........... |
All 47 extracted rows balance: appropriations, earlier balances and repayments add up to the total available, and so do payments, the amount carried to surplus and the closing balance.
One thing to note is that the quality of the parsing is closely tied to how well you will answer your questions. On scanned pages where the numbers can barely be made out, incorrect parsing can throw off your model, and a single misread digit leads to a wrong answer that looks right.
At Parsewise, we have been evaluating and optimising this pre-processing for a long time, across a few dimensions. First, it needs to be quick, since Navi and Parsewise agents can't start reasoning until the text is parsed. Second, it needs the highest possible accuracy, so that your figures match up even on the toughest scans. Finally, we have to balance both against the cost of the model.
A dedicated harness unlocks SOTA accuracy
By this point you might be wondering where the extra 10 points of accuracy come from. It isn't the extracted text on its own, since that didn't change Opus 5.5 High's accuracy in Claude Code. As you will see below, it's the harness: with the right context and guidance, a data-centric agent still beats the underlying model by a wide margin.
One way Navi does that is by understanding the underlying data much better. Even with our preferred parsing model, we found that data on some scanned pages was close to unreadable. When answering a question, Navi keeps track of the figures it uses in its calculations and checks the ones that don't add up. To do that, it can zoom into the page image itself or ask a stronger model to re-read the figures on that page. It can also look at the surrounding pages, totals and neighbouring years to work out what the value must be.
Checking a figure that doesn't add up

This is also where the revised figure published five years later comes back in. Searchable text makes it findable, but Navi still has to go looking for it. So Navi understands the scope of the corpus and is tasked with keeping a good balance between exploration and exploitation. It won't automatically pick the first data source it finds. Instead, it actively looks for revisions and other values that might contradict what it found, and only answers the question once it is confident in its findings.
Looking past the first answer
"What were the Patent Office receipts for March 1908?"
- 1908
- 1909
- 1910
- 1911
- 1912
- 1913
Answers $191,075, the revised figure
What we also observed is that answering the question once you have all the data points is a lot less straightforward than it might seem. That's less surprising when you consider that some questions in the benchmark carry an underlying ambiguity. Take a question like "How much did the government spend on the Patent Office in the 1920s?". An analyst might mean the calendar years 1920 to 1929 or the fiscal years that cover them, the figures as first published or as later revised, and gross spending or spending net of the office's own receipts.
To handle this, Navi approaches the question as an analyst would: before calculating anything, it works out what the question is really asking and what the numbers mean. If web search is enabled, it may even look online to get more context on the question. Finally, when a question allows more than one reading, it picks the most likely one and states the alternative alongside its answer.
Working out what the question means
"How much did the government spend on the Patent Office in the 1920s?"
- Which years?Treasury reports run by fiscal year.
- Which figures?Revisions win unless the question says otherwise.
- Gross or net?Spending means what the office paid out.
A hypothetical question with illustrative totals.
Is there a place for a human in the loop?
This ambiguity raises an interesting question. When colleagues work together, not every request is a one-shot question with a single clear answer. By talking it through and considering alternatives, you build a much better picture of what the question means and why it was asked. We think working with AI should be no different.
To test how close agents already get on their own, we allowed Navi to provide up to 5 alternative answers based on different question interpretations. This raised accuracy by about 6 more points. It would be interesting to explore a version of this benchmark where a human could steer the agent to reduce this uncertainty further.
Steering Navi to the right reading
How much did the government spend on the Patent Office in the 1920s?
$29.4M
Fiscal years, revised figures, gross spending
Treasury reports run by fiscal year, so I used those. Did you mean calendar years?
Yes, calendar years
$30.1M
Calendar years, revised figures, gross spending
What comes next
As we mentioned at the start, this run used Navi alone. Navi works like an analyst on top of a project's processed data: it explores the documents, reasons across them and runs the calculations. We have spent over a year getting it to where it is now.
On a corpus this large, Navi doesn't read every page. It explores the data and reasons over it, so it can miss small details. That's what Parsewise agents are for. Users or Navi create them, and each agent runs a deep search over the whole corpus, page by page. Agents are one of the foundations of the platform, and they guarantee users complete traceability over the data.
In a follow-up post, we will publish the results of a run with Navi and agents together. We expect it to raise accuracy further.

