Think about the last time you handed a document to an AI tool. Maybe it was a financial report. An earnings summary. A stack of contracts or invoices you needed pulled into a spreadsheet. You uploaded it, typed your request, and watched the output arrive, clean, confident, formatted exactly the way you asked.
You probably didn’t check every number. Why would you? The whole point was to save time.
But here’s the bitter truth nobody tells you: the tool wasn’t reading that document. It was performing a very convincing impression of reading it. And somewhere in that performance, it made mistakes. Quietly. Without flagging them. Without any sign that something had gone wrong.
This isn’t a fringe problem. It’s not something that happens to careless users or badly worded prompts. It’s a structural issue baked into how general-purpose AI tools process documents, and it’s costing people real time, real money, and real trust in work that they believed was already done.
The cruel irony is that the better the tool looks, the harder the problem is to catch. A rough output makes you skeptical. A polished one makes you trust it. And the tools producing the most professional-looking outputs from documents are often the ones with the least reliable extraction underneath.
The Specific Ways It’s Failing You Right Now
If you work with documents (financial reports, contracts, invoices, compliance filings, anything with structured data in it) you’ve almost certainly experienced one of these. You just may not have known the AI was the cause.
It reads pages, not structure
Most AI tools process a document the way you’d read a novel left to right, top to bottom, one page at a time. That works fine for prose. It falls apart completely with tables that span two pages, columns that carry different units, or figures that only make sense when you pair them with a footnote three pages later.
The AI doesn’t know the footnote exists. It just picks a number and moves on. And because the number it picked looks plausible it’s in the right range, it’s formatted correctly you have no reason to question it.
It guesses when it’s uncertain and doesn’t tell you
When a general-purpose AI hits an ambiguous field a column header that’s cut off, a figure that could be in millions or thousands, a date formatted inconsistently it doesn’t stop. It doesn’t flag the ambiguity. It makes a call, formats it neatly, and presents it to you with the same confidence it uses for everything else.
“A figure reported in millions vs. thousands is a 1,000x difference. If the AI picks the wrong
unit and you don’t catch it, that error travels into every calculation downstream.”
It loses the thread across long documents
Feed a 60-page quarterly report to most AI tools and something quietly breaks in the middle. Context from page 12 doesn’t reliably connect to data on page 48. Numbers get attributed to the wrong section. A restatement that changes a prior figure gets ignored because the model didn’t hold the connection between the original and the correction.
For short documents, this rarely matters. For the documents that actually drive decisions at work the dense ones, the ones with cross-references and multi-period comparatives it matters every single time.
It can’t handle multiple documents at once
You need to compare this quarter to last quarter. Or reconcile figures across three subsidiary reports. Or pull consistent metrics from four vendor contracts. Most AI tools force you to do this one document at a time, then stitch the results together yourself which is exactly the manual work you were trying to escape. And every stitch is another place for an error to hide.
It has no memory of what it was uncertain about
Even when an AI tool does extract data correctly, it produces no record of where it struggled. You receive an output with no asterisks, no confidence scores, no indication that one figure required a judgment call and another was unambiguous. Every data point looks equally authoritative. That uniformity is not a feature it’s a design choice that actively hides the risk in your output.
We Tested It And The Results Were Uncomfortable
We didn’t want to make claims without proof. So we ran a test designed to reflect what real document work actually looks like not a clean, cherry-picked demo file, but four consecutive NVIDIA quarterly financial reports pulled directly from public filings. Over 200 pages total. Dense tables, footnote-adjusted figures, multi-segment revenue breakdowns, GAAP and non-GAAP reconciliations sitting side by side.
We gave the same task to three different AI setups: extract the key financial metrics across all four quarters, identify trends, flag anomalies, and produce a structured report. No shortcuts. No pre-processing. No feeding documents one at a time to make it easier. Same files, same task, same standard for what counts as correct.
Here’s an honest account of what each one did:
ChatGPT: Fast, Polished, and 20% Wrong
We started with ChatGPT because it’s where most people start. The interface is familiar, the upload feature exists, and the model is genuinely impressive at a wide range of tasks. The problem surfaced immediately: the platform imposed a file size and page limit that made processing all four reports simultaneously impossible.
So we adapted. We uploaded one report at a time, ran the analysis, then uploaded the remaining, and asked the model to synthesize across both sessions. This is a workaround that real users actually employ and it’s worth being honest about what it introduces into the workflow.
When we compared the output to the source documents, figure by figure, it was not good.
| What Went Wrong | What That Means for You |
|---|---|
| R&D; figures reported in millions when source used thousands | Numbers off by a factor of 1,000 |
| Q2 figures carried forward incorrectly into Q3 synthesis | Year-over-year trend completely inverted |
| Restated prior-year revenue used alongside current figures | Growth rate fabricated from bad inputs |
| Entire revenue segment missing from two quarters | Segment analysis built on incomplete data |
| Wrong EPS column pulled basic instead of diluted | Per-share figure overstated |
The most unsettling part wasn’t the errors. It was the presentation. Every mistake came wrapped in clean formatting and professional language. No hesitation. No ‘I’m not certain about this figure.’ No asterisk. The wrong numbers looked exactly like the right ones.
Claude Code: Smarter, But Still Quietly Wrong
Claude Code is a legitimately excellent tool. We use it ourselves. For coding tasks, technical reasoning, and building complex pipelines, it earns every bit of the enthusiasm the developer community has thrown at it. We went into this test genuinely expecting it to perform better on financial documents than a conversational chat interface.
It did perform better. It also failed in ways that were harder to catch.
Claude Code was able to ingest all four reports without the session-splitting workaround. It reasoned through the documents methodically, identified the right table structures, and produced output that was more consistently formatted than the ChatGPT run. The error rate was lower around 12% of extracted figures had meaningful discrepancies but the errors it did make were subtler and therefore more dangerous. However it did mess up:
- Transposed quarterly figures: In one instance, Q1 and Q2 data center revenue were swapped. The numbers were real numbers from real pages just assigned to the wrong period.
- Footnote-adjusted figures ignored: NVIDIA’s reports include figures that are GAAP and non-GAAP, with adjustments explained in footnotes. Claude Code consistently pulled the unadjusted figures and labeled them as the comparable metric.
- Trend narrative vs. actual trend: In two cases, the written trend analysis described movement in the opposite direction of the actual numbers the model had reasoned correctly about direction from context clues but the underlying data was wrong.
- Confidence without verification: Like ChatGPT, Claude Code produced no uncertainty signals. A 12% error rate delivered with 100% confidence is a particularly bad combination.
To be clear: Claude Code was doing exactly what it was built to do. It was reasoning about documents, not parsing them as structured data objects. The architecture is the constraint, not the capability. When the task requires precision extraction from complex financial layouts, that architectural difference is not a minor detail.
The Kudra Agent: Built to Actually Read Documents
The Kudra agent was built on a different premise entirely. Where the other two agents approached the NVIDIA reports the way a knowledgeable analyst might (reading, inferring, reasoning) the Kudra agent approached them as structured data objects that need to be parsed before they can be understood.
The pipeline works in layers. First, Kudra’s document perception stack processes the PDFs: OCR calibrated for financial layouts, table detection that preserves multi-column header hierarchies, footnote linking that maintains cross-references between annotated figures and their explanatory text, and unit normalization that resolves the millions-vs-thousands ambiguity before any data reaches the analysis layer. The language model receives clean, validated, schema-conformant data. It doesn’t have to guess what a table says. It knows.
We fed all four NVIDIA quarterly reports simultaneously, no batching, no session splitting, no manual stitching. 200+ pages, in one pass.
What came out the other side:
Zero figure transpositions: Every revenue number matched its correct quarter. Every comparison column mapped to its correct period. Q1 was Q1. Q4 was Q4.
Complete segment coverage: All revenue segments captured across all four quarters with no omissions.
GAAP/non-GAAP correctly separated: Adjusted and unadjusted figures were extracted, labeled, and kept distinct throughout the analysis. No silent metric mixing.
Footnote-referenced adjustments included: Where a figure carried a footnote that changed its interpretation, the Kudra agent surfaced both the figure and the qualifying context.
Restatements flagged, not silently overwritten: In one instance where NVIDIA restated a prior-quarter figure, the agent flagged the discrepancy rather than picking one number and moving on.
Accurate trend analysis: Quarter-over-quarter and year-over-year calculations matched our manual source-document verification across all metrics.
What was particularly amazing is that, although the document did not clearly label capital expenditure for the prior year, the agent managed to locate it in the document as purchases related to property and equipment and intangible assets for the year 2022. Remarkably, in a 50-page document with a complex format, it was able to pinpoint this specific, small detail with precision.
The results are clear:
| Metric | ChatGPT | Claude Code | Kudra Agent |
|---|---|---|---|
| All 4 reports in one pass | No | Yes | Yes |
| Figure error rate | 20% | 12% | 0% |
| Segment coverage | Partial | Near-complete | Complete |
| GAAP/non-GAAP handling | Mixed | Inconsistent | Separated & labeled |
| Footnote integration | Missed | Partial | Full |
| Restatement detection | None | None | Flagged |
| Uncertainty signals | None | None | Where relevant |
Why Your Current Tool Isn’t Lying to You: It Just Wasn’t Built for This
To be fair to the tools we tested: they weren’t designed for precision document extraction. They were designed to reason, generate, and converse. For those tasks they’re genuinely excellent and we use them for exactly that.
The problem is that document processing got quietly folded into their feature sets without the underlying architecture changing to support it. The tool can accept a PDF upload. It can produce formatted output from that PDF. But between the upload and the output, something that looks like reading is actually a form of sophisticated inference and inference fails in ways that reading doesn’t.
Here is the clearest way to explain the difference.
Reading a document means knowing that column 3 in the table on page 34 is connected to the footnote on page 36, and that footnote changes how column 3 should be interpreted. It means recognising that the number in row 7 is in thousands, not millions, because of a header note that appeared six rows above. It means understanding that a figure marked with an asterisk is not the final figure until you’ve read what the asterisk says.
Inference means making a reasonable, pattern-based guess about what each field probably contains based on what similar documents look like. It’s impressive. It’s often right. And it fails in exactly the situations where precision matters most: complex layouts, unusual formatting, dense cross-references, restated figures.
For most tasks, inference is good enough. For documents where every number matters, it isn’t and the tool will not tell you when it has crossed from one category into the other.
Who This Actually Matters To
We want to be specific rather than vague, because “financial document analysis” can mean very different things to very different people.
- Investment analysts: You’re running analysis on 10–50 companies per quarter. Each one has a 60–150 page report. You can’t afford to manually verify every extracted figure. You need a pipeline where the numbers that go into your model are the numbers on the page.
- Corporate finance teams: You’re consolidating reports from multiple subsidiaries or portfolio companies. A misread restatement in one document propagates through every comparative analysis downstream. You need extraction that catches discrepancies before they travel.
- Accounting firms and auditors: The entire value proposition of your work rests on accuracy. Using an AI tool that has a 12–20% figure error rate — and doesn’t tell you when it’s wrong — is not a productivity gain. It’s an audit liability.
- Founders and executives: You’re reading competitor filings, sector reports, or your own historical financials to build strategic context. You need the right numbers, not confidently formatted wrong ones.
- Developers building fintech tools: You’re building pipelines that need to ingest financial documents at scale. The question is whether you build on an extraction layer that’s accurate by architecture or one that’s accurate enough until it isn’t.
Final Thoughts
We’re not here to tell you AI doesn’t work. It works. The productivity gains are real, the use cases are expanding, and the tools are genuinely getting better every quarter.
But there’s a gap between what AI tools are marketed to do and what they actually do when you hand them a 150-page financial filing and ask for the numbers. That gap is being quietly filled with confident-looking errors that nobody warned you to look for because the tools have every incentive to look capable and no incentive to look uncertain.
The people who discover the gap tend to discover it the same way: after the report went out, after the model was built, after the decision was made. At which point the cost isn’t just fixing the number it’s rebuilding the work and explaining to someone why the first version was wrong.
Four NVIDIA quarterly reports. 200 pages. One tool got every single number right. The other two got most of them right and gave you no way to know which ones they missed.
The right tool doesn’t just work faster. It doesn’t fail quietly. And it tells you the truth.
If you are looking to explore other useful AI tools, check out this website: AI Pulse – AI Tools Directory
