When building production RAG systems, document extraction accuracy isn’t a nice-to-have, it’s the difference between a system users trust and one they abandon. A single misextracted table cell in a financial report can propagate a $10M error through your analytics. A wrongly parsed medication dosage in medical documents can have life-threatening consequences. A garbled legal clause can invalidate contracts.
Yet most teams building RAG systems treat extraction as solved: “just use LlamaIndex” or “throw it at an OCR API.” This assumption is dangerous. We benchmarked Kudra against LlamaIndex—the most popular document parsing library in the RAG ecosystem—on complex, real-world documents. The results reveal why extraction accuracy is the silent killer of RAG systems.
Findings:
- LlamaIndex fails catastrophically on nested table structures (tables within tables)
- Visual grounding shows Kudra achieves pixel-perfect field localization; LlamaIndex produces misaligned, incorrectly formatted output
- On complex tax forms (1040), Kudra extracts 100% of tabular data correctly; LlamaIndex produces corrupted values and structural errors
- For production systems where accuracy isn’t negotiable, the choice is clear
This article presents side-by-side benchmark results, explains why LlamaIndex’s architecture fails on complex documents, and demonstrates why teams building mission-critical RAG systems choose Kudra.
Want Better Extraction Quality?
Benchmark 1: IRS Form 1040 (Tables Within Tables)
Why Form 1040?
We selected IRS Form 1040 (U.S. Individual Income Tax Return) as our benchmark document because it represents the worst-case scenario for extraction systems:
Challenge 1: Nested Table Structure The “Dependents” section contains a table embedded within the larger form structure. This table-within-table design is common in government forms, legal documents, and financial filings—but breaks most extraction tools.
Challenge 2: Multi-Column Layout Form 1040 uses complex multi-column layouts where spatial positioning determines meaning. Fields are tightly packed with minimal whitespace.
Challenge 3: Semantic Ambiguity Labels like “(1) First name” and “(2) Social security number” use numbering that looks like data values. Extraction systems often confuse labels with content.
Challenge 4: Dense Information The dependents table contains:
- Multiple rows (each dependent)
- Multiple columns (first name, last name, SSN, relationship, child tax credit, credit for other dependents)
- Checkboxes (binary data)
- Mixed data types (text, numbers, boolean)
If a system can extract this correctly (preserving row-column relationships, distinguishing labels from data, maintaining nested structure) it can handle production document complexity.
The Document: Visual Analysis
The Nested Table Challenge:
The form contains 7 distinct structural layers, nested checkboxes, multi-part field labels, and tightly-packed data. This is representative of real-world document complexity in finance, healthcare, and legal domains.
LlamaIndex Output (FAILED)
Raw extraction:
Analysis of Failure:
- Row boundaries obliterated: Can’t distinguish Giuseppe’s data from Mayra’s data
- Column relationships lost: SSN “083 94 1905” split incorrectly (should be “083-94-1905” for Giuseppe)
- Checkbox values garbled: “:unselected:” and “:selected:” notation is non-standard and unreliable
- Labels mixed with data: “(1) First name Giuseppe” suggests the label is part of the value
- Spatial meaning destroyed: Impossible to tell which checkbox corresponds to which dependent
Real-World Impact: If this were ingested into a RAG system:
- Query: “Does Mayra qualify for child tax credit?”
- Retrieved chunk contains garbled text
- LLM cannot reliably extract boolean value
- Result: System unusable for tax preparation applications
Issue 2: Value Corruption
Examining the SSN extraction:
- Correct: Giuseppe SSN = 083-94-1905, Mayra SSN = 573-98-1070
- LlamaIndex extracted: “083 94 1905” (spacing incorrect, no hyphenation), “573 98 1070”
Why this matters: SSN format validation would fail. If this data flows to a compliance system expecting standard SSN format (XXX-XX-XXXX), it would reject the record.
Issue 3: Missing Data & Hallucinations
LlamaIndex sometimes drops fields entirely or invents structure that doesn’t exist. In complex tables, columns can simply vanish from the output, or the parser might create spurious rows to “fill gaps” in its understanding.
Kudra Output: Perfect Structure Preservation
When we ran the same Form 1040 through Kudra, the extraction maintained complete structural fidelity:
What went right:
- Perfect table structure preservation
- Clean row separation (each dependent is discrete object)
- Column alignment maintained (every field maps to correct header)
- Checkboxes properly parsed (
true/falsebooleans) - Format standardized (SSN in XXX-XX-XXXX format)
- Type-aware (strings, booleans correctly identified)
Impact on RAG:
Query: “Does Mayra qualify for child tax credit?”
Direct lookup: rows[1]['child_tax_credit'] == true
System is production-ready.
Benchmark 2: Visual Grounding
Visual grounding means linking each extracted value to its exact pixel location in the source document. This is critical for:
- Audit trails (regulatory compliance)
- Error detection (verify extraction correctness)
- Debugging (trace wrong answers to extraction errors)
LlamaIndex: No Visual Grounding
Output: Plain text with no spatial information
- Cannot verify which pixels this came from
- Cannot audit extraction for compliance
- Cannot debug when RAG gives wrong answers
- Black box: You get text output, no provenance
For regulated industries (finance, healthcare, legal): This is disqualifying. You cannot prove the extracted value came from the source document.
Kudra: Pixel-Perfect Visual Grounding
Output: For the same image every field has color-coded bounding boxes on source image
- Verify extraction visually (see exactly where each value came from)
- Audit trail ready (screenshot visual grounding for compliance)
- Debug RAG errors (trace wrong answers: was extraction bad or retrieval bad?)
- Confidence scores (Kudra shows per-field confidence for human review)
Example use case: Your RAG system gives a wrong answer about a dependent’s SSN. With Kudra, you can:
- Look at visual grounding
- See the SSN was correctly extracted from pixel coordinates (340, 120)
- Confirm the error was in retrieval/generation, not extraction
- Fix the actual problem
With LlamaIndex: You have no idea where the error came from.
Why Kudra Succeeds Where LlamaIndex Fails
Architecture Difference
LlamaIndex (Language-Model-First):
Document → VLM → Text/Markdown → Parse
- VLM infers table structure from text patterns
- Markdown cannot represent nested tables accurately
- No explicit layout detection
- No visual grounding by design
Kudra (Vision-First, Structure-Aware):
Document → Layout Analysis (CV) → Explicit Table Detection → VLM → Specialized Extractors → Structured JSON + Bounding Boxes
- Computer vision detects table boundaries using pixel geometry
- Understands nested structures (table-in-form hierarchy)
- Specialized extractors for tables, checkboxes, text
- Visual grounding built-in (every field has coordinates)
Result: Kudra preserves structure.
The Benchmark Verdict
Our head-to-head comparison on IRS Form 1040 reveals stark differences:
| Dimension | LlamaIndex | Kudra |
|---|---|---|
| Nested Table Handling | Failed (structure destroyed) | Perfect (preserved all relationships) |
| Visual Grounding | None (black box output) | Pixel-perfect (color-coded bounding boxes) |
| Value Accuracy | Errors (wrong SSN format, garbled checkboxes) | 100% (correct format, proper boolean parsing) |
| Structural Fidelity | Markdown approximation (lost hierarchy) | JSON with full hierarchy (table-in-form) |
| Audit Trail | No (cannot verify extraction provenance) | Yes (every field linked to source pixels) |
| Production-Ready | No (requires manual verification) | Yes (structured output, confidence scores) |
Conclusion: For simple documents, LlamaIndex may suffice. For complex, high-value documents (the kind that matter most in production RAG systems) Kudra is the only reliable choice.
Why This Matters for Your RAG System
If you’re building:
- Financial RAG: Parsing 10-Qs, balance sheets, analyst reports → You need Kudra’s table accuracy
- Medical RAG: Extracting dosage tables, lab results, clinical guidelines → You need Kudra’s structural precision
- Legal RAG: Analyzing contracts, compliance docs, regulatory filings → You need Kudra’s audit trails
- Enterprise RAG: Processing invoices, forms, reports with complex layouts → You need Kudra’s nested table support
The documents that contain the highest-value information are precisely the ones that break generic extraction tools. Your RAG system’s value is capped by your extraction quality.
