A comparative analysis of document understanding solutions tested on a welfare assistance form. This project evaluates how various AI models and services extract information from complex documents.
- Claude (Anthropic)
- Deepseek
- OpenAI GPT
- LLM Whisperer - Layout & text analysis
- Donut - Document understanding model
- DocuPanda - Form extraction tool
- Google Document AI - Cloud document processing
Each solution's results are available in their respective directories:
claude/- Claude's JSON outputdeepseek/- Deepseek's JSON outputopenai/- OpenAI's JSON outputllmwhisperer/- Layout and text analysisdonut/- Donut model resultsdocupanda/- DocuPanda resultsdocument-ai/- Google Document AI results and visualization
The solutions were evaluated on their ability to:
- Extract text accurately
- Identify form fields
- Structure data appropriately
- Understand document layout
- Handle checkboxes and tables
See notes.md for detailed comparison and analysis.
MIT License