E-invoice Extraction
A web app that connects to a mailbox over IMAP, pulls the compressed attachments of Colombian electronic invoices, unpacks them safely and extracts the CUFE and the full invoice detail from the DIAN UBL 2.1 XML, scoped per company and exportable to Excel.
Request path in production
- React wizardmailbox and date range
- FastAPIJWT, background task
- IMAP over SSLdate search, batches of 10
- archive layersafety checks, pluggable formats
- UBL 2.1 parsingordered XPath, no model
- PostgreSQLscoped per company
- Excel exportthree sheets
What runs where
API
FastAPI
Authentication, mailbox runs, results and exports. Long runs execute as background tasks so the request returns immediately.
Archive layer
Pluggable handlers
ZIP and TAR by default, with 7Z and RAR handlers behind feature flags, all going through the same safety checks.
Extraction
lxml + XPath
The CUFE first, then the invoice detail: issuer, totals, taxes, line items, allowances and charges.
Web app
React + TypeScript
A step-by-step wizard to run a mailbox, a dashboard with results and an admin area for users and companies.
Architecture decisions
- From four services to one
- It started as four microservices talking over HTTP. I merged them into one backend: direct calls instead of network hops, one container to deploy and one place to test.
- Stages driven by status
- Each stage picks up the work that is pending for it (downloaded, extracted, CUFE found, detail parsed), so one broken email or file never stops the batch.
- Deterministic over generative
- I built an LLM fallback with schema-checked output and cost logging, then took it out of the live path: XPath over the UBL structure proved more reliable and costs nothing per invoice.
- Files are temporary
- Attachments live in temporary storage only for the length of a run and are deleted after the database commit.
- Mailbox credentials are never stored
- They are sent for one run, used for one IMAP session and discarded.
- Moving to clean architecture
- Domain, application and infrastructure layers with repository interfaces and a dependency-injection container, migrated module by module.
Reading Colombian e-invoices
- UBL 2.1 and DIAN namespaces
- The parser knows the UBL common components and the DIAN extensions, instead of guessing at text.
- CUFE, with fallbacks
- Ordered, confidence-scored XPath first; then the document key inside the invoice QR data; text search only as a last resort.
- Wrapped documents
- Handles the attached-document envelope that carries the actual invoice inside it.
- Full invoice detail
- Issuer, amounts and taxes, line items, and allowances and charges, linked to the email they came from.
Unpacking untrusted archives
- Size limits
- A cap on archive size and on the total uncompressed size.
- Zip-bomb check
- Archives with an extreme compression ratio are rejected before extraction.
- File limits
- A maximum number of extracted files and an allow-list of extensions.
- Path traversal
- Relative and absolute escapes are rejected, and every target path is resolved to confirm it stays inside the extraction folder.
Users, companies and delivery
- Accounts and roles
- JWT sessions, bcrypt passwords, and separate user and admin roles.
- Per-company data
- Results and exports are filtered to the signed-in user's company.
- Containers
- A Docker image for the API and an nginx-served build of the web app, deployed on Render, with a scale-to-zero serverless variant.
- Tests
- 55 automated tests covering XML extraction, mailbox processing against a mocked IMAP server, repositories and services.