LLM Enrichment Pipeline
Two services that pull records from an external system, enrich them with an LLM and write them back safely: a serverless function app for ingestion and write-back, and a containerized worker for enrichment, sharing PostgreSQL and a queue. Under NDA, so this covers the architecture only.
Architecture
- Claim-check messaging
- Only a batch identifier travels on the queue; the payload stays in PostgreSQL.
- Staged workers
- Timer-triggered workers each advance one stage, claiming rows with FOR UPDATE SKIP LOCKED and tracking status per stage, so failures retry without blocking the rest.
- Singleton ingestion
- PostgreSQL advisory locks keep ingestion to one instance at a time.
- Distributed rate limiting
- A token bucket stored in PostgreSQL, on the database's clock, so every instance shares one limit.
- Circuit breaker
- A closed / open / half-open breaker protects write-back, and 429 responses honour Retry-After.
- Hexagonal layering
- Ports, adapters and orchestrators, with import rules enforced automatically.
- LLM guardrails
- Schema-constrained output, cost tracking, a daily spend cap and deduplication of recent work.
- Privacy by design
- Personal data is stripped before storage and never travels on the queue.
Main processing path
- scheduled triggertimer
- external readerrate-limited
- PostgreSQL + queueclaim-check
- queue workercontainerized
- LLM enrichmentschema-constrained
- write-backcircuit breaker
Simplified, with generic component names. Highlighted steps use a model; the rest is software.