Intake & file-type routing
An intake form receives the files, and a router sorts each one by type so a spreadsheet, a PDF, an image, and a ZIP each follow the right path through the pipeline instead of a one-size-fits-all parser.
📊AI vendor reporting pipeline
Engage Media: vendor reporting that ingests itself
We built Engage Media, a media agency handling offline, OOH, broadcast, and digital vendors, an end-to-end pipeline that turned fully manual, error-prone data ingestion from ~20 vendors into an automated, validated flow with a human check on anything uncertain.
Engage Media received performance data from around 20 vendors in wildly inconsistent formats: Excel files, CSVs, PDF screenshots, even ZIP folders of billboard photos. All of it was ingested into the reporting dashboards 100% by hand, which is slow and easy to get wrong. We built an end-to-end n8n pipeline that routes each file by type, reads it with AI, de-duplicates and validates it, resolves campaign names against a canonical list, and merges clean records into the reporting warehouse. Anything the AI is not confident about is quarantined and sent to a person to review, so the dashboards stay trustworthy.
🎯The problem
Twenty vendors, twenty formats, all keyed by hand
Media reporting is only as good as the data feeding it, and Engage Media's data arrived in every shape imaginable. Around 20 active vendors sent performance data as Excel workbooks, CSV exports, PDF screenshots, and in some cases ZIP archives full of billboard photos. No two vendors agreed on a layout, a header row, or even a file type.
Every one of those files was ingested into the reporting dashboards manually. Someone had to open each file, figure out which columns meant what, match it to the right campaign, watch for duplicate co-op invoices, and type it all into the warehouse. At that volume and variety, the work was slow and error-prone, and a single mistyped figure or a misread screenshot could quietly skew a client's dashboard.
🧭The approach
Let AI read the mess, let rules guard the warehouse
The variety was the whole challenge, so the design leans on AI exactly where it is strong, reading inconsistent documents and images, and on deterministic rules everywhere else. AI extracts and maps. Validation, de-duplication, and campaign matching are rule-driven against a canonical reference. And crucially, the pipeline scores its own confidence: anything it is unsure about is quarantined and routed to a human before it can touch the dashboards. That way automation handles the volume without ever silently feeding bad data into a client report.
⚙️What we built
An end-to-end ingestion pipeline
Built on n8n, BigQuery, Gemini, Looker Studio, Google Cloud Storage, and Gmail, plus a direct daily connector for one streaming-TV vendor. The AI reads documents and images, the rules validate and route, and uncertain extractions go to human review.
AI extraction
Gemini Vision reads images and photos, and Gemini extraction handles spreadsheets and PDFs, detecting header rows and mapping columns. The system turns each vendor's idiosyncratic layout into a common structured shape.
De-dup, aggregate & resolve
Co-op invoices are de-duplicated, broadcast data is aggregated to a daily grain, and campaign names are resolved against a canonical list held in BigQuery, so messy vendor naming lines up with the agency's real campaigns.
Validate & quarantine
Records are validated, and any low-confidence extraction is routed to a quarantine path for human review instead of flowing straight through. People only ever look at the uncertain cases, not every file.
Merge & report
Clean records merge into the metrics warehouse, which feeds a unified view to Looker Studio with AI insights. A direct daily connector pulls one streaming-TV vendor straight in, no manual step at all.
📈The outcome
A fully manual, error-prone ingestion process became an automated, validated pipeline with a human check on anything uncertain. The team stopped re-keying every vendor file by hand, and the reporting dashboards now draw from data that has been de-duplicated, campaign-matched, and confidence-checked before it lands.
~20 vendors
of inconsistent formats automated into one validated reporting warehouse
- Every format handled Excel, CSV, PDF screenshots, and ZIPs of billboard photos routed and read automatically instead of by hand
- Confidence-scored quarantine low-confidence extractions go to human review, so nothing uncertain reaches a client dashboard
- De-duped and canonically matched co-op invoices de-duplicated and campaign names resolved against a single canonical list
- Unified reporting clean data merged into one warehouse feeding a single Looker Studio view with AI insights
"Bastien was great to work with. He is very responsive, supportive, and takes strong initiative throughout the project. He is amazing at what he does and the value he offers. He did strong research, stayed accountable, and handled his responsibilities very well. He also completed tasks before the deadline, which made the whole process smooth and reliable. I strongly recommend Bastien to any client looking for someone dependable, proactive, and professional."
Verified on Upwork
🔗Services used
Built with
📬Contact
Drowning in vendor files every month?
Book a free audit. We will map how your data comes in and show you exactly what to automate first, with a human check kept on anything uncertain. No spam, ever. Your details stay confidential.
Book My Free Audit bastien@otimizagency.com