Gideon: Drop a CSV, Get a Live ML Dashboard

Most ML demos have a dirty secret: someone is babysitting them. There's a notebook that has to be re-run, a cell that has to be edited when the data changes, a person quietly regenerating the dashboard before the meeting. I wanted to know how far I could push the opposite: drop a file in a folder, and everything else — cleaning, feature engineering, training, evaluation, deployment, dashboard — happens without anyone touching code.

That experiment became Gideon: a local, zero-cloud, zero-API automated ML pipeline. Drop a CSV (or JSON, or Parquet) into inbox/, and Gideon takes it from raw file to a deployed XGBoost model with a live Streamlit dashboard. Drop a newer file, and the whole pipeline reruns and the dashboard updates. Everything runs on your machine — no network calls, no external services, no keys.

Eight agents, one rule

The interesting part isn't the ML. It's the architecture, and it comes straight from my day job building ETL pipelines. Gideon is eight specialist agents, each owning exactly one task:

ingestor → cleaner → feature_eng → trainer →
evaluator → deployer → dashboard_gen → monitor

The rule that makes it work: one agent, one input, one output — no shared in-memory state. Agents communicate only by reading and writing files in artifacts/. The ingestor writes raw.parquet; the cleaner reads it and writes cleaned.parquet plus a cleaning report; the trainer writes model.joblib; and so on down the line. A boss orchestrates the sequence and writes a run manifest, and a watcher (built on watchdog) triggers the boss when a new file lands.

If you've read my Medallion pipeline case study, you'll recognize the instinct — this is Bronze/Silver/Gold thinking applied to an ML system. Files as contracts buy you the same things layers buy you in a warehouse: every stage is replayable, every intermediate result is inspectable, and when something breaks you can run one agent standalone (python -m agents.cleaner) against the exact artifact that made it fail. Debugging a pipeline where state lives in files is a search; debugging one where state lives in memory is an archaeology dig.

Making "automatic" honest

"Nobody touches any code" is only true if the pipeline can make its own decisions about an arbitrary dataset. Gideon handles that with a few deliberately boring heuristics:

• Target detection — use a well-known column name (target, label, y, class, outcome, result) if one exists, otherwise the last column.
• Task detection — non-numeric or few integer-like values means classification; otherwise regression.
• Feature engineering — datetime expansion, one-hot encoding for low-cardinality categoricals, frequency encoding for high-cardinality ones.
• Modeling — XGBoost with sensible defaults, no tuning theatre.

None of these decisions is clever, and that's the point. Sensible defaults that are right 90% of the time beat configurable options that require a human 100% of the time — a human in the loop was the exact thing I was trying to remove.

The unglamorous parts are the product

What I ended up most proud of isn't visible in a screenshot. It's the failure modes that don't happen:

• Artifacts are written atomically — temp file, then rename — so the dashboard can never read a half-written file. There is no window where a chart renders from a model that's mid-save.
• Runs are serialised. Drop a second CSV while the first is processing and it queues; it doesn't corrupt the run in flight.
• The watcher de-duplicates by content hash, so re-saving the same file doesn't burn a pointless retrain. Only genuinely new data triggers the pipeline.
• Emptying the inbox resets the dashboard to a clean waiting state instead of showing stale results from data that no longer exists.

Every one of those is a lesson production data engineering teaches you eventually, usually at 2 a.m. Race conditions, duplicate triggers, and stale state are exactly how "automated" systems lose people's trust — and once a stakeholder catches one wrong number, they re-check everything by hand forever after. The plumbing is the product.

The dashboard does the talking

The Streamlit dashboard renders whatever the pipeline produced, and it adapts to the dataset. A dataset with a date column gets month-over-month and year-over-year trends, best and worst months, and auto-discovered correlations described in plain language. A regression target gets a forecast tab with a configurable horizon and a ±1.5×RMSE confidence band. Predictions that deviate more than 2σ from actuals get flagged red as anomalies. And a what-if tab builds one slider per influential feature and runs a live model.predict() as you move them, showing the delta against the baseline.

That last tab is the one that lands with non-technical people. Metrics cards tell an analyst the model is fine; dragging a slider and watching the prediction move is what makes a stakeholder believe the model is real.

Where it stops

Gideon is not AutoML and doesn't want to be. It won't beat a tuned model built by someone who sat with the data, and heuristics that are right most of the time are still wrong sometimes — that's the tax on "zero config". What it demonstrates is the part I care about professionally: that the reliability patterns from warehouse engineering — single-responsibility stages, file contracts, atomic writes, idempotent triggers — are exactly what most ML tooling is missing between the notebook and production.

The code is on GitHub. Clone it, run uv sync, start the watcher, and drop a CSV in. Nobody touches any code — that was the whole idea.