Most ML demos have a dirty secret. Someone is babysitting them.
There's a notebook that has to be rerun, a parameter that gets tweaked when the data changes, or someone quietly refreshing the dashboard before the meeting.
I wanted to see how far I could push the opposite. What if you could drop a dataset into a folder and walk away? The pipeline should clean the data, engineer features, train a model, evaluate it, deploy it, and update a dashboard without anyone touching the code.
That experiment became Gideon.
It's a local, zero-cloud, zero-API automated ML pipeline. Drop a CSV, JSON, or Parquet file into inbox/, and Gideon takes it from raw data to a deployed XGBoost model with a live Streamlit dashboard. Drop in a newer version later and the whole pipeline runs again automatically.
Everything runs locally. No network calls. No external services. No API keys.
Eight agents, one rule
The interesting part isn't the ML itself. It's the architecture.
The design comes straight from the ETL systems I've built professionally. Gideon consists of eight specialist agents, each responsible for exactly one job.
ingestor → cleaner → feature_eng → trainer →
evaluator → deployer → dashboard_gen → monitorEach agent follows one simple rule: one input, one output.
There is no shared in-memory state. Agents communicate only through files stored in artifacts/.
The ingestor writes raw.parquet. The cleaner reads that file and produces cleaned.parquet plus a cleaning report. The trainer writes model.joblib. Each stage reads the output from the previous one and produces the next artifact.
A boss orchestrator runs the agents in sequence and writes a run manifest, while a filesystem watcher built with watchdog starts the whole process whenever a new dataset appears.
If you've read my Medallion pipeline case study, you'll probably notice the influence. This is Bronze, Silver, and Gold thinking applied to an ML pipeline.
Using files as contracts gives you the same benefits that layered data warehouses do. Every stage is replayable. Every intermediate result is easy to inspect. And if something breaks, you can run a single agent against the exact artifact that caused the failure.
Debugging becomes much simpler when the state lives in files instead of memory.
Making "automatic" actually automatic
Saying "nobody touches any code" only works if the pipeline can make reasonable decisions about an unfamiliar dataset.
Gideon uses a handful of deliberately simple heuristics.
- Target detection: Look for common names like
target,label,class,outcome,result, ory. If none exist, use the last column. - Task detection: Treat categorical or low-cardinality integer targets as classification. Everything else becomes regression.
- Feature engineering: Expand datetime columns, one-hot encode low-cardinality categoricals, and frequency encode high-cardinality ones.
- Modeling: Train an XGBoost model with sensible default settings.
None of these ideas are groundbreaking.
That was intentional.
Defaults that work well most of the time are far more useful here than dozens of configuration options. The whole goal was to remove the human from the loop.
The boring parts matter most
The part I'm happiest with isn't something you notice in a screenshot.
It's all the things that don't go wrong.
Artifacts are written atomically. Every file is written to a temporary location first and then renamed, so the dashboard never reads a half-written model or dataset.
Runs are processed one at a time. If another CSV arrives while one is already running, it waits its turn instead of colliding with the current run.
The watcher compares content hashes, so saving the exact same file twice doesn't trigger another training run.
If you empty the inbox, the dashboard resets to a clean waiting state instead of showing stale results from data that no longer exists.
These are the kinds of lessons production data engineering teaches you, usually after you've been burned once or twice.
Race conditions, duplicate triggers, and stale state are exactly how automated systems lose people's trust. Once someone spots one incorrect number, they start double-checking everything manually.
The plumbing matters just as much as the model.
Letting the dashboard tell the story
The Streamlit dashboard adapts itself to whatever dataset the pipeline produces.
If it finds a date column, it automatically generates month-over-month and year-over-year trends, highlights the best and worst periods, and surfaces interesting correlations in plain English.
For regression models, it includes a forecasting tab with a configurable prediction horizon and a confidence band based on ±1.5×RMSE.
Predictions that differ from the actual value by more than two standard deviations are highlighted as potential anomalies.
There's also a what-if analysis tab.
For the most influential features, the dashboard creates interactive sliders. As you move them, it runs model.predict() in real time and shows how the prediction changes.
That's usually the feature that resonates most with non-technical users.
Performance metrics are useful, but watching a prediction change as you move a slider makes the model feel tangible.
What Gideon isn't
A carefully tuned model built by someone who understands the dataset will almost always outperform the default pipeline. And heuristics that work most of the time will occasionally make the wrong call.
That's the trade-off you make for zero configuration.
For me, that's not really the point.
What Gideon demonstrates is something I've learned from building production data systems. The reliability patterns that make warehouse pipelines dependable also make ML pipelines dependable.
- Single-responsibility stages.
- File-based contracts.
- Atomic writes.
- Idempotent triggers.
Those ideas are common in data engineering, but they're still surprisingly rare in ML projects that sit somewhere between a notebook and production.
It's also worth mentioning that Gideon is still a work in progress. I built the entire project as a vibe coding experiment to see how far I could push Anthropic's Sonnet 5 as the primary developer. Rather than writing every line myself, I focused on designing the architecture, defining the requirements, reviewing the implementation, and iterating on the system while Sonnet generated the code.
The goal wasn't to build the next AutoML platform. It was to answer a simple question: How much of a production-style ML pipeline can be built when AI does most of the implementation?
The answer surprised me. While there's still plenty to improve, the project has already reached the point where it's genuinely useful. It's a solid foundation, and there's a long roadmap ahead.
The code is on GitHub.
Clone it, run uv sync, start the watcher, and drop a dataset into inbox/.
Everything else should just happen.