Multi-Agent Orchestration with Open-Source Models: A Practical Pattern

The video demonstrates a practical attempt to replicate Andrej Karpathy's AutoResearch project—where a single agent systematically improves a training script—by decomposing the task across multiple, role-specific agents running open-source models. The setup uses the OpenCode harness and defines a Researcher (finds papers, proposes hypotheses), a Planner (maintains an experiment queue), Workers (patch and execute scripts), and a Reporter (aggregates results). This division of labor lets weaker open-source models handle narrower tasks reliably, which a single agent might fail at over long runs. Jobs share a Hugging Face Hub cache bucket, avoiding repeated asset downloads. Traecheo ties the multi-agent swarm together with anomaly alerts and delta-vs-master metric tracking, making it possible to oversee a churning batch of parallel experiments without sifting through logs. The video is essentially a walk-through and a repo tour, not a rigorous comparative study. There’s no head-to-head against a single-agent baseline or different model families. The presenter's claim that role separation 'makes the task slightly easier' is plausible but anecdotal—the evidence is that the system ran and found some improvements, not that it outperformed alternatives. Long-running stability issues with open-source models are noted and worked around with extra prompting, which is a real-world detail teams will recognize. For engineering leaders, the interesting part is the architecture pattern: specialized sub-agents, shared state through a results ledger, and observability hooks. This resembles incident-response or CI pipeline automation more than 'general AI.' The hype is low; limitations around model reliability and the fragility of prompt-defined agents are transparent. However, the video glosses over failure modes of multi-agent coordination when experiments conflict or planner logic hallucinates. The repository provides templates for agent instructions, which is useful for teams wanting to experiment with this pattern quickly, but the underlying problem (optimizing hyperparams of a small model) is narrow. The broader applicability to real production systems is not explored.