How to Run Karpathy's Autoresearch on Vast H100s
Andrej Karpathy recently released autoresearch -- a framework that lets an AI agent autonomously run ML research experiments while you sleep. Point it at a GPU, give it a training script, and it will iterate on model architecture and hyperparameters for hours, keeping improvements and discarding regressions. No human in the loop.
The idea is simple but powerful: a single H100 running overnight can execute ~100 five-minute training experiments. Each one modifies the model code, trains, evaluates, and decides whether to keep or revert. By morning you have a log of findings from every experiment.
Why This Matters
ML research has always been bottlenecked by the experiment loop -- tweak, train, evaluate, repeat. Autoresearch collapses that loop by handing it to an AI agent (Claude Code) that can run continuously. It's not replacing the researcher -- it's running the tedious overnight grid searches and ablations so you can focus on the ideas that require human intuition.
Autoresearch includes a simplified single-GPU implementation of nanochat -- a small GPT trained on FineWeb-Edu and evaluated on validation bits-per-byte (val_bpb). The agent has full freedom to modify the training script -- architecture, optimizer, learning rate schedules, attention patterns -- within a fixed 5-minute time budget per experiment.
Running It on Vast
We put together a step-by-step guide for running autoresearch on Vast.ai. It covers everything from renting a GPU instance to launching the autonomous research loop:
Autonomous AI Research with Autoresearch on Vast.ai ->
The guide walks through:
- Renting a GPU instance with the Vast CLI
- Installing dependencies (
uv, Node.js, Claude Code) - Preparing the training data and running a baseline experiment
- Configuring Claude Code for unattended overnight operation
- Monitoring experiments as they run
All you need is a Vast.ai account and a Claude Code account. The whole setup takes about 15 minutes before you can kick off the research loop and walk away.
