agent-mapreduce

Autoresearch as recursive agentic map-reduce.

sachinkesiraju/agent-mapreduce

Python·★ 0·pushed

Last week I was inspired by this launch for Devin's new Security Swarm product https://devin.ai/blog/agentic-map-reduce

Most notably, to build this the Cognition team adapted a simple idea from distributed computing to divide and reduce work with agents. I decided to apply it to open-ended research tasks and have found it extremely helpful.

Simply put, Karpathy's autoresearch loop tests one idea at a time:

one idea -> eval -> keep/revert -> repeat

Agentic map reduce tests multiple ideas per generation:

K ideas -> K isolated worktrees -> keep top B -> repeat

The loop:

1. Run the baseline twice. The spread becomes the noise margin.

2. Propose K ideas from the current frontier.

3. Create one git worktree per idea.

4. Send one worker agent into each worktree. It implements one idea and runs the eval.

5. The orchestrator greps scores from artifacts and logs every result.

6. amr.py reduce keeps candidates that beat their own parent by more than the margin.

7. The top B survivors become the next frontier.

8. Repeat.

Why use this over autoresearch?

Traditional autoresearch is a greedy hill-climber:

greedy: keep the first thing that improves

This is beam search over research branches:

beam: compare many ideas, keep the best few, recurse

This helps when one idea at a time is too path-dependent. Agentic map reduce allows you to compare sibling ideas from the same parent, avoid advancing on tiny noisy wins, keep a few competing branches alive, and discover paths where A alone is weak but A+B wins.

Use this when:

• you want to avoid local optima

• you care about comparing sibling ideas fairly

• you want a tree of research, not a linear walk

• you can afford K evaluations per generation

• useful changes may only pay off in combination

Generally, you may want to stick with traditional autoresearch when evals are expensive, when there is one obvious direction forward, or you want the smallest possible loop.

I firmly believe we are in the early innings of new agent paradigms that can divide and conquer long horizon tasks and generate novel research leaps.

We've already seen how new models like Fable when paired with the right goal-oriented harness can make impressive progress. I believe more token-abundant paradigms that allow agents to self-improve and hill climb on goals will be incredibly transformative.