Adaptive-Machine-Patterns/amp-tadpole

AMP Tadpole

A small vision-language-action model that drives a robot arm from three camera images and a plain-English instruction. It has 6.5M parameters and runs on a laptop CPU (and, through ONNX, in a web browser).

AMP Tadpole is trained from scratch on the AMP Toy Robotics Benchmark, and is the first model of Adaptive Machine Patterns.

Try it in your browser: the AMP Embodied AI Workbench Demo runs AMP Tadpole live (through ONNX), and shows it inside, layer by layer.

v0.2.0 is a preview. Later versions will be published under new tags (see Versions).

What it does

Every 8 steps the model is given:

• the top, side and isometric camera images (512 × 512 RGB each);
• the arm's state: 6 joint angles and the gripper opening;
• one instruction clause, e.g. "Pick up the lime star".

It returns the next 8 steps of joint angles and gripper opening. It is never told where the objects are: it has to find them in the pictures and work out which one the words mean. The benchmark's closed loop executes the steps and asks again, until the model asks for no further motion.

Instructions: pick up the X, move to the X, move to the midpoint between X and Y, open the gripper, close the gripper, return home, chained with "then".

Results

On the benchmark's standard 300 held-out scenes (never seen in training), with the model driving the arm itself:

clauses	pick	move	midpoint	open	close	home	all	path vs teacher
898	93%	94%	75%	100%	100%	100%	93.7%	1.03

The reference teacher, a scripted controller that knows where everything is, scores 99.4% on the same scenes.

Grounding tests (100 further held-out scenes; see the benchmark's README):

named object	other objects	image swap	instruction swap	blank instruction
0.98	0.31	1.00	1.00	0.95

Told "move to the X", its motion points the way the teacher's does (0.98, where 1 is the same direction), and not toward the other objects in the scene (0.31). When two objects swap places, or the instruction names another object, its motion changes as much as the teacher's does (1.00). So it uses both the cameras and the words.

These results are the same under the benchmark's version 0.2.0, which adds a joint speed limit (no joint faster than 0.12 rad per step): it slowed 0.13% of AMP Tadpole's steps, and no clause came out differently. AMP Tadpole takes 1.13 times the teacher's steps on the clauses both complete, and its path is 1.96 times a straight line (the teacher's: 1.90).

How the checkpoint was chosen. The training run checked itself every 2,500 steps on 100 held-out scenes, and those turned out to be the first 100 of the 300 above (seeds 9,010,000 to 9,010,099); v0.1.0 was picked that way. For v0.2.0 the two final candidates (steps 300,000 and 330,000) were compared on 300 separate validation scenes (seeds from 9,100,000), where they tied at 94.2%; by a rule set beforehand, the tie went to the longer-trained one. On the 200 benchmark scenes that no choice ever looked at (608 clauses), v0.2.0 scores 93.1%.

These numbers are a line in the sand, not a ceiling. They are published as a reference point for small models on the benchmark: something to beat.

These come from amp-bench eval --policy amp-tadpole on a CPU (about 40 minutes). Other hardware can differ in the last digits of the arithmetic, so a clause or two may come out differently.

Use it

With the benchmark (pip install "amp-toy-robotics[packages]"):

hf download Adaptive-Machine-Patterns/amp-tadpole --revision v0.2.1 --local-dir amp-tadpole
amp-bench eval --policy amp-tadpole

In Python, this repository is a model package:

import importlib.util
from safetensors.torch import load_file

spec = importlib.util.spec_from_file_location("tadpole", "amp-tadpole/model.py")
tadpole = importlib.util.module_from_spec(spec); spec.loader.exec_module(tadpole)
model = tadpole.build()
model.load_state_dict(load_file("amp-tadpole/weights.safetensors"))
model.eval()

chunk = tadpole.predict(model, images, state, "Pick up the lime star")   # (8, 7)

With LeRobot: install lerobot-policy-tadpole and load amp-tadpole-lerobot, the same weights in LeRobot's format (--policy.path=Adaptive-Machine-Patterns/amp-tadpole-lerobot).

With ONNX Runtime (any language, or the browser): amp-tadpole.onnx takes images uint8 (1, 3, 3, 512, 512), tokens int64 (1, 48), token_mask bool (1, 48) and state float32 (1, 7), and returns chunk (1, 8, 7). instructions.json gives the tokens for every canonical clause.

How it works

part	what it does
CNN, shared by the cameras	512 × 512 pixels (plus two pixel-coordinate channels) to a 16 × 16 grid of tokens per camera
word embeddings	the clause's tokens (up to 48)
state token	the 7 numbers, with sines and cosines of the joint angles
8 action tokens	learned; one per output step
transformer	6 layers, width 256, 8 heads, over all 825 tokens
head	each action token gives one step's change; the changes are added up

It predicts the 8 steps directly in one pass (no sampling), which suits a teacher whose action is a fixed function of the state.

Training

• Data: generated on the fly from scene seeds by the benchmark (amp_toy_robotics.data), labelled by the scripted teacher. No external data, no dataset files, no human demonstrations. Scenes from seed 9,000,000 up were never used.
• Inputs only: the images, the state and the text, under the benchmark's rules. No object positions, no progress reward, no path penalty.
• Recipe: 330,000 steps, batch 64, AdamW (peak learning rate 3e-4, 1,000-step warm-up, cosine decay; the run was extended and resumed several times), the state hidden from half the samples (so the model must use the cameras), and an auxiliary loss in which each image token redraws its patch of the image. The redraw head is used only in training and is not included.
• The word table was cut to the 32 tokens the instructions use. This changes no output, and takes the model from 19.2M to 6.5M parameters.

Limitations

• It knows only this simulated world: one arm, three fixed cameras, 8 colours and 5 shapes. It has not seen a real camera.
• It reads only canonical clauses (instructions.json); the benchmark rewrites free text into canonical form first.
• The midpoint instruction is its weakest (75%; see Results). It is not overfitting: on teacher-labelled samples its errors are the same on scenes it trained on and scenes it never saw. Midpoint is simply less well learned. Its target is an empty spot between two objects, so there is nothing to aim at in the pictures, and it is 10% of the training clauses; its error was still shrinking when training ended.
• What these results do not show. The teacher is deterministic and unlimited and the cameras are fixed: that is why a 6.5M-parameter model does well here. It is not evidence against larger VLAs on benchmarks of human demonstrations, which are varied, finite and often use a wrist camera.
• All results come from one training run; it has not been repeated with other random seeds.

Versions

version	tag	what changed
0.2.1	v0.2.1	model.py now finds instructions.json when loaded from the Hugging Face cache; weights unchanged. AMP Tadpole for LeRobot: amp-tadpole-lerobot
0.2.0	v0.2.0	step 330,000, the end of the training run: 93.7% (from 88.2%), midpoint 75% (from 63%); checkpoint chosen on separate validation scenes
0.1.0	v0.1.0	first preview: step 250,000. Its card said the 300 scenes were not used to choose the checkpoint; the training run's own checks used 100 of them (see Results)

Pin a version with revision="v0.2.1" (or --revision v0.2.1).

Licence

Free for non-commercial use under CC BY-NC 4.0 (see LICENSE and NOTICE). For commercial use, licences are available from Adaptive Machine Patterns Ltd: adaptive-machine-patterns.com/embodied-ai-workbench or amp@adaptive-machine-patterns.com.

No warranty. AMP Tadpole is a research preview, provided "as is", without warranty of any kind, and Adaptive Machine Patterns Ltd accepts no liability for its use, as set out in sections 5 and 6 of the licence. It is not suitable for controlling real machinery, or for any use where a wrong movement could cause harm or damage.

Credits

• The token ids in instructions.json come from the tokenizer of SmolVLM2-500M-Video-Instruct (Apache-2.0; Marafioti et al., SmolVLM: Redefining small and efficient multimodal models, arXiv:2504.05299). The model's weights are AMP's own, trained from scratch.
• Written with the help of AI coding assistants (Anthropic Claude, Qwen, OpenAI ChatGPT).

Cite

@misc{amp_tadpole_2026,
  author    = {Whyte, Andy and {Adaptive Machine Patterns Ltd}},
  title     = {{AMP Tadpole}: a small vision-language-action model},
  year      = {2026},
  version   = {0.2.1},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/Adaptive-Machine-Patterns/amp-tadpole}
}

© 2026 Adaptive Machine Patterns Ltd, registered in England & Wales, no. 15618869.