从 10 便士硬币到机器人 Chip
你好，我是 Jonny。我在布莱克浦 Dickson Road 经营一家小店，店里不忙的时候就研究 AI。
小时候我几乎住在游乐场里，伴随着噪音、灯光和一口袋 10 便士硬币长大。成年后，我一直着迷于科技，也着迷于 AI。
2015 年，我读到一台电脑只凭画面和分数自学 Atari 游戏的报道，于是也想试试。当时需要用 USB 启动 Linux、配置双系统，每次安装和调整都是工程噩梦。修环境的时间比看 AI 学习的时间还多。
后来，我被其他突然冒出来的爱好带走了几年。我的记性不太好，又有覆盖旧文件的习惯，所以早期那段历史已经有些模糊。
2026 年 9 月，我重新投入这个项目，像着了魔一样。我错误地从旧处接着做，没有先看看技术进步了多少，结果浪费了大量时间。电脑里有个叫 Do Not Use 的文件夹，还有七个不同版本的 Track & Field。我的 Space Invaders 机器人发现角落最安全，就躲在那里对空开火。失败了一长串，然后终于突破。大部分过程都是边做边摸索。
它最初运行在 Windows 内置的 Linux 中。后来我想：做一个便携版，让任何人都能尝试、都能参与。不用 Linux，不用安装。解压、双击、选游戏。
一定有人比我更懂电脑和 AI，却被复杂的配置吓退了。这就是为他们做的。哪怕只让一个人爱上它，这个项目就值了。
这大概会遗传。我女儿大学学的就是机器学习，现在也在 AI 行业工作。向她解释我那些早期进化机器人，是这个项目的一半原因；另一半是我的复古街机俱乐部。我们每月聚一次，我答应过要拿点成果给他们看。
谢谢，Jonny
FOR THE CURIOUS: HOW CHIP ACTUALLY LEARNSNo equations, promise. Just how it works, the way I'd explain it over the counter.
He looks at the screen, like you do
Chip doesn't get any secret information from the game. He sees the same picture you'd see, shrunk down, plus the score. He looks at the last few pictures together, like a flick-book, so he can tell which way the ball's going and where the bullets are heading.
He has a go, then looks back
He plays for a little while, then pauses and has a think: did that go better or worse than I expected? Whatever went better, he does a bit more of. Whatever went worse, a bit less. Never too much at once, so he doesn't forget what he already knows. Then he plays again. Millions of times.
It's like a boxer watching the tape of their last fight. Same boxer next time, just a better one.
He learns to be patient
My early bots only cared about points, so they grabbed the quickest points they could. In Space Invaders that meant sitting in the corner and firing at nothing. The fix was making a lost life really count. Camping in the corner gets you shot more, so he learns the only way to keep his lives is to move, dodge and get good. Points now are nice. Still being alive later is better.
Get the balance wrong the other way and he turns coward. One night I paid him a little for every second he stayed alive and went to bed. By morning he was hiding under the bases, not firing at all.
He gets curious
In some games the points are miles away. A bot that only chases points never finds them, so it just wanders about. Pick the exploration setting and Chip gets a little reward every time he finds a screen he's never seen before. Curiosity, basically. It's his best chance in something like Pitfall.
Before the breakthrough: breeding bots
I bred them first. An Atari has 128 bytes of memory, so I made each bot a list of 12,800 numbers (12,806 if you're counting). I liked the symmetry. Each bot picked one of six moves at a time. All 896 of them played three games each, the best survived, and the next lot were copies of the winners with every number nudged slightly at random. They learned fast, then hit a wall. Well, a corner.
Evolution mode: a family of Chips
The kit's second way to learn is the grown-up version of my old breeding idea. A small family of Chips take turns practising. After each round, the weakest gets swapped for a copy of the best, with a few of its learning settings nudged up or down at random. I call those nudges mutations. No idea if that's the right word. The hope is a monster in a surprisingly short time; I haven't raced it against the one-brain version yet.
He can learn by watching
Both ways of learning can learn by watching you play. Show him how, and next time he trains he copies you first, then learns to beat you. You can even let one bot show a fresh Chip how it plays. When I tried that, he seemed to fly ahead, and the teacher's style stuck around long after. Part of that's by design (the pull towards the teacher fades slowly, over 5 million moves), so I want to measure it properly. It still baffles me.
What's next: playing in his head
The cleverest AIs now work out how a game works, then practise inside their own imagination, like a snooker player lining up shots in their head the night before a match. Then there's learning from videos of experts playing. That's where I'm heading next. No promises on dates.
THANK YOU: THE CLEVER PEOPLE AND PAPERS BEHIND ITI'm self-taught and I learn as I go. These are the people whose work I've read, borrowed from or bumped into along the way.
- DeepMind · 2015Human-level control through deep reinforcement learning
The one that started it all for me. A computer learning dozens of Atari games from nothing but the screen and the score.
- OpenAI · 2017Proximal Policy Optimization
Chip's way of practising: have a go, look back, do a bit more of what worked, but never change too much at once.
- DeepMind · 2018IMPALA
Where Chip's eyes come from: the part of his brain that looks at the screen.
- Uber AI · 2017Deep Neuroevolution
Proof that breeding the best bots can hold its own on Atari games. Read loads on it.
- Uber AI · 2019Go-Explore
Clever ideas for games where the points are miles away and you have to go exploring first.
- OpenAI · 2018Exploration by Random Network Distillation
Where Chip's curiosity comes from: a little reward for finding screens he's never seen.
- NVIDIA · 2019CuLE: Atari games on a graphics card
Running thousands of Atari games at once on a graphics card. Blew my mind.
- DeepMind · 2018Playing hard exploration games by watching YouTube
An AI that learns from videos of people playing. Next on my list.
- DeepMind · 2019MuZero
Works out the rules for itself, then plans its moves in its head.
- Danijar Hafner and friends · 2023DreamerV3
Dreams up the game in its head and practises in its dreams. One day, Chip.
- The Farama Foundation and friendsArcade Learning Environment
The Atari 2600 playground AI researchers use. The kit uses it for the Atari games.
- The emulator teamsFinalBurn Neo and MAME 2003-Plus
Years of volunteers' work keeping arcade machines alive. They run the arcade games inside the kit, under their own licences.
- Everyone behindPyTorch and Python
Free tools that do the heavy lifting under the bonnet.
And thanks to my retro arcade club, for listening to me go on about it.
