Smart-Fly

A fruit fly’s brain, learning to play Mario.

The eye and visual system below are simulated neuron by neuron from the fruit-fly connectome, the complete wiring diagram of a real fly brain. It sees only the game’s pixels. A small learned network turns what those neurons do into button presses, and practice is the only teacher: moving forward is rewarded, dying is punished.

These are recorded runs. The fly keeps training on a laptop; the recordings and the numbers below are refreshed from time to time.

Learning now
—
Levels mastered
—
Practice so far
—
Neurons simulated
92,743
The game as the fly is playing it right now
Connecting…
—

How it sees

Left to right is the path every frame takes, about fifteen times a second.

  1. 1 What reaches the eye

    The game reduced to the 24 × 15 patches the eye covers. Bright marks are edges.

  2. 2 Photoreceptors

    The 3,354 light-sensing neurons themselves, responding. This is the brain’s own activity, not the picture.

  3. 3 Deeper optic lobe

    The rest of the visual system. The picture blurs here; the wiring is real, the connection strengths are not known.

  4. 4 What decided it

    Which parts of its view moved this choice most. Ideally the next obstacle.

480 optic-lobe neurons over the last few seconds, top of the view to bottom. Each mark is a neuron strongly active at that moment.

Reward and punishment

The only feedback it learns from. Each bar is one decision.

The whole game

It moves on once it reaches the flag in at least 8 of 10 test runs, each starting slightly differently, choosing only its top button. Levels it can’t pass yet are set aside and revisited.

Share of recent practice runs that reached the flag
Average distance of recent practice runs

Tests