Artificial intelligence reveals what neurons in the visual cortex prefer to look at
Why do our eyes tend to be drawn more to some shapes, colors, and silhouettes than others?
For more than half a century, researchers have known that neurons in
the brain's visual system respond unequally to different images — a
feature that is critical for the ability to recognize, understand, and
interpret the multitude of visual clues surrounding us. For example,
specific populations of visual neurons in an area of the brain known as
the inferior temporal cortex fire more when people or other primates —
animals with highly attuned visual systems — look at faces, places,
objects, or text. But exactly what these neurons are responding to has
Now a small study in macaques led by investigators in the Blavatnik
Institute at Harvard Medical School has generated some valuable clues
based on an artificial intelligence system that can reliably determine
what neurons in the brain's visual cortex prefer to see.
The vast majority of experiments to date that attempted to measure
neuronal preferences have used real images. But real images carry an
inherent bias: They are limited to stimuli available in the real world
and to the images that researchers choose to test. The AI-based program
overcomes this hurdle by creating synthetic images tailored to the
preference of each neuron.
Will Xiao, a graduate student at Harvard University, designed a
computer program that uses a form of responsive artificial intelligence
to create self-adjusting images based on neural responses obtained from
six macaque monkeys. To do so, he and his colleagues measured the firing
rates from individual visual neurons in the brains of the animals as
they watched images on a computer screen.
Over the course of a few hours, the animals were shown images in
100-millisecond blips generated by Xiao's program. The images started
out with a random textural pattern in grayscale. Based on how much the
monitored neurons fired, the program gradually introduced shapes and
colors, morphing over time into a final image that fully embodied a
neuron's preference. Because each of these images is synthetic, Xiao
said, it avoids the bias that researchers have traditionally introduced
by only using natural images.
“At the end of each experiment,” he said, “this program generates a super-stimulus for these cells.”
The results of these experiments were consistent over separate runs, explained senior investigator Margaret Livingstone: Specific neurons tended to evolve images through the program that weren't identical but were remarkably similar.
Some of these images were in line with what Livingstone, the Takeda
Professor of Neurobiology at HMS, and her colleagues expected. For
example, a neuron that they suspected might respond to faces evolved
round pink images with two big black dots akin to eyes. Others were more
surprising. A neuron in one of the animals consistently generated
images that looked like the body of a monkey, but with a red splotch
near its neck. The researchers eventually realized that this monkey was
housed near another that always wore a red collar.
“We think this neuron responded preferentially not just to monkey bodies but to a specific monkey,” Livingstone said.
Not every final image looked like something recognizable, Xiao added.
One monkey's neuron evolved a small black square. Another evolved an
amorphous black shape with orange below.
Livingstone notes that research from her lab and others has shown
that the responses of these neurons are not innate — instead, they are
learned through consistent exposure to visual stimuli over time. When
during development this ability to recognize and fire preferentially to
certain images arises is unknown, Livingstone said. She and her
colleagues plan to investigate this question in future studies.
Learning how the visual system responds to images could be key to
better understanding the basic mechanisms that drive cognitive issues
ranging from learning disabilities to autism spectrum disorders, which
are often marked by impairments in a child's ability to recognize faces
and process facial cues.
“This malfunction in the visual processing apparatus of the brain can
interfere with a child's ability to connect, communicate, and interpret
basic cues,” said Livingstone. “By studying those cells that respond
preferentially to faces, for example, we could uncover clues to how
social development takes place and what might sometimes go awry.”
The research was funded by National Institutes of Health and National Science Foundation.