A team of scientists at the Weizmann Institute of Science in Rehovot, Israel, has developed an artificial intelligence system capable of reconstructing images from brain scans with a level of detail that surpasses previous tools. The breakthrough, presented last month at the Cognitive Computational Neuroscience conference in New York, reopens the debate on how far technology that interprets human brain activity can go.
The research is led by computer scientist Michal Irani and proposes applications ranging from communication with paralyzed patients to understanding phenomena like post-traumatic stress flashbacks. However, it also raises questions about mental privacy at a time when AI systems are advancing faster than the regulations that accompany them.
Two branches for a more faithful image
Previous reconstruction systems using functional magnetic resonance imaging (fMRI) generated blurry or imprecise images. According to Irani, although the quality of scanners and analysis tools has improved over the years, existing models still failed to reach the level necessary to be useful.
The researcher uses a banana as an example: previous models could generate the image of a banana, but not necessarily with the same shape, color, or position as the original seen by the scanned person.
To solve this, the team trained what they call a brain decoder using public data from eight people who were shown about 9,000 images inside a high-resolution scanner. This decoder works with two simultaneous branches: one predicts the visual structure of the image, such as the arrangement of colors, and the other predicts the content, for example, a bunch of bananas on a plate. Both predictions then feed into a diffusion model, the type of AI that generates images by gradually cleaning up visual noise until a sharp result is obtained.
The trick to training with more data than available
The main obstacle in this type of research is cost. Scanning a person with fMRI is expensive, and machine time limits how many images can be shown during an experiment.
Irani's team solved this limitation by training a second model, an encoder that performs the reverse process of the decoder: it predicts what the brain activity of someone observing a specific image would look like. The system works as follows: a new photograph is taken, for example, a leopard, and the encoder predicts what brain activity that image would generate. Then, the decoder attempts to reconstruct the original photograph from that simulated activity.
At first, as Irani acknowledges, the result barely resembles a leopard. But repeating this feedback cycle significantly improves both tools. This allows the system to be trained with many more images than were actually shown in a real scanner. In fact, Irani estimates that about 70% of the training data comes from images that were never originally paired with an MRI scan.
From 40 hours of scanning to just one
By combining data from different studies, the researchers detected brain regions that appear to perform similar functions in all people: one area responds to images of food and another to images related to sports, for example. This allowed for the construction of what the team calls a universal brain encoder, capable of adapting to a new person's scan with minimal calibration.
This is the most relevant practical difference of the work. According to Irani, other tools usually need about 40 hours of fMRI data from each new person before they can predict what they are seeing. Her team's system only requires one hour of scanning.
Tommy Sprague, a neuroscientist at the University of California, Santa Barbara, who was not involved in the study, highlights the impact of this reduction for laboratories: no research team can afford 40 hours of imaging for every new subject. According to his calculations, one hour of scanning costs between $600 and $1,000.
The limits of the system
The tool still makes obvious errors. Irani has shown examples where the system reconstructed a cake as a stack of three sandwiches, or turned a dog in a bathtub into a goat of the same color in the same bathtub.
Even with those flaws, in comparative tests, the system clearly outperformed the tools described in the scientific literature to date. Irani prefers not to use the term mind reading, which she considers more of a catchy claim than an accurate description of what the system actually does.
The goal: video, audio, and even dreams
The laboratory's next step is to move from reconstructing still images to reconstructing video and audio. Irani aspires for the technology to eventually reproduce what a person thinks, imagines, or even dreams, although she admits that this goal has not yet been reached.
Among the most promising applications is the possibility that people with locked-in syndrome, completely paralyzed but conscious, could communicate solely through their brain activity. The technology could also help better understand phenomena like flashbacks associated with post-traumatic stress disorder.
Judy Illes, a neuroethicist and professor of neurology at the University of British Columbia in Canada, who did not participate in the research, has described the work as extraordinary and highlighted the therapeutic potential of applying this approach to people with neurological conditions.
The pending debate on mental privacy
Advances like this revive an uncomfortable question: what would happen if someone could reconstruct others' thoughts or memories without consent? Sprague acknowledges that a decade ago he would have dismissed that possibility out of hand, given how difficult it is to get a person to actively cooperate inside a scanner during an experiment.
However, Irani's own team and other research groups are already working with electroencephalography (EEG), a technology that measures brain electrical activity using electrode caps or even headsets, which is much more accessible than magnetic resonance imaging. As AI models improve, analyzing that type of data will become increasingly simple.
Sprague warns that if a way to covertly extract information about what a person is thinking ever exists, scenarios that for 150 years belonged to science fiction could materialize at any moment. That is why he calls for ethical considerations regarding this type of technology to be taken more seriously.
What this breakthrough means in the short term
The most immediate impact will be in neuroscience laboratories themselves. Reducing the scan time required per volunteer from 40 hours to one significantly lowers the cost of these types of studies, which could open the door to research with tighter budgets.
For patients with severe paralysis, the long-term promise is a form of assistive communication, although Irani herself acknowledges that this goal is not yet solved. And for the general public, the breakthrough forces us to closely follow the debate on the regulation of brain data, a discussion that today depends on voluntary cooperation inside a scanner, but which could change if more accessible EEG technology continues to be perfected.
Frequently Asked Questions
What exactly does this artificial intelligence do?
It reconstructs images that a person has seen from their brain activity recorded via functional magnetic resonance imaging, and it can also predict what brain activity a given image would generate.
Who is leading this research?
The project is led by Michal Irani, a computer scientist at the Weizmann Institute of Science in Rehovot, Israel.
How much scan time does a new person need to use the system?
According to Irani, her decoder only requires one hour of fMRI data from a new person, compared to the 40 hours that other similar tools usually need.
Can the technology read dreams or thoughts?
Not yet. The team is working to extend the system to video, audio, and eventually to thoughts or dreams, but that capability does not yet exist.
What errors does the system currently make?
Among the documented flaws, the system has confused a cake with a stack of sandwiches and a dog in a bathtub with a goat of the same color in the same environment.
What privacy risks does this technology pose?
Experts like Tommy Sprague warn that if it were possible in the future to extract mental information covertly, serious ethical concerns would arise, especially as it is combined with more accessible technologies like EEG.
Can this technology already help paralyzed patients?
It is one of the applications the laboratory is pursuing, intended for people with locked-in syndrome, but Irani acknowledges that such a solution is not yet developed.
