Japan scientists create world’s first mental images with AI tech

Japanese scientists said they have succeeded in creating the world’s first mental images of objects and landscapes from human brain activity by using artificial intelligence technology.

The team of scientists from the National Institutes for Quantum Science and Technology, another national institute and Osaka University was able to produce rough images of a leopard, with a recognizable mouth, ears and spotted pattern, as well as objects like an airplane with red lights on its wings.

The technology, dubbed “brain decoding,” enables the visualization of perceptual contents based on brain activity and could be applied to the medical and welfare fields.

The findings were recently published online in the international scientific journal Neural Networks.

Previous studies had shown that images seen by human participants could be reconstructed from brain activity measured using functional magnetic resonance imaging, or fMRI, although they were limited to specific domains such as alphabetical letters.

Based on previous methods, the team developed a technology that quantifies brain activity and allows generative AI to draw images in combination with predictive techniques to re-create complex objects.

During research, participants were shown 1,200 images of objects and landscapes, with the relationship between their brain signals and the images analyzed and quantified using fMRI. The same images were inputted into the generative AI to learn their correspondence with the brain activity.

The technology could be used in the development of communication devices and to gain an understanding of the brain mechanisms of hallucinations and dreams, according to the article.