Reconstructing the Mind's Eye

The Gallant lab at UC Berkeley may be close to figuring out how to decode and reconstruct brain activity.  This may possibly allow one to look at the mind of a coma patient and even watch a dream. This video shows a reconstruction of video from recoded brain activity. Brain activity was recorded using functional MRI, the left side is the video shown to subjects while in the machine, while the right side is the reconstruction from brain activity data.
The procedure starts with, the recording of brain activity for several hours, while a subject watches a set of movies. A dictionary is then made to translate between the shapes, edges, and motion in the movies and brain activity. A separate dictionary is constructed for each several thousand points at which brain activity was measured. A new set of movies is shown to the patients and the dictionaries made are tested to see the quality of their translations. The lab then built a random library of about 5000 hours of video downloaded from YouTube. These clips were put through the dictionaries to generate predictions about brain activity. The 100 clips that predicted the brain activity the best were averaged together to make up the reconstruction dictionary.


Inception may be right around the corner...


Gallant Lab
Interview