Brain-IT: Image Reconstruction from fMRI via Brain-Interaction
  Transformer
Reconstructing images seen by people from their fMRI brain recordings
provides a non-invasive window into the human brain. Despite recent progress
enabled by diffusion models, current methods often lack faithfulness to the
actual seen images. We present "Brain-IT", a brain-inspired approach that
addresses this challenge through a Brain Interaction Transformer (BIT),
allowing effective interactions between clusters of functionally-similar
brain-voxels. These functional-clusters are shared by all subjects, serving as
building blocks for integrating information both within and across brains. All
model components are shared by all clusters & subjects, allowing efficient
training with a limited amount of data. To guide the image reconstruction, BIT
predicts two complementary localized patch-level image features: (i)high-level
semantic features which steer the diffusion model toward the correct semantic
content of the image; and (ii)low-level structural features which help to
initialize the diffusion process with the correct coarse layout of the image.
BIT's design enables direct flow of information from brain-voxel clusters to
localized image features. Through these principles, our method achieves image
reconstructions from fMRI that faithfully reconstruct the seen images, and
surpass current SotA approaches both visually and by standard objective
metrics. Moreover, with only 1-hour of fMRI data from a new subject, we achieve
results comparable to current methods trained on full 40-hour recordings.
6905a52db26ae8b17699fd22
Roman Beliy
claimed_verified
2025-11-05T16:25:16.961Z
6698688dd723c35c0ea9f16d
/avatars/0748274382d0814bb52b9491bb0a329c.svg
Roman Beliy
RomanBeliy
user
RomanBeliy
6905a52db26ae8b17699fd23
Amit Zalcher
6905a52db26ae8b17699fd24
Jonathan Kogman
6905a52db26ae8b17699fd25
Navve Wasserman
claimed_verified
2025-12-11T10:14:16.566Z
6492c419702103104f9450c4
/avatars/d6700e4bb1b9f172096ea31ba15a83b2.svg
navve wasserman
navvew
user
navvew
6905a52db26ae8b17699fd26
Michal Irani
Brain-IT: Image Reconstruction from fMRI via Brain-Interaction
  Transformer
Reconstructing images seen by people from their fMRI brain recordings
provides a non-invasive window into the human brain. Despite recent progress
enabled by diffusion models, current methods often lack faithfulness to the
actual seen images. We present "Brain-IT", a brain-inspired approach that
addresses this challenge through a Brain Interaction Transformer (BIT),
allowing effective interactions between clusters of functionally-similar
brain-voxels. These functional-clusters are shared by all subjects, serving as
building blocks for integrating information both within and across brains. All
model components are shared by all clusters & subjects, allowing efficient
training with a limited amount of data. To guide the image reconstruction, BIT
predicts two complementary localized patch-level image features: (i)high-level
semantic features which steer the diffusion model toward the correct semantic
content of the image; and (ii)low-level structural features which help to
initialize the diffusion process with the correct coarse layout of the image.
BIT's design enables direct flow of information from brain-voxel clusters to
localized image features. Through these principles, our method achieves image
reconstructions from fMRI that faithfully reconstruct the seen images, and
surpass current SotA approaches both visually and by standard objective
metrics. Moreover, with only 1-hour of fMRI data from a new subject, we achieve
results comparable to current methods trained on full 40-hour recordings.
Brain-IT uses a Brain Interaction Transformer to reconstruct images from fMRI data with high fidelity, surpassing current methods and requiring less training data.
diffusion models Brain Interaction Transformer BIT brain-voxels functional-clusters high-level semantic features low-level structural features image reconstruction fMRI standard objective metrics