MMFormalizer: Multimodal Autoformalization in the Wild
Autoformalization, which translates natural language mathematics into formal statements to enable machine reasoning, faces fundamental challenges in the wild due to the multimodal nature of the physical world, where physics requires inferring hidden constraints (e.g., mass or energy) from visual elements. To address this, we propose MMFormalizer, which extends autoformalization beyond text by integrating adaptive grounding with entities from real-world mathematical and physical domains. MMFormalizer recursively constructs formal propositions from perceptually grounded primitives through recursive grounding and axiom composition, with adaptive recursive termination ensuring that every abstraction is supported by visual evidence and anchored in dimensional or axiomatic grounding. We evaluate MMFormalizer on a new benchmark, PhyX-AF, comprising 115 curated samples from MathVerse, PhyX, Synthetic Geometry, and Analytic Geometry, covering diverse multimodal autoformalization tasks. Results show that frontier models such as GPT-5 and Gemini-3-Pro achieve the highest compile and semantic accuracy, with GPT-5 excelling in physical reasoning, while geometry remains the most challenging domain. Overall, MMFormalizer provides a scalable framework for unified multimodal autoformalization, bridging perception and formal reasoning. To the best of our knowledge, this is the first multimodal autoformalization method capable of handling classical mechanics (derived from the Hamiltonian), as well as relativity, quantum mechanics, and thermodynamics. More details are available on our project page: MMFormalizer.github.io
696488cc138cc47cbd765365
Jing Xiong
696488cc138cc47cbd765366
Qi Han
696488cc138cc47cbd765367
Yunta Hsieh
696488cc138cc47cbd765368
Hui Shen
claimed_verified
2026-01-13T08:24:41.913Z
640feb91b0ee289c8583dd59
https://cdn-avatars.huggingface.co/v1/production/uploads/640feb91b0ee289c8583dd59/CA4UZj2mEsJO2UR0NnN5H.jpeg
Hui Shen
Cloudriver
user
Cloudriver
696488cc138cc47cbd765369
Huajian Xin
696488cc138cc47cbd76536a
Chaofan Tao
696488cc138cc47cbd76536b
Chenyang Zhao
696488cc138cc47cbd76536c
Hengyuan Zhang
696488cc138cc47cbd76536d
Taiqiang Wu
claimed_verified
2026-02-25T17:33:20.463Z
6621cea88850e38ffbb1854f
https://cdn-avatars.huggingface.co/v1/production/uploads/6621cea88850e38ffbb1854f/qmKHts7YCUylxqzPHk4uT.jpeg
Taki WU
taki555
user
taki555
696488cc138cc47cbd76536e
Zhen Zhang
696488cc138cc47cbd76536f
Haochen Wang
696488cc138cc47cbd765370
Zhongwei Wan
696488cc138cc47cbd765371
Lingpeng Kong
696488cc138cc47cbd765372
Ngai Wong
MMFormalizer: Multimodal Autoformalization in the Wild
Autoformalization, which translates natural language mathematics into formal statements to enable machine reasoning, faces fundamental challenges in the wild due to the multimodal nature of the physical world, where physics requires inferring hidden constraints (e.g., mass or energy) from visual elements. To address this, we propose MMFormalizer, which extends autoformalization beyond text by integrating adaptive grounding with entities from real-world mathematical and physical domains. MMFormalizer recursively constructs formal propositions from perceptually grounded primitives through recursive grounding and axiom composition, with adaptive recursive termination ensuring that every abstraction is supported by visual evidence and anchored in dimensional or axiomatic grounding. We evaluate MMFormalizer on a new benchmark, PhyX-AF, comprising 115 curated samples from MathVerse, PhyX, Synthetic Geometry, and Analytic Geometry, covering diverse multimodal autoformalization tasks. Results show that frontier models such as GPT-5 and Gemini-3-Pro achieve the highest compile and semantic accuracy, with GPT-5 excelling in physical reasoning, while geometry remains the most challenging domain. Overall, MMFormalizer provides a scalable framework for unified multimodal autoformalization, bridging perception and formal reasoning. To the best of our knowledge, this is the first multimodal autoformalization method capable of handling classical mechanics (derived from the Hamiltonian), as well as relativity, quantum mechanics, and thermodynamics. More details are available on our project page: MMFormalizer.github.io
MMFormalizer enables multimodal autoformalization by integrating visual perception with formal mathematical reasoning, supporting complex physical domains from classical mechanics to quantum mechanics.
autoformalization multimodal perceptually grounded primitives recursive grounding axiom composition adaptive recursive termination dimensional grounding axiomatic grounding PhyX-AF MathVerse PhyX Synthetic Geometry Analytic Geometry GPT-5 Gemini-3-Pro classical mechanics relativity quantum mechanics thermodynamics