I am a research scientist at Google DeepMind, Japan, working mainly on multimodal AI agents and alignment for diffusion models. I received Ph.D. at The University of Tokyo, advised by Yutaka Matsuo. I also received BEng and MEng at The University of Tokyo, advised by Yutaka Matsuo and closely collaborated with Shixiang Shane Gu. I was a Student Researcher at Google DeepMind, hosted by Heiga Zen and Izzeddin Gur.

My recent research interest has focused on AI agents driven by multimodal LLMs for real-world applications (e.g., robotics, web navigation), AI alignment through deep reinforcement learning, and mechanistic interpretablebility of LLMs.

Recent Preprints

1. Hiroki Furuta, Heiga Zen, Dale Schuurmans, Aleksandra Faust, Yutaka Matsuo, Percy Liang, Sherry Yang.
   Improving Dynamic Object Interactions in Text-to-Video Generation with AI Feedback
   arXiv preprint arXiv:2412.02617, 2024.
   [arxiv] [website]

Conference Publications

1. Gouki Minegishi, Hiroki Furuta, Takeshi Kojima, Yusuke Iwasawa, Yutaka Matsuo.
   Topology of Reasoning: Understanding Large Reasoning Models through Reasoning Graph Properties
   Neural Information Processing Systems (NeurIPS 2025).
   [arxiv]

2. Yuta Oshima, Masahiro Suzuki, Yutaka Matsuo, Hiroki Furuta.
   Inference-Time Text-to-Video Alignment with Diffusion Latent Beam Search
   Neural Information Processing Systems (NeurIPS 2025).
   [arxiv]

3. Gouki Minegishi, Hiroki Furuta, Shohei Taniguchi, Yusuke Iwasawa, Yutaka Matsuo.
   Beyond Induction Heads: In-Context Meta Learning Induces Multi-Phase Circuit Emergence
   International Conference on Machine Learning (ICML 2025).
   [arxiv]

4. Lutfi Eren Erdogan, Nicholas Lee, Sehoon Kim, Suhong Moon, Hiroki Furuta, Gopala Anumanchipalli, Kurt Keutzer, Amir Gholami.
   Plan-and-Act: Improving Planning of Agents for Long-Horizon Tasks
   International Conference on Machine Learning (ICML 2025).
   [arxiv]

5. Gouki Minegishi, Hiroki Furuta, Yusuke Iwasawa, Yutaka Matsuo.
   Rethinking Evaluation of Sparse Autoencoders through the Representation of Polysemous Words
   International Conference on Learning Representations (ICLR 2025).
   [arxiv] [code]

6. Hiroki Furuta, Kuang-Huei Lee, Shixiang Shane Gu, Yutaka Matsuo, Aleksandra Faust, Heiga Zen, Izzeddin Gur.
   Geometric-Averaged Preference Optimization for Soft Preference Labels
   Neural Information Processing Systems (NeurIPS 2024).
   [arxiv]

7. Kuang-Huei Lee, Xinyun Chen, Hiroki Furuta, John Canny, Ian Fischer.
   A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts
   International Conference on Machine Learning (ICML 2024).
   [arxiv] [website]

8. Open X-Embodiment Collaboration, et al. (including Hiroki Furuta)
   Open X-Embodiment: Robotic Learning Datasets and RT-X Models
   IEEE International Conference on Robotics and Automation (ICRA 2024) (Best Conference Paper Award).
   [arxiv] [website]

9. Izzeddin Gur*, Hiroki Furuta*, Austin Huang, Mustafa Safdari, Yutaka Matsuo, Douglas Eck, Aleksandra Faust. (*Equal Contribution)
   A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis
   International Conference on Learning Representations (ICLR 2024) (Oral, 1.2% of 7262 submissions).
   [arxiv]

10. Hiroki Furuta, Kuang-Huei Lee, Ofir Nachum, Yutaka Matsuo, Aleksandra Faust, Shixiang Shane Gu, Izzeddin Gur.
    Multimodal Web Navigation with Instruction-Finetuned Foundation Models
    International Conference on Learning Representations (ICLR 2024).
    [arxiv] [website]

11. Hiroki Furuta, Yusuke Iwasawa, Yutaka Matsuo, Shixiang Shane Gu.
    A System for Morphology-Task Generalization via Unified Representation and Behavior Distillation
    International Conference on Learning Representations (ICLR 2023) (Notable-top-25%, 8% of 4966 submissions).
    [arxiv] [code] [website]

12. Hiroki Furuta, Yutaka Matsuo, Shixiang Shane Gu.
    Generalized Decision Transformer for Offline Hindsight Information Matching
    International Conference on Learning Representations (ICLR 2022) (Spotlight, 5% of 3391 submissions).
    [arxiv] [code] [website]

13. Hiroki Furuta, Tadashi Kozuno, Tatsuya Matsushima, Yutaka Matsuo, Shixiang Shane Gu.
    Co-Adaptation of Algorithmic and Implementational Innovations in Inference-based Deep Reinforcement Learning
    Neural Information Processing Systems (NeurIPS 2021).
    [arxiv] [code]

14. Hiroki Furuta, Tatsuya Matsushima, Tadashi Kozuno, Yutaka Matsuo, Sergey Levine, Ofir Nachum, Shixiang Shane Gu.
    Policy Information Capacity: Information-Theoretic Measure for Task Complexity in Deep Reinforcement Learning
    International Conference on Machine Learning (ICML 2021).
    [arxiv] [code]

15. Tatsuya Matsushima*, Hiroki Furuta*, Yutaka Matsuo, Ofir Nachum, Shixiang Gu. (*Equal Contribution)
    Deployment-Efficient Reinforcement Learning via Model-Based Offline Optimization
    International Conference on Learning Representations (ICLR 2021).
    [openreview] [code]

Journal Publications

1. Hiroki Furuta, Yutaka Matsuo, Aleksandra Faust, Izzeddin Gur.
   Exposing Limitations of Language Model Agents in Sequential-Task Compositions on the Web
   Transactions on Machine Learning Research (TMLR), 2024.
   [arxiv] [code]

2. Hiroki Furuta, Gouki Minegishi, Yusuke Iwasawa, Yutaka Matsuo.
   Towards Empirical Interpretation of Internal Circuits and Properties in Grokked Transformers on Modular Polynomials
   Transactions on Machine Learning Research (TMLR), 2024.
   [arxiv] [code]

3. So Kuroki, Tatsuya Matsushima, Junpei Arima, Hiroki Furuta, Yutaka Matsuo, Shixiang Shane Gu, Yujin Tang.
   Collective Intelligence for 2D Push Manipulations With Mobile Robots
   IEEE Robotics and Automation Letters (RA-L), 2023.
   [paper]

Please check Publications for further details.

Talks

1. Hiroki Furuta. “Opportunities and Challenges of Language Model Agents in Web Automation”. Berkeley Artificial Intelligence Research Lab, 2023.

2. Hiroki Furuta. “Co-Adaptation of Algorithmic and Implementational Innovations in Inference-based Deep Reinforcement Learning”. NeurIPS Meetup Japan 2021 $^{*}$, 2021.

Academic Activitites

1. Reviewer for Neural Information Processing Systems (NeurIPS), 2021, 2022 (Top Reviewer), 2023 (Top Reviewer), 2024, 2025.

2. Reviewer for International Conference on Learning Representations (ICLR), 2022 (Highlighted Reviewer), 2023, 2024, 2025, 2026.

3. Reviewer for International Conference on Machine Learning (ICML), 2021, 2022, 2023, 2024, 2025.

4. Reviewer for IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.

5. Reviewer for International Conference on Computer Vision (ICCV), 2025.

6. Reviewer for Association for Computational Linguistics (ACL) Rolling Review, 2025.

7. Reviewer for Transactions on Machine Learning Research (TMLR).

8. Reviewer for Advanced Robotics (AR).

9. Co-organizer for Workshop on Robotics World Modeling at CoRL 2025.

10. Co-organizer for Workshop on Building Physically Plausible World Models at ICML 2025.

11. Co-organizer for Ecological Theory of RL Workshop at NeurIPS 2021.

12. Program Committee for Foundation Models for Decision Making Workshop at NeurIPS 2022, 2023.

Honors & Awards

• Dean’s Award (Ph.D.) (from Graduate School of Engineering, The University of Tokyo, 2025)
• Forbes JAPAN 30 UNDER 30 2023 (August, 2023)
• The Japan Society for the Promotion of Science Research Fellow (DC1) (April, 2022 - March, 2025)
• Dean’s Award (Master) (from Graduate School of Engineering, The University of Tokyo, 2022)
• Toyota/Dwango Scholarship for Advanced Artificial Intelligence Researcher (April, 2021 - March, 2022)

Education & Experience

• Research Scientist at Google DeepMind (Jan, 2025 - Present)
• Ph.D. from The University of Tokyo (March, 2025)
• Student Researcher at Google DeepMind (May, 2023 - Jan, 2025)
• Student Researcher at Google Research, Brain Team (July, 2022 - May, 2023)
• MEng from The University of Tokyo (March, 2022)
• BEng from The University of Tokyo (March, 2020)