Mimo Shirasaka

Hi! I am a master's student in the Computer Vision Group at The University of Tokyo, advised by Prof. Yoichi Sato. I am also a research engineer intern at AIRoA.

I received my bachelor's degree in engineering from UTokyo in March 2026, where I was advised by Prof. Jun Ota. During my undergraduate studies, I interned at the Tokyo Robotics and AI Lab (TRAIL) as a research assistant, advised by Dr. Tatsuya Matsushima. I also interned at OMRON SINIC X as a researcher and at Mercari as an ML engineer.

I am working closely with Prof. Yonatan Bisk at Carnegie Mellon University.

Before joining UTokyo, I worked with Prof. Norihiko Sugimoto on the atmosphere of Venus as a high school research scholar in the JST Global Science Campus Program.

profile photo

Research Engineer Intern May'26-

Research Intern (remote) Mar'25-May'26

Undergrad Apr'22-Mar'26

STEP '25

Build@Mercari '25
ML Engineer Intern '25

Research Intern '24-'25


News

[Aug 2026] Paper "Contextual Observer Grounding" got accepted to EMNLP 2026 Findings!
[May 2026] Started a research engineer internship at AIRoA.
[Apr 2026] Joined Computer Vision Group at UTokyo.
[Mar 2026] Obtained BEng with Dean's Award (Faculty of Engineering) from UTokyo!
[Mar 2026] Received Hatakeyama Award from The Japan Society of Mechanical Engineers!
[Feb 2026] Paper "INSIGHT" got accepted to Augmented Humans 2026.
[Jan 2026] Paper "Compliance-enabled Failure Recovery" got accepted to ICRA 2026!
[Apr 2025] Joined Ota Lab at UTokyo.
[Jul 2024] Received Best Poster Award at RSS 2024 Workshop!
[Jul 2024] Received Best Poster Award at RoboCup 2024!
[Jun 2024] Paper "SPARK" got accepted to RSS 2024 Workshop.
[May 2024] Received Outstanding Research Award from The Robotics Society of Japan!
[Jan 2024] Paper "Self-recovery prompting" got accepted to ICRA 2024.
[Sep 2023] Presented "Self-recovery prompting" at CoRL 2023.
[Aug 2023] Started an research internship at TRAIL, UTokyo.


Research

povbench Contextual Observer Grounding: Evaluating Situated Spatial Reasoning in Vision-Language Models
Mimo Shirasaka, Haochen Zhang, and Yonatan Bisk
EMNLP Findings, 2026
Website / Code / Dataset

POVBench evaluates contextual observer grounding: recovering a speaker's viewpoint from contextual cues and localizing an object they described from that viewpoint. State-of-the-art VLMs struggle, and explicitly decomposing the reasoning improves localization.
insight INSIGHT: A Feedback System for Learning Expert Reasoning-Action Sequences in Industrial Plant Inspection
Mimo Shirasaka, Kohei Kaminishi, Shinji Kanda, Yasushi Umeda, Masayoshi Kinoshita, Risa Koda, Shota Shimizu, and Jun Ota
Augmented Humans, 2026 Poster and Demo
Paper

INSIGHT combines VR-based industrial plant inspection with post-task action-reasoning feedback, helping novices learn expert reasoning-action sequences and improve anomaly detection performance.
Robust and Resilient Soft Robotic Object Insertion with Compliance-Enabled Contact Formation and Failure Recovery
Mimo Shirasaka, Cristian C. Beltran-Hernandez, Masashi Hamaya, and Yoshitaka Ushiku
ICRA, 2026
Paper

The Compliance Enabled Failure Recovery system integrates automated failure recovery strategies into compliance-enabled soft robotic object insertion, making the system robust and resilient to uncertainty and environmental variations.
spark SPARK: Graph-Based Online Semantic Integration System for Robot Task Planning
Mimo Shirasaka, Yuya Ikeda, Tatsuya Matsushima, Yutaka Matsuo, and Yusuke Iwasawa
RSS Workshop, 2024 Poster — Best Poster Award
Paper

The SPARK system extracts semantic information from environment cues (verbal, written text, and gesture) and updates the scene graph online for robot task planning.
selfrecovery Self-Recovery Prompting: Promptable General Purpose Service Robot System with Foundation Models and Self-Recovery
Mimo Shirasaka, Tatsuya Matsushima, Soshi Tsunashima, Yuya Ikeda, Aoi Horo, So Ikoma, Chikaha Tsuji, Hikaru Wada, Tsunekazu Omija, Dai Komukai, Yutaka Matsuo, and Yusuke Iwasawa
ICRA, 2024 Oral, Poster, Demo
Paper / Website

The self-recovery pipeline's automatic prompt updates enable failure recovery and task completion in general-purpose home robot tasks.

Projects

Tracking the Seasonal Journey of Migratory Birds - Source Data

Analyzed and visualized large-scale migration data from 2021 to 2024 to reveal how wild birds move south for wintering and return north in spring, as the final project for a data science class.
Tech: Python, Matplotlib, Data Organization and Analysis
What's in Your Fridge? - Healthy Recipe Generator - Code

Designed a bilingual (EN/JP) mobile app that generates simple recipes from on-hand ingredients to encourage elderly people with little cooking experience to prepare healthy meals at home.
Tech: React Native, Python, FastAPI, OpenAI API, Expo Go
Simple Mercari - Code

Developed a full-stack web app (frontend & backend) for Build@Mercari 2025 final project.
Tech: Python, API, SQLite, Docker
arlissdrone Small Satellite System Development

Developed a small satellite drone system from scratch in a team of eleven.
Contribution: Control systems
Tech: Python, Raspberry Pi zero2W etc.
netball NetBall: AI Robot System for Nature Preservation - Website

Developed an original quadruped transformable robot from scratch in a team of four.
Contribution: Team lead, design, and image recognition
Tech: Python, YOLOv5, 3D printing, Raspberry Pi 4B etc.
cadrobot Designing a Humanoid Robot with 3D CAD

Designed and rendered a humanoid robot from scratch as the final project for a CAD class.
Tech: 3DCAD (Fusion)
tr Designing a teleoperation robot system - Demo video

Designed the gripper and master interface for a teleoperated robot.
The project was done in this course.
Tech: 3D CAD (tinkercad), Keyestudio (compatible with Arduino), servo motor, 3D Printer, potentiometers, soldering
stem_app App Development for Enjoyable STEM Learning - Video (in Japanese)

Developed an app designed to make learning STEM enjoyable, participating in an app contest hosted by The University of Electro-Communications during high school.
Tech: Python, tkinter
web_endeavor Website Development

Designed and developed a website for a fictional research and education institute. (Note: All content on the webpage is purely fictional.)
This was an individual project completed during high school.
Tech: HTML, CSS

Source code taken from Jon Barron's site.