Researchers from the University of Tokyo recently unveiled a cutting-edge cooking robot system that can autonomously read recipes, identify ingredients, and cook meals.

The robot, known as the PR2, uses a dual-wheeled design and sophisticated vision technology to navigate the kitchen and locate ingredients. Its onboard camera system enables it to identify objects like eggs or butter, determining their exact position for easy pickup and preparation.

This cooking system relies on Large Language Models (LLMs), which allow the robot to read and understand recipes written in natural language. It then breaks down these instructions into step-by-step actions, such as chopping, frying, or baking.

The robot uses Vision-Language Models (VLMs) to monitor the food's condition, such as recognizing when butter has melted or when eggs are cooked to the desired level. Throughout the cooking process, the PR2 robot continuously adjusts its actions based on what it "sees," for example, turning off the stove when the food is ready.

Looking ahead, researchers are working on expanding the robot’s capabilities to handle more complex dishes. This could involve multi-step recipes that require a higher degree of precision and coordination, pushing the boundaries of what robotic systems can achieve in real-world cooking environments.