Watch out, Boston Dynamics! Mentee Robotics unveils ‘AI-first’ robot

Today, Israel-based Mentee Robotics, the startup co-founded by Mobileye and AI21 Labs founder Amnon Shashua, emerged from stealth by showcasing what it has been working on for the last two years – a humanoid named Menteebot.

While still a prototype, Menteebot has been designed to target household and warehouse applications with AI technologies, including transformer-based large language models (LLMs) popularized by OpenAI’s ChatGPT and others, working across all operational layers.

This company describes it as an AI-first robot that could complete complex tasks end-to-end. Most other players in the space have evolved their products to leverage AI, rather than building the product with an AI-first approach right from the start.

Mentee even released a video that shows the AI robot taking a verbal command and processing the task required with a response accompanied by locomotion, scene understanding, object detection and localization and grasping. See below:

What makes Menteebot humanoid unique?

While humanoid robots have been around for years, the work has largely focused on improving how the machines interact with the physical world, covering aspects like mimicking human-like motion and dexterity.

Most robots back then were either pre-programmed or controlled with a software platform to execute a specific task, like dropping boxes in a controlled environment. 

Then, a few years ago, language and embodied learning models came to the scene. Many robot makers soon adopted the technology (largely through partnerships) to give their humanoids the ability to understand what the user asks in natural language and perform that task while learning over time.

Mentee is also doing something similar, although rather than bringing AI into an existing in-development humanoid, the company is working to build a humanoid with AI at all layers of its operations. 

The three levels of Menteebot

According to the company, the Menteebot prototype turns commands given by humans into complex real-world actions by using AI at three main levels. 

First, it uses transformer-based LLMs to interpret commands and “think through” the required steps for completing a task.

Then, using NeRF-based algorithms, it builds a cognitive 3D map of the environment in real-time, complete with semantic information about objects and items, and localizes itself in the map while planning a dynamic path to avoid obstacles.

Finally, it executes the planned steps on the planned path with a simulator-to-reality (Sim2Real) machine-learning approach that defines the required locomotion in a simulated environment and adopts it in the real world with gait and hand movements.

“We are on the cusp of a convergence of computer vision, natural language understanding, strong and detailed simulators, and methodologies on and for transferring from simulation to the real world,” Shashua said in a statement. “At Mentee Robotics, we see this convergence as the starting point for designing the future general-purpose bi-pedal robot that can move everywhere (as a human) with the brains to perform household tasks and learn through imitation tasks it was not previously trained for.”

While the robot showcased in the demo video does seem to execute the basic task of going into a kitchen and moving fruit items from one place to another, it is important to note that it did not do the task with one single command. The user instructed the robot to go and wait in the kitchen and then gave another command to pick and drop fruits at the next place. It remains to be seen if the robot can do the same task in one go.

That said, given this is a prototype, we can expect the robot to improve over time and gain the ability to handle complex commands without step-by-step instructions. This would be critical for delivering practical applications in household and warehouse environments.

Mentee, on its part, says that the final production-ready version of the humanoid will be powered by camera-only sensing, proprietary electric motors that support unprecedented dexterity and a fully integrated AI. It will be ready for deployment in the first quarter of 2025, although the company has not confirmed which segment it plans to target first.

Other players building AI-driven humanoids

While Shashua and his team’s experience in computer vision and LLMs sure gives Mentee an edge in this space, it is not going to be an easy win for the company. Multiple other players are moving aggressively in the space of AI-powered humanoids, including Elon Musk’s Tesla and OpenAI-backed Figure AI and 1X Technologies. 

Nvidia has also unveiled Project GR00T, a general-purpose foundation model for humanoids, and is providing it to many players in the industry, including Agility Robotics, Apptronik, Boston Dynamics, Fourier Intelligence, Sanctuary AI, Unitree Robotics and XPENG Robotics. 

Speaking of Boston Dynamics, the longstanding robotics company now operating under Hyundai just today unveiled its new all-electric Atlas humanoid robot targeting automotive and industrial applications as well.

It will be interesting to see how effectively and quickly Mentee manages to deploy its AI-first humanoid in this race.