Figure released a video showing its new Helix VLA model running on a pair of Figure 02 humanoids cooperating on a household task.
The post Figure humanoid robots use Helix VLA model to demonstrate household chores appeared first on The Robot Report.
Figure AI Inc. yesterday demonstrated the capabilities of its Helix visual-language-action, or VLA, model in a simple household task: putting away the groceries. Initiated with a single prompt from a human, the robots in the company’s video above visually evaluate the scene and then cooperate to identify each object and move the object to a proper location in the kitchen.
There were couple of noteworthy takeaways from the video: First, the robots work independently on the items placed in front of them, until it becomes evident that one needs to hand off a few items to a destination within the other’s reach.
Second, the robots don’t communicate verbally, but there are noticeable pauses in their interactions where they stare at each other in an uncanny “telepathic” interaction. Figure said the supervisory AI architecture breaks the overall goal into smaller subtasks while controlling each robot independently.
This was one of the first times that we’ve seen two humanoids working collaboratively.
Figure demonstrated the capability of the Figure humanoid robots to work collaboratively in handling grocery items. | Credit: The Robot Report
To finish the requested task, the robots close a drawer, close the refrigerator door, and place a bowl at the side of the counter. These are subtasks that are intuitive to humans, but they weren’t specified in the request. Figure said this demonstrated the completeness of the training actions.
In a separate blog article, the company explained the architecture for the supervisory Helix system for the robots in the demo. The heart of the Helix system is the VLA model, which it said is emerging as a key technology for all humanoid robot manufacturers.
Scaling curves for different approaches to acquiring new robot skills. In conventional heuristic manipulation, skills grow with Ph.D.s who manually script them. In conventional robot imitation learning, skills scale with data collected. With Helix, new skills can be specified on the fly with language. | Credit: Figure AI
The Robot Report saw its first demonstration of LLM-based robot guidance on stage at the 2023 RoboBusiness event in Santa Clara, Calif. In a keynote, Pras Velagapudi, chief technology officer at Agility Robotics, wowed the audience with a video showing the Digit humanoid cleaning up a cluttered room by responding to a simple verbal command “Clean up the room.”
This demonstration of Figure Helix is no less impressive a year and a half later because it’s now running onboard the robots, and because the VLA is fully coded and tested. Figure said it generated the VLA by collecting about 500 hours of high-quality, multi-robot, multi-operator dataset of diverse teleoperated behaviors .
To generate natural language-conditioned training pairs, the company used an auto-labeling VLA to generate hindsight instructions. The VLA processed segmented video clips from the onboard robot cameras and prompted with: “What instruction would you have given the robot to get the action seen in this video?”
Figure shows Helix functionality
Figure’s Helix VLA model represents a significant advancement in the field of robotics and AI, particularly in how it differs from previous VLA models. Here’s a breakdown of its key distinctions:
1. Full upper-body control
-
- Dexterity: Figure demonstrated the Helix VLA model providing high-rate, continuous control over an entire humanoid upper body. This includes the torso, head, wrists, and individual fingers, boasting 35 degrees of freedom (DoF). This level of dexterity appears to be more complex and nuanced manipulation of objects compared to previous models.
- Human-like movements: The ability to control the entire upper body enables Helix to perform tasks with more human-like movements and coordination. For instance, it can track its hands with its head for visual alignment and adjust its torso for optimal reach while maintaining precise finger movements for grasping.
2. Multi-robot collaboration
- Cooperative tasks: Figure demonstrated Helix operating simultaneously on two robots, enabling them to collaborate on shared tasks. This opens up possibilities for more complex and coordinated actions, such as two robots working together to put away groceries or assemble a piece of furniture.
- Zero-shot generalization: Based on the setup of the demonstration, the robots collaborate on tasks involving objects they have never (supposedly) encountered before. If the robots have never seen these objects, then it demonstrates the model’s ability to generalize and adapt to new situations.
3. ‘Pick up anything’ capability
- General object recognition: The demonstration shows how Helix enables Figure humanoids to identify and manipulate a wide range of household objects. The use of VLA’s is an improvement over previous models that often required specific training for each object.
- Natural language prompts: The robot demonstrated understanding and response to natural language commands, allowing users to instruct it to “pick up the desert item” or “hand the bag of cookies to the robot on your right” without needing to provide detailed instructions.
4. Unified neural network
- Single model for all behaviors: Unlike previous approaches that often required separate models for different tasks, Helix appears to use a single set of neural network weights to handle all behaviors. This simplifies the model and makes it more efficient.
- No task-specific fine-tuning: Helix can perform a wide range of tasks without needing to be fine-tuned for each specific task. This makes it more adaptable and easier to use in different environments, said Figure.
5. Commercial readiness:
- Onboard processing: Helix runs entirely on embedded GPUs in the Figure 02 humanoid with low power consumption, making it suitable for real-world deployment without relying on external computing resources. This is a crucial step towards making humanoid robots commercially viable for use in homes and other environments.
- Reduced latency: Onboard processing reduces latency, allowing the robot to respond quickly to commands and interact with its environment in real-time.
Production trials under way
Figure announced in late 2024 that its robots are moving development and trials to commercial use and that it has delivered its Figure 02 systems to a paying customer.
Figure AI won a 2024 RBR50 award for its rapid pace of innovation. Since emerging from stealth in January 2023, the Sunnyvale, Calif.-based company has built and iterated on a working humanoid and tested its robot on a production line.
Last month, Figure said it planned to certify its robot’s battery, functional safety control system, and electrical system to industrial safety standards. The company also asserted that it intends to ship 100,000 humanoid robots over the next four years and is reportedly in talks to raise $1.5 billion.
Learn about humanoids at the Robotics Summit
Humanoids will be prominent at the Robotics Summit & Expo, which runs from April 30 to May 1 in Boston and is produced by WTWH Media, parent of The Robot Report. Aaron Saunders, CTO of Boston Dynamics, will give the opening keynote on Day 2 of the event. He will discuss the recently redesigned Atlas robot and share his thoughts about the future humanoids.
The first day of the show will feature a panel about the state of humanoids with Velagapudi; Aaron Prather, director of robotics and autonomous systems at ASTM International; and Al Makke, director of engineering at Schaeffler. The panel will explore the technical and business challenges shaping the development of humanoids. It will also share insights from early deployments, what’s on the horizon, and the ongoing efforts to establish safety standards.
The Robotics Summit & Expo will bring together more than 5,000 developers focused on building robots for a variety of commercial industries. Attendees can gain insights into the latest enabling technologies, engineering best practices, and emerging trends.
The event will feature over 200 exhibitors, 70-plus speakers on stage, 10+ hours of dedicated networking time, a Women in Robotics Breakfast, a career fair, startup showcase, and more. Returning to the show are the RBR50 Pavilion and RBR50 Awards Dinner, which will honor the winners of the annual RBR50 Robotics Innovation Awards.
Register today to save 40% on conference passes!
The post Figure humanoid robots use Helix VLA model to demonstrate household chores appeared first on The Robot Report.
Figure AI Inc. yesterday demonstrated the capabilities of its Helix visual-language-action, or VLA, model in a simple household task: putting away the groceries. Initiated with a single prompt from a human, the robots in the company’s video above visually evaluate the scene and then cooperate to identify each object and move the object to a proper location in the kitchen.
There were couple of noteworthy takeaways from the video: First, the robots work independently on the items placed in front of them, until it becomes evident that one needs to hand off a few items to a destination within the other’s reach.
Second, the robots don’t communicate verbally, but there are noticeable pauses in their interactions where they stare at each other in an uncanny “telepathic” interaction. Figure said the supervisory AI architecture breaks the overall goal into smaller subtasks while controlling each robot independently.
This was one of the first times that we’ve seen two humanoids working collaboratively.
Figure demonstrated the capability of the Figure humanoid robots to work collaboratively in handling grocery items. | Credit: The Robot Report
To finish the requested task, the robots close a drawer, close the refrigerator door, and place a bowl at the side of the counter. These are subtasks that are intuitive to humans, but they weren’t specified in the request. Figure said this demonstrated the completeness of the training actions.
In a separate blog article, the company explained the architecture for the supervisory Helix system for the robots in the demo. The heart of the Helix system is the VLA model, which it said is emerging as a key technology for all humanoid robot manufacturers.
Scaling curves for different approaches to acquiring new robot skills. In conventional heuristic manipulation, skills grow with Ph.D.s who manually script them. In conventional robot imitation learning, skills scale with data collected. With Helix, new skills can be specified on the fly with language. | Credit: Figure AI
The Robot Report saw its first demonstration of LLM-based robot guidance on stage at the 2023 RoboBusiness event in Santa Clara, Calif. In a keynote, Pras Velagapudi, chief technology officer at Agility Robotics, wowed the audience with a video showing the Digit humanoid cleaning up a cluttered room by responding to a simple verbal command “Clean up the room.”
This demonstration of Figure Helix is no less impressive a year and a half later because it’s now running onboard the robots, and because the VLA is fully coded and tested. Figure said it generated the VLA by collecting about 500 hours of high-quality, multi-robot, multi-operator dataset of diverse teleoperated behaviors .
To generate natural language-conditioned training pairs, the company used an auto-labeling VLA to generate hindsight instructions. The VLA processed segmented video clips from the onboard robot cameras and prompted with: “What instruction would you have given the robot to get the action seen in this video?”
Figure shows Helix functionality
Figure’s Helix VLA model represents a significant advancement in the field of robotics and AI, particularly in how it differs from previous VLA models. Here’s a breakdown of its key distinctions:
1. Full upper-body control
-
- Dexterity: Figure demonstrated the Helix VLA model providing high-rate, continuous control over an entire humanoid upper body. This includes the torso, head, wrists, and individual fingers, boasting 35 degrees of freedom (DoF). This level of dexterity appears to be more complex and nuanced manipulation of objects compared to previous models.
- Human-like movements: The ability to control the entire upper body enables Helix to perform tasks with more human-like movements and coordination. For instance, it can track its hands with its head for visual alignment and adjust its torso for optimal reach while maintaining precise finger movements for grasping.
2. Multi-robot collaboration
- Cooperative tasks: Figure demonstrated Helix operating simultaneously on two robots, enabling them to collaborate on shared tasks. This opens up possibilities for more complex and coordinated actions, such as two robots working together to put away groceries or assemble a piece of furniture.
- Zero-shot generalization: Based on the setup of the demonstration, the robots collaborate on tasks involving objects they have never (supposedly) encountered before. If the robots have never seen these objects, then it demonstrates the model’s ability to generalize and adapt to new situations.
3. ‘Pick up anything’ capability
- General object recognition: The demonstration shows how Helix enables Figure humanoids to identify and manipulate a wide range of household objects. The use of VLA’s is an improvement over previous models that often required specific training for each object.
- Natural language prompts: The robot demonstrated understanding and response to natural language commands, allowing users to instruct it to “pick up the desert item” or “hand the bag of cookies to the robot on your right” without needing to provide detailed instructions.
4. Unified neural network
- Single model for all behaviors: Unlike previous approaches that often required separate models for different tasks, Helix appears to use a single set of neural network weights to handle all behaviors. This simplifies the model and makes it more efficient.
- No task-specific fine-tuning: Helix can perform a wide range of tasks without needing to be fine-tuned for each specific task. This makes it more adaptable and easier to use in different environments, said Figure.
5. Commercial readiness:
- Onboard processing: Helix runs entirely on embedded GPUs in the Figure 02 humanoid with low power consumption, making it suitable for real-world deployment without relying on external computing resources. This is a crucial step towards making humanoid robots commercially viable for use in homes and other environments.
- Reduced latency: Onboard processing reduces latency, allowing the robot to respond quickly to commands and interact with its environment in real-time.
Production trials under way
Figure announced in late 2024 that its robots are moving development and trials to commercial use and that it has delivered its Figure 02 systems to a paying customer.
Figure AI won a 2024 RBR50 award for its rapid pace of innovation. Since emerging from stealth in January 2023, the Sunnyvale, Calif.-based company has built and iterated on a working humanoid and tested its robot on a production line.
Last month, Figure said it planned to certify its robot’s battery, functional safety control system, and electrical system to industrial safety standards. The company also asserted that it intends to ship 100,000 humanoid robots over the next four years and is reportedly in talks to raise $1.5 billion.
Learn about humanoids at the Robotics Summit
Humanoids will be prominent at the Robotics Summit & Expo, which runs from April 30 to May 1 in Boston and is produced by WTWH Media, parent of The Robot Report. Aaron Saunders, CTO of Boston Dynamics, will give the opening keynote on Day 2 of the event. He will discuss the recently redesigned Atlas robot and share his thoughts about the future humanoids.
The first day of the show will feature a panel about the state of humanoids with Velagapudi; Aaron Prather, director of robotics and autonomous systems at ASTM International; and Al Makke, director of engineering at Schaeffler. The panel will explore the technical and business challenges shaping the development of humanoids. It will also share insights from early deployments, what’s on the horizon, and the ongoing efforts to establish safety standards.
The Robotics Summit & Expo will bring together more than 5,000 developers focused on building robots for a variety of commercial industries. Attendees can gain insights into the latest enabling technologies, engineering best practices, and emerging trends.
The event will feature over 200 exhibitors, 70-plus speakers on stage, 10+ hours of dedicated networking time, a Women in Robotics Breakfast, a career fair, startup showcase, and more. Returning to the show are the RBR50 Pavilion and RBR50 Awards Dinner, which will honor the winners of the annual RBR50 Robotics Innovation Awards.
Register today to save 40% on conference passes!
The post Figure humanoid robots use Helix VLA model to demonstrate household chores appeared first on The Robot Report.
About The Author
