Reported by the Heart of the Machine
Editors: Du Wei, Chen Chen
"The next wave of AI is robotics, and one of the most exciting developments is humanoid robots," Huang said. Now, Project GR00T has taken another important step.
Yesterday, NVIDIA founder Jensen Huang spoke about its universal base model for humanoid robots, Project GR00T, in a SIGGRAPH 2024 Keynote presentation. The model has received a series of updates in terms of functionality.
Yuke Zhu, an assistant professor at the University of Texas at Austin and a senior research scientist at NVIDIA, tweeted how NVIDIA integrated RoboCasa and MimicGen systems, a large-scale simulation training framework for general-purpose housework robots, into the NVIDIA Omniverse platform and the Isaac robot development platform.
Source: https://x.com/yukez/status/1818092679936299373
The video covers three of NVIDIA's own computing platforms, including AI, Omniverse, and Jetson Thor, to simplify and accelerate developer workflows. Through the joint empowerment of these computing platforms, we are expected to enter the era of humanoid robots powered by physical AI.
One of the biggest highlights is that developers will be able to use Apple's Vision Pro to remotely control humanoid robots to perform tasks.
Meanwhile, Jim Fan, another senior research scientist at Nvidia, said the update to Project GR00T is exciting. NVIDIA solves the toughest challenges in robotics with a systematic approach to extending robotics data.
The idea is simple: humans collect demo data on real robots, and NVIDIA expands that data by a thousand-fold or more in simulation. With GPU-accelerated simulation, people can now trade computing power for time-consuming, labor-intensive, and cost-intensive human-collected data.
He talked about how not so long ago he thought that remote control was fundamentally non-scalable, because in the atomic world, we were always limited by 24 hours/robots/days. Nvidia's new synthetic data pipeline on the GR00T breaks this limitation in the world of bits.
Source: https://x.com/DrJimFan/status/1818302152982343983
Regarding NVIDIA's latest progress in the field of humanoid robots, some netizens said that Apple's Vision Pro has found the coolest use case.
Loading...
Nvidia is starting to lead the next wave: physical AI
Nvidia also detailed the technical process of accelerating humanoid robots in a blog post, which is as follows:
To accelerate the development of humanoid robots worldwide, NVIDIA announced a suite of services, models, and computing platforms for the world's leading robotics manufacturers, AI model developers, and software manufacturers to develop, train, and build the next generation of humanoid robots.
The suite includes new NVIDIA NIM microservices and frameworks for robotics simulation and learning, NVIDIA OSMO orchestration services for running multi-stage robotics workloads, and AI and simulation-enabled remote operations workflows that allow developers to train robots with small amounts of human demo data.
"The next wave of AI is robotics, and one of the most exciting developments is humanoid robots," Huang said. We're advancing the entire NVIDIA robotics stack with open access for humanoid robot developers and companies around the world, giving them access to the platforms, acceleration libraries, and AI models that best meet their needs."
借助 NVIDIA NIM 和 OSMO 加速开发
NIM microservices provide pre-built containers powered by NVIDIA inference software, enabling developers to reduce deployment time from weeks to minutes.
Two new AI microservices will allow roboticists to enhance generative physics-based AI simulation workflows in NVIDIA Isaac Sim.
The MimicGen NIM microservice generates synthetic motion data based on remote data recorded from spatial computing devices, such as Apple Vision Pro. Robocasa NIM microservices generate robot tasks and simulation environments in OpenUSD.
NVIDIA OSMO, a cloud-native managed service, is now available, allowing users to orchestrate and scale complex robotics development workflows across distributed computing resources, whether on-premises or in the cloud. The advent of OSMO has dramatically simplified robot training and simulation workflows, reducing deployment and development cycles from months to less than a week.
Advanced data capture workflows for humanoid robot developers
Training the underlying model behind a humanoid robot requires a lot of data. One way to get human presentation data is to use remote operations, but this is becoming more expensive and lengthy.
Researchers and AI developers are able to generate massive amounts of synthetic motion and perception data from a very small number of remotely captured human presentations with NVIDIA AI and Omniverse Remote Operations Reference Workflows, showcased at SIGGRAPH.
First, developers use Apple Vision Pro to capture a small number of remote presentations. They then simulated the recordings in NVIDIA Isaac Sim and used the MimicGen NIM microservice to generate a synthetic dataset from the recordings.
Developers use real-world and synthetic data to train the Project GR00T humanoid robot base model, saving significant time and reducing costs. They then used the Robocasa NIM microservice in Isaac Lab, a robot learning framework, to generate experiences to retrain the robot model. Throughout the workflow, NVIDIA OSMO seamlessly allocates compute tasks to different resources, reducing weeks of administrative effort for developers.
Expand access to NVIDIA humanoid robot developer technology
NVIDIA offers three computing platforms to simplify the development of humanoid robots: the NVIDIA AI supercomputer for training models; NVIDIA Isaac Sim, built on Omniverse, allows robots to learn and refine their skills in a simulated world; and the NVIDIA Jetson Thor humanoid robot computer used to run the model. Developers can access and use all or part of the platform according to their specific needs.
With the new NVIDIA Humanoid Robot Developer Program, developers have early access to new products and the latest versions of NVIDIA Isaac Sim, NVIDIA Isaac Lab, Jetson Thor, and Project GR00T universal humanoid robot base models.
1x、波士顿动力、字节跳动、Field AI、Figure、Fourier、Galbot、LimX Dynamics、Mentee、Neura Robotics、RobotEra 和 Skild AI 是首批加入早期访问计划的公司。
Developers can now join the NVIDIA Humanoid Robotics Developer Program to gain access to NVIDIA OSMO and the Isaac Lab, and will soon gain access to NVIDIA NIM microservices.
Blog Links:
https://nvidianews.nvidia.com/news/nvidia-accelerates-worldwide-humanoid-robotics-development