Nvidia Releases Four Open Cosmos 3 Models for Driving and Robot Simulation

The company published the checkpoints on Hugging Face on Thursday. All four are post-trained versions of Cosmos 3 Nano, each with 16 billion trainable parameters, and are released under the OpenMDW 1.1 licence, which allows commercial and non-commercial use under its terms. Nvidia also published a technical report and post-training recipes so others can adapt the models to their own sensors and environments.

The release goes after one of the costliest bottlenecks in physical AI: gathering enough varied, labelled sensor data to train and test driving and robotics systems, especially for rare conditions. Nvidia said in its announcement that when generating driving scenes, visual appearance and 3D geometry must agree, and that its models now generate camera and lidar output together. It also said the hard work is still ahead: measuring where the models can be trusted for training and evaluating control software, and where they fall short.

Two Models for Driving

Cosmos3-Nano-Transfer-Auto generates synthetic driving data offline. Nvidia said it can produce synchronised video from up to 11 camera views, including fisheye lenses, along with matching lidar. Developers lay out roads and traffic movement with per-camera scene maps and use text prompts to change weather, lighting and time of day. Nvidia presents it as the base model that carmakers and autonomy developers can retrain on their own sensor layouts and driving regions.

Cosmos3-Nano-Sim-Auto is a faster version distilled from Transfer-Auto. It produces camera video and lidar in short chunks over a few denoising steps, so a driving policy can be tested in long, interactive runs. Nvidia said closed-loop use still needs a separate simulation runtime and renderer to feed driving inputs back into the model.

Two Models for Robots

Cosmos3-Nano-Sim-Transfer takes depth, segmentation, edge or blur maps from a conventional physics simulator, plus a text prompt, and turns them into photorealistic video that a robot's control software can use. Physics and ground-truth state remain with the simulator. Cosmos3-Nano-Sim-Bimanual skips the physics engine and predicts the next video frames directly from a two-armed robot's actions or camera moves. Nvidia said it supports the YAM and AgiBot robot designs and can be driven by a policy or by a human with joysticks.

Nvidia launched Cosmos 3 earlier this year at GTC Taipei, with Nano and larger Super versions, as part of a wider push to sell hardware and software for robotics and autonomous vehicles alongside its data-centre chips. The new checkpoints need Linux, an Nvidia GPU, Docker and the Nvidia Container Toolkit, and the company has published notebooks and agent "skills" for running inference on each one.