Featured on the NVIDIA Omniverse Blog: A Voice-Controlled VR Simulation Built With AI Agents
NVIDIA’s “Into the Omniverse” series shows how developers combine frontier AI agents with NVIDIA Omniverse libraries to build simulation applications. Our CEO Michael Wagner is one of the developers featured in the latest edition, “How Developers Turn Ideas Into Simulations With Frontier AI Agents”.
What he built
Michael directed GPT-6 Astra to model a study environment in Blender from two reference images, balancing visual quality with performance in VR. He then used Claude Code to build an application that lets users find SimReady Central assets by voice and interact with them in an immersive VR scene. The agent worked for about eight hours, and the application came together over a weekend on an NVIDIA RTX 6000 Ada workstation with a Meta Quest 3 headset.
The application runs on NVIDIA Omniverse libraries: ovstage shares the OpenUSD scene, ovrtx renders it, ovxr enables VR and ovphysx runs the physics. A second version uses ovnewton to compare physics libraries.
The lesson
Automated tests missed problems you can only feel inside the headset, such as objects shaking when grabbed. Michael reported the symptoms, guided the agent through the changes and verified the interaction himself. His takeaway: give agents renders and logs to inspect, then check the experience directly.
It is the same pattern we build for at SyncTwin: agents do the work, people direct and check it, all on an open OpenUSD foundation.
Read the full article on the NVIDIA Blog