Vibe Coding XR: Google’s Attempt to Make XR Prototyping Not a Nightmare

If you’ve ever tried to prototype anything for extended reality, you know the pain. It’s not just the fragmented perception pipelines or the game engine complexity — it’s the sheer friction of getting something spatial to run on a headset before you’ve lost the creative spark.

Google Research is trying to fix that with Vibe Coding XR, announced today. The idea is straightforward: describe what you want in plain English, and let Gemini turn that into a working, physics-aware WebXR app for Android XR. No XR SDK knowledge required. No wrestling with Unity or Unreal. Just type, wait a minute, and pinch your way into a spatial experience.

I’ve been playing with the live demo, and I have to say — it’s surprisingly not terrible.

How it actually works

The workflow is refreshingly simple. You open the XR Blocks Gem in Chrome — either on an Android XR headset like the Galaxy XR or on your desktop — and type something like “Create a beautiful dandelion.” Gemini then uses its multi-step planning to configure the scene, set up perception and interaction logic, and generate a complete XR Blocks application. In under 60 seconds, you get an interactive dandelion that blows away when you pinch it.

What I like is the iteration loop. You can test on desktop in a simulated reality environment first, then deploy to the headset with one click. That’s a workflow that actually respects your time.

The technical bits worth knowing

Under the hood, Vibe Coding XR combines Gemini’s long-context reasoning with specialized system prompts and curated code templates from XR Blocks — Google’s open-source framework for building WebXR applications. The system handles spatial logic automatically, which is the part that usually kills prototypes. Depth sensing, hand tracking, physics — all wired up without you touching a single line of boilerplate.

XR Blocks itself has been around for a while, but this is the first time Google has tightly integrated it with an LLM for vibe coding. The result is something that feels more like a creative partner than a code generator.

What’s missing (because nothing is perfect)

Let’s be real: this isn’t going to replace professional XR development. The generated experiences are simple — think interactive objects, basic physics, and straightforward spatial UI. If you need complex multiplayer, custom shaders, or anything requiring fine-grained performance tuning, you’re still looking at traditional tooling.

Also, the best perceptual features — depth sensing, hands interaction, robust physics — are only fully available on Android XR headsets. The desktop simulator gives you a taste, but it’s not the same.

Why this matters

Despite the limitations, Vibe Coding XR addresses a real pain point. XR prototyping has been gatekept by technical complexity for too long. Lowering the barrier means more people can test spatial ideas before committing to full builds. That’s good for everyone.

Google is presenting this at ACM CHI 2026, and the live demo is available now. If you have an Android XR headset, give it a shot. If not, the desktop simulator still gives you a feel for what’s possible.

I’m curious to see where this goes. The vibe coding trend in 2D web development has already changed how I prototype. If Google can bring that same fluidity to XR, we might finally see spatial computing become accessible to more than just the hardcore Unity crowd.