PostHole
Compose
Login
Turn creative prompts into interactive XR experiences with Gemini
The Android XR team is using Gemini's Canvas feature to make creating immersive extended reality (XR) experiences more accessible. This allows developers to rapidly prototype interactive 3D environments and models on a Samsung Galaxy XR headset using simple creative prompts.
Gemini
Turn creative prompts into interactive XR experiences with Gemini
FEB. 19, 2026
Benjamin Hersh
Product Manager
Ruofei Du
Interactive Perception & Graphics Lead
[Gemini_Generated_Image_gasezsgasezsgase]
With the release of Canvas in the Gemini web app, our Android XR team began to explore how we might use it to make immersive computing accessible to developers and users at large. We noticed that Gemini is particularly adept at generating interactive 3D web graphics, and asked a simple question: What if you could experience Gemini’s 3D web creations in extended reality?
To test this, our team collaborated to build an immersive biology simulation. Gemini first modeled the complex interactions between blood cells and rendered them visually using WebGL and Three.js. It looked like a living textbook illustration, but the shift occurred when we converted that content to XR using WebXR APIs. A button appeared on the screen: “Enter XR”. Exploring car-sized blood cells at a sub-cellular scale demonstrated Gemini’s capability to build fully realized, interactive XR experiences.
This led to the development of tools that allow you to build complex 3D prototypes and interactive models significantly faster than traditional methods. Here’s how you can give this a try yourself and turn your ideas into functional 3D experiences with ease.
Cellular simulation
Get started building 3D experiences with Gemini
While high-fidelity XR development traditionally requires a broad range of skills, Gemini offers a new way to rapidly prototype and explore interactive 3D concepts in real-time. This approach allows you to move quickly from a creative spark to a functional experience you can test immediately. To begin building your own interactive environments, you will need a Samsung Galaxy XR headset and access to Gemini 3 Pro.
- Set up your XR Gem: You can start with the pre-built XR Blocks Gem, or build your own version by downloading the XR Blocks ultra-prompt to give Gemini advanced perception capabilities for realistic AR physics. Upload it to a new Gem in the Gemini web interface and add a description like "you are a creative and resourceful WebXR developer with superb aesthetic taste and technical execution," and any other information to capture your vision. This persona allows Gemini to interpret your visual scene and use image-generation tools to create polished 3D textures.
- Launch the experience: Open Gemini in Chrome on your Galaxy XR, start a new chat with your XR Gem, and select Canvas. This setup allows you to build interactive components, like a dandelion that reacts to your touch. Try prompting Gemini to embed Gemini Live within the experience. This allows you to talk to Gemini directly, or even continue creating without leaving the immersive session.
Left: Senior XR Engineer - 1 day, Right: Canvas, < 1 min
3. Build and iterate: Think of an idea and ask Gemini to build it. You can say "create a volcanic environment" to change your surroundings instantly, or "add a butterfly there" to place specific objects. You can even give characters backstories and agency within your scene. If the result isn't perfect, try again or layer simple ideas together to create complexity. When you’re finished, your creation becomes a web link that you can share with anyone.
Screen capture of a user creating a volcanic world and adding dinosaurs.
Ideas to get you building today
We have a few recommended starter prompts if you want to try something simple that you can build on:
- “Make a pen that draws rainbows in 3D”
- “Make a bunch of bubbles that pop when I touch them.”
- “Make an origami bird that flies around the room for a few seconds and then lands on my hand. When I move my hand, it flies away and repeats.”
If you are feeling adventurous, we have more advanced challenges for you to try.
- Create a smart planetarium with real star data, ex: NASA exoplanets, Smithsonian star catalog, European Space Agency’s 3D map of the galaxy.
- Create a clone of your favorite 80s arcade game, and re-skin it to be cat-themed.
- Create a tool that turns your 3D doodles into lifelike sculptures.
Create something great and share your Canvas so other people can try it and build on it as well. Browse XR Blocks demos to get inspired with live demos designed for Android XR. Head over to gemini.google.com to get started today using the steps above!
posted in:
Gemini
Web
Tutorials
How-To Guides
Best Practices
Industry Trends
Explore
Learn
Generative XR
Related Posts
[A Guide to Fine-Tuning FunctionGemma]
Tutorials
A Guide to Fine-Tuning FunctionGemma
JAN. 16, 2026
[Get ready for Google I/O 2026]
Gemini
Google AI Studio
Events
Get ready for Google I/O 2026
FEB. 17, 2026
[Beyond the terminal: Gemini CLI comes to Zed]
Gemini
Announcements
Beyond the terminal: Gemini CLI comes to Zed
AUG. 27, 2025
[Introducing A2UI: An open project for agent-driven interfaces]
Mobile
Web
Announcements
Solutions
Introducing A2UI: An open project for agent-driven interfaces
DEC. 15, 2025
[LiteRT: The Universal Framework for On-Device AI]
Mobile
Web
Announcements
LiteRT: The Universal Framework for On-Device AI
JAN. 28, 2026
[Easy FunctionGemma finetuning with Tunix on Google TPUs]
Tutorials
Easy FunctionGemma finetuning with Tunix on Google TPUs
FEB. 3, 2026