In this post, we ask ChatGPT to write code for a VR website. After my last blog post about the latest A-Frame and IPFS developments and the tools we used during a library workshop in 2019 called the VR Hackfest, I thought it would be worthwhile to show how easy this process really is. Watch and learn!
The following code examples are all direct copies from ChatGPT 4. In this blog post if you read bolded text, that's what I wrote into the chat prompt. Any lines that start with an ">" symbol indicate ChatGPT's reply. Also included are embeded links to my copy/pasted code hosted on Codepen.
How This All Works
Quite literally, you can ask ChatGPT to build and refine VR scenes using plain language. It will output working code that event has comments. It's great. Just yesterday it created an idea I've had for 2 years in less than 5 seconds. That's an amazing and humbling experience. Although the project isn't done, this process has given me an incredible jump start. Just say the magic words!
Out of habit, I like to prime my bots with some context before jumping into the actual question, but you don't have to do that. That first question was "I'd like some help today with creating a 3D scene using A-frame code, are you ready?" Notice ChatGPT came back with its own question too.
> Sure, I'd be happy to help! What kind of 3D scene are you looking to create? And do you have any specific requirements or constraints in mind?
First I need a scene with blue sky and green grass
Great! Here's a simple example code for creating a 3D scene with a blue sky and green grass:
>This code creates a blue sky using an
<a-sky>element with acolorattribute set to#8CE6F0, and a green grass ground using an<a-plane>element with acolorattribute set to#7BC8A4. The camera is positioned at0 1.6 0and set to look straight ahead.>You can adjust the colors and dimensions of the sky and ground by changing the
color,width, andheightattributes of the<a-sky>and<a-plane>elements.
Refining Your Code
After creating the initial scene, I asked ChatGPT "let's add a mushroom but make it look like the one from Super Mario with a red cap and white spots". This works, but ChatGPT wanted to use a premade glTF model, and I wanted to keep things simple. After some back and forth with ChatGPT trying to fix the position of the red spots (which also turned out to be more trouble than it was worth), I came back and said "Can we add a mushroom to this scene using primatives?"
>Sure! Instead of using a GLTF model, we can create a mushroom using primitives in A-Frame. Here's an example code:
>In this code, we've created a mushroom using two primitives - a
<a-cylinder>element to represent the stem of the mushroom, and a<a-sphere>element to represent the cap. Thepositionattribute of the mushroom sets the initial position of the entire entity, and thecolorattribute of the cylinder and sphere sets the colors of the stem and cap, respectively.>Note that the position of the sphere is set to
0 0.3 0to align the center of the sphere with the top of the cylinder, and the radius of the sphere is set to0.6to create a spherical cap with a diameter of 1.2 units (twice the radius of the cylinder). You can adjust the dimensions and colors of the mushroom by changing the attributes of the cylinder and sphere elements.
Wow! This is of course a very rudimentary example, but just imagine how easy this will be in the future when we're combining AI image generation (like MidJourney or Stable Diffusion) to make 3D shapes and textures on the fly! The fact that it not only outputs the answer, but explains how it works is also very helpful.
You can even get really fancy and generate "randomized" objects so your scenes look more natural and less "blocky" like Minecraft. Here's another prompt idea—what if instead of one mushroom it was an entire field of mushrooms? How about a real supershroom situation like the one we're having in California this year? I asked ChatGPT to create 20 mushrooms with random sizes, shapes, positions, and colors in the same scene:
>To create 20 mushrooms with random sizes, shapes, positions, and colors, you can use JavaScript to generate the required entities with random properties. Add a script element inside the head of your HTML file and then write the JavaScript code to create the mushrooms.
>Here's the modified code to create 20 mushrooms with random properties
>This code uses JavaScript to create 20 mushrooms with random properties in the A-Frame scene. The mushrooms will have random positions within a specified range, random stem heights, and random cap sizes. The cap colors are also randomly generated.
>When you load this page, you should see a scene with 20 randomly generated mushrooms.
Are you starting to see how easy this is? Maybe in the future I'll have that random function do some math based on the position of the VR headset or controllers. I've never done that before, but I know it's possible. "Presto change-o! Abracadabra! Watch me pull a rabbit out of the chat".
Finally, knowing how easy it was to ask ChatGPT to write Javascript for me, I had to take it further. The next prompt went like this "based on the relative position of each mushroom in this scene (generation 1), create 20 more mushrooms that blend the the characteristics of these mushrooms, call that generation 2".
>To create a second generation of mushrooms that blend the characteristics of the first generation, you can first store the properties of the first generation mushrooms in an array. Then, for each new mushroom in the second generation, you can randomly select two parent mushrooms from the first generation and blend their properties. Here's how you can modify the existing code to achieve this:
ProTips & Lessons Learned
- If you're going to try this out, keep in mind the current public version of ChatGPT does not have access to information past 2021, so you're stuck with A-Frame features in version 1.2.0. My hope is that eventually we'll have a ChatGPT plugins specifically for A-Frame that will always be up to date, but we're not there yet.
- You may need to view these code examples in a different web browser if you're only seeing a white screen. I think due to a recent Chrome update this just stopped working completely, but it does work in Safari, or at least on my machine it does.
- Don't forget to place a camera in your A-Frame scene! This stumped me for the longest time because the objects were directly below the default camera position @ 0 0 0, rather than slightly in front of the camera when the page loaded. So even when I had tip #2 figured out, I was still not seeing anything.
- When you're working with ChatGPT, eventually your chat session is going to run out of tokens. The number of tokens you're working with depends on what version of ChatGPT you're using, but basically this is a unit of memory or understanding that gets refreshed with each new chat session. This is less like human memory and more like the gas tank in your car. If you knew you only had 1/4 of a tank left, that could mean picking a different route to get home. The same is true for ChatGPT—I have noticed you're more likely to get nonsense replies towards the end of a conversation when it has reached it's limit. The easiest way to get around this limit is to start a new chat sessions with the code you're already using, or start thinking about your VR scenes in individual "chunks" that can be developed individually. Recombining is just a copy/paste away.
- Similar to #4, once you start doing more complex A-Frame prompts with ChatGPT, you might find that it reaches the upper limit of what it can say back to you at one time. This is another bug that will probably change in the future, but also a friendly reminder that you still need to read and understand what you're doing when coding with ChatGPT. If this happens to you, just follow up with a question like "it looks like this got cut off, was there more?" You'll probably get a friendly robo-reply ?
>Yes, I apologize for that. Here's the rest of the code
Add a comment to: Creating VR Scenes with AI