Virtual Reality Worlds in Tilt Brush VR

Virtual Reality is a growing medium for teaching and learning programmes and at Raranga Matihiko, we've been using VR to create collaborative worlds where students could add their own content to. 

For our HTC VIVE sets, we installed the Tilt Brush by Google app. Tilt Brush has the functionality of users adding their own images, 3D objects and videos.

(Note: In 2021 Google announced that it would end development on Tilt Brush so these instructions work on the final version of Tiltbrush and the hope is that any future versions of the programme will still enable you to use these functions).

Examples of work are by actual students from 5 to 15 years old who have attended the Raranga Matihiko | Weaving Digital Futures project in Aotearoa New Zealand.

Supplies

Keep in mind there are many alternatives for the following:

• Virtual Reality Set - HTC VIVE with a VR-ready computer!
• Painting Creation app - Tilt Brush
• 2D drawing program - Paint 3D
• Computer-aided Design tools - SculptGL and Tinkercad

Step 1: Decide What Content You Will Be Creating

For many of our example worlds, students designed and made digital recreations of traditional Māori ways of telling local stories and passing down important information. This included whakairo (carvings) and kōwhaiwhai (rafter paintings) that adorn our wharenui (traditional meeting house), similar to the one pictured.  We have also built virtual worlds around many other topics including ones on Whakapapa, Patterns, Identity, Sustainability, School Pride and Te Whare Tapa Whā.

The world is your (virtual) oyster, get creative!

Step 2: Choose What Types of Content You Will Make...

...and what you will use Tilt Brush brushes for!

When planning to create content for your virtual world, think about where and how the object/image will be viewed and what you want to do with it once it’s inside the Tilt Brush environment.

Also, think about which digital tool would be best for the most effective outcome; we could easily draw a 3D-looking building in a digital drawing app, but if the building is to go into a virtual world, it would be much better to construct a Tinkercad building out of 3D blocks so we can actually stand inside it!

Step 3: Create 2D Images With Photos

We use photos of students in our virtual worlds so it looks like they are actually there!

• Discuss with students how they can pose for a photo to make the scene appear more realistic (looking like they are interacting with an object in the world - holding the object, sitting on it or inside it)
• Take clear photos of each of your subjects in front of a blank wall
• Run the photo through a photo editing tool to remove the background. We use remove.bg for its ease of use and availability to teachers through any browser

Hot tip: you could also take photos of real-life objects, paintings or students' art pieces to use in virtual worlds!

Step 4: Create 2D Images With Digital Art

Use your favourite 2D drawing programme or app to get creative! Our students use Microsoft Paint3D but other free software like Google Drawings and Pixlr have also been used in some of our worlds.

Always save your image without a background or you could use an image editing tool like we did in Step 3.

Step 5: Create 3D Objects With SculptGL

SculptGL is a free digital sculpting and modelling app, so it works well to emulate whakairo (wood carving). Our students use Microsoft Surface Pro laptops with a stylus pen, making it easier to ‘carve’ curved designs into their objects.

Save your completed object as a .obj file.

Currently, any painting on the object won't show once imported into Tilt Brush but you could try editing the object in another app to do this, we use Tinkercad.

Step 6: Create 3D Objects With Tinkercad

Tinkercad is a free web app and our ‘go to’ for creating all sorts of 3D objects, from wharenui, sustainable housing, helper robots, native plants and animals, solar-powered cars and patterns.

The user has a 360 degree view when inside the virtual world, so before downloading your object from Tinkercad, use the viewing cube (or right-click and hold on a trackpad/mouse) to check out your object from all sides and make sure everything lines up.

Use the align tool to get corners and sides perfectly situated.

Make creating patterns way easier by using the Duplicate and Repeat tool!

Once finished, you’ll need to export your Tinkercad object to your computer: Click the Export button at the top right, then choose .obj. This will save your object in a zipped folder containing two files. You’ll need to extract these files to use in Tiltbrush, but keep both files in the same folder. One file (.obj) is the object itself, the other (.mtl) tells the computer which colours and textures to use on your object.

The Tinkercad community is widespread and extremely helpful and if you're just starting out, go to the Learn How to Tinker page where you can follow a very easy step-by-step process to begin your Tinkercad journey!

Step 7: Create Movies

There are many inventive ways to use videos inside a Tilt Brush virtual world. The video will have most of the properties that a 2D image does in the Tilt Brush world (flat and transparent when viewed from the back).

In the example above: students painted 3D fish in Paint3D then recorded them moving around in Mixed Reality in front of a green screen. We then added a background with the DoInk app before saving as a .mp4 file and importing into our virtual underwater world.

Step 8: Import Your Content Into Tilt Brush

Now that you have created all your 2D and 3D objects for your virtual world, it’s time to move all the files you have saved, into the correct folders on your computer.

All elements for your Tilt Brush world will go into separate folders, depending on what type of file they are. Go to:

This PC > Documents > Tilt Brush > Media Library

and drop your file into its new folder, either > Images, > Models or > Videos

For any files that have been saved as a zipped file (like Tinkercad does), make sure you extract these and keep the .obj and .mtl files inside their original folder before transferring into the Tilt Brush Media Library.

Step 9: Get Familiar With the VR Equipment

Once your VR is set up and you have the Tilt Brush app running, you can put on your headset and start building the world!

The following steps instruct you to use the Tilt Brush menu panels which you'll only be able to see on your VIVE controllers while in the VR view.

We’ll refer to the two controllers as the palette-controller (contains all the tools, colours and brushes on separate panels) and the painting-controller (for holding and using the selected tool/brush).

To use the controllers, make a selection by hovering your painting-controller over an icon on your palette-controller. A dashed line will connect the two controllers and you will see the icon light up and enlarge slightly. To ‘click’, pull the trigger on your painting-controller while it is hovering over an icon. This will act as if you have ‘clicked’ that icon.

Hot Tip: Take a screenshot of these instructions and save in the images folder. Once in the VR view, you can take it out of the Local Images folder and follow along without having to remove your headset!

Step 10: Access the Local Media Library

a) On your palette-controller, enable Advanced mode by clicking on the beginner/advanced-mode toggle

b) Click on the Sketchbook icon

c) Click on the More Options ellipsis, then the Labs flask

d) This will pop out the Labs panel where you can click the Local Media Library icon

You should now have your Library panel open and will be able to see that all your items have been sorted into their folders: Local Models, Local Images and Local Videos

e) By clicking on one of these folders and then an object in that folder, you will insert that object into your world

Step 11: Move and Resize Your Objects/images

While touching an inserted object with your whole controller (not just pointing the dashed line), the border of that object (or image) will light up and you will feel some haptic feedback through the controller.

While selecting the object in this way, press and hold the grip button on the side of the controller.

As long as you are still gripping, you can move your image/object around.

Letting go of the grip button releases the image/object.

By selecting the object with one controller and then holding down the grip buttons on both controllers, you can move your hands closer together or further apart to resize the image/object.

Hot tip: Gripping with both controllers, without holding an object, will help you resize the entire world.

Step 12: Pin Your Objects/images

Don’t miss this step, you’ll thank me later…

a) Grip the object with one controller (as above)

b) Pull the trigger on the same controller while holding that object in place

A large pin will stab your object, preventing it from being moved!

c) Pull the trigger again to remove the pin and move the object freely

Alternatively:

a) Place the object where you want it in the world

b) Swipe the track pad on the palette-controller to the tools panel

c) Use the painting-controller to point and click on the 'pin icon'

d) Move the painting-controller to the object and pull the trigger again to insert the pin

e) To remove the pin, swipe the trackpad sideways to show the minus and pull the trigger on your object

Now that you can pin objects, if you accidentally grip the object when moving things around in your world, it won’t move!

Continue this process of adding, resizing and pinning every object and image until your world has all the 2D and 3D elements imported.

Step 13: Use Brushes to Highlight and Decorate

Brushes enable you to create movement in your world and make objects or areas stand out.

a) On your palette-controller, swipe the trackpad sideways until you get to the brushes panel

b) Select a brush by pointing and clicking with your painting-controller

c) By pointing the painting-controller away from the panels on the palette-controller, you can hold down the trigger to activate the selected brush

d) Swipe the painting-controller trackpad left or right to resize the brush (the size of some brushes can't be changed in this way)

e) Swipe the palette-controller trackpad to find the colours panel. Click with your painting-controller to change the colour output of your brush

Hot tip: Use the hull brushes to create the base for the world by holding the trigger while dragging across a large area. You can also use this method to create mountains and large areas of land or water.

Step 14: Record Video and Take Snapshots of Your World

a) On the palette-controller, swipe the trackpad sideways to the tools panel and click on the camera icon

Your painting-controller is now a camera!

b) Swipe the trackpad on your painting-controller left or right to choose between taking a snapshot, an Auto-Gif, a 5 second Gif or a Video

c) Hold the controller up so that the scene you want recorded is within the camera lines

d) Pull the trigger to take the snapshot, Gif or video

Step 15: Access Your Recorded Content

Anything you record with the camera is saved in the same Tilt Brush folder as the elements you imported.

On your computer, go to: 

This PC > Documents > Tilt Brush 

and then:

> Snapshots 

or

> Videos

Step 16: Share With the Real World!

Now you’re ready to share your virtual world with the real world! Your snapshots and videos will save as .png and .mp4 files so you can upload or share them almost anywhere!

For more ideas on teaching, learning and creating digital content for teachers and learners, go to www.rarangamatihiko.com or search Raranga Matihiko on You Tube.