Locket of Love: From Kinect to 3D Printing

This Instructable shows you how to create a heart-shape locket with 3D relief of myself (But it could be a profile of any one like you too).

With the available free software, MeshMixer08, Processing, combine with Kinect Sensor (XBOX 360 was not needed) and a laptop or desktop I can create this precious locket to be a great gift to give as a present to the love one. 

Why don't I use 123D Catch to capture myself? You might ask!
In case of there is nobody else around to help you out with the photos shooting of you, like I was. Kinect Sensor (which I have one in my procession) was a good option to make this happen. And it was not really hard to do either.

Here's how I made it.

Step 1: Stuffs

Stuffs we needed!

A Person (In this case me!)


Hardwares

A Computer (MacBook Air was used in this project)

Kinect Sensor and Kinect Power Cable


Software and Libraries

MeshMixer08

Processing and Libraries

• OpenNI Processing Library
• Modelbuilder Processing Library

Optional

123D App. (see Step 8)

3D Printer

Step 2: OpenNI Processing Library

OpenNI Processing Library

We needed this library because it is a simple OpenNI and NITE wrapper for Processing. This library allows us to use Kinect Sensor in Processing programing environment.

Download and Install OpenNI

1 Go to the SimpleOpenNI Google Code website.
2 Find Mac OSX installation section. Then click on the download the installer.
3 Unzipped the downloaded file by double clicks on it. A directory called OpenNI_NITE_Installer_OSX appeared. (Image 1)
4 Launch the Terminal app. from the Utilities icon on the desktop. (Image 2).
   Or from the Finder tab go to Applications->Utilities->Terminal.
5 Change directory to the downloaded folder by type the following
    cd ./Downloads/OpeNI_NITE_Installer-OSX [and press Enter key] (Image 3)
6 Run the installer with the command
   sudo ./install.sh [press Enter key] (Image 4)
7 If the program ask for the password just enter your password.

This should successfully complete the installation of OpenNI on your Mac.


Installing the SimpleOpenNI Processing Library 

Next, Install OpenNI Processing Library on Mac OSX. Easiest platform to install OpenNI

1 Visit SimpleOpenNI download page on SimpleOpenNI Google Code.
2 Select the version of SimpleOpenNI for Mac OSX
3 Unzip the download file 
4 Locate the Processing Libraries folder on the Mac OSX (Image 5)
5 Drag the unzipped SimpleOpenNI directory to your Processing Libraries folder, (Documents->Processing->libraries->SimpleOpenNI) 

6 Quit and restart the Processing
7 Go to Sketch on main menu bar, select Import Libraries...
   Notice that the library SimpleOpenNI was added to the libraries list. (Image 6)

And we move on to test the application in the next Step.

Step 3: Kinect

In this Step I am going to show how to set up the Kinect with the Mac OSX

Kinect Sensor

Kinect Sensor and the Windows OS go well together, since there both are from Microsoft. But I was surprised that it i was also a lot easier than I thought to use Kinect Sensor on the Mac OS.

Other reason I was using Kinect Sensor with the Mac was that my PC (It's kind of old!) does not have OpenGL Frame Buffer support, which is necessary to run MeshMixer08. I use MeshMixer08 to edit and combine the scanned 3D-model of my profile and the perfect heart together.

Kinect Connection

Kinect that is bundled with XBox 360 does not required an extra power cable. But to use Kinect Sensor with computer, we needed to have extra power supply cable for Kinect Sensor. I got mine from Amazon.com. (Search with key word 'kinect sensor power supply').

What we needed to do is plug the female connector from the power supply (has the orange color), to Kinect male connector.
Then connect the USB male connector from power supply cable to USB port on the computer.

Plug the power cable connector to wall power outlet.

And we have the Kinect ready.


Test

Place the kinect on a convenience place that is also facing you.


Run the examples called DepthImage

The result of running the DepthImage sketch. The depth of the image capture by the camera displayed on the left and the RGB image is displayed on the right. (Image 8)

Step 4: Scan Yourself With Kinect

In this Step I am going to show how to scan myself with Kinect and send the file to next Step.

Modelbuilder Processing Library

Modelbuilder provides tools that make our life a lot easier to create and manipulate 3D geometry in Processing.

The library to be used with the scanning process is called ModelBuilder designed by Marius Watz, which can be obtained here.
Install it like OpenNI Processing library(in the previous Step) by moving its folder into your Processing libraries folder.

Usually, Documents->Processing->libraries (see image 2)

The sketch I used for scanning myself was from the type out from the book 'Making Things See Books' in chapter 5, The full code doesn't available anywhere else. 
The book  gives the details explanation of the sketch and how it works. This is a very good book, I recommended!

I did modified a few lines of the sketch to make it suitable what I'm doing for the project.

Here is the sketch:

import processing.opengl.*;

import unlekker.util.*;
import unlekker.modelbuilder.*;

import SimpleOpenNI.*;

SimpleOpenNI kinect;

boolean scanning = false;

//int maxZ = 2000;
int maxZ = 800;
int spacing = 3;

UGeometry model;
UVertexList vertexList;

void setup() {
  //size(1024, 768, OPENGL);
  size(640, 480, OPENGL);
  kinect = new SimpleOpenNI(this);
  kinect.enableDepth();
  
  model = new UGeometry();
  vertexList = new UVertexList();
  
}

void draw() {
  background(0);
  
  kinect.update();
  
  translate(width / 2, height / 2, -1000);
  rotateX(radians(180));
  
  if(scanning) {
    model.beginShape(TRIANGLES);
  }
  
  PVector[] depthPoints = kinect.depthMapRealWorld();
  
  // cleanup pass
  for(int y = 0; y < 480; y += spacing) {
    for(int x = 0; x < 640; x += spacing) {
      int i  = y * 640 + x;
      PVector p = depthPoints[i];
      
     // if the point is on the edge or if it has no depth
     if(p.z < 10 || p.z > maxZ ||
        y == 0 || y == 480 - spacing ||
        x == 0 || x == 640 - spacing) {
          // replace it with a point the depth of the backplane (i.e. maxZ)
          PVector realWorld = new PVector();
          PVector projective = new PVector(x, y, maxZ);
          
          // to get the point in the right place, we need to translate
          // from x/y to realWorld coordinates to match our other points:
          kinect.convertProjectiveToRealWorld(projective, realWorld);
          depthPoints[i] = realWorld;
          
        }
    }
  }
  
  for(int y = 0; y < 480 - spacing; y += spacing) {
    for(int x = 0; x < 640 - spacing; x += spacing) {
      if(scanning) {
        int nw = x + y * 640;
        int ne = (x + spacing) + y * 640;
        int sw = x + (y + spacing) * 640;
        int se = (x + spacing) + (y + spacing) * 640;
        
        model.addFace(new UVec3(depthPoints[nw].x, depthPoints[nw].y, depthPoints[nw].z),
                      new UVec3(depthPoints[ne].x, depthPoints[ne].y, depthPoints[ne].z),
                      new UVec3(depthPoints[sw].x, depthPoints[sw].y, depthPoints[sw].z));

        model.addFace(new UVec3(depthPoints[ne].x, depthPoints[ne].y, depthPoints[ne].z),
                      new UVec3(depthPoints[se].x, depthPoints[se].y, depthPoints[se].z),
                      new UVec3(depthPoints[sw].x, depthPoints[sw].y, depthPoints[sw].z));

      }
      else {
        stroke(255);
        int i = y * 640 + x;
        
        PVector currentPoint = depthPoints[i];
        if(currentPoint.z < maxZ) {
          point(currentPoint.x, currentPoint.y, currentPoint.z);
        }
      }
    }
  }
  if(scanning) {
    model.calcBounds();
    model.translate(0, 0, -maxZ);
    
    float modelWidth = (model.bb.max.x - model.bb.min.x);
    float modelHeight = (model.bb.max.y - model.bb.min.y);
    
    UGeometry backing = Primitive.box(modelWidth / 2, modelHeight / 2, 10);
    model.add(backing);
    model.scale(0.01);
    model.rotateY(radians(180));
    model.toOrigin();
    model.endShape();
     
    String filename = "Scan02.stl";
    //for (int i = 0; i <100; i++) {
    //   filename.substring(4) = '0' + 1/10;
    //   filename.substring(5) = '0' + 1%10;
    //create file if it does not exist on SD Card
    // if (!SD.exist(filename)) {
    //    break;
    //  }
    //} 
    
    //SimpleDateFormat logFileFmt = new SimpleDateFormat("'scan_'yyyyMMddHHmmss'.stl'");
    //model.writeSTL(this, logFileFmt.format(new Date()));
    model.writeSTL(this, filename);
    
    scanning = false;
  }
}

void keyPressed() {
  println(maxZ);
  if(keyCode == UP) {
    maxZ += 100;
  }
  
  if(keyCode == DOWN) {
    maxZ -= 100;
  }
  
  if(key == ' ') {
    scanning = true;
    model.reset();
  }
}

Step 5: MeshMixer

How to use MeshMixer to edit(fix) the model.

First, I am going to show you how to edit the model of my head or profile.
Open the Application
Import the head scan file from last Step (Step 4)

Click Inspector button on the Main Menu Strip
After the head model is loaded.
You'll see  colored spheres (either Blue, Red, or Magenta) attached to the model. (image 5)

Blue spheres represent holes
Red spheres represent non-manifold areas
Magenta spheres represent disconnected components

These are the areas that needed to be repaired. It sounds so difficult, doesn't it?
But it is not!

'Left Click' to allow MeshMixer to fix the problem for you.

'Right Click' on the sphere to edit the problem area.
By picking the 'Modify Selection' on the Main Menu Bar. The option are,
Select All
Expand to Connected
Expand to Groups
Invert
Invert (Connected)
Smooth Boundary


Then Press Accepted button. 

Or picking the 'Deformation' Button for a choice to edit the model.
Edit  & Fill
Discard 
Reduce
Remesh
Extrude
Extract
Bridge / Handle
Joint
Weld Boundaries
Convert to Parts
Separate

When done. Export the file out as STL format.

Note: See Step 10 for MeshMixer Tutorials.

Step 6: Editing Scanned Model

Download MeshMixer08 if you haven't done so. See previous Step (Step 5)

Image 2. Import your scanned file into MeshMixer. 

Image 3. Select Inspector command from the Analyse tab on the main menu bar.

Image 4. Number of spheres show up with the lines connected to the model.
Blue spheres represent holes.
Red spheres represent non-manifold areas.
Left click on each sphere to tell MeshMixer to fix the problem on each area. (Image 5)

Image 6. Next I used the brush tool to select the unwanted part to eliminate, the brown area was the area that I already select.

Image 7 to 9. I also used mouse to draw the line to cover the area I want to erase.
After select the area I used Discard Command in the Edit tab on the main menu to get rid of the selected area.

Image 10. shows  all the backplane had been deleted. I saved the file at the halfway point of editing.

I will be using only the head portion of the profile. So I continue deleting the part below the neck.

Image 12. Using the brush tool to select.

Image 13 and 14. Select the polygons on the sides at the bottom.

Select Discard command to delete the unwanted polygons.

Image 15 shows the profile of only head that's needed!

Image 16. Then save the file in stl format.

Next step, we are going to merge this head profile to the heart.

Step 7: Editing Scanned Model (Con't)

Image 1. SmoothBrush/1 tool for smoothing out the surface.

image 2. Adjust the Smoothing Brush Tool Properties to use suitable brush size. The appearance of the Smoothing Brush. Move the brush around the surface to smooth out the surface.

Image 3. I used the Volume Brush to fill up some crease in this area.

Image 4. Return to Smoothing Brush to smooth out the surface in the neck area.

Image 5. After I was happy with the surface of my profile. I used Inspector command from the main menu tab to check the model out if there was any more problem. And there was only one on the outline on the back. This is fine, since I'm going to trim out some more surface!

Image 6 to 11. I used Plane Cut command I used to erase the unwanted surface.

Image 12. Export the finished model into stl format. So we can use it in case we mess it up.

Step 8: Create Perfect Heart in 123D (Optional)

I just want to show how I create my perfect heart using 123D app. This heart model will be used in Step 9, with the 3D model of the scanned 3D model of my head or profile deepen upon with one will look better!

If you are not familiar with 123D, but would like to learn more, take a look at my tutorial, 123D Modeling for 3D Printing. Or you can use any 3D Modeling program that you are familiar with to create your own heart, or just use the heart model provided and skip this Step.

NOTE: The 3D model of a perfect heart (Image 1) is provided below. If any one want to use it to make your own locket, feel free to download it.


Create Perfect Heart

Following are  the steps I used to create my perfect heart:

Image 2. Start 123D app.
Image 3. Use Pen Tool to DRAW the straight  line of 200 mm long.
Image 4. Continue draw an ARC of 100 mm radius, start from end of straight line drawn earlier.
Image 5. Draw another ARC, with the same radius (100mm) start from end point of previous ARC.
Image 6. DRAW a straight line, start from the end of ARC and end at the start point of the first line.

Now, we have the outline of the heart.
Next, create another heart outline inside the first heart outline.

Image 7. Start to draw a line inside the first heart outline (20, 180)
Image 8. Draw the straight line parallel to the outside border of the heart.
Image 9. Draw the ARC with the same center as the outside ARC of the heart.
Image 10. Draw a short straight line downward 20 mm long. And draw another short straight line perpendicular to the first one with the length of 20 mm also.
Image 11. Continue draw an ARC using the same center point with the outside arc.
Image 12. Draw the last line, starting from the end point of the ARC and end the line at the start point of the inner heart.

Now, we have double outline of the perfect heart. 

Image 13. Use the EXTRUDE command to extrude the inner heart to 60 mm in height.
Image 14. Select the external heart ring (as shown) Then extrude the selection to the height of 45 mm.
Image 15. We now have the 3D model of perfect heart. Save the file as STL format.

(Note: If you could not save the file in STL format. That might be because you haven't sign in when you start the 123D app!)

Step 9: Assemble the Locket for 3D Printing

Image 1. Import the head model into MeshMixer, or Open the Head model that saved in MeshMixer format.

Image 2. And Import the perfect heart from Step 8 into MeshMixer.

Image 3. MeshMixer wanted to know if we wanted to replace or append the importing 'heart model' to the face. Select Append.

Image 4. MeshMixer knew that the heart model is kind of larger than the face model. And asked whether we want MeshMixer to fix the problem for us. Click Yes.

Image 5. The models of the scaled heart and face overlapped. The dark area was the face model hidden behind and it was inactive.

Image 6. Select the Transform command from Edit menu tab to move the heart away.

Image 7. Still in the Transform command, I clicked at the blue arc to rotate the heart 45 deg.

Image 8. Select the face by clicking at it. This make the heart inactive, the color of the heart became darker. (Image 9)

Image 10. Hold down the middle mouse to rotate the scene or view. Then move the face to fit on the heart top surface by using the transform arrows.

Image 11 and 12. Click at the small white cube in the middle will scale the selected surface (face model) up or down.

Image 13. I turned on the Wireframe mode. (W key is the shortcut for wireframe command.) And select the heart model.
Note: you will notice that the heart surfaces have the different polygon sizes. We need to adjust the amount and size of the polygon on the surface to match the polygons of the face.

Image 14. Use the Brush Tool to select the surface of the heart.

Image 15. Use command Remesh to adjust the polygon of the selected surface.

Image 16. Click Accept to 'remesh' the surface.

Image 17. Repeat the 'Remesh' a few time  until the polygon sizes about the same. Then 'Accept' the change.

Image 18. Select the whole heart, and do the final transformation, so the face model is totally contain inside the heart.

Image 19. Then I scaled down the heart along the red axis (X) to make the heart propotionally fit with the face model.

Image 20. I select both models, face and heart, by pressing the Shift Key and use mouse to select both models.

Image 21. I used command Boolean Union  in Edits... command to merge both models together.

Image 22. Finally, I used Inspector command to check if the final model is 'water tight' or ready for 3D Print. And it was good! No Blue, Red or Magenta sphere!

Step 10: Conclusion

And now you have it! A locket of love that you could print it yourself if you have a 3D Printer in your procession.
Since I do not have a 3D Printer, I am not able to show you my actual 3D print of this 'locket of love' right now.
But I will post it as soon as I receive the 3D Print from the fabricator.

The title image was the quick rendering that I did in 3DS Max 8, which I also import my model in to check for the "water thigh" too.

Below, are some reference and tutorrial of the MeshMixer app. There are very good and easy to follow. 
I learn a lot from these tutorial.

meshmixer reference:
http://meshmixer.com/help/index.html

MeshMixer7 Tutorials Series:

meshmixer model/print tut 1/6

meshmixer model/print tut 2/6

meshmixer model/print tut 3/6

meshmixer model/print tut 4/6

meshmixer model/print tut 5/6

meshmixer model/print tut 6/6

Easier Capping and Booleans Tutorial