Introduction: Kinect Desktop Control for Linux

This project covers the integration of a Xbox 360 (version 1) Kinect with a Linux computer.

Topics include:

  • RequirementsOS installation
  • Additional Information for Libraries
  • Testing with Examples
  • Controlling the Desktop
  • Suggestions
  • Credits

This guide will not cover the attempt to run Skeltrack on a Raspberry Pi 3 B. This is simply because the Pi does not have the processing power for this.

Step 1: Abstract

Step 2: Requirements

Firstly,

You must have a system that has the following specifications:

- At least 3GB of RAM

-2.0Ghz or greater CPU, preferably multi-core

-Integrated or discrete GPU compatible with OpenGL 3.1

To find out if you meet these requirements, you can google the specifications of your computer online. You can also open Run->"DxDiag" in a windows environment to see what you have.

Secondly,

You must have a Xbox 360 Kinect, not the Xbox One Kinect. You can use a Xbox One Kinect but this guide only covers the 360 version.

Step 3: OS Installation

One of the ABSOLUTE first things to be done is to install a Operating System on to the computer. We chose to use Raspbian at first (trying to install on a Pi) but this was fruitless, and could get us as far as using the Kinect as a fancy Webcam. So to make sure we had a functioning control, we upgraded to a new system and installed a better operating system, thus Ubuntu.

When using Ubuntu we chose the Gnome3 environment, which is what the Skeltrack-Desktop-Control application, (later used) will require.

You can download Ubuntu straight from Cannonical, https://www.ubuntu.com/download/desktop .

Then you can either A) Burn the .iso image to a disk, or B) burn it to a USB.

Both option result in the same turn out, and are subjected to preference only. You can burn the image using any disk writing software that supports ISO images. You can also use rufus https://rufus.akeo.ie/ . This will create a bootable USB device with the Ubuntu image on there.

Now you will have to configure the BIOS on the computer to boot from the CD drive, or USB. This will be accomplished by watching the computer boot, then pressing the function key (F5,F4,...) that is responsible for the system BIOS options. Once there navigate to the boot options, and select your medium.

Then reboot, and select install or LiveCD. If you choose LiveCD, the install is not pertinent and will not be persistent after shutting down the system.

Step 4: Additional Information for Libraries

Initially we followed this guide: http://tayyabnaseer.blogspot.com/2012/05/installin... . We found that this guide worked quite well with our experience level, but still let us have large room for error, and once finished left us in the dark on how to install the Desktop Control program. In order to make the guide more comprehensive we have made this Inscrutables, and will compile it into one. We had initially ran into some issues and were able to resolve them, and have included the work arounds in this document.

Step 5: Installing Dependencies (1)

First we will install the Libfreenect library and its dependencies.

sudo apt-get install git-core cmake freeglut3-dev pkg-config build-essential libxmu-dev libxi-dev libusb-1.0-0-dev 
sudo apt-get install libglib-3.0
sudo apt-get install libgtk2.0-dev
sudo apt-get install libgtk2.0-doc
git clone  http://github.com/OpenKinect/libfreenect.git 

cd libfreenect
mkdir build
cd build
cmake ..make
sudo make install
sudo ldconfig /usr/local/lib64/ 
cd ../..  
sudo aptitude install gobject-introspection gtk-doc-tools

Step 6: Installing Dependencies (2)

For GFreenect:

PKG_CONFIG_PATH=/usr/local/lib64/pkgconfig/:/usr/local/lib/pkgconfig:/usr/lib/pkgconfig
export PKG_CONFIG_PATH
git clone https://github.com/elima/GFreenect.git
cd GFreenect
git clean -f
libtoolize -v --copy --install
aclocal
autoconf
./autogen.sh
make
sudo make install
sudo ldconfig /usr/local/lib/
cd ..

Step 7: Dependencies (3)

sudo aptitude 
install libclutter-1.0-dev
export PKG_CONFIG_PATH
git clone <a href="https://github.com/joaquimrocha/Skeltrack.gitcd"> https://github.com/joaquimrocha/Skeltrack.git</a>
<a href="https://github.com/joaquimrocha/Skeltrack.gitcd">cd </a> Skeltrack
git clean -f
libtoolize -v --copy --install
aclocal
autoconf
./autogen.sh
./configure --enable-examples=yes

Once you are here STOP. This is something that came to be a problem every time we isntalled Skeltrack. There is an issue with the "Makefile.in" In order to fix the issue do this:

<strong>sudo nano Makefile.in</strong>

Once you have opened the file in nano, do the following:

Edit "Makefile.ini" by
delete this line "ENABLE_TEST_TRUE"@am_append_1=tests"change the following two lines to-- "BUILD_GTK_DOC_TRUE"@am_append_1=doc" "ENABLE_EXAMPLES_TRUE"@am_append_2=examples"
make
sudo make install
cd examples
./test-kinect
If the ./test-kinect does not work, check the resulting dialog to make sure you have met all additional requirements. 

Step 8: Installing Skeltrack-Desktop-Control

Now that we have installed all the dependencies we can install the Control we need!

cd /
git clone  https://github.com/joaquimrocha/Skeltrack-Desktop...
aclocal
autoconf
./autogen.sh
./configure LIBS="-lm"
make
make install
cd src
skeltrack-desktop*

Enjoy!

Step 9: Testing With Examples

Testing the installation is an important part to verify functionality.

To do this in the Skeltrack-Desktop-Control check the src folder. Inside of that folder there should be an executable called skeltrack-desktop-control. Type the name into the terminal and it should run. If it doesn't read the resulting error and start googling.

You can test the other libraries by checking the examples folders in each of the libraries. There are python, C++, C, ruby ...wrappers for each test. You can run these. For example type python filename.py this will execute the python program. You can also compile them into executables by typing chmod+x filename.py

Inside of the Skeltrack Libary you can go to examples, then find the file called testkinect. You can run this by typing ./testkinect

Step 10: Controlling the Desktop

Actions to control the Desktop:

First start the skeltrack-desktop-control from /Skeltrack-Desktop-Control/src . Once this is done, it will load.

Control Area:

  • This is the area kinect will recognize for controls and input. This is about 20cm from your hear, or full arms length away from your body. Also note you should be about 3 feet from the kinect sensor.
  • Anything that happens outside this area, will not affect the control.
  • Anything that happens inside or in front of this area will affect the control.

There are two separate modes of control and one main set of controls.You can change mode by pressing tab, when viewing the skeltrak-desktop-control GUI.

  • Main controls
    • You can use the movement of one hand as the mouse input to the computer
      • Moving your hand left or right results in a change on the desktop
    • Click
      • Quick move the hand, behind your back, into the same area as the cursor hand. Return to back immediately.
        • This motion performs a right click of the mouse.
    • Click-Hold
      • Perform the above motion except leave the second hand in the control area. Drag the selected object around by moving the original hand.
  • Pinch
    • Functions the same as Steering in most respects except:
      • The Pinch method uses Control + Scoll wheel to zoom in an out (much like zooming in a google maps on android).
        • You do this by starting in the Wheel position and moving in and out.
  • Steering Wheel
    • To use this mode, you put your hands in the position of holding a steering wheel. You're arms must be fully extended.
      • This mode changes input when you move to the right or left with the wheel (like a car).
      • This mode maps the movements to Forward, and which ever direction you turn.

Step 11: Suggestions

The Skeltrack-Desktop-Control program is not fully robust. This is because the of the limited number of action items you can preform with the program. Although the program works fantastically, more could be added at a later date by anyone willing to give it a shot.

Because of the lack of functionality, some suggestions for use include:

  • Simple Games: Mouse based only, Slow paced is better
  • Example from the Author: SuperTuxKart (run sudo apt-get install supertuxkart)
    • This racing game requires the OpenGL 3.1 library and will run horribly with out it. It is very difficult to control.
  • Point and shoot games
  • Simple motion deactivated Displays.

Step 12: Credits

Tayyab Naseer: Blogger with a very helpful and useful guide to installing up to Skeltrack libraries.

Joaquim Rocha: Author of the Skeltrack Library, and Skeltrack-Desktop-Control

http://www.joaquimrocha.com

USU ECE Department for awesome funding, and classes.

Step 13: Calibration of the System for Games

Initially, operating the games may be uncomfortable and frustrating. For this project we tested the kinect interface with several games. Here, we discuss about a python based brick breaker game.We can calibrate the game by editing the game codes.

Download the game from the download link from http://codentronix.com/2011/04/14/game-programming...

You may have to install python and pygame module for your distribution, if already not installed, which is simple with just following two commands.

For debain based systems: sudo apt-get install python, sudo apt-get install pygame

Installing pygame module may not be straight forward for some distributions. You may install it directly from binaries, instructions for which can be found at its official website link http://www.pygame.org/download.shtml.

From our initial observation, the game was not all controllable; the speed at which the paddle and ball moved were too fast. We can control the size and speeds of the objects.

The ball velocity is changed from the line

self.ball_vel = [5,-5] //change 5, -5 to lesser values

To change the paddle velocity we changed the lines

if keys[pygame.K_LEFT]:
self.paddle.left -= 5 //change 5 to lesser values

if keys[pygame.K_RIGHT]:
self.paddle.left += 5 //change 5 to lesser values

The window sizes also can be adjusted from different defined parameters in the code. We can download other python games. Raspberry pi inbuilt games are good choices which can directly be downloaded from https://github.com/KenT2/python-games