Introduction: 3-DOF Gripper Controlled Through Computer Vision

By Facebook
About: I'm Electrical Engineer from Pakistan institute of Engineering and Applied sciences Islamabad, Currently working as a Research Assistant at Hong Kong Polytechnic University. My area of Interests are Biomedical…

In my previous tutorial I made "3-DOF Gripper Manually Controlled by Joystick", this is just the upgraded version of that system in this tutorial instead of controlling that system manually I'm going to use computer vision technique to control the gripper.
Prerequisite for this tutorial is OpenCV, the reader must have bit of a knowledge about OpenCV or at least he/she should go through it.

Step 1: How It Works?

First we need to understand the technique that I have implemented in this project, which is very simple.
Before that i strongly recommend you to see my previous tutorial because this tutorial is totally linked with it.
In this project we will use the camera as it's obvious from the computer vision, we will apply the color detection technique for our project, but how? Our goal is that our gripper moves to the target's position and grasp it. For that i have used the color detection technique. Technique is discussed below.

Steps are as follows:

  • I will place one circular object(Reference Object) of certain color(in my case blue) on the gripper.
  • I will place a ball(Target Object) of some color(in my case green) on the ground underneath the gripper.
  • With the help of OpenCV I will detect these two objects on the basis of their colors and will identify there positions (pixel position).
  • Then first my motor will move in forward or backward direction and will come align to the ball.
  • Then second motor will move left or right, such that the ball come exactly under the gripper.
  • At last the third motor will move down to grasp the ball and will move back to home co-ordinates.

Step 2: Camera

Step 3: Reference Object

Step 4: Target Object

Step 5: Object Detection

Step 6: Camera Eye

Step 7: Code:

Code is bit long so i haven't discuss it here but you can download it from this given link, its very easy to understand it has two parts.

  • Python code (to handle the OpenCV)
  • Arduino code (to handle the control of system)

    In case of any query feel free to ask in comment section.

Link: Code (Leave the manual)