Introduction: The Vision Based Pick-and-place With UArm
Recently, most of the queries we hear from you are about the vision based applications of uArm, like the object recognition, camera-uArm correlation, etc. Actually we have worked on that for a period of time.
We have conducted a simple vision based pick-and-place project with uArm, where uArm and the camera are set up independently of each other. In this system, uArm is somehow intelligent that it can distinguish different objects, colors, sizes, and conduct the pick-and-place task automatically.
Step 1: Components
A quick briefing of components--
A uArm,
A camera,
A PC
Software platforms--
Windows
Visual Studio C++
Libraries-- Opencv3.0, Aruco1.3, QT5.5
Specifically, uArm will separate two different types of coins, 1Yuan and 1Jiao of Chinese coins, and put them into the corresponding transparent small boxes (The diameter of 1Yuan is about 25mm and that of 1Jiao is about 19mm).
Step 2: Let’s Get a Mind-map of This Project.
Here you can see three module which are three major steps. They will be discussed later.
Step 3: Module One-- Recognition.
First of all, we need to locate the coin. This visual application is actually not that much complicated here, so morphology and edge threshhold methods are capable to locate the target coin.
Next step is recognition. The size of 1Yuan and 1Jiao are easy to tell the difference. If you have some other color coins or different shape, you can share your idea with us. :)
Step 4: Module Two-- Coordinate Conversions
With image processing, we can locate the position of the coin in the Pixel Coordinates, however we need the coordinates of coins in the frame axes of uArm. We can actually get those data based on pixels’ and camera’s coordinates.
Step 5: Last Module-- Adjust Sucker Position and Right Get the Target!
How do we deal with the accuracy problem as to grasp the target coin? Let’s see!
Once the coordinate conversions is completed for the first time, uArm will start to run the commands. After this movement, uArm will transfer his actual position information back and compare this actual position with the coin, thus we can get an error in reality. Last, by adding this error with the same coordinate, uArm will receive a new command and start to move again. As a result, uArm will keep running this circulation until the sucker moves to the right upper position of the coin!
Step 6: This Video Performs the Whole Process of Collecting Coins and Distinguishing Them.
Thanks for your interest. Hope this video helps and have fun!
Any question, please talk in the forum--
:)
Poppy in uArm