Course Home
Administration
Syllabus
References
Lecture Notes
Readings
Assignments
Projects
Resources


Comments? Idea?























































































































































































































































































































































Object Recognition in ROS

Final project by

Dolev Shapira

shdolev@post.bgu.ac.il


Introduction

To us, humans, object recognition seems to be a simple task.
We usually neglect the "primitive" process of recognizing a pattern familiar to us in a given scene.
Close your eyes and think of a door. You probably just saw a door, because our visual system is very powerful. It links between images and concepts, and sometimes, between images and other images.
Once you think of a door, you usually think of a specific kind, however there are all kinds and shapes of doors, but you think of a specific one.
One can always invent a door that opens in such a bizzare way, contrary to any previous experiences, which justifies the classification of properties of objects.
The goal for this project was to implement a simple, modular mechanism which is able to recognize an object in a scene by a specific kind of property, given by images, which will be called "Templates", therefore simulating a situation where a familiar object is to be recognized in a given scene.
The project is implemented as a ROS node subscribed to a control topic and a topic of which the scene is sampled, and publishing the result of the recognition to a result topic.

Approach and Method

To achieve this goal, I tried several methods, out of which pattern recognition proved itself to be the most accurate. The pattern recognition can be found in libraries within OpenCV versions 2.4.9-2.4.11 called "features2d" and "nonfree".
These libraries allowed the use of FeatureDescriptors, FeatureExtractors, and Matchers, as well as the findHomography method, which is crucial in finding the projection of the template in the scene.
This addressed one of our problems but created another. The object recognition process was time consuming, and needed a lot of CPU time, which made us design the node in a way that uses a control topic to control the node - whether it should be active or not.

Results

The tests were performed in two parts:
1. Taking photos of an elevator panel using a camera (Nikon D5200) on a tripod from different distances and angles, generating 9 templates out of them and checking how many successes were achieved using these templates, and how many false positives were achieved. All photos were reduced to a resolution of 1536*1024, since the system was designed to work with the Komodo's ASUS camera which should have a maximum resolution of 1280*1024.
We achieved no false positives, and almost 70% success rate. One should know however that the elevator panel is not optimal for this kind of object recognition since it is a reflective surface, which makes the recognition using the pattern matching method less successful.
2. Many tests were performed using the Komodo robot in a lab, which was badly lit, using out of focus templates, and to make it even worse, a low resolution scene images published by the ASUS camera node.
These conditions make it work less effectively, and made a recognition possible only up to half a meter. The better the resolution is, and the more lit the scene is (less noise), the better results we shall get.
Here you can see an example of how the recognition worked for a "Coca-Cola" bottle:

The Templates:

some_text
some_text

The Result:

some_text

Conclusions

Looking back to the goals, the code is built in a very modular way, and is simple to understand, and the recognition itself functions relatively well, assuming sufficient conditions.
Once one examines the code, it is obvious that some malfunctions were experienced when working with the Komodo robot, such as problems with the publishing mechanism, and imwrite failiure for unknown reasons. These problems are considered external to the object recognition, however affect the operation of the node as a whole (as one can see).
Leaving these problems aside, the object recognition functions relatively well under better working assumptions, such as in focus templates, good lighting, and higher resolution images of the scenes, and achieved the main goals of this project.

Additional Information

References