Pets Recognition

Just like face recognition algorithms. Pets recognition is an experiment using  the FaceRecognizer.Fisherfaces / Eigenfaces algorithms to find specific faces of animals in a set of trained data. the original idea came as an improvement for the picky kitty project. to have the camera identify a face of a specific cat and only open the cat dish if that cat is around, eliminating the access of any other predator to the cat food. similar to what you see in this video:

The idea in this experiment was to test if a trained set of objects that are not traditional faces will return reliable results using traditional face recognition algorithms such as Fisherfaces or Eigenfaces which are common in opencv. 

a quick note before we get started, I choose to skip the general object training – detection phase in this experiment though it is recommended to incorporate it, this would have ensured that if we are looking to detect a specific cat in an image, first we know that we have  a cat (any cat) in that specific image, you can read more about it in my previous post – personal trainer.  for simplicity I’ll skip this phase here and define the pets face area manually in this experiment.

Here is the code for the full experiment if you’ll like to take a look – https://github.com/screename/Pets-Recognition.git (obviously that this should work for any object, not just a pet).

General architecture

The app will work in 3 modes:

  1. COLLECT_MODE – collecting object to recognize from a video file,  save the trained object into a directory and a log file
  2. TRAIN_MODE – train the model, there is no real use for this mode aside testing for errors in the training process
  3. RECOGNIZE_MODE – load an image, train the model based on the data collected in the log file and see if it finds any similar objects

 Collection Mode

 in the collection mode, we’ll load a video file, you can use a similar method to train from a collection of images as well, just like in the personal trainer app. we’ll let the user define the object and save it to a directory and a log file.

 if a user defined a rect area in the image, they will have to type a key to assign it to a specific object, the more objects the user define the more accurate our training will be. 

train

the onMouse method will handle the mouse events and draw the rectangle on the image (or reset it to the original one), similar to this image of my cat chachi while training.

before we save the image, first we’ll check that it is different enough from the previous image we saved using the getSimilarity method which calculate the square root of sum of squared error. 

 if the image is different enough we’ll clean it and save it. to clean the image we’ll follow these steps:

1. convert it to a grayscale mat.

2. adjust the histogram of the image based on two of its sides, often in images one side has a lot of light while the other is dark, splitting the adjustment process into two helps in not over burning the image (or not adjusting it enough) since the value of the pixels on one side balance the other so the original look adjusted while it is not.

3. reduce the noise using  a bilateralFilter filter.

 This will apply the following transformation.

Captured

Captured

 
Gray

Gray

Equalized

Equalized

 
Filtered

Filtered

 while the histogram adjustment will combine the left side with the right side using the original historgam adjustment of the whole picture.

Equalized - left

Equalized – left

 
Equalized whole (center)

Equalized whole (center)

 
Equalized - right

Equalized – right

 

 Train Mode

 To train the algorithm, first we’ll load the data from our log file, convert it into Mat objects and sort them in our objects array with a co-responding index for a label.

 the method getIndexFromFileName simply split the file path with the delimiters “/” and “_” to get the index we used to save it before, so if a file was saved to “somedir/data/1_2.jpg” it will return 1 for its index

 Once the data is loaded we’ll feed it to our training model using either the FaceRecognizer.Fisherfaces or FaceRecognizer.Eigenfaces algorithms. note that in order to do that we need to make sure the countrib module is available in the opencv version we use. 

 Recognize Mode

when we load the app in recognize mode, first we’ll repeat the trainData method to train our model from the saved data, then we’ll load an image and let the user define an area which they will like to check for similar object.  in a real app we’ll use similar cascade classifier object to detect if we have an object that qualify to our standards first, just like we did in the personal trainer app.

The trainData function will load the data from the log file and pass it to the model, using one of the training algorithms we choose.

when a user define an area on the picture which he’ll like to test if the algorithm identify as an object, we’ll crop the selection and pass it as a Mat to the recognizeObject function.

This is working pretty well, but I’ll like to do more tests and see how it perform with a large amount of data, as it still throw some errors (when I don’t pass enough images for training). another thing that will improve the performance is to have a process of detection before the recognition that will ensure that an object belong to a specific group of objects, similar to what I did in this post

I’ll also like to try an alternative approach of knocking out the objects  – in this case the cat face out of the background, while aligning all images to the position of the right ear of the cat and rotating them so both eyes are on the same y position. hopefully this will provide a more accurate average cat face.Credit –  in the Mastering OpenCV with Practical Computer Vision Projects book.

 

 

Resources:

source code – https://github.com/screename/Pets-Recognition.git

Mastering OpenCV with Practical Computer Vision Project – http://www.amazon.com/Mastering-OpenCV-Practical-Computer-Projects/dp/1849517827 – much of the approach of my application is based on the last chapter.

 Cat database –  http://137.189.35.203/WebUI/CatDatabase/catData.html

Cat detection research –  http://research.microsoft.com/pubs/80582/ECCV_CAT_PROC.pdf

Computer Vision: Algorithms and Applications

http://szeliski.org/Book/drafts/SzeliskiBook_20100903_draft.pdf

http://research.microsoft.com/en-us/um/people/szeliski/

Viola–Jones object detection framework – http://en.wikipedia.org/wiki/Viola%E2%80%93Jones_object_detection_framework

Viola Jones face detection and tracking explained- https://www.youtube.com/watch?v=WfdYYNamHZ8

http://ramsrigoutham.com/

Template Matching Techniques in Computer Vision: Theory and Practice – http://www.amazon.com/dp/0470517069/?tag=stackoverfl08-20

http://homepages.inf.ed.ac.uk/rbf/IAPR/researchers/D2PAGES/TUTORIALS/VisMac2008.pdf

How do facial recognition programs work – http://stackoverflow.com/questions/4978630/how-do-facial-recognition-programs-work

Google’s image recognition software improves search http://www.technologyreview.com/fromthelabs/428910/self-taught-software/

FaceU – pets facial recognition software – http://techcrunch.com/2010/12/16/face-recognition-for-dogs-cats-and-birds-theres-software-for-that-video/

 face recognition algorithm – http://www.face-rec.org/algorithms/#Image

adaboost algorithm 

Weka 3: Data Mining Software in Java – http://www.cs.waikato.ac.nz/ml/weka/

face recognition development blog – http://apart-dev.blogspot.com/

Fast and Robust Face / Rare Event Detection width source code – http://www.c2i.ntu.edu.sg/jianxin/RareEvent/rare_event.htm

 Gaborgas algorithm

OpenIMAJ – Open Intelligent Multimedia Analysis toolkit for Java (OpenIMAJ). – http://www.openimaj.org/

 facial detection – https://www.youtube.com/watch?v=sWTvK72-SPU

lucas kanade algorithm – predict movement  – http://en.wikipedia.org/wiki/Lucas%E2%80%93Kanade_method

Javascript face recognition – http://blog.mashape.com/post/45712257463/face-recognition-using-javascript-and-mashape

Introduction to Face Detection and Face Recognition: http://www.shervinemami.info/faceRecognition.html

OpenCV face recornizer http://docs.opencv.org/modules/contrib/doc/facerec/facerec_tutorial.html

http://docs.opencv.org/doc/tutorials/objdetect/cascade_classifier/cascade_classifier.html#cascade-classifier
http://www.prodigyproductionsllc.com/articles/programming/detect-your-own-objects-with-opencv-and-c/
http://www.prodigyproductionsllc.com/articles/programming/how-to-train-opencv-haar-classifiers/
http://www.youtube.com/watch?v=WEzm7L5zoZE
http://www.packtpub.com/content/opencv-computer-vision-application-programming/video
http://achuwilson.wordpress.com/2011/08/05/object-detection-using-surf-in-opencv-part-1/
http://docs.opencv.org/doc/tutorials/features2d/feature_homography/feature_homography.html
http://www.intorobotics.com/how-to-detect-and-track-object-with-opencv/