The Scanner

Often I been asked to fax something, or to print a form, sign it and email it, these are things that become more and more challenging for me, since I certainly do not own a fax or a printer, what I usually do is take a picture of the form with my phone, open photoshop, correct it, re-size it and send it. I wanted to build an app that will take care of these steps for me. so I came up with the following.

The scanner is an app that will automatically recognize a document in your picture, crop it, color correct it and save it for you.

The-Scanner-home   The-Scanner-Scan

Lines VS Contours

Obviously  I was going to use opencv to handle these tasks, I found that there are two approaches to do this, the first one is based on finding the most prominent lines in the picture and create a quadrilateral estimate of the main rectangle shape in the image. using function similar to this:

 followed by a function to find the intersection points between the lines and define them as corners. I find that this approach works well on images where the main rectangle are has a lot of contrast to the background, while in normal images, often the lines gets blurry and the function fail to recognize them.

The second approach is to find the contours in the image, search inside of them for the biggest contour and define its four corners and the shape we’ll like to manipulate, here is how it works with a local file.

The setup

 We take a typical document image, in different settings, duplicate it and convert it to grayscale tones.

scan3   chair

Image Processing

with the contourCrop method we start with equalizing the histogram of the gray image, blur it a bit so any noise will disappear and apply an adaptive threshold filter to increase the contrast.

 So our image has just gone through the following transformation.

source   gray   equalized
blurred   treshold    

Looking for contours 

Then we look for contours in the treshold image

 This will find all the contours in the image, if we’ll run the last 2 lines of the code just so we can see what the contours are, we’ll get an image somewhat similar to this, where all the contours have been drawn on.

contours

 Next we’ll look for the most substantial contour area we found, assuming that if we scan a document it will be the largest object in the image. The function contourArea gives us the measurement of an area which we use to compere to the largest area we encounter so far, if the new area is bigger, we assume the largest area to be the new area.

 This is how the largest area in our image will look like if we draw it, using the selected id we found for the largest contour area.

 contour

 we can also retrieve the bounding rect of the area we found.

 which will look like this

 rect

 Nobody puts baby in the corner

 The challenge at this point is to find the points in our contour vector which are the actual corners of the document. to do that I choose to follow this plan.

1. find the center point of the contour and define it as the center.

2. run a loop through all the contours and see how far they are from the center, knowing their position in regard to the center to assume that they are a top left or a bottom left corner, if a distance between a point is bigger then the current top left when the point x is smaller than the center x and the point y is smaller than the center y we assume that it is the top left corner.

 we’ll apply the distance formula to find to compere the distance between the center and the current point

Error: http://lab.onetwoclick.com/wp-content/plugins/wpmathpub/phpmathpublisher/img/ must have write access Read the official wpmathpub plugin FAQ for more details

 implemented in the the getDistance formula

 drawing these points will show us where the the most four distance points from the center are

 corners

Put it in perspective

now all we need to do is transform these point in to the shape we want (a rectangle?) we define a Mat and a vector with our ideal points

 and pass it to opencv to do the magic

 cropped

 now we just need to apply some color corrections, and pass it back to our mobile environment, it’s been a fun app to work on, again – I have seen this done using the first approach of finding lines, but from my tests you’ll get better results looking for contours, makes me wonder if this has a bigger meaning in life :) 

Resources: