Scale Invariant Braille Translator

Final project by

Yaniv Tocker

ytocker@gmail.com

Introduction

Optical Character Recognition ( a.k.a OCR) is the process of translating letter/digits that appear in images and being able to convert them into a format that a computer can manipulate (like strings & ASCII code).

OCR is one of the most active fields in computer vision research since its applications seem to be endless, from automatically reading car plates digits to recognizing street signs in different languages.

 

Using OCR on braille language is a natural application to explore the importance of OCR.

Braille language is a way to let the blind read, while each set of 6 six ( 3 rows & 2 columns) dots represent a certain letter, therefore words are simply  a row of 6 dot block .

The  string 'Braile Language' writtrn in braille leanguage

 

In order to assist the blind, braille reading is getting more common in public places such as bus stops. Although braille is used in public the blind don't know that it is there.

Therefore being able to detect & interpret  braille language in a scene could be a life-changing ability for the vision impaired.

I decided to focus on the 2nd challenge – to interpret braille language to ASCII code.

A system which does so correctly can be used to teach the blind Braille language and of course to be the 2nd phase  after locating braille writing in a scene.

 

Objectives:

-          Main goal : to design a software system that receives images containing braille language as input and after processing the image the output will be an image with the letters interpreted to English.

-          Make the system robust to different appearances of braille language in varying scales – a dynamic system.

-          Detect circles in a scene.

-          Eliminate circles that don't belong to the braille template – 6 dots arranged in 2 columns and 3 rows.

Approach and Method

In order to achieve main goal – interpreting braille language from images, we used the common braille template found in images – 6 circles for each letter (black and white circles).

 

System's overflow:

1.     Circle detection

Applying a simple Circular Hough Transform (CHT) is used in a large radius scale – 5:30 pixels,  in order to detect circles in different scales.

 

2.     Radius Histogram

To make the system dynamic & to eliminate circles that don't belong to the braille template, the radius histogram of the previous step was calculated.

The detected circles which have a radius which is far from the common one were deleted.

Another feature to eliminate non-desired circles is to use the 'k-nearest neighbors' method – choosing k=3. This means that if a circle didn't have at least 3 nearest circles where the distance between the center of the checked circle to its neighbors is less than 4 times the common radius were eliminated.

3.     Find areas of braille templates

We would like to find areas in the image that match the braille template – 6 dots arranged in 2 columns & 3 rows, and mark their center.

In order to do so the following steps were taken:

-          Create a mask from the result of step 2.

-          Create a dynamic patch – an artificial braille template which is constructed by using the common radius value. The chosen patch dimension after many tries were taken to be

(9*common radius) X (6*common radius).

-          Applying convolution on the previous steps will indicate where in the image braille templates are, since the patch is symmetric.

Convolution with a template is like applying hough transform, since pixels are 'voting' for their likelihood to belong to a certain template.

Local maxima with point above a certain threshold were taken as centers of braille templates.

 

4.     Create patches

 A patch is a part of the image.

We would like to take parts of the image that are match template of the previous section.

Therefore we used the following steps:

-          Create a mask for 'Braille meaningful circle' – basically these circles are simply black circles from the original image.

-          Mark the template centers.

-          Cut around each center according to the dimension of the artificial patch.

 

Applying the steps – black & white pixels – the mask, orange circles – template centers, red squares – examples of patches

 

5.      Create a dictionary

A dictionary is letter representation in braille language, as shown in the following key:

 B

            A dictionary was built based on the common radius size & patch dimension used previously – this is important in order to keep the system robust to scale changes.

The result is a braille representation for each letter in the same size as the patches.

 

6.     Letter Matching

Each patch is classified as a letter by the following steps:

-          Each patch & letters of the dictionary are reshaped into vectors.

-          For a certain patch its normalized inner product with each letter is calculated.

-          Taking the maximum result of the previous step is the patch classification as a specific letter.

In the figure we can see an example for the steps : a parch is on the left while on the right we can see the letter representation in braille method whole the title for each representation is the corresponding letter and the inner product result. The patch was classified as 'M' since it gave the maximum result.

Results

 Applying the method on the original image yields the following result:

 

Conclusions

-          A novel way to analyze braille language has been proposed.

-          The suggested system is robust to scale changes, which is a required skill from such a system since braille can appear in varying sizes.

Future Work

-          The system is designed to translate braille writing that appear in the most common way that images of braille are available online- each letter is represented with 6 dots. Not always braille appears this way, since the circles that aren't 'braille meaningful' (white circles) don't always appear. The system isn't design to deal with this scenario, but it is a great platform and lays down the foundations to develop such a system.

-          Future development could be to be able to detect braille in a scene from images (like in figure 3) as a prior phase to the suggested system, which could truly make this system a life changing feature for the vision impaired

Additional Information

-          Use the following website

-          Save the image with windows snipping tool

-          Interpret it using the GUI (there's a manual  in the 'Full project report')

References

1.  OCR: http://en.wikipedia.org/wiki/Optical_character_recognition

2. Braille OCR

http://en.wikipedia.org/wiki/Optical_braille_recognition

3. Braille OCR example

http://www.ni.com/white-paper/6470/en/

4. Braille code generator

http://braille.compelo.com/generate/

5. X. fernanadez et al, “A braille O.C.R for the blind”

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.141.7727&rep=rep1&type=pdf

6. J.Mennans et al,”Optical recognition of braillle writing using standard equipment”.

http://bauhaus.ece.curtin.edu.au/~iain/PhD%20BU/A_Phd%20docs/To%20read/Accessibility%20info/Research/Braille_Articles/OCR%20of%20Braille.pdf

7.  O. ben-shahar – lecture notes from ICBV 2014 –

    - object classification

    - hough transform