Optical Character Recognition using Pytesseract

pytesseract.pytesseract.tesseract_cmd = r'C:Program FilesTesseract-OCRtesseract.exe'

Note: Above command will set the path of the tesseract library in a system configuration if the path is not set according to the system configuration then even if the tesseract is installed then too it will throw an error.

Here we will be extracting the text from the image with custom configuration.

# Simply extracting text from image
custom_config = r'-l eng --oem 3 --psm 6' 
text = pytesseract.image_to_string(image,config=custom_config)
print(text)

Output:

 

Here in the custom configuration you can see the “eng” which indicates the English language i.e it will recognize the English letters you can also add multiple languages and “PSM” means Page segmentation which set the configuration of how the chunks will recognize the characters and “OEM” is the default configuration.

Now we will remove unwanted symbols from the text we extracted by replacing the symbol with an empty string.

# Extracting text from image and removing irrelevant symbols from characters
try:
    text=pytesseract.image_to_string(image,lang="eng")
    characters_to_remove = "!()@—*“>+-/,'|£#%$&^_~"
    new_string = text
    for character in characters_to_remove:
        new_string = new_string.replace(character, "")
    print(new_string)
except IOError as e:
    print("Error (%s)." % e)

Output:

 

In the below cell, we are reading the image into OpenCV format to process it further. This is required when we need to extract the text from complex images.

Now we will perform OpenCV operations to get a text from complex images.

image = cv2.imread('sample.png') # will read in the array format

Output:

 

Converting the images to grayscale so that it becomes less complex to process as it will have only two values 0 and 1. Here we are using the cv2.cvtColor() method to convert the colored image into the grayscale format and cv2.cvtColor can actually help in the 150 color conversion of the images.

Grayscale image

def get_grayscale(image):
    return cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
gray = get_grayscale(image)
Image.fromarray(gray)

Output:

 

Now we will blur the image so that we can remove the noise from the image. Here, we are using the function cv2.medianBlur() function with an aim of reducing the noise from the image, blurring is basically the technique that smoothens the images by applying relevant smoothing filters is one of the widely used methods in image processing.

Noise removal

def remove_noise(image):
    return cv2.medianBlur(image,5)
noise = remove_noise(gray)
Image.fromarray(gray)

Output:

 

We will perform threshold transformation here. cv2. Thresholding works on the simple concept i.e whenever the pixel value is lower than the given threshold value then the color is white otherwise the pixel color would be just opposite which is black. The function used is cv2.threshold.

Thresholding

def thresholding(image):
                        # source image,  grayscale image
    return cv2.threshold(image, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]
thresh = thresholding(gray)
Image.fromarray(thresh)

Output:

Here we are doing erode transformation. Erode transformation is one of the most basic and important steps in image transformation in erode transformation it usually fits the missing shapes and lattices in the images which later helps in recognizing the character in the images when it is slightly blurred or distorted. Here for eroding transformation we are using the erode() function from the cv2 library.

Erosion

def erode(image):
    kernel = np.ones((5,5),np.uint8)
    return cv2.erode(image, kernel, iterations = 1)
erode = erode(gray)
Image.fromarray(erode)

Output:

 

Here we will perform the morphological transformation. Morphological transformation is one of the best-suited techniques for binary images where it sorts the image according to its pixel values rather than going for numerical values of the images keeping into account the threshold values too.

Morphology

def opening(image):
    kernel = np.ones((5,5),np.uint8)
    return cv2.morphologyEx(image, cv2.MORPH_OPEN, kernel)
opening = opening(gray)
Image.fromarray(opening)

Output:

 

Here we are trying to match the image. As we are passing the same image for matching we got the similarity of 99.99%. Here, template matching is a method for searching and finding the location of a template image in a larger image. For template matching, we are using the match template() function from the cv2 library.

Template matching

def match_template(image, template):
    return cv2.matchTemplate(image, template, cv2.TM_CCOEFF_NORMED)
match = match_template(gray, gray)
match

Output:

array([[1.]], dtype=float32)

Now we will segregate every character in the text by creating a rectangle around it.

# Drawing rectangle around text
img = cv2.imread('sample.png')
h, w, c = img.shape
boxes = pytesseract.image_to_boxes(img) 
for b in boxes.splitlines():
    b = b.split(' ')
    img = cv2.rectangle(img, (int(b[1]), h - int(b[2])), (int(b[3]), h - int(b[4])), (0, 255, 0), 2)
Image.fromarray(img)

Output:

 

Finally, we can draw rectangles around a specific pattern or word.

# Drawing pattern on specific pattern or word
img = cv2.imread('sample.png')
d = pytesseract.image_to_data(img, output_type=Output.DICT)
keys = list(d.keys())

date_pattern = 'artificially'

n_boxes = len(d['text'])
for i in range(n_boxes):
    if float(d['conf'][i]) > 60:
        if re.match(date_pattern, d['text'][i]):
            (x, y, w, h) = (d['left'][i], d['top'][i], d['width'][i], d['height'][i])
            img = cv2.rectangle(img, (x, y), (x + w, y + h), (0, 255, 0), 2)
Image.fromarray(img)

Output: