使用tesseract识别车牌

I'm developing an app which can recognize license plates (ANPR). The first step is to extract the licenses plates from the image. I am using OpenCV to detect the plates based on width/height ratio and this works pretty well:

我正在开发一款可识别车牌（ANPR）的应用程序。第一步是从图像中提取牌照。我正在使用OpenCV来检测基于宽高比的印版，这非常有效：

But as you can see, the OCR results are pretty bad.

但正如您所看到的，OCR结果非常糟糕。

I am using tesseract in my Objective C (iOS) environment. These are my init variables when starting the engine:

我在Objective C（iOS）环境中使用tesseract。这些是启动引擎时的初始化变量：

// init the tesseract engine.
    tesseract = new tesseract::TessBaseAPI();
    int initRet=tesseract->Init([dataPath cStringUsingEncoding:NSUTF8StringEncoding], [language UTF8String]);
    tesseract->SetVariable("tessedit_char_whitelist", "BCDFGHJKLMNPQRSTVWXYZ0123456789-");
    tesseract->SetVariable("language_model_penalty_non_freq_dict_word", "1");
    tesseract->SetVariable("language_model_penalty_non_dict_word ", "1");
    tesseract->SetVariable("load_system_dawg", "0");

How can I improve the results? Do I need to let OpenCV do more image manipulation? Or is there something I can improve with tesseract?

如何改善结果？我是否需要让OpenCV进行更多的图像处理？或者有什么我可以通过tesseract改善？

5 个解决方案

#1

Two things will fix this completely:

有两件事可以完全解决这个问题

1. Remove everything which is not text from the image. You need to use some CV to find the plate area (for example by color, etc) and then mask out all of the background. You want the input to tesseract to be black and white, where text is black and everything else is white

   从图像中删除不是文本的所有内容。您需要使用一些CV来查找板区域（例如通过颜色等），然后屏蔽掉所有背景。您希望tesseract的输入为黑色和白色，其中文本为黑色，其他所有内容均为白色

2. Remove skew (as mentioned by FrankPI above). tesseract is actually supposed to work okay with skew (see "Tesseract OCR Engine" overview by R. Smith) but on the other hand it doesn't always work, especially if you have a single line as opposed to a few paragraphs. So removing skew manually first is always good, if you can do it reliably. You will probably know the exact shape of the bounding trapezoid of the plate from step 1, so this should not be too hard. In the process of removing skew, you can also remove perspective: all license plates (usually) have the same font, and if you scale them to the same (perspective-free) shape the letter shapes would be exactly the same, that would help text recognition.

   消除偏斜（如上面的FrankPI所述）。 tesseract实际上应该可以正常工作（参见R. Smith的“Tesseract OCR Engine”概述），但另一方面它并不总是有效，特别是如果你有一行而不是几段。因此，如果您可以可靠地执行此操作，首先手动删除倾斜总是好的。你可能会知道步骤1中板块的边界梯形的确切形状，所以这不应该太难。在删除歪斜的过程中，您还可以删除透视：所有牌照（通常）都具有相同的字体，如果您将它们缩放到相同（无透视）的形状，字母形状将完全相同，这将有助于文字识别。

Some further pointers...

一些进一步的指示......

Don't try to code this at first: take a really easy to OCR (ie: from directly in front, no perspective) picture of a plate, edit it in photoshop (or gimp) and run it through tesseract on the commandline. Keep editing in different ways until this works. For example: select by color (or flood select the letter shapes), fill with black, invert selection, fill with white, perspective transform so corners of plate are a rectangle, etc. Take a bunch of pictures, some harder (maybe from odd angles, etc). Do this with all of them. Once this works completely, think about how to make a CV algorithm that does the same thing you did in photoshop :)

首先不要尝试编写代码：从一个非常容易的OCR（即：直接在前面，没有透视）板的图片，在photoshop（或gimp）中编辑它并通过命令行上的tesseract运行它。以不同的方式继续编辑，直到这样做。例如：按颜色选择（或选择字母形状），填充黑色，反转选择，填充白色，透视变换，因此板的角是矩形等。拍摄一堆图片，有些更难（可能来自奇数）角度等）。与所有人一起做这件事。一旦完成这个工作，想想如何制作一个与你在photoshop中做同样事情的CV算法:)

P.S. Also, it is better to start with higher resolution image if possible. It looks like the text in your example is around 14 pixels tall. tesseract works pretty well with 12 point text at 300 dpi, this is about 50 pixels tall, and it works much better at 600 dpi. Try to make your letter size be at least 50 preferably 100 pixels.

附：此外，如果可能，最好从更高分辨率的图像开始。看起来您的示例中的文本大约有14像素高。 tesseract与300 dpi的12点文本相当不错，大约50像素高，在600 dpi时效果更好。尽量让你的信件大小至少为50，最好是100像素。

P.P.S. Are you doing anything to train tesseract? I think you have to do that, the font here is different enough to be a problem. You probably also need something to recognize (and not penalize) dashes which will be very common in your texts, looks like in the second example "T-" is recognized as H.

P.P.S.你在做什么训练tesseract？我认为你必须这样做，这里的字体不同就足以成为一个问题。你可能还需要一些东西来识别（而不是惩罚）破折号，这些破折号在你的文本中很常见，看起来在第二个例子中“T-”被识别为H.

#2

I don't know tesseract too much, but I have some information about OCR. Here we go.

我不太了解tesseract，但我有一些关于OCR的信息。开始了。

• In an OCR task you need to be sure that, your train data has the same font that you are trying to recognize. Or if you are trying to recognize multiple fonts, be sure that you have those fonts in your train data to get best performance.
• 在OCR任务中，您需要确保您的列车数据与您尝试识别的字体相同。或者，如果您尝试识别多种字体，请确保您的列车数据中包含这些字体以获得最佳性能。
• As far as I know, tesseract applies OCR in few different ways: One, you give an image which has multiple letters in it and let tesseract do the segmentation. And other, you give segmented letters to tesseract and only expect it to recognize the letter. Maybe you can try to change the one which you are using.
• 据我所知，tesseract以几种不同的方式应用OCR：一，你给出一个有多个字母的图像，让tesseract进行分割。另外，你给tesseract分段的字母，只希望它能识别这封信。也许你可以尝试改变你正在使用的那个。
• If you are training recognizer by yourself be sure that you have enough and equally amount of each letter in your train data.
• 如果您正在训练识别器，请确保您的列车数据中每个字母都有足够的数量。

Hope this helps.

希望这可以帮助。

#3

I've been working on an iOS app, if you need to improve the results you should train tesseract OCR, this improved 90% for me. Before tranning, OCR results were pretty bad.

我一直在研究iOS应用程序，如果你需要改进你应该训练tesseract OCR的结果，这对我来说提高了90％。在转换之前，OCR结果非常糟糕。

So, I used this gist in the past to train tesseract ORC with a licence plate font.

所以，我过去用这个要点来训练带有车牌字体的tesseract ORC。

If you are interested, I open-sourced this project some weeks ago on github

如果你有兴趣，我几周前在github上开源了这个项目

#4

Here is my real world example with trying out OCR from my old power meter. I would like to use your OpenCV code so that OpenCV does automatic cropping of image, and I'll do image cleaning scripts.

这是我从旧功率计中尝试OCR的真实世界示例。我想使用你的OpenCV代码，以便OpenCV自动裁剪图像，我会做图像清理脚本。

• First image is original image (croped power meter numbers)
• 第一张图片是原始图像（弯曲功率计数字）
• Second image is slightly cleaned up image in GIMP, around 50% OCR accuracy in tesseract
• 第二张图像是在GIMP中略微清理的图像，在tesseract中约为50％的OCR精度
• Third image is completely cleaned image - 100% OCR recognized without any training!
• 第三张图像是完全清洁的图像 - 100％OCR无需任何培训即可识别！

#5

Now License Plate can be easily recognized by mlmodel. I have created the core model you can find it here . You just need to split characters in 28*28 resolution through vision framework and send this image to VNImageRequestHandler like given below-

现在，mlmodel可以轻松识别车牌。我已经创建了你可以在这里找到它的核心模型。您只需要通过视觉框架以28 * 28分辨率分割字符，并将此图像发送到VNImageRequestHandler，如下所示 -

let handler = VNImageRequestHandler(cgImage: imageUI.cgImage!, options: [:])

you will get desired results by using my core mlmodel. Use this link for better clarification but use my model for better results in license plate recognition. I have also created the mlmodel for License Plate Recognition.

你将通过使用我的核心mlmodel获得所需的结果。使用此链接可以更好地说明，但使用我的模型可以更好地识别车牌。我还创建了用于车牌识别的mlmodel。