又到了一年一度的靈魂拷問季：
年會中獎了沒？被年終獎砸暈了沒？
從本節起，我們開始嘗試做一下車牌識别中的算法部分。從上一節的基本架構圖中，可以看到，要想做車牌識别，第一步還是要知道車牌在圖檔中的位置！
是以，萬裡長征第一步，我們先從車牌定位開始吧。
車牌定位
尋找車牌對于人腦來說真是小事一樁，這也是經過千錘百煉的結果。但是對于計算機來說可能就沒有這麼簡單了。我們先來看看在實體世界什麼是車牌，以及他們有什麼特征。
我們以中國車牌為例，車牌的種類也是繁雜得很。從汽車類型上分有：
- 小型車号牌
- 大型車号牌
- 挂車号牌
- 使、領館汽車号牌
- 港澳出境車号牌
- 教練号牌
- 警車号牌
- 消防号牌
- 等等。。。
從車牌底色上看有:
- 藍色
- 黃色
- 綠色
- 白色
- 黑色
- 黃色+綠色
面對如此衆多的分類，最怕的就是一開始就想做一個大而全的系統。靈活開發才是王道，我們以其中一個最普通的小型車号牌+藍色為例，找一找它的特征點：
1. 尺寸
寬440mm×高140mm的矩形
2. 顔色
背景為藍色，顯示内容為白色
3. 内容
以“滬A-88888”為例，格式為“漢字（省/直轄市縮寫）”+“大寫字母（市/區縮寫）”+“點（-）”+“5位大寫字母和數字的組合（随機車牌号）”
好了，了解過了車牌的基本内容，我們就要開始思考如何在一張數字圖像上找到車牌。這裡我們隻利用兩個有用資訊尺寸和顔色（内容部分比較難，放在後面）。
尺寸因為圖檔大小和車牌遠近的問題，隻能用到它的比例和矩形特征。我們可以嘗試找到符合寬高比在（2, 4）之間的矩形。那麼車牌就在這些矩形裡了。
顔色部分可以用來做精調，可以在上面的矩形裡找到符合藍色覆寫比例的部分。這樣一可以剔除那些非藍色的矩形，而來可以縮小矩形範圍提煉精确的車牌内容。
為了實作上面兩個大思路，再具體一些可以分成如下七步：
1. 圖檔縮放到固定的大小
由于加載圖檔大小的差異，縮放到固定大小的最重要的原因是友善後面的模糊、開、閉操作，可以用一個統一的核心大小處理不同的圖檔了。
def zoom(w, h, wMax, hMax):
# if w <= wMax and h <= hMax:
# return w, h
widthScale = 1.0 * wMax / w
heightScale = 1.0 * hMax / h
scale = min(widthScale, heightScale)
resizeWidth = int(w * scale)
resizeHeight = int(h * scale)
return resizeWidth, resizeHeight
# Step1: Resize
img = np.copy(self.imgOri)
h, w = img.shape[:2]
imgWidth, imgHeight = zoom(w, h, self.maxLength, self.maxLength)
print(w, h, imgWidth, imgHeight)
img =cv.resize(img, (imgWidth, imgHeight), interpolation=cv.INTER_AREA)
cv.imshow("imgResize", img) 2. 圖檔預處理
圖檔預處理部分是最重要的，這裡面所有做的操作都是給有效地尋找包絡服務的，其中用到了高斯模糊來降低噪聲，開操作和權重來強化對比度，二值化和Canny邊緣檢測來找到物體輪廓，用先閉後開操作找到整塊整塊的矩形。
# Step2: Prepare to find contours
img = cv.GaussianBlur(img, (3, 3), 0)
imgGary = cv.cvtColor(img, cv.COLOR_BGR2GRAY)
cv.imshow("imgGary", imgGary)
kernel = np.ones((20, 20), np.uint8)
imgOpen = cv.morphologyEx(imgGary, cv.MORPH_OPEN, kernel)
cv.imshow("imgOpen", imgOpen)
imgOpenWeight = cv.addWeighted(imgGary, 1, imgOpen, -1, 0)
cv.imshow("imgOpenWeight", imgOpenWeight)
ret, imgBin = cv.threshold(imgOpenWeight, 0, 255, cv.THRESH_OTSU + cv.THRESH_BINARY)
cv.imshow("imgBin", imgBin)
imgEdge = cv.Canny(imgBin, 100, 200)
cv.imshow("imgEdge", imgEdge)
kernel = np.ones((10, 19), np.uint8)
imgEdge = cv.morphologyEx(imgEdge, cv.MORPH_CLOSE, kernel)
imgEdge = cv.morphologyEx(imgEdge, cv.MORPH_OPEN, kernel)
cv.imshow("imgEdgeProcessed", imgEdge) 3. 尋找包絡
有了上面的處理，尋找包絡就簡單多了。OpenCV的一個接口findContours就搞定！
# Step3: Find Contours
image, contours, hierarchy = cv.findContours(imgEdge, cv.RETR_TREE, cv.CHAIN_APPROX_SIMPLE)
contours = [cnt for cnt in contours if cv.contourArea(cnt) > self.minArea] 4. 删除一些實體尺寸不滿足的包絡
輪詢所有包絡，通過minAreaRect找到他們對應的最小矩形。先通過寬、高比來删除一些不符合條件的。
# Step4: Delete some rects
carPlateList = []
imgDark = np.zeros(img.shape, dtype = img.dtype)
for index, contour in enumerate(contours):
rect = cv.minAreaRect(contour) # [中心(x,y), (寬,高), 旋轉角度]
w, h = rect[1]
if w < h:
w, h = h, w
scale = w/h
if scale > 2 and scale < 4:
# color = (random.randint(0, 255), random.randint(0, 255), random.randint(0, 255))
color = (255, 255, 255)
carPlateList.append(rect)
cv.drawContours(imgDark, contours, index, color, 1, 8)
box = cv.boxPoints(rect) # Peak Coordinate
box = np.int0(box)
# Draw them out
cv.drawContours(imgDark, [box], 0, (0, 0, 255), 1)
cv.imshow("imgGaryContour", imgDark)
print("Vehicle number: ", len(carPlateList)) 5. 重映射
這裡做的是仿射變換，為什麼要做這個呢？原因是因為拍攝角度的原因，我們得到的矩形通常是由寫偏角的，這裡希望把它們擺正。
# Step5: Rect rectify
imgPlatList = []
for index, carPlat in enumerate(carPlateList):
if carPlat[2] > -1 and carPlat[2] < 1:
angle = 1
else:
angle = carPlat[2]
carPlat = (carPlat[0], (carPlat[1][0] + 5, carPlat[1][1] + 5), angle)
box = cv.boxPoints(carPlat)
# Which point is Left/Right/Top/Bottom
w, h = carPlat[1][0], carPlat[1][1]
if w > h:
LT = box[1]
LB = box[0]
RT = box[2]
RB = box[3]
else:
LT = box[2]
LB = box[1]
RT = box[3]
RB = box[0]
for point in [LT, LB, RT, RB]:
pointLimit(point, imgWidth, imgHeight)
# Do warpAffine
newLB = [LT[0], LB[1]]
newRB = [RB[0], LB[1]]
oldTriangle = np.float32([LT, LB, RB])
newTriangle = np.float32([LT, newLB, newRB])
warpMat = cv.getAffineTransform(oldTriangle, newTriangle)
imgAffine = cv.warpAffine(img, warpMat, (imgWidth, imgHeight))
cv.imshow("imgAffine" + str(index), imgAffine)
print("Index: ", index)
imgPlat = imgAffine[int(LT[1]):int(newLB[1]), int(newLB[0]):int(newRB[0])]
imgPlatList.append(imgPlat)
cv.imshow("imgPlat" + str(index), imgPlat) 需要注意的是這裡用了boxPoints接口擷取了矩形的四個點的坐标，需要通過這四個點坐标對應矩形的左上、右上、左下、右下四個點，才能給後面的warpAffine仿射變換做鋪墊。
函數 cv2.minAreaRect() 傳回一個Box2D結構rect：（最小外接矩形的中心（x，y），（寬度，高度），旋轉角度），但是要繪制這個矩形，我們需要矩形的4個頂點坐标box, 通過函數 cv2.cv.BoxPoints() 獲得，傳回形式[ [x0,y0], [x1,y1], [x2,y2], [x3,y3] ]。得到的最小外接矩形的4個頂點順序、中心坐标、寬度、高度、旋轉角度（是度數形式，不是弧度數）的對應關系如下：
6. 定車牌顔色
基本思路就是把上面重映射後的圖檔轉換到HSV空間，然後通過統計全部像素的個數以及單個顔色對應的個數，如果滿足藍色占了全部像素的1/3及以上的時候，就認為這是一個藍色車牌。
#Step6: Find correct place by color.
colorList = []
for index, imgPlat in enumerate(imgPlatList):
green = yellow = blue = 0
imgHsv = cv.cvtColor(imgPlat, cv.COLOR_BGR2HSV)
rows, cols = imgHsv.shape[:2]
imgSize = cols * rows
color = None
for row in range(rows):
for col in range(cols):
H = imgHsv.item(row, col, 0)
S = imgHsv.item(row, col, 1)
V = imgHsv.item(row, col, 2)
if 11 < H <= 34 and S > 34:
yellow += 1
elif 35 < H <= 99 and S > 34:
green += 1
elif 99 < H <= 124 and S > 34:
blue += 1
limit1 = limit2 = 0
if yellow * 3 >= imgSize:
color = "yellow"
limit1 = 11
limit2 = 34
elif green * 3 >= imgSize:
color = "green"
limit1 = 35
limit2 = 99
elif blue * 3 >= imgSize:
color = "blue"
limit1 = 100
limit2 = 124
print("Image Index[", index, '], Color：', color)
colorList.append(color)
print(blue, green, yellow, imgSize)
if color is None:
continue 附：
HSV空間下的顔色判斷關系表。
7. 根據顔色重新裁剪、篩選圖檔
我們知道了車牌顔色之後，就可以通過逐行、逐列掃描，把車牌精确到更小的範圍，這樣還可以通過寬高比剔除一些不正确的矩形，而且還得到了精确唯一車牌圖像内容！
def accurate_place(self, imgHsv, limit1, limit2, color):
rows, cols = imgHsv.shape[:2]
left = cols
right = 0
top = rows
bottom = 0
# rowsLimit = 21
rowsLimit = rows * 0.8 if color != "green" else rows * 0.5 # 綠色有漸變
colsLimit = cols * 0.8 if color != "green" else cols * 0.5 # 綠色有漸變
for row in range(rows):
count = 0
for col in range(cols):
H = imgHsv.item(row, col, 0)
S = imgHsv.item(row, col, 1)
V = imgHsv.item(row, col, 2)
if limit1 < H <= limit2 and 34 < S:# and 46 < V:
count += 1
if count > colsLimit:
if top > row:
top = row
if bottom < row:
bottom = row
for col in range(cols):
count = 0
for row in range(rows):
H = imgHsv.item(row, col, 0)
S = imgHsv.item(row, col, 1)
V = imgHsv.item(row, col, 2)
if limit1 < H <= limit2 and 34 < S:# and 46 < V:
count += 1
if count > rowsLimit:
if left > col:
left = col
if right < col:
right = col
return left, right, top, bottom
# Step7: Resize vehicle img.
left, right, top, bottom = self.accurate_place(imgHsv, limit1, limit2, color)
w = right - left
h = bottom - top
if left == right or top == bottom:
continue
scale = w/h
if scale < 2 or scale > 4:
continue
needAccurate = False
if top >= bottom:
top = 0
bottom = rows
needAccurate = True
if left >= right:
left = 0
right = cols
needAccurate = True
# imgPlat[index] = imgPlat[top:bottom, left:right] \
# if color != "green" or top < (bottom - top) // 4 \
# else imgPlat[top - (bottom - top) // 4:bottom, left:right]
imgPlatList[index] = imgPlat[top:bottom, left:right]
cv.imshow("Vehicle Image " + str(index), imgPlatList[index]) 好了，我們終于拿到了最終結果，下一步就是把這裡面的内容提取出來吧！
import cv2 as cv
import numpy as np
from numpy.linalg import norm
import matplotlib.pyplot as plt
import sys, os, json, random
class LPRAlg:
maxLength = 700
minArea = 2000
def __init__(self, imgPath = None):
if imgPath is None:
print("Please input correct path!")
return None
self.imgOri = cv.imread(imgPath)
if self.imgOri is None:
print("Cannot load this picture!")
return None
# cv.imshow("imgOri", self.imgOri)
def accurate_place(self, imgHsv, limit1, limit2, color):
rows, cols = imgHsv.shape[:2]
left = cols
right = 0
top = rows
bottom = 0
# rowsLimit = 21
rowsLimit = rows * 0.8 if color != "green" else rows * 0.5 # 綠色有漸變
colsLimit = cols * 0.8 if color != "green" else cols * 0.5 # 綠色有漸變
for row in range(rows):
count = 0
for col in range(cols):
H = imgHsv.item(row, col, 0)
S = imgHsv.item(row, col, 1)
V = imgHsv.item(row, col, 2)
if limit1 < H <= limit2 and 34 < S:# and 46 < V:
count += 1
if count > colsLimit:
if top > row:
top = row
if bottom < row:
bottom = row
for col in range(cols):
count = 0
for row in range(rows):
H = imgHsv.item(row, col, 0)
S = imgHsv.item(row, col, 1)
V = imgHsv.item(row, col, 2)
if limit1 < H <= limit2 and 34 < S:# and 46 < V:
count += 1
if count > rowsLimit:
if left > col:
left = col
if right < col:
right = col
return left, right, top, bottom
def findVehiclePlate(self):
def zoom(w, h, wMax, hMax):
# if w <= wMax and h <= hMax:
# return w, h
widthScale = 1.0 * wMax / w
heightScale = 1.0 * hMax / h
scale = min(widthScale, heightScale)
resizeWidth = int(w * scale)
resizeHeight = int(h * scale)
return resizeWidth, resizeHeight
def pointLimit(point, maxWidth, maxHeight):
if point[0] < 0:
point[0] = 0
if point[0] > maxWidth:
point[0] = maxWidth
if point[1] < 0:
point[1] = 0
if point[1] > maxHeight:
point[1] = maxHeight
if self.imgOri is None:
print("Please load picture frist!")
return False
# Step1: Resize
img = np.copy(self.imgOri)
h, w = img.shape[:2]
imgWidth, imgHeight = zoom(w, h, self.maxLength, self.maxLength)
print(w, h, imgWidth, imgHeight)
img =cv.resize(img, (imgWidth, imgHeight), interpolation=cv.INTER_AREA)
cv.imshow("imgResize", img)
# Step2: Prepare to find contours
img = cv.GaussianBlur(img, (3, 3), 0)
imgGary = cv.cvtColor(img, cv.COLOR_BGR2GRAY)
cv.imshow("imgGary", imgGary)
kernel = np.ones((20, 20), np.uint8)
imgOpen = cv.morphologyEx(imgGary, cv.MORPH_OPEN, kernel)
cv.imshow("imgOpen", imgOpen)
imgOpenWeight = cv.addWeighted(imgGary, 1, imgOpen, -1, 0)
cv.imshow("imgOpenWeight", imgOpenWeight)
ret, imgBin = cv.threshold(imgOpenWeight, 0, 255, cv.THRESH_OTSU + cv.THRESH_BINARY)
cv.imshow("imgBin", imgBin)
imgEdge = cv.Canny(imgBin, 100, 200)
cv.imshow("imgEdge", imgEdge)
kernel = np.ones((10, 19), np.uint8)
imgEdge = cv.morphologyEx(imgEdge, cv.MORPH_CLOSE, kernel)
imgEdge = cv.morphologyEx(imgEdge, cv.MORPH_OPEN, kernel)
cv.imshow("imgEdgeProcessed", imgEdge)
# Step3: Find Contours
image, contours, hierarchy = cv.findContours(imgEdge, cv.RETR_TREE, cv.CHAIN_APPROX_SIMPLE)
contours = [cnt for cnt in contours if cv.contourArea(cnt) > self.minArea]
# Step4: Delete some rects
carPlateList = []
imgDark = np.zeros(img.shape, dtype = img.dtype)
for index, contour in enumerate(contours):
rect = cv.minAreaRect(contour) # [中心(x,y), (寬,高), 旋轉角度]
w, h = rect[1]
if w < h:
w, h = h, w
scale = w/h
if scale > 2 and scale < 4:
# color = (random.randint(0, 255), random.randint(0, 255), random.randint(0, 255))
color = (255, 255, 255)
carPlateList.append(rect)
cv.drawContours(imgDark, contours, index, color, 1, 8)
box = cv.boxPoints(rect) # Peak Coordinate
box = np.int0(box)
# Draw them out
cv.drawContours(imgDark, [box], 0, (0, 0, 255), 1)
cv.imshow("imgGaryContour", imgDark)
print("Vehicle number: ", len(carPlateList))
# Step5: Rect rectify
imgPlatList = []
for index, carPlat in enumerate(carPlateList):
if carPlat[2] > -1 and carPlat[2] < 1:
angle = 1
else:
angle = carPlat[2]
carPlat = (carPlat[0], (carPlat[1][0] + 5, carPlat[1][1] + 5), angle)
box = cv.boxPoints(carPlat)
# Which point is Left/Right/Top/Bottom
w, h = carPlat[1][0], carPlat[1][1]
if w > h:
LT = box[1]
LB = box[0]
RT = box[2]
RB = box[3]
else:
LT = box[2]
LB = box[1]
RT = box[3]
RB = box[0]
for point in [LT, LB, RT, RB]:
pointLimit(point, imgWidth, imgHeight)
# Do warpAffine
newLB = [LT[0], LB[1]]
newRB = [RB[0], LB[1]]
oldTriangle = np.float32([LT, LB, RB])
newTriangle = np.float32([LT, newLB, newRB])
warpMat = cv.getAffineTransform(oldTriangle, newTriangle)
imgAffine = cv.warpAffine(img, warpMat, (imgWidth, imgHeight))
cv.imshow("imgAffine" + str(index), imgAffine)
print("Index: ", index)
imgPlat = imgAffine[int(LT[1]):int(newLB[1]), int(newLB[0]):int(newRB[0])]
imgPlatList.append(imgPlat)
cv.imshow("imgPlat" + str(index), imgPlat)
#Step6: Find correct place by color.
colorList = []
for index, imgPlat in enumerate(imgPlatList):
green = yellow = blue = 0
imgHsv = cv.cvtColor(imgPlat, cv.COLOR_BGR2HSV)
rows, cols = imgHsv.shape[:2]
imgSize = cols * rows
color = None
for row in range(rows):
for col in range(cols):
H = imgHsv.item(row, col, 0)
S = imgHsv.item(row, col, 1)
V = imgHsv.item(row, col, 2)
if 11 < H <= 34 and S > 34:
yellow += 1
elif 35 < H <= 99 and S > 34:
green += 1
elif 99 < H <= 124 and S > 34:
blue += 1
limit1 = limit2 = 0
if yellow * 3 >= imgSize:
color = "yellow"
limit1 = 11
limit2 = 34
elif green * 3 >= imgSize:
color = "green"
limit1 = 35
limit2 = 99
elif blue * 3 >= imgSize:
color = "blue"
limit1 = 100
limit2 = 124
print("Image Index[", index, '], Color：', color)
colorList.append(color)
print(blue, green, yellow, imgSize)
if color is None:
continue
# Step7: Resize vehicle img.
left, right, top, bottom = self.accurate_place(imgHsv, limit1, limit2, color)
w = right - left
h = bottom - top
if left == right or top == bottom:
continue
scale = w/h
if scale < 2 or scale > 4:
continue
needAccurate = False
if top >= bottom:
top = 0
bottom = rows
needAccurate = True
if left >= right:
left = 0
right = cols
needAccurate = True
# imgPlat[index] = imgPlat[top:bottom, left:right] \
# if color != "green" or top < (bottom - top) // 4 \
# else imgPlat[top - (bottom - top) // 4:bottom, left:right]
imgPlatList[index] = imgPlat[top:bottom, left:right]
cv.imshow("Vehicle Image " + str(index), imgPlatList[index])
if __name__ == '__main__':
L = LPRAlg("3.jfif")
L.findVehiclePlate()
cv.waitKey(0)