垃圾碩士的第一篇博文，最近可能不太做圖像處理這塊了，做個總結吧。我寫代碼的時間不長，代碼也非常的混亂，如果有想交流批評指教的同學留言就好。（害怕臉）
寒假前被老闆叫去做機械臂和雙目視覺（我專業是航天工程啊喂！），反正有的沒的做了做，雖然也遇到了一些問題但老實說都不是大問題，總之就是參考了很多大牛們的代碼和方法。結果這次彙報老闆居然對我說。。。說。。。你做的這些東西都是自己想做的吧。。。。。。人上了年紀記性都這麼好的嗎。。。怕是要把我這個學生給忘了。。。
我這個項目要求是通過雙目攝像頭來對物體進行定位，然後抓取，都做爛了哈哈（尴尬的微笑）。
對于雙目視覺，我個人的了解，雙目視覺分為這幾步：
1、擷取攝像頭圖像
2、雙目攝像頭标定
3、攝像頭畸變矯正
4、目辨別别
5、計算位置
---------------------------------------------------------分割線------------------------------------------------------
硬體裝置需求 ：
1、雙目攝像頭
雙目攝像頭是某寶上買的那種現成，插上直接識别為兩個攝像頭，還是蠻好用的。
2、一台你心愛的電腦，出bug或者速度太慢的時候不至于把它砸了。
3、一塊足夠平的标定闆，闆不平何以平天下！
---------------------------------------------------------分割線------------------------------------------------------
接下來是一步步的過程哈！
實作的軟體：主要由VS2017+OPENCV3.4完成，Matlab Calib工具箱進行标定工作。感謝這幫做開源的人，真心感謝！！！！！！！
1、擷取攝像頭圖像
單目相機直接用VideoCapture就能讀了，但是雙目要稍微注意一下順序（我的是筆記本電腦，如果是桌上型電腦大概就是0和1而不是1和2了）：
#define wi 320
#define he 240
VideoCapture camera0(1);
camera0.set(CV_CAP_PROP_FRAME_WIDTH, wi);
camera0.set(CV_CAP_PROP_FRAME_HEIGHT, he);
VideoCapture camera1(2);
camera1.set(CV_CAP_PROP_FRAME_WIDTH, wi);
camera1.set(CV_CAP_PROP_FRAME_HEIGHT, he);
攝像頭程式初始化如上，這邊值得注意的是如果我寫成：
#define wi 320
#define he 240
VideoCapture camera0(1);
VideoCapture camera1(2);
camera0.set(CV_CAP_PROP_FRAME_WIDTH, wi);
camera0.set(CV_CAP_PROP_FRAME_HEIGHT, he);
camera1.set(CV_CAP_PROP_FRAME_WIDTH, wi);
camera1.set(CV_CAP_PROP_FRAME_HEIGHT, he);
程式就會報錯。一定要設定好第一個攝像頭後立刻設定它的長寬。推測的原因是買的攝像頭是USB2.0，不支援太大的圖像傳輸，如果直接這樣一下子設定兩個會導緻圖像傳輸來不及。聽說DirectShow可以傳輸的更大一些，沒有試過，有畫質要求的同學可以試試。
然後就是截圖，把你舉着标定闆各種奇怪的姿勢拍下來。
while(1)
{
capture0 >> frame0;
capture1 >> frame1;
imshow("L",frame0);
imshow("R",frame1);
if (waitKey(30) == 's')
{
ss1 << i << "L.jpg" << endl;
ss1 >> str1;
imwrite(str1, frame0); //延時30ms
ss2 << i << "R.jpg" << endl;
ss2 >> str2;
imwrite(str2, frame1); //延時30ms
}
}
這段時截圖的代碼，按下‘s’就會截圖儲存。也是拿别人的代碼改的，渣渣碩cpp水準真的是0。。被matlab寵慣了根本不會寫這些代碼。。。
主要是截圖的程式被我不小心删掉了，，這裡就不貼出來了。放兩張舉着标定闆的英姿。
就像這樣各個角度多來幾張，大概L、R各20多張吧，就可以先把标定闆收起來了。以後看誰不爽就抄起标定闆莽他！
---------------------------------------------------------分割線------------------------------------------------------
2、相機标定
有請Matlab閃亮登場！（把握好登場的這段時間吧。。）這邊不用OPENCV做标定是因為看到很多資料說OPENCV的标定并不好用，而且以前我也用過Matlab這套标定，算是老相識了，确實比較好用。
用了Matlab Calib工具箱，這裡不多做贅述了，就是比較麻煩，而且要很小心！心有猛虎細嗅薔薇的機會來了！寫這塊标定的大神已經很多了，感謝他們！
這邊我對大神們做個補充，就是如何把Matlab得出的資料填入OPENCV所需的XML檔案中。
<?xml version="1.0"?>
<opencv_storage>
<calibrationDate>"2015.5.31 14:23"</calibrationDate>
<Intrinsic_Camera_L type_id="opencv-matrix">
<rows>3</rows>
<cols>3</cols>
<dt>d</dt>
<data>
2.3892563e+002 0. 1.5892866e+002
0. 2.3865967e+002 1.3122136e+002
0. 0. 1.
</data></Intrinsic_Camera_L>
<Intrinsic_Distortion_L type_id="opencv-matrix">
<rows>4</rows>
<cols>1</cols>
<dt>d</dt>
<data>
-0.43373 0.16967 0.00095 -0.00027
</data>
</Intrinsic_Distortion_L>
<Intrinsic_Camera_R type_id="opencv-matrix">
<rows>3</rows>
<cols>3</cols>
<dt>d</dt>
<data>
2.3892563e+002 0. 1.6092744e+002
0. 2.3865967e+002 1.2619615e+002
0. 0. 1.
</data>
</Intrinsic_Camera_R>
<Intrinsic_Distortion_R type_id="opencv-matrix">
<rows>4</rows>
<cols>1</cols>
<dt>d</dt>
<data>
-0.43376 0.16801 0.00007 -0.00042
</data>
</Intrinsic_Distortion_R>
<Extrinsic_Rotation_vector type_id="opencv-matrix">
<rows>3</rows>
<cols>1</cols>
<dt>d</dt>
<data>
0.00870 -0.00792 0.00152
</data>
</Extrinsic_Rotation_vector>
<Extrinsic_Translation_vector type_id="opencv-matrix">
<rows>3</rows>
<cols>1</cols>
<dt>d</dt>
<data>
-0.06184 -0.00025 0.00081
</data>
</Extrinsic_Translation_vector>
</opencv_storage>
裡面包括左相機内部參數，左相機畸變參數，右相機内部參數，右相機畸變參數，兩個相機的旋轉矩陣，平移矩陣。大家可以對号入座，把相應的參數填入這個檔案中。
---------------------------------------------------------分割線------------------------------------------------------
3、攝像頭畸變矯正
雖然不懂原理但是就是那麼用的。。
Mat cameraMatrix_Left = Mat::eye(3, 3, CV_64F);
cameraMatrix_Left.at<double>(0, 0) = 2.3892563e+002;
cameraMatrix_Left.at<double>(0, 2) = 1.5892866e+002;
cameraMatrix_Left.at<double>(1, 1) = 2.3865967e+002;
cameraMatrix_Left.at<double>(1, 2) = 1.3122136e+002;
Mat distCoeffs_Left = Mat::zeros(5, 1, CV_64F);
distCoeffs_Left.at<double>(0, 0) = -0.43373;
distCoeffs_Left.at<double>(1, 0) = 0.16967;
distCoeffs_Left.at<double>(2, 0) = 0.00095;
distCoeffs_Left.at<double>(3, 0) = -0.00027;
distCoeffs_Left.at<double>(4, 0) = 0;
Mat cameraMatrix_Right = Mat::eye(3, 3, CV_64F);
cameraMatrix_Right.at<double>(0, 0) = 2.3892563e+002;
cameraMatrix_Right.at<double>(0, 2) = 1.6092744e+002;
cameraMatrix_Right.at<double>(1, 1) = 2.3865967e+002;
cameraMatrix_Right.at<double>(1, 2) = 1.2619615e+002;
Mat distCoeffs_Right = Mat::zeros(5, 1, CV_64F);
distCoeffs_Right.at<double>(0, 0) = -0.43376;
distCoeffs_Right.at<double>(1, 0) = 0.16801;
distCoeffs_Right.at<double>(2, 0) = 0.00007;
distCoeffs_Right.at<double>(3, 0) = -0.00042;
distCoeffs_Right.at<double>(4, 0) = 0;
Mat R = Mat::zeros(3, 1, CV_64F);
Mat T = Mat::zeros(3, 1, CV_64F);
R.at<double>(0, 0) = 0.00870;
R.at<double>(1, 0) = -0.00792;
R.at<double>(2, 0) = 0.00152;
T.at<double>(0, 0) = -61.84;
T.at<double>(1, 0) = -0.00025;
T.at<double>(2, 0) = 0.0008;
camera0 >> frame0;
camera1 >> frame1;
Size imageSize;
imageSize = frame0.size();
Rect roi3, roi2;
Mat Q, R1, P1, R2, P2, BWL, BWR;
stereoRectify(cameraMatrix_Left, distCoeffs_Left, cameraMatrix_Right, distCoeffs_Right, imageSize, R, T, R1, R2, P1, P2, Q, CALIB_ZERO_DISPARITY, -1, imageSize, &roi2, &roi3);
initUndistortRectifyMap(cameraMatrix_Left, distCoeffs_Left, R1, P1, imageSize, CV_16SC2, map1, map2);
initUndistortRectifyMap(cameraMatrix_Right, distCoeffs_Right, R2, P2, imageSize, CV_16SC2, map3, map4);
while (true)
{
camera0 >> frame0;
camera1 >> frame1;
remap(frame1, frame1_ud, map1, map2, INTER_LINEAR);
remap(frame0, frame0_ud, map3, map4, INTER_LINEAR);
imshow("R", frame1_ud);
imshow("L", frame0_ud);
}
這邊我是抄了一個大腿的代碼，小小修改了一下，這邊我沒有直接把XML讀進來，而是自己定義了這些參數。。都行吧。。。
上點結果圖
---------------------------------------------------------分割線------------------------------------------------------
目辨別别+定位：
為了擷取目标物體的位置，就要知道物體在兩張圖像中的位置；為了知道物體在兩張圖像中的位置，就要能識别物體出來。在OPENCV下面大概是有這些方法來識别：1、霍夫變換找圓之類的，簡單快捷，效果一般2、SIFT，效果好到爆炸，速度也慢到爆炸，不适用于實時系統（18.4.11看到https://blog.csdn.net/chenyusiyuan/article/details/8715129裡面處理速度竟然能達到20ms一張圖，可能還是有點慢，但是已經很不錯了，改天試試）3、級聯分類器，速度挺快，就是訓練難度有點大。
這邊識别我拿霍夫找圓來做的，後面補充級聯分類器訓練的方法。因為這個效果說真的真不怎麼樣。。
cvtColor(frame1_ud, frame1_ud, CV_BGR2GRAY);
cvtColor(frame0_ud, frame0_ud, CV_BGR2GRAY);
HoughCircles(frame0_ud, circles0, CV_HOUGH_GRADIENT, 1.5, 10, 200, 100, 0, 0);
HoughCircles(frame1_ud, circles1, CV_HOUGH_GRADIENT, 1.5, 10, 200, 100, 0, 0);
for (size_t i = 0; i < circles0.size(); i++)
{
center_l.x = cvRound(circles0[i][0]);
center_l.y= cvRound(circles0[i][1]);
int radius = cvRound(circles0[i][2]);
//繪制圓心
circle(frame0_ud, center_l, 3, Scalar(0, 255, 0), -1, 8, 0);
//繪制圓輪廓
circle(frame0_ud, center_l, radius, Scalar(155, 50, 255), 3, 8, 0);
std::cout << center_l.x <<","<< center_l.y << endl;
}
for (size_t i = 0; i < circles1.size(); i++)
{
center_r.x = cvRound(circles1[i][0]);
center_r.y = cvRound(circles1[i][1]);
int radius = cvRound(circles1[i][2]);
//繪制圓心
circle(frame1_ud, center_r, 3, Scalar(0, 255, 0), -1, 8, 0);
//繪制圓輪廓
circle(frame1_ud, center_r, radius, Scalar(155, 50, 255), 3, 8, 0);
std::cout << center_r.x << "," << center_r.y << endl;
}
關于目标定位其實就是一個國中數學的計算，我是看了大神們的說明，然後還是自己算了算，用了自己算的公式：
int disparity = center_l.x -center_r.x;
//基線/視差 w = B/d
double w = -T.at<double>(0, 0) / disparity;
int m_point_3d_strcut_camZ = cameraMatrix_Left.at<double>(0, 0) * w;
int m_point_3d_strcut_camX = (center_l.x - 160) * w ;
int m_point_3d_strcut_camY = (120 - center_l.y ) * w ;
std::cout << "at " << m_point_3d_strcut_camX << "," << m_point_3d_strcut_camY << "," << m_point_3d_strcut_camZ << endl;
順便提一句，我一開始用BM，SGBM等雙目比對算法效果很不好，不知道為什麼，是以才出此下策做了這個識别來确定位置。
這個是效果，at後面是物體的XYZ坐标。x,y軸不好測隻測了z軸（即深度）的精度，老實說精度其實還可以！
下面是我識别的全部代碼，因為當時試了比較多的方案又不舍得删掉，導緻代碼看起來很長其實有用的很少，如果能幫到大家那再好不過了。
#include <opencv2/opencv.hpp>
#include "opencv2/calib3d/calib3d.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/highgui/highgui.hpp"
#define wi 320
#define he 240
using namespace std;
using namespace cv;
stringstream ss1, ss2;
string str1, str2;
int i = 0;
CvMat *m_IntrinsicMat_L; //左相機 - 内參矩陣
CvMat *m_DistortionMat_L; //左相機 - 扭曲矩陣
CvMat *m_IntrinsicMat_R;
CvMat *m_DistortionMat_R;
CvMat *m_TranslateMat; //平移矩陣
double m_F_L; //左相機焦距
double m_T; //基線長
double m_Cx_L;
Rect leftROI, rightROI;
Mat disparity,xyz;
Mat imgDisparity8U, imgDisparity16S;
int flag = true;
Point center_l, center_r;
void detectAndDraw_l(Mat& img, CascadeClassifier& cascade,
CascadeClassifier& nestedCascade,
double scale, bool tryflip);
void detectAndDraw_r(Mat& img, CascadeClassifier& cascade,
CascadeClassifier& nestedCascade,
double scale, bool tryflip);
//void onMouse(int event, int x, int y, int flags, void* param);
int main()
{
//initialize and allocate memory to load the video stream from camera
VideoCapture camera0(1);
camera0.set(CV_CAP_PROP_FRAME_WIDTH, wi);
camera0.set(CV_CAP_PROP_FRAME_HEIGHT, he);
VideoCapture camera1(2);
camera1.set(CV_CAP_PROP_FRAME_WIDTH, wi);
camera1.set(CV_CAP_PROP_FRAME_HEIGHT, he);
Mat frame0, frame1, frame0_ud, frame1_ud, map1, map2, map3, map4;
if (!camera0.isOpened()) return 1;
if (!camera1.isOpened()) return 1;
CascadeClassifier cascade, nestedCascade;
bool stop = false;
//訓練好的檔案名稱，放置在可執行檔案同目錄下
cascade.load("haarcascade_frontalface_alt.xml");
nestedCascade.load("haarcascade_eye.xml");
//CvFileStorage* fs = cvOpenFileStorage("/home/calib.xml", 0, CV_STORAGE_READ);
//m_IntrinsicMat_L = (CvMat *)cvReadByName(fs, 0, "Intrinsic_Camera_L"); //相機内參矩陣
//m_DistortionMat_L = (CvMat *)cvReadByName(fs, 0, "Intrinsic_Distortion_L"); //扭曲矩陣
//m_IntrinsicMat_R = (CvMat *)cvReadByName(fs, 0, "Intrinsic_Camera_R");
//m_DistortionMat_R = (CvMat *)cvReadByName(fs, 0, "Intrinsic_Distortion_R");
//m_TranslateMat = (CvMat *)cvReadByName(fs, 0, "Extrinsic_Translation_vector");
//m_T = CV_MAT_ELEM(*m_TranslateMat, double, 0, 0) * -1; //基線
//m_F_L = CV_MAT_ELEM(*m_IntrinsicMat_L, double, 0, 0); //左相機焦距
//m_Cx_L = CV_MAT_ELEM(*m_IntrinsicMat_L, double, 0, 2);
Mat cameraMatrix_Left = Mat::eye(3, 3, CV_64F);
cameraMatrix_Left.at<double>(0, 0) = 2.3892563e+002;
cameraMatrix_Left.at<double>(0, 2) = 1.5892866e+002;
cameraMatrix_Left.at<double>(1, 1) = 2.3865967e+002;
cameraMatrix_Left.at<double>(1, 2) = 1.3122136e+002;
Mat distCoeffs_Left = Mat::zeros(5, 1, CV_64F);
distCoeffs_Left.at<double>(0, 0) = -0.43373;
distCoeffs_Left.at<double>(1, 0) = 0.16967;
distCoeffs_Left.at<double>(2, 0) = 0.00095;
distCoeffs_Left.at<double>(3, 0) = -0.00027;
distCoeffs_Left.at<double>(4, 0) = 0;
Mat cameraMatrix_Right = Mat::eye(3, 3, CV_64F);
cameraMatrix_Right.at<double>(0, 0) = 2.3892563e+002;
cameraMatrix_Right.at<double>(0, 2) = 1.6092744e+002;
cameraMatrix_Right.at<double>(1, 1) = 2.3865967e+002;
cameraMatrix_Right.at<double>(1, 2) = 1.2619615e+002;
Mat distCoeffs_Right = Mat::zeros(5, 1, CV_64F);
distCoeffs_Right.at<double>(0, 0) = -0.43376;
distCoeffs_Right.at<double>(1, 0) = 0.16801;
distCoeffs_Right.at<double>(2, 0) = 0.00007;
distCoeffs_Right.at<double>(3, 0) = -0.00042;
distCoeffs_Right.at<double>(4, 0) = 0;
Mat R = Mat::zeros(3, 1, CV_64F);
Mat T = Mat::zeros(3, 1, CV_64F);
R.at<double>(0, 0) = 0.00870;
R.at<double>(1, 0) = -0.00792;
R.at<double>(2, 0) = 0.00152;
T.at<double>(0, 0) = -61.84;
T.at<double>(1, 0) = -0.00025;
T.at<double>(2, 0) = 0.0008;
int ndisparities = 16; /**< Range of disparity */
int SADWindowSize = 9; /**< Size of the block window. Must be odd */
int slider_pos=75;
Mat alineImgL;
vector<Vec3f> circles0;
vector<Vec3f> circles1;
camera0 >> frame0;
camera1 >> frame1;
Size imageSize;
imageSize = frame0.size();
Rect roi3, roi2;
Mat Q, R1, P1, R2, P2, BWL, BWR;
stereoRectify(cameraMatrix_Left, distCoeffs_Left, cameraMatrix_Right, distCoeffs_Right, imageSize, R, T, R1, R2, P1, P2, Q, CALIB_ZERO_DISPARITY, -1, imageSize, &roi2, &roi3);
initUndistortRectifyMap(cameraMatrix_Left, distCoeffs_Left, R1, P1, imageSize, CV_16SC2, map1, map2);
initUndistortRectifyMap(cameraMatrix_Right, distCoeffs_Right, R2, P2, imageSize, CV_16SC2, map3, map4);
// std::cout << frame1.rows() << std::endl;
//wait for 40 milliseconds
while (true)
{
camera0 >> frame0;
camera1 >> frame1;
remap(frame1, frame1_ud, map1, map2, INTER_LINEAR);
remap(frame0, frame0_ud, map3, map4, INTER_LINEAR);
cvtColor(frame1_ud, frame1_ud, CV_BGR2GRAY);
cvtColor(frame0_ud, frame0_ud, CV_BGR2GRAY);
//Mat roi1 = frame1_ud;
//Mat roi0 = frame0_ud;
//threshold(frame0_ud, BWL ,slider_pos , 255 , THRESH_BINARY);
//threshold(frame1_ud, BWR ,slider_pos , 255 , THRESH_BINARY);
//Mat roi1(frame1_ud, Rect(30, 30, 260, 180));
//Mat roi0(frame0_ud, Rect(30, 30, 260, 180));
//GaussianBlur(roi0 ,roi0, Size(7, 7), 2, 2);
//GaussianBlur(roi1 ,roi1, Size(7, 7), 2, 2);
HoughCircles(frame0_ud, circles0, CV_HOUGH_GRADIENT, 1.5, 10, 200, 100, 0, 0);
HoughCircles(frame1_ud, circles1, CV_HOUGH_GRADIENT, 1.5, 10, 200, 100, 0, 0);
for (size_t i = 0; i < circles0.size(); i++)
{
center_l.x = cvRound(circles0[i][0]);
center_l.y= cvRound(circles0[i][1]);
int radius = cvRound(circles0[i][2]);
//繪制圓心
circle(frame0_ud, center_l, 3, Scalar(0, 255, 0), -1, 8, 0);
//繪制圓輪廓
circle(frame0_ud, center_l, radius, Scalar(155, 50, 255), 3, 8, 0);
std::cout << center_l.x <<","<< center_l.y << endl;
}
for (size_t i = 0; i < circles1.size(); i++)
{
center_r.x = cvRound(circles1[i][0]);
center_r.y = cvRound(circles1[i][1]);
int radius = cvRound(circles1[i][2]);
//繪制圓心
circle(frame1_ud, center_r, 3, Scalar(0, 255, 0), -1, 8, 0);
//繪制圓輪廓
circle(frame1_ud, center_r, radius, Scalar(155, 50, 255), 3, 8, 0);
std::cout << center_r.x << "," << center_r.y << endl;
}
//detectAndDraw_l(roi0, cascade, nestedCascade, 1, 0);
//detectAndDraw_r(roi1, cascade, nestedCascade, 1, 0);
//————————————————————————以下是BM算法————————————————————————————
//imgDisparity16S = Mat(roi1.rows, roi1.cols, CV_16S);
//imgDisparity8U = Mat(roi1.rows, roi1.cols, CV_8UC1);
//Ptr<StereoBM> bm = StereoBM::create(64, 21);
bm->setPreFilterType(CV_STEREO_BM_XSOBEL);
bm->setPreFilterSize(9);
bm->setPreFilterCap(20);
//bm->setBlockSize(7);//1,15 2,21
bm->setMinDisparity(-16);//1,0
//bm->setNumDisparities(64);//2,64
//bm->setTextureThreshold(10);
//bm->setUniquenessRatio(20);//2,8
//bm->setSpeckleWindowSize(100);
//bm->setSpeckleRange(32);
//bm->setROI1(roi2);
//bm->setROI2(roi3);
-- 3. Calculate the disparity image
//bm->compute(roi1, roi0, imgDisparity16S);
-- Check its extreme values
//double minVal; double maxVal;
//minMaxLoc(imgDisparity16S, &minVal, &maxVal);
//printf("X: %d Y: %d \n", center.x, center.y);
-- 4. Display it as a CV_8UC1 image
//imgDisparity16S.convertTo(imgDisparity8U, CV_8UC1, 255 / (maxVal - minVal));
//reprojectImageTo3D(imgDisparity16S, xyz, Q, true);
//xyz = xyz * 16;
//cout << "in world coordinate is: " << xyz.at<Vec3f>(center) << endl;
//namedWindow("Disparity", 0);
setMouseCallback("Disparity", onMouse, reinterpret_cast<void*> (&alineImgL));
//imshow("Disparity", imgDisparity8U);
//——————————————————————————————————————————————————
//————————————————————————以下是SGBM算法————————————————————————————
//Mat imgDisparity16S = Mat(roi1.rows, roi1.cols, CV_16S);
//Mat imgDisparity8U = Mat(roi1.rows, roi1.cols, CV_8UC1);
//Ptr<StereoSGBM> sgbm = StereoSGBM::create(0,16,3,0,0,0,0,0,0,0,StereoSGBM::MODE_SGBM); //int minDisparity = 0,
// // int numDisparities = 16,
// // int blockSize = 3,
// // int P1 = 0,
// // int P2 = 0,
// // int disp12MaxDiff = 0,
// // int preFilterCap = 0,
// // int uniquenessRatio = 0,
// // int speckleWindowSize = 0,
// // int speckleRange = 0,
// // int mode = StereoSGBM::MODE_SGBM
//sgbm->setP1(8 * SADWindowSize*SADWindowSize);
//sgbm->setP2(32 * SADWindowSize*SADWindowSize);
//
//sgbm->setUniquenessRatio ( 10);
//sgbm->setSpeckleWindowSize ( 100);
//sgbm->setSpeckleRange(32);
//sgbm->setDisp12MaxDiff( 1);
//sgbm->compute(roi1, roi0, imgDisparity16S);
//double minVal; double maxVal;
//minMaxLoc(imgDisparity16S, &minVal, &maxVal);
//printf("Min disp: %f Max value: %f \n", minVal, maxVal);
//cv::normalize(imgDisparity16S, imgDisparity8U, 0, 256, cv::NORM_MINMAX, CV_8U);
-- 4. Display it as a CV_8UC1 image
//namedWindow("Disparity", 0);
//setMouseCallback("Disparity", onMouse, reinterpret_cast<void*> (&alineImgL));
//imshow("Disparity", imgDisparity8U);
//——————————————————————————————————————————————————
//————————————————————————以下是不比對直接計算的方法————————————————————————————
int disparity = center_l.x -center_r.x;
//基線/視差 w = B/d
double w = -T.at<double>(0, 0) / disparity;
int m_point_3d_strcut_camZ = cameraMatrix_Left.at<double>(0, 0) * w/1.5;
int m_point_3d_strcut_camX = (center_l.x - 160) * w ;
int m_point_3d_strcut_camY = (120 - center_l.y ) * w ;
std::cout << "at " << m_point_3d_strcut_camX << "," << m_point_3d_strcut_camY << "," << m_point_3d_strcut_camZ << endl;
//——————————————————————————————————————————————————
imshow("R", frame1_ud);
imshow("L", frame0_ud);
//imshow("R_Undistort", roi1);
//imshow("L_Undistort", roi0);
/*imshow("disp", disparity);*/
//grab and retrieve each frames of the video sequentially
if (waitKey(10) == 's')
{
ss1 << i << "R.jpg" << endl;
ss1 >> str1;
imwrite(str1, frame1); //延時30ms
ss1 << i << "R_Undistort.jpg" << endl;
ss1 >> str1;
imwrite(str1, frame1_ud); //延時30ms
ss2 << i << "L.jpg" << endl;
ss2 >> str2;
imwrite(str2, frame0); //延時30ms
ss2 << i << "L_Undistort.jpg" << endl;
ss2 >> str2;
imwrite(str2, frame0_ud); //延時30ms
i++;
}
}
return 0;
}
//void onMouse(int event, int x, int y, int flags, void * param)
//{
// int valuess;
// Mat *im = reinterpret_cast<Mat*>(param);
// switch (event)
// {
// case CV_EVENT_LBUTTONDOWN: //滑鼠左鍵按下響應：傳回坐标和灰階
// {
// valuess = imgDisparity8U.at<uchar>(x, y);
// std::cout << "at(" << x << "," << y << ")value is: SGM: " << valuess
// << endl;
// }
//
//
// flag = false;
// break;
// }
//}
void detectAndDraw_l(Mat& img, CascadeClassifier& cascade,
CascadeClassifier& nestedCascade,
double scale, bool tryflip)
{
int i = 0;
double t = 0;
//建立用于存放人臉的向量容器
vector<Rect> faces, faces2;
//定義一些顔色，用來标示不同的人臉
const static Scalar colors[] = {
CV_RGB(0,0,255),
CV_RGB(0,128,255),
CV_RGB(0,255,255),
CV_RGB(0,255,0),
CV_RGB(255,128,0),
CV_RGB(255,255,0),
CV_RGB(255,0,0),
CV_RGB(255,0,255) };
//建立縮小的圖檔，加快檢測速度
//nt cvRound (double value) 對一個double型的數進行四舍五入，并傳回一個整型數！
Mat gray, smallImg(cvRound(img.rows / scale), cvRound(img.cols / scale), CV_8UC1);
//轉成灰階圖像，Harr特征基于灰階圖
//cvtColor(img, gray, CV_BGR2GRAY);
gray = img;
//改變圖像大小，使用雙線性內插補點
resize(gray, smallImg, smallImg.size(), 0, 0, INTER_LINEAR);
//imshow("縮小尺寸", smallImg);
//變換後的圖像進行直方圖均值化處理
equalizeHist(smallImg, smallImg);
//imshow("直方圖均值處理", smallImg);
//程式開始和結束插入此函數擷取時間，經過計算求得算法執行時間
t = (double)cvGetTickCount();
//檢測人臉
//detectMultiScale函數中smallImg表示的是要檢測的輸入圖像為smallImg，faces表示檢測到的人臉目标序列，1.1表示
//每次圖像尺寸減小的比例為1.1，2表示每一個目标至少要被檢測到3次才算是真的目标(因為周圍的像素和不同的視窗大
//小都可以檢測到人臉),CV_HAAR_SCALE_IMAGE表示不是縮放分類器來檢測，而是縮放圖像，Size(30, 30)為目标的
//最小最大尺寸
cascade.detectMultiScale(smallImg, faces,
1.1, 2, 0
//|CV_HAAR_FIND_BIGGEST_OBJECT
//|CV_HAAR_DO_ROUGH_SEARCH
| CV_HAAR_SCALE_IMAGE
, Size(30, 30));
//如果使能，翻轉圖像繼續檢測
if (tryflip)
{
flip(smallImg, smallImg, 1);
//imshow("反轉圖像", smallImg);
cascade.detectMultiScale(smallImg, faces2,
1.1, 2, 0
//|CV_HAAR_FIND_BIGGEST_OBJECT
//|CV_HAAR_DO_ROUGH_SEARCH
| CV_HAAR_SCALE_IMAGE
, Size(30, 30));
for (vector<Rect>::const_iterator r = faces2.begin(); r != faces2.end(); r++)
{
faces.push_back(Rect(smallImg.cols - r->x - r->width, r->y, r->width, r->height));
}
}
t = (double)cvGetTickCount() - t;
// qDebug( "detection time = %g ms\n", t/((double)cvGetTickFrequency()*1000.) );
for (vector<Rect>::const_iterator r = faces.begin(); r != faces.end(); r++, i++)
{
Mat smallImgROI, img_crop;
vector<Rect> nestedObjects;
Point pt1, pt2;
Scalar color = colors[i % 8];
int radius;
double aspect_ratio = (double)r->width / r->height;
if (0.75 < aspect_ratio && aspect_ratio < 1.3)
{
//标示人臉時在縮小之前的圖像上标示，是以這裡根據縮放比例換算回去
pt1.x = cvRound(r->x*scale);
pt1.y = cvRound(r->y*scale);
pt2.x = cvRound(r->x*scale + r->width*scale);
pt2.y = cvRound(r->y*scale + r->height*scale);
center_l.x = cvRound(r->x*scale + r->width*scale/2);
center_l.y = cvRound(r->y*scale + r->height*scale/2);
//radius = cvRound((r->width + r->height)*0.25*scale);
//img_crop = img(Range(pt1.y, pt2.y), Range(pt1.x, pt2.x));
//circle(img, center, radius, color, 3, 8, 0);
cv::rectangle(img, pt1, pt2, Scalar(255, 0, 0), 2);
}
else
rectangle(img, cvPoint(cvRound(r->x*scale), cvRound(r->y*scale)),
cvPoint(cvRound((r->x + r->width - 1)*scale), cvRound((r->y + r->height - 1)*scale)),
color, 3, 8, 0);
//if (nestedCascade.empty())
// continue;
//smallImgROI = smallImg(*r);
同樣方法檢測人眼
//nestedCascade.detectMultiScale(smallImgROI, nestedObjects,
// 1.1, 2, 0
// //|CV_HAAR_FIND_BIGGEST_OBJECT
// //|CV_HAAR_DO_ROUGH_SEARCH
// //|CV_HAAR_DO_CANNY_PRUNING
// | CV_HAAR_SCALE_IMAGE
// , Size(30, 30));
//for (vector<Rect>::const_iterator nr = nestedObjects.begin(); nr != nestedObjects.end(); nr++)
//{
// center.x = cvRound((r->x + nr->x + nr->width*0.5)*scale);
// center.y = cvRound((r->y + nr->y + nr->height*0.5)*scale);
// radius = cvRound((nr->width + nr->height)*0.25*scale);
// circle(img, center, radius, color, 3, 8, 0);
//}
}
imshow("識别結果", img);
}
void detectAndDraw_r(Mat& img, CascadeClassifier& cascade,
CascadeClassifier& nestedCascade,
double scale, bool tryflip)
{
int i = 0;
double t = 0;
//建立用于存放人臉的向量容器
vector<Rect> faces, faces2;
//定義一些顔色，用來标示不同的人臉
const static Scalar colors[] = {
CV_RGB(0,0,255),
CV_RGB(0,128,255),
CV_RGB(0,255,255),
CV_RGB(0,255,0),
CV_RGB(255,128,0),
CV_RGB(255,255,0),
CV_RGB(255,0,0),
CV_RGB(255,0,255) };
//建立縮小的圖檔，加快檢測速度
//nt cvRound (double value) 對一個double型的數進行四舍五入，并傳回一個整型數！
Mat gray, smallImg(cvRound(img.rows / scale), cvRound(img.cols / scale), CV_8UC1);
//轉成灰階圖像，Harr特征基于灰階圖
//cvtColor(img, gray, CV_BGR2GRAY);
gray = img;
//imshow("灰階", gray);
//改變圖像大小，使用雙線性內插補點
resize(gray, smallImg, smallImg.size(), 0, 0, INTER_LINEAR);
//imshow("縮小尺寸", smallImg);
//變換後的圖像進行直方圖均值化處理
equalizeHist(smallImg, smallImg);
//imshow("直方圖均值處理", smallImg);
//程式開始和結束插入此函數擷取時間，經過計算求得算法執行時間
t = (double)cvGetTickCount();
//檢測人臉
//detectMultiScale函數中smallImg表示的是要檢測的輸入圖像為smallImg，faces表示檢測到的人臉目标序列，1.1表示
//每次圖像尺寸減小的比例為1.1，2表示每一個目标至少要被檢測到3次才算是真的目标(因為周圍的像素和不同的視窗大
//小都可以檢測到人臉),CV_HAAR_SCALE_IMAGE表示不是縮放分類器來檢測，而是縮放圖像，Size(30, 30)為目标的
//最小最大尺寸
cascade.detectMultiScale(smallImg, faces,
1.1, 2, 0
//|CV_HAAR_FIND_BIGGEST_OBJECT
//|CV_HAAR_DO_ROUGH_SEARCH
| CV_HAAR_SCALE_IMAGE
, Size(30, 30));
//如果使能，翻轉圖像繼續檢測
if (tryflip)
{
flip(smallImg, smallImg, 1);
//imshow("反轉圖像", smallImg);
cascade.detectMultiScale(smallImg, faces2,
1.1, 2, 0
//|CV_HAAR_FIND_BIGGEST_OBJECT
//|CV_HAAR_DO_ROUGH_SEARCH
| CV_HAAR_SCALE_IMAGE
, Size(30, 30));
for (vector<Rect>::const_iterator r = faces2.begin(); r != faces2.end(); r++)
{
faces.push_back(Rect(smallImg.cols - r->x - r->width, r->y, r->width, r->height));
}
}
t = (double)cvGetTickCount() - t;
// qDebug( "detection time = %g ms\n", t/((double)cvGetTickFrequency()*1000.) );
for (vector<Rect>::const_iterator r = faces.begin(); r != faces.end(); r++, i++)
{
Mat smallImgROI, img_crop;
vector<Rect> nestedObjects;
Point pt1, pt2;
Scalar color = colors[i % 8];
int radius;
double aspect_ratio = (double)r->width / r->height;
if (0.75 < aspect_ratio && aspect_ratio < 1.3)
{
//标示人臉時在縮小之前的圖像上标示，是以這裡根據縮放比例換算回去
pt1.x = cvRound(r->x*scale);
pt1.y = cvRound(r->y*scale);
pt2.x = cvRound(r->x*scale + r->width*scale);
pt2.y = cvRound(r->y*scale + r->height*scale);
center_r.x = cvRound(r->x*scale + r->width*scale / 2);
center_r.y = cvRound(r->y*scale + r->height*scale / 2);
//radius = cvRound((r->width + r->height)*0.25*scale);
//img_crop = img(Range(pt1.y, pt2.y), Range(pt1.x, pt2.x));
//circle(img, center, radius, color, 3, 8, 0);
cv::rectangle(img, pt1, pt2, Scalar(255, 0, 0), 2);
}
else
rectangle(img, cvPoint(cvRound(r->x*scale), cvRound(r->y*scale)),
cvPoint(cvRound((r->x + r->width - 1)*scale), cvRound((r->y + r->height - 1)*scale)),
color, 3, 8, 0);
//if (nestedCascade.empty())
// continue;
//smallImgROI = smallImg(*r);
同樣方法檢測人眼
//nestedCascade.detectMultiScale(smallImgROI, nestedObjects,
// 1.1, 2, 0
// //|CV_HAAR_FIND_BIGGEST_OBJECT
// //|CV_HAAR_DO_ROUGH_SEARCH
// //|CV_HAAR_DO_CANNY_PRUNING
// | CV_HAAR_SCALE_IMAGE
// , Size(30, 30));
//for (vector<Rect>::const_iterator nr = nestedObjects.begin(); nr != nestedObjects.end(); nr++)
//{
// center.x = cvRound((r->x + nr->x + nr->width*0.5)*scale);
// center.y = cvRound((r->y + nr->y + nr->height*0.5)*scale);
// radius = cvRound((nr->width + nr->height)*0.25*scale);
// circle(img, center, radius, color, 3, 8, 0);
//}
}
imshow("識别結果", img);
}
裡面人臉檢測的部分是用來試級聯分類器好不好用的，那到底好不好用呢，反正OPENCV自帶的那幾個人臉分類器特别好用，我自己訓練的特别不好用，可能也是因為我的正樣本有點少。
---------------------------------------------------------分割線------------------------------------------------------
級聯分類器我基本參考了https://blog.csdn.net/qq_27063119/article/details/79247266大神的文章，大神介紹的很詳細了，由于不會java，我寫了matlab的幾個腳本作為大神的代替。
批量圖檔重命名：
Files = dir(fullfile('D:\matlab folder\opencvca_file','*.jpg'));
time=size(Files);
for i=1:time(1)
oldfilename=Files(i).name;
newfilename=[num2str(i),'.jpg'];
system(['rename ' oldfilename ' ' newfilename]);
end
制作灰色正樣本圖（讀取檔案夾下全部的圖檔，截取出正樣本，并儲存為灰色圖檔）：
Files = dir(fullfile('D:\matlab folder\cul_area','*.jpg'));
time=size(Files);
for i=1:time(1)
A=imread(Files(i).name);
I=rgb2gray(A);
imshow(I);
[x,y] = ginput(2);%确定圖像上的兩點利用ginput函數，傳回值是兩點的坐标
I = imcrop(I,[x(1),y(1),abs(x(1)-x(2)),abs(y(1)-y(2))]);
I=imresize(I,[40 40],'nearest');
newfilename=[num2str(i),'.jpg'];
imwrite(I,newfilename);
end
制作正樣本txt
for i=1:24
a=['pos/',num2str(i),'.jpg 1 0 0 40 40'];
fid=fopen('pos.txt','a');
fprintf(fid,'%s\r',a);
fclose(fid);
end
制作負樣本txt
for i=1:2483
a=['neg/',num2str(i),'.jpg'];
fid=fopen('neg.txt','a');
fprintf(fid,'%s\r',a);
fclose(fid);
end
但是效果一般，識别效果上圖
隻能識别出來一個。。。
大緻就是以上了！大家交流！！