企业官网建设流程全解析

做网站公司深,如何做一个app软件需要多少钱,沙田镇网站建设公司,asp作业做购物网站代码文章目录 1 简介2 传统机器视觉的手势检测2.1 轮廓检测法2.2 算法结果2.3 整体代码实现2.3.1 算法流程 3 深度学习方法做手势识别3.1 经典的卷积神经网络3.2 YOLO系列3.3 SSD3.4 实现步骤3.4.1 数据集3.4.2 图像预处理3.4.3 构建卷积神经网络结构3.4.4 实验训练过程及结果 3.5 …文章目录1 简介2 传统机器视觉的手势检测2.1 轮廓检测法2.2 算法结果2.3 整体代码实现2.3.1 算法流程3 深度学习方法做手势识别3.1 经典的卷积神经网络3.2 YOLO系列3.3 SSD3.4 实现步骤3.4.1 数据集3.4.2 图像预处理3.4.3 构建卷积神经网络结构3.4.4 实验训练过程及结果3.5 关键代码4 实现手势交互1 简介今天学长向大家介绍一个机器视觉项目基于机器视觉opencv的手势检测 手势识别 算法选题指导, 项目分享见文末2 传统机器视觉的手势检测普通机器视觉手势检测的基本流程如下其中轮廓的提取多边形拟合曲线的求法凸包集和凹陷集的求法都是采用opencv中自带的函数。手势数字的识别是利用凸包点以及凹陷点和手部中心点的几何关系简单的做了下逻辑判别了可以肯定的是这种方法很烂具体的做法是先在手部定位出2个中心点坐标这2个中心点坐标之间的距离阈值由程序设定其中一个中心点就是利用OpenNI跟踪得到的手部位置。有了这2个中心点的坐标在程序中就可以分别计算出在这2个中心点坐标上的凸凹点的个数。当然了这样做的前提是用人在做手势表示数字的同时应该是将手指的方向朝上因为没有像机器学习那样通过样本来训练所以使用时条件要苛刻很多。利用上面求出的4种点的个数(另外程序中还设置了2个辅助计算点的个数具体见代码部分)和简单的逻辑判断就可以识别出数字0~5了。其它的数字可以依照具体的逻辑去设计还可以设计出多位数字的识别只是数字越多设计起来越复杂因为要考虑到它们之间的干扰性且这种不通用的设计方法也没有太多的实际意义。2.1 轮廓检测法使用 void convexityDefects(InputArray contour, InputArray convexhull, OutputArray convexityDefects) 方法该函数的作用是对输入的轮廓contour凸包集合来检测其轮廓的凸型缺陷一个凸型缺陷结构体包括4个元素缺陷起点坐标缺陷终点坐标缺陷中离凸包线距离最远的点的坐标以及此时最远的距离。参数3即其输出的凸型缺陷结构体向量。其凸型缺陷的示意图如下所示第1个参数虽然写的是contour字面意思是轮廓但是本人实验过很多次发现如果该参数为目标通过轮廓检测得到的原始轮廓的话则程序运行到onvexityDefects()函数时会报内存错误。因此本程序中采用的不是物体原始的轮廓而是经过多项式曲线拟合后的轮廓即多项式曲线这样程序就会顺利地运行得很好。另外由于在手势识别过程中可能某一帧检测出来的轮廓非常小由于某种原因以致于少到只有1个点这时候如果程序运行到onvexityDefects()函数时就会报如下的错误intMat::checkVector(int_elemChannels,int_depth,bool _requireContinuous)const{return(depth()_depth||_depth0)(isContinuous()||!_requireContinuous)((dims2(((rows1||cols1)channels()_elemChannels)||(cols_elemChannels)))||(dims3channels()1size.p[2]_elemChannels(size.p[0]1||size.p[1]1)(isContinuous()||step.p[1]step.p[2]*size.p[2])))?(int)(total()*channels()/_elemChannels):-1;}该函数源码大概意思就是说对应的Mat矩阵如果其深度连续性通道数行列式满足一定条件的话就返回Mat元素的个数和其通道数的乘积否则返回-1而本文是要求其返回值大于3有得知此处输入多边形曲线(即参数1)的通道数为2所以还需要求其元素的个数大于1.5即大于2才满足ptnum 3。简单的说就是用convexityDefects()函数来对多边形曲线进行凹陷检测时必须要求参数1曲线本身至少有2个点(也不知道这样分析对不对)。因此本人在本次程序convexityDefects()函数前加入了if(Mat(approx_poly_curve).checkVector(2, CV_32S) 3)来判断只有满足该if条件才会进行后面的凹陷检测。这样程序就不会再出现类似的bug了。第2个参数一般是由opencv中的函数convexHull()获得的一般情况下该参数里面存的是凸包集合中的点在多项式曲线点中的位置索引且该参数以vector的形式存在因此参数convexhull中其元素的类型为unsigned int。在本次凹陷点检测函数convexityDefects()里面根据文档要求该参数为Mat型。因此在使用convexityDefects()的参数2时一般将vector直接转换Mat型。参数3是一个含有4个元素的结构体的集合如果在c的版本中该参数可以直接用vector来代替Vec4i中的4个元素分别表示凹陷曲线段的起始坐标索引终点坐标索引离凸包集曲线最远点的坐标索引以及此时的最远距离值这4个值都是整数。在c版本的opencv中一般不是保存的索引而是坐标值。2.2 算法结果数字“0”的识别结果数字“1”的识别结果数字“2”的识别结果数字“3”的识别结果数字“4”的识别结果数字“5”的识别结果2.3 整体代码实现2.3.1 算法流程学长实现过程和上面的系统流程图类似大概过程如下1. 求出手部的掩膜2. 求出掩膜的轮廓3. 求出轮廓的多变形拟合曲线4. 求出多边形拟合曲线的凸包集找出凸点5. 求出多变形拟合曲线的凹陷集找出凹点6. 利用上面的凸凹点和手部中心点的几何关系来做简单的数字手势识别(这里用的是C语言写的这个代码是学长早期写的同学们需要的话学长出一个python版本的)#includeiostream#includeopencv2/highgui/highgui.hpp#includeopencv2/imgproc/imgproc.hpp#includeopencv2/core/core.hpp#includecopenni.cpp#includeiostream#defineDEPTH_SCALE_FACTOR255./4096.#defineROI_HAND_WIDTH140#defineROI_HAND_HEIGHT140#defineMEDIAN_BLUR_K5#defineXRES640#defineYRES480#defineDEPTH_SEGMENT_THRESH5#defineMAX_HANDS_COLOR10#defineMAX_HANDS_NUMBER10#defineHAND_LIKELY_AREA2000#defineDELTA_POINT_DISTENCE25//手部中心点1和中心点2距离的阈值#defineSEGMENT_POINT1_DISTANCE27//凸点与手部中心点1远近距离的阈值#defineSEGMENT_POINT2_DISTANCE30//凸点与手部中心点2远近距离的阈值using namespace cv;using namespace xn;using namespace std;intmain(intargc,char**argv){unsignedintconvex_number_above_point10;unsignedintconcave_number_above_point10;unsignedintconvex_number_above_point20;unsignedintconcave_number_above_point20;unsignedintconvex_assist_above_point10;unsignedintconvex_assist_above_point20;unsignedintpoint_y10;unsignedintpoint_y20;intnumber_result-1;bool recognition_flagfalse;//开始手部数字识别的标志vectorScalarcolor_array;//采用默认的10种颜色{color_array.push_back(Scalar(255,0,0));color_array.push_back(Scalar(0,255,0));color_array.push_back(Scalar(0,0,255));color_array.push_back(Scalar(255,0,255));color_array.push_back(Scalar(255,255,0));color_array.push_back(Scalar(0,255,255));color_array.push_back(Scalar(128,255,0));color_array.push_back(Scalar(0,128,255));color_array.push_back(Scalar(255,0,128));color_array.push_back(Scalar(255,128,255));}vectorunsignedinthand_depth(MAX_HANDS_NUMBER,0);vectorRecthands_roi(MAX_HANDS_NUMBER,Rect(XRES/2,YRES/2,ROI_HAND_WIDTH,ROI_HAND_HEIGHT));namedWindow(color image,CV_WINDOW_AUTOSIZE);namedWindow(depth image,CV_WINDOW_AUTOSIZE);namedWindow(hand_segment,CV_WINDOW_AUTOSIZE);//显示分割出来的手的区域namedWindow(handrecognition,CV_WINDOW_AUTOSIZE);//显示0~5数字识别的图像COpenNI openni;if(!openni.Initial())return1;if(!openni.Start())return1;while(1){if(!openni.UpdateData()){return1;}/*获取并显示色彩图像*/Matcolor_image_src(openni.image_metadata_.YRes(),openni.image_metadata_.XRes(),CV_8UC3,(char*)openni.image_metadata_.Data());Mat color_image;cvtColor(color_image_src,color_image,CV_RGB2BGR);Mathand_segment_mask(color_image.size(),CV_8UC1,Scalar::all(0));for(autoitUseropenni.hand_points_.cbegin();itUser!openni.hand_points_.cend();itUser){point_y1itUser-second.Y;point_y2itUser-second.YDELTA_POINT_DISTENCE;circle(color_image,Point(itUser-second.X,itUser-second.Y),5,color_array.at(itUser-first%color_array.size()),3,8);/*设置不同手部的深度*/hand_depth.at(itUser-first%MAX_HANDS_COLOR)(unsignedint)(itUser-second.Z*DEPTH_SCALE_FACTOR);//itUser-first会导致程序出现bug/*设置不同手部的不同感兴趣区域*/hands_roi.at(itUser-first%MAX_HANDS_NUMBER)Rect(itUser-second.X-ROI_HAND_WIDTH/2,itUser-second.Y-ROI_HAND_HEIGHT/2,ROI_HAND_WIDTH,ROI_HAND_HEIGHT);hands_roi.at(itUser-first%MAX_HANDS_NUMBER).xitUser-second.X-ROI_HAND_WIDTH/2;hands_roi.at(itUser-first%MAX_HANDS_NUMBER).yitUser-second.Y-ROI_HAND_HEIGHT/2;hands_roi.at(itUser-first%MAX_HANDS_NUMBER).widthROI_HAND_WIDTH;hands_roi.at(itUser-first%MAX_HANDS_NUMBER).heightROI_HAND_HEIGHT;if(hands_roi.at(itUser-first%MAX_HANDS_NUMBER).x0)hands_roi.at(itUser-first%MAX_HANDS_NUMBER).x0;if(hands_roi.at(itUser-first%MAX_HANDS_NUMBER).xXRES)hands_roi.at(itUser-first%MAX_HANDS_NUMBER).xXRES;if(hands_roi.at(itUser-first%MAX_HANDS_NUMBER).y0)hands_roi.at(itUser-first%MAX_HANDS_NUMBER).y0;if(hands_roi.at(itUser-first%MAX_HANDS_NUMBER).yYRES)hands_roi.at(itUser-first%MAX_HANDS_NUMBER).yYRES;}imshow(color image,color_image);/*获取并显示深度图像*/Matdepth_image_src(openni.depth_metadata_.YRes(),openni.depth_metadata_.XRes(),CV_16UC1,(char*)openni.depth_metadata_.Data());//因为kinect获取到的深度图像实际上是无符号的16位数据Mat depth_image;depth_image_src.convertTo(depth_image,CV_8U,DEPTH_SCALE_FACTOR);imshow(depth image,depth_image);//取出手的mask部分//不管原图像时多少通道的mask矩阵声明为单通道就okfor(autoitUseropenni.hand_points_.cbegin();itUser!openni.hand_points_.cend();itUser){for(intihands_roi.at(itUser-first%MAX_HANDS_NUMBER).x;istd::min(hands_roi.at(itUser-first%MAX_HANDS_NUMBER).xhands_roi.at(itUser-first%MAX_HANDS_NUMBER).width,XRES);i)for(intjhands_roi.at(itUser-first%MAX_HANDS_NUMBER).y;jstd::min(hands_roi.at(itUser-first%MAX_HANDS_NUMBER).yhands_roi.at(itUser-first%MAX_HANDS_NUMBER).height,YRES);j){hand_segment_mask.atunsignedchar(j,i)((hand_depth.at(itUser-first%MAX_HANDS_NUMBER)-DEPTH_SEGMENT_THRESH)depth_image.atunsignedchar(j,i))((hand_depth.at(itUser-first%MAX_HANDS_NUMBER)DEPTH_SEGMENT_THRESH)depth_image.atunsignedchar(j,i));}}medianBlur(hand_segment_mask,hand_segment_mask,MEDIAN_BLUR_K);Mathand_segment(color_image.size(),CV_8UC3);color_image.copyTo(hand_segment,hand_segment_mask);/*对mask图像进行轮廓提取,并在手势识别图像中画出来*/std::vectorstd::vectorPointcontours;findContours(hand_segment_mask,contours,CV_RETR_LIST,CV_CHAIN_APPROX_SIMPLE);//找出mask图像的轮廓Mat hand_recognition_imageMat::zeros(color_image.rows,color_image.cols,CV_8UC3);for(inti0;icontours.size();i){//只有在检测到轮廓时才会去求它的多边形凸包集凹陷集recognition_flagtrue;/*找出轮廓图像多边形拟合曲线*/Mat contour_matMat(contours[i]);if(contourArea(contour_mat)HAND_LIKELY_AREA){//比较有可能像手的区域std::vectorPointapprox_poly_curve;approxPolyDP(contour_mat,approx_poly_curve,10,true);//找出轮廓的多边形拟合曲线std::vectorstd::vectorPointapprox_poly_curve_debug;approx_poly_curve_debug.push_back(approx_poly_curve);drawContours(hand_recognition_image,contours,i,Scalar(255,0,0),1,8);//画出轮廓// drawContours(hand_recognition_image, approx_poly_curve_debug, 0, Scalar(256, 128, 128), 1, 8); //画出多边形拟合曲线/*对求出的多边形拟合曲线求出其凸包集*/vectorinthull;convexHull(Mat(approx_poly_curve),hull,true);for(inti0;ihull.size();i){circle(hand_recognition_image,approx_poly_curve[hull[i]],2,Scalar(0,255,0),2,8);/*统计在中心点1以上凸点的个数*/if(approx_poly_curve[hull[i]].ypoint_y1){/*统计凸点与中心点1的y轴距离*/longdis_point1abs(long(point_y1-approx_poly_curve[hull[i]].y));intdis1point_y1-approx_poly_curve[hull[i]].y;if(dis_point1SEGMENT_POINT1_DISTANCEdis10){convex_assist_above_point1;}convex_number_above_point1;}/*统计在中心点2以上凸点的个数*/if(approx_poly_curve[hull[i]].ypoint_y2){/*统计凸点与中心点1的y轴距离*/longdis_point2abs(long(point_y2-approx_poly_curve[hull[i]].y));intdis2point_y2-approx_poly_curve[hull[i]].y;if(dis_point2SEGMENT_POINT2_DISTANCEdis20){convex_assist_above_point2;}convex_number_above_point2;}}// /*对求出的多边形拟合曲线求出凹陷集*/std::vectorVec4iconvexity_defects;if(Mat(approx_poly_curve).checkVector(2,CV_32S)3)convexityDefects(approx_poly_curve,Mat(hull),convexity_defects);for(inti0;iconvexity_defects.size();i){circle(hand_recognition_image,approx_poly_curve[convexity_defects[i][2]],2,Scalar(0,0,255),2,8);/*统计在中心点1以上凹陷点的个数*/if(approx_poly_curve[convexity_defects[i][2]].ypoint_y1)concave_number_above_point1;/*统计在中心点2以上凹陷点的个数*/if(approx_poly_curve[convexity_defects[i][2]].ypoint_y2)concave_number_above_point2;}}}/**画出手势的中心点**/for(autoitUseropenni.hand_points_.cbegin();itUser!openni.hand_points_.cend();itUser){circle(hand_recognition_image,Point(itUser-second.X,itUser-second.Y),3,Scalar(0,255,255),3,8);circle(hand_recognition_image,Point(itUser-second.X,itUser-second.Y25),3,Scalar(255,0,255),3,8);}/*手势数字0~5的识别*///0的识别if((convex_assist_above_point10convex_number_above_point22convex_number_above_point23concave_number_above_point21concave_number_above_point11)||(concave_number_above_point10||concave_number_above_point20)recognition_flagtrue)number_result0;//1的识别if(convex_assist_above_point11convex_number_above_point11convex_number_above_point12convex_number_above_point22convex_assist_above_point21)number_result1;//2的识别if(convex_number_above_point12concave_number_above_point11convex_assist_above_point22/*convex_assist_above_point1 1*/concave_number_above_point21)number_result2;//3的识别if(convex_number_above_point13concave_number_above_point13concave_number_above_point11convex_number_above_point23convex_number_above_point24convex_assist_above_point23)number_result3;//4的识别if(convex_number_above_point14concave_number_above_point13concave_number_above_point12convex_number_above_point24)number_result4;//5的识别if(convex_number_above_point14convex_number_above_point25concave_number_above_point23convex_number_above_point24)number_result5;if(number_result!0number_result!1number_result!2number_result!3number_result!4number_result!5)number_result-1;/*在手势识别图上显示匹配的数字*/std::stringstream number_str;number_strnumber_result;putText(hand_recognition_image,Match: ,Point(0,60),4,1,Scalar(0,255,0),2,0);if(number_result-1)putText(hand_recognition_image, ,Point(120,60),4,2,Scalar(255,0,0),2,0);elseputText(hand_recognition_image,number_str.str(),Point(150,60),4,2,Scalar(255,0,0),2,0);imshow(handrecognition,hand_recognition_image);imshow(hand_segment,hand_segment);/*一个循环中对有些变量进行初始化操作*/convex_number_above_point10;convex_number_above_point20;concave_number_above_point10;concave_number_above_point20;convex_assist_above_point10;convex_assist_above_point20;number_result-1;recognition_flagfalse;number_str.clear();waitKey(20);}}#includecopenni.h#includeXnCppWrapper.h#includeiostream#includemapusing namespace xn;using namespace std;COpenNI::COpenNI(){}COpenNI::~COpenNI(){}bool COpenNI::Initial(){status_context_.Init();if(CheckError(Context initial failed!)){returnfalse;}context_.SetGlobalMirror(true);//设置镜像xmode_.nXRes640;xmode_.nYRes480;xmode_.nFPS30;//产生颜色nodestatus_image_generator_.Create(context_);if(CheckError(Create image generator error!)){returnfalse;}//设置颜色图片输出模式status_image_generator_.SetMapOutputMode(xmode_);if(CheckError(SetMapOutputMdoe error!)){returnfalse;}//产生深度nodestatus_depth_generator_.Create(context_);if(CheckError(Create depth generator error!)){returnfalse;}//设置深度图片输出模式status_depth_generator_.SetMapOutputMode(xmode_);if(CheckError(SetMapOutputMdoe error!)){returnfalse;}//产生手势nodestatus_gesture_generator_.Create(context_);if(CheckError(Create gesture generator error!)){returnfalse;}/*添加手势识别的种类*/gesture_generator_.AddGesture(Wave,NULL);gesture_generator_.AddGesture(click,NULL);gesture_generator_.AddGesture(RaiseHand,NULL);gesture_generator_.AddGesture(MovingHand,NULL);//产生手部的nodestatus_hand_generator_.Create(context_);if(CheckError(Create hand generaotr error!)){returnfalse;}//产生人体nodestatus_user_generator_.Create(context_);if(CheckError(Create gesturen generator error!)){returnfalse;}//视角校正status_depth_generator_.GetAlternativeViewPointCap().SetViewPoint(image_generator_);if(CheckError(Cant set the alternative view point on depth generator!)){returnfalse;}//设置与手势有关的回调函数XnCallbackHandle gesture_cb;gesture_generator_.RegisterGestureCallbacks(CBGestureRecognized,CBGestureProgress,this,gesture_cb);//设置于手部有关的回调函数XnCallbackHandle hands_cb;hand_generator_.RegisterHandCallbacks(HandCreate,HandUpdate,HandDestroy,this,hands_cb);//设置有人进入视野的回调函数XnCallbackHandle new_user_handle;user_generator_.RegisterUserCallbacks(CBNewUser,NULL,NULL,new_user_handle);user_generator_.GetSkeletonCap().SetSkeletonProfile(XN_SKEL_PROFILE_ALL);//设定使用所有关节共15个//设置骨骼校正完成的回调函数XnCallbackHandle calibration_complete;user_generator_.GetSkeletonCap().RegisterToCalibrationComplete(CBCalibrationComplete,this,calibration_complete);returntrue;}bool COpenNI::Start(){status_context_.StartGeneratingAll();if(CheckError(Start generating error!)){returnfalse;}returntrue;}bool COpenNI::UpdateData(){status_context_.WaitNoneUpdateAll();if(CheckError(Update date error!)){returnfalse;}//获取数据image_generator_.GetMetaData(image_metadata_);depth_generator_.GetMetaData(depth_metadata_);returntrue;}ImageGeneratorCOpenNI::getImageGenerator(){returnimage_generator_;}DepthGeneratorCOpenNI::getDepthGenerator(){returndepth_generator_;}UserGeneratorCOpenNI::getUserGenerator(){returnuser_generator_;}GestureGeneratorCOpenNI::getGestureGenerator(){returngesture_generator_;}HandsGeneratorCOpenNI::getHandGenerator(){returnhand_generator_;}bool COpenNI::CheckError(constchar*error){if(status_!XN_STATUS_OK){cerrerror: xnGetStatusString(status_)endl;returntrue;}returnfalse;}voidCOpenNI::CBNewUser(UserGeneratorgenerator,XnUserID user,void*p_cookie){//得到skeleton的capability,并调用RequestCalibration函数设置对新检测到的人进行骨骼校正generator.GetSkeletonCap().RequestCalibration(user,true);}voidCOpenNI::CBCalibrationComplete(SkeletonCapabilityskeleton,XnUserID user,XnCalibrationStatus calibration_error,void*p_cookie){if(calibration_errorXN_CALIBRATION_STATUS_OK){skeleton.StartTracking(user);//骨骼校正完成后就开始进行人体跟踪了}else{UserGenerator*p_user(UserGenerator*)p_cookie;skeleton.RequestCalibration(user,true);//骨骼校正失败时重新设置对人体骨骼继续进行校正}}voidCOpenNI::CBGestureRecognized(GestureGeneratorgenerator,constXnChar*strGesture,constXnPoint3D*pIDPosition,constXnPoint3D*pEndPosition,void*pCookie){COpenNI*openni(COpenNI*)pCookie;openni-hand_generator_.StartTracking(*pEndPosition);}voidCOpenNI::CBGestureProgress(GestureGeneratorgenerator,constXnChar*strGesture,constXnPoint3D*pPosition,XnFloat fProgress,void*pCookie){}voidCOpenNI::HandCreate(HandsGeneratorrHands,XnUserID xUID,constXnPoint3D*pPosition,XnFloat fTime,void*pCookie){COpenNI*openni(COpenNI*)pCookie;XnPoint3D project_pos;openni-depth_generator_.ConvertRealWorldToProjective(1,pPosition,project_pos);pairXnUserID,XnPoint3Dhand_point_pair(xUID,XnPoint3D());//在进行pair类型的定义时可以将第2个设置为空hand_point_pair.secondproject_pos;openni-hand_points_.insert(hand_point_pair);//将检测到的手部存入map类型的hand_points_中。pairXnUserID,vectorXnPoint3Dhand_track_point(xUID,vectorXnPoint3D());hand_track_point.second.push_back(project_pos);openni-hands_track_points_.insert(hand_track_point);}voidCOpenNI::HandUpdate(HandsGeneratorrHands,XnUserID xUID,constXnPoint3D*pPosition,XnFloat fTime,void*pCookie){COpenNI*openni(COpenNI*)pCookie;XnPoint3D project_pos;openni-depth_generator_.ConvertRealWorldToProjective(1,pPosition,project_pos);openni-hand_points_.find(xUID)-secondproject_pos;openni-hands_track_points_.find(xUID)-second.push_back(project_pos);}voidCOpenNI::HandDestroy(HandsGeneratorrHands,XnUserID xUID,XnFloat fTime,void*pCookie){COpenNI*openni(COpenNI*)pCookie;openni-hand_points_.erase(openni-hand_points_.find(xUID));openni-hands_track_points_.erase(openni-hands_track_points_.find(xUID));}3 深度学习方法做手势识别3.1 经典的卷积神经网络卷积神经网络的优势就在于它能够从常见的视觉任务中自动学习目 标数据的特征 然后将这些特征用于某种特定任务的模型。 随着时代的发展 深度学习也形成了一些经典的卷积神经网络。3.2 YOLO系列YOLO 系列的网络模型最早源于 2016 年 之后几年经过不断改进相继推出YOLOv2、 YOLOv3 等网络直到今日yoloV5也诞生了不得不感慨一句darknet是真的肝。最具代表性的yolov3的结构3.3 SSDSSD 作为典型的一阶段网络模型 具有更高的操作性 端到端的学习模式同样受到众多研究者的喜爱3.4 实现步骤3.4.1 数据集手势识别的数据集来自于丹成学长实验室由于中国手势表示3的手势根据地区有略微差异按照这个数据集的手势训练与测试即可。图像大小100*100 像素颜色空间RGB 种类图片种类6 种(0,1,2,3,4,5)每种图片数量200 张一共6种手势每种手势200张图片共1200张图片100x100RGB3.4.2 图像预处理实际图片处理展示resize前先高斯模糊提取边缘后可以根据实际需要增加一次中值滤波去噪3.4.3 构建卷积神经网络结构使用tensorflow的框架构建一个简单的网络结构Dropout 增加鲁棒性帮助正则化和避免过拟合一个相关的早期使用这种技术的论文ImageNet Classification with Deep Convolutional Neural Networks, by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton (2012).中启发性的dropout解释是:因为一个神经元不能依赖其他特定的神经元。因此不得不去学习随机子集神经元间的鲁棒性的有用连接。换句话说。想象我们的神经元作为要给预测的模型dropout是一种方式可以确保我们的模型在丢失一个个体线索的情况下保持健壮的模型。在这种情况下可以说他的作用和L1和L2范式正则化是相同的。都是来减少权重连接然后增加网络模型在缺失个体连接信息情况下的鲁棒性。在提高神经网络表现方面效果较好。3.4.4 实验训练过程及结果经过约4800轮的训练后loss基本收敛在0.6左右在120份的测试样本上的模型准确率能够达到约96%3.5 关键代码# 作者丹成学长 Q746876041 需要完整代码联系学长获取importtensorflowastf IMAGE_SIZE100NUM_CHANNELS1CONV1_SIZE4CONV1_KERNEL_NUM8CONV2_SIZE2CONV2_KERNEL_NUM16FC_SIZE512OUTPUT_NODE6defget_weight(shape,regularizer):wtf.Variable(tf.truncated_normal(shape,stddev0.1))ifregularizer!None:tf.add_to_collection(losses,tf.contrib.layers.l2_regularizer(regularizer)(w))returnwdefget_bias(shape):btf.Variable(tf.zeros(shape))returnbdefconv2d(x,w):returntf.nn.conv2d(x,w,strides[1,1,1,1],paddingSAME)defmax_pool_8x8(x):returntf.nn.max_pool(x,ksize[1,8,8,1],strides[1,4,4,1],paddingSAME)defmax_pool_4x4(x):returntf.nn.max_pool(x,ksize[1,4,4,1],strides[1,2,2,1],paddingSAME)defforward(x,train,regularizer):conv1_wget_weight([CONV1_SIZE,CONV1_SIZE,NUM_CHANNELS,CONV1_KERNEL_NUM],regularizer)conv1_bget_bias([CONV1_KERNEL_NUM])conv1conv2d(x,conv1_w)relu1tf.nn.relu(tf.nn.bias_add(conv1,conv1_b))pool1max_pool_8x8(relu1)conv2_wget_weight([CONV2_SIZE,CONV2_SIZE,CONV1_KERNEL_NUM,CONV2_KERNEL_NUM],regularizer)conv2_bget_bias([CONV2_KERNEL_NUM])conv2conv2d(pool1,conv2_w)relu2tf.nn.relu(tf.nn.bias_add(conv2,conv2_b))pool2max_pool_4x4(relu2)pool_shapepool2.get_shape().as_list()nodespool_shape[1]*pool_shape[2]*pool_shape[3]reshapedtf.reshape(pool2,[pool_shape[0],nodes])fc1_wget_weight([nodes,FC_SIZE],regularizer)fc1_bget_bias([FC_SIZE])fc1tf.nn.relu(tf.matmul(reshaped,fc1_w)fc1_b)iftrain:fc1tf.nn.dropout(fc1,0.5)fc2_wget_weight([FC_SIZE,OUTPUT_NODE],regularizer)fc2_bget_bias([OUTPUT_NODE])ytf.matmul(fc1,fc2_w)fc2_breturny# 作者丹成学长 Q746876041 需要完整代码联系学长获取importtensorflowastfimportnumpyasnpimportgesture_forwardimportgesture_backwardfromimage_processingimportfunc5,func6importcv2defrestore_model(testPicArr):withtf.Graph().as_default()astg:xtf.placeholder(tf.float32,[1,gesture_forward.IMAGE_SIZE,gesture_forward.IMAGE_SIZE,gesture_forward.NUM_CHANNELS])#y_ tf.placeholder(tf.float32, [None, mnist_lenet5_forward.OUTPUT_NODE])ygesture_forward.forward(x,False,None)preValuetf.argmax(y,1)variable_averagestf.train.ExponentialMovingAverage(gesture_backward.MOVING_AVERAGE_DECAY)variables_to_restorevariable_averages.variables_to_restore()savertf.train.Saver(variables_to_restore)withtf.Session()assess:ckpttf.train.get_checkpoint_state(gesture_backward.MODEL_SAVE_PATH)ifckptandckpt.model_checkpoint_path:saver.restore(sess,ckpt.model_checkpoint_path)#global_step ckpt.model_checkpoint_path.split(/)[-1].split(-)[-1]preValuesess.run(preValue,feed_dict{x:testPicArr})returnpreValueelse:print(No checkpoint file found)return-1defapplication01():testNuminput(input the number of test pictures:)testNumint(testNum)foriinrange(testNum):testPicinput(the path of test picture:)imgfunc5(testPic)cv2.imwrite(str(i)ttt.jpg,img)# cv2.waitKey(0)# cv2.destroyAllWindows()imgimg.reshape([1,100,100,1])imgimg.astype(np.float32)imgnp.multiply(img,1.0/255.0)# print(img.shape)# print(type(img))preValuerestore_model(img)print(The prediction number is:,preValue)defapplication02():#vc cv2.VideoCapture(testVideo.mp4)vccv2.VideoCapture(0)# 设置每秒传输帧数fpsvc.get(cv2.CAP_PROP_FPS)# 获取视频的大小size(int(vc.get(cv2.CAP_PROP_FRAME_WIDTH)),int(vc.get(cv2.CAP_PROP_FRAME_HEIGHT)))# 生成一个空的视频文件# 视频编码类型# cv2.VideoWriter_fourcc(X,V,I,D) MPEG-4 编码类型# cv2.VideoWriter_fourcc(I,4,2,0) YUY编码类型# cv2.VideoWriter_fourcc(P,I,M,I) MPEG-1 编码类型# cv2.VideoWriter_fourcc(T,H,E,O) Ogg Vorbis类型文件名为.ogv# cv2.VideoWriter_fourcc(F,L,V,1) Flask视频文件名为.flv#vw cv2.VideoWriter(ges_pro.avi,cv2.VideoWriter_fourcc(X,V,I,D), fps, size)# 读取视频第一帧的内容success,framevc.read()# rows frame.shape[0]# cols frame.shape[1]# t1 int((cols-rows)/2)# t2 int(cols-t1)# M cv2.getRotationMatrix2D((cols/2,rows/2),90,1)# frame cv2.warpAffine(frame,M,(cols,rows))# frame frame[0:rows, t1:t2]# cv2.imshow(sd,frame)# cv2.waitKey(0)# cv2.destroyAllWindows()whilesuccess:#90度旋转# img cv2.warpAffine(frame,M,(cols,rows))# img img[0:rows, t1:t2]imgfunc6(frame)imgimg.reshape([1,100,100,1])imgimg.astype(np.float32)imgnp.multiply(img,1.0/255.0)preValuerestore_model(img)# 写入视频cv2.putText(frame,Gesture:str(preValue),(50,50),cv2.FONT_HERSHEY_PLAIN,2.0,(0,0,255),1)#vw.write(frame)cv2.imshow(gesture,frame)ifcv2.waitKey(1)0xFFord(q):break# 读取视频下一帧的内容success,framevc.read()vc.release()cv2.destroyAllWindows()print(viedo app over!)defmain():#application01()application02()if__name____main__:main()4 实现手势交互我们还可以通过手势检测和识别实现软件交互学长录了一个视频效果如下计算机毕业设计基于深度学习的手势识别 项目分享:大家可自取用于参考学习获取方式见文末!