The computer engineer researching how they identify the face of a human in an image. For this, we need to identify first where the human face is located in the whole image. The face detector is the method which locates the face of a human in an image and returns as a bounding box or rectangle box values.
After getting the face position in an image and next we have to find out small features of the face like eyebrows, lips, etc. The facial landmark detection tells all the required features of a human face which we want.
The below image is an example of a Dlib's 68 points model. There we can see that points from 1 to 68. But sometimes we don't need all 68 feature points, then for that, we will do in the next post, how we can customize those points according to our requirements. In this post, we only going to see about 68 Dlib's points for clear understanding.
There are mostly two steps to detect face landmarks in an image which are given below:
Face detection: Face detection is the first methods which locate a human face and return a value in x,y,w,h which is a rectangle.
Face landmark: After getting the location of a face in an image, then we have to through points inside of that rectangle.
There are many methods of face detector but we focus in this post only one which is Dlib's method. Like, Opencv uses methods LBP cascades and HAAR and Dlib's use methods HOG (Histogram of Oriented Gradients) and SVM (Support Vector Machine).
The code in python is given below and same code you can download from here
All codes are given with proper comment so that you can understand each and every line of code easily way.
This python code file name is facial_68_landmark.py
Dlib has already a pre-built model which can detect the face. That's why in the below python code facial_68_landmark.py line number 25, we are just accessing directly that model and creating an object
faceLandmarkDetector. In the below code, we are first uploading an image than trying to face on that whole image. After getting the face position from the image, we return the rectangle value where face resides. And on that rectangle is called detection of face. Now, in code line number 54 we are using that rectangle value and image inside of the function to detect face landmarks.
#USAGE: python facial_68_Landmark.py import dlib,cv2 import numpy as np from facePoints import facePoints def writeFaceLandmarksToLocalFile(faceLandmarks, fileName): with open(fileName, 'w') as f: for p in faceLandmarks.parts(): f.write("%s %s\n" %(int(p.x),int(p.y))) f.close() # location of the model (path of the model). Model_PATH = "shape_predictor_68_face_landmarks.dat" # now from the dlib we are extracting the method get_frontal_face_detector() # and assign that object result to frontalFaceDetector to detect face from the image with # the help of the 68_face_landmarks.dat model frontalFaceDetector = dlib.get_frontal_face_detector() # Now the dlip shape_predictor class will take model and with the help of that, it will show faceLandmarkDetector = dlib.shape_predictor(Model_PATH) # We now reading image on which we applied our face detector image = "test.jpg" # Now we are reading image using openCV img= cv2.imread(image) imageRGB = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) # landmarks of the face image will be stored in output/image_k.txt faceLandmarksOuput= "output/image" # Now this line will try to detect all faces in an image either 1 or 2 or more faces allFaces = frontalFaceDetector(imageRGB, 0) print("List of all faces detected: ",len(allFaces)) # List to store landmarks of all detected faces allFacesLandmark =  # Below loop we will use to detect all faces one by one and apply landmarks on them for k in range(0, len(allFaces)): # dlib rectangle class will detecting face so that landmark can apply inside of that area faceRectangleDlib = dlib.rectangle(int(allFaces[k].left()),int(allFaces[k].top()), int(allFaces[k].right()),int(allFaces[k].bottom())) # Now we are running loop on every detected face and putting landmark on that with the help of faceLandmarkDetector detectedLandmarks = faceLandmarkDetector(imageRGB, faceRectangleDlib) # count number of landmarks we actually detected on image if k==0: print("Total number of face landmarks detected ",len(detectedLandmarks.parts())) # Svaing the landmark one by one to the output folder allFacesLandmark.append(detectedLandmarks) # Now finally we drawing landmarks on face facePoints(img, detectedLandmarks) fileName = faceLandmarksOuput +"_"+ str(k)+ ".txt" print("Lanmdark is save into ", fileName) # Write landmarks to disk writeFaceLandmarksToLocalFile(detectedLandmarks, fileName) #Name of the output file outputNameofImage = "output/image.jpg" print("Saving output image to", outputNameofImage) cv2.imwrite(outputNameofImage, img) cv2.imshow("Face landmark result", img) # Pause screen to wait key from user to see result cv2.waitKey(0) cv2.destroyAllWindows()
In the code below we have defined the method
facePoints which is called in the python code above.
Now to draw landmarks on the face of the detected rectangle, we are passing the landmarks values and image to the facePoints. In the below code, we are passing landmarks and image as a parameter to a method called
drawPoints which accessing the coordinates(x,y) of the ith landmarks points using the
part(i).y. All landmarks points are saved in a numpy array and then pass these points to in-built
cv2.polylines method to draw the lines on the face using the startpoint and endpoint parameters.
Here is the basic syntax of the
cv2.polylines(img, array of points in int32 format, bool isClosed, color, lineThickness, lineType)
Now let's see the code,
import cv2 import numpy as np # This below mehtod will draw all those points which are from 0 to 67 on face one by one. def drawPoints(image, faceLandmarks, startpoint, endpoint, isClosed=False): points =  for i in range(startpoint, endpoint+1): point = [faceLandmarks.part(i).x, faceLandmarks.part(i).y] points.append(point) points = np.array(points, dtype=np.int32) cv2.polylines(image, [points], isClosed, (255, 200, 0), thickness=2, lineType=cv2.LINE_8) # Use this function for 70-points facial landmark detector model # we are checking if points are exactly equal to 68, then we draw all those points on face one by one def facePoints(image, faceLandmarks): assert(faceLandmarks.num_parts == 68) drawPoints(image, faceLandmarks, 0, 16) # Jaw line drawPoints(image, faceLandmarks, 17, 21) # Left eyebrow drawPoints(image, faceLandmarks, 22, 26) # Right eyebrow drawPoints(image, faceLandmarks, 27, 30) # Nose bridge drawPoints(image, faceLandmarks, 30, 35, True) # Lower nose drawPoints(image, faceLandmarks, 36, 41, True) # Left eye drawPoints(image, faceLandmarks, 42, 47, True) # Right Eye drawPoints(image, faceLandmarks, 48, 59, True) # Outer lip drawPoints(image, faceLandmarks, 60, 67, True) # Inner lip # Use this function for any model other than # 70 points facial_landmark detector model def facePoints2(image, faceLandmarks, color=(0, 255, 0), radius=4): for p in faceLandmarks.parts(): cv2.circle(im, (p.x, p.y), radius, color, -1)
The complete code of the above post you can download from the below link:
The 68-Dlib's point model not included in that because of the heavy size. So that, we can download from the below link and keep inside of that folder and you can also set the path of the model from the code.
Dlib's 68-face landmark model shows how we can access the face features like eyes, eyebrows, nose, etc. But some times, we don't want to access all features of the face and want only some features likes, lips for lipstick application. So that is also possible using custom training of the Dlib's 68-landmark models and you will get details of that in the next blog.