Signup/Sign In

Dlib 68 points Face landmark Detection with OpenCV and Python

Posted in Machine Learning   LAST UPDATED: MAY 22, 2023

    The computer engineer researched how they identify the face of a human in an image. For this, we need to identify first where the human face is located in the whole image. The face detector is the method that locates the face of a human in an image and returns it as a bounding box or rectangle box value.

    dlib 68 point face landmark

    After getting the face position in an image and next, we have to find out small features of the face like eyebrows, lips, etc. Facial landmark detection tells all the required features of a human face which we want.

    Dlib's 68 Face Features

    The below image is an example of a Dlib's 68 points model. There we can see that points from 1 to 68. But sometimes we don't need all 68 feature points, then for that, we will do in the next post, how we can customize those points according to our requirements. In this post, we only going to see about 68 Dlib's points for clear understanding.

    dlib 68 point face feature

    Facial landmark detection with Dlib's 68 Model

    There are mostly two steps to detect face landmarks in an image which are given below:

    1. Face detection: Face detection is the first method that locates a human face and returns a value in x,y,w,h which is a rectangle.

    2. Face landmark: After getting the location of a face in an image, then we have to through points inside of that rectangle.

    There are many methods of face detector but we focus in this post on only one which is Dlib's method. Like, Opencv uses methods of LBP cascades, and HAAR and Dlib use methods of HOG (Histogram of Oriented Gradients) and SVM (Support Vector Machine).

    Dlib's 68 Facial Landmark Detection in Python

    The code in Python is given below and same code you can download it from here

    All codes are given with proper comments so that you can understand each and every line of code easily way.

    Python: facial_68_landmark.py

    This Python code file name is facial_68_landmark.py

    Dlib has already a pre-built model which can detect the face. That's why in the below python code facial_68_landmark.py line number 25, we are just accessing directly that model and creating an object faceLandmarkDetector. In the below code, we are first uploading an image and then trying to face that whole image. After getting the face position from the image, we return the rectangle value where the face resides. And on that rectangle is called detection of face. Now, in code line number 54, we are using that rectangle value and image inside of the function to detect face landmarks.

    #USAGE: python facial_68_Landmark.py
    
    import dlib,cv2
    import numpy as np
    from facePoints import facePoints
    
    def writeFaceLandmarksToLocalFile(faceLandmarks, fileName):
      with open(fileName, 'w') as f:
        for p in faceLandmarks.parts():
          f.write("%s %s\n" %(int(p.x),int(p.y)))
    
      f.close()
    
    # location of the model (path of the model).
    Model_PATH = "shape_predictor_68_face_landmarks.dat"
    
    
    # now from the dlib we are extracting the method get_frontal_face_detector()
    # and assign that object result to frontalFaceDetector to detect face from the image with 
    # the help of the 68_face_landmarks.dat model
    frontalFaceDetector = dlib.get_frontal_face_detector()
    
    
    # Now the dlip shape_predictor class will take model and with the help of that, it will show 
    faceLandmarkDetector = dlib.shape_predictor(Model_PATH)
    
    # We now reading image on which we applied our face detector
    image = "test.jpg"
    
    # Now we are reading image using openCV
    img= cv2.imread(image)
    imageRGB = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
    
    # landmarks of the face image  will be stored in output/image_k.txt
    faceLandmarksOuput= "output/image"
    
    # Now this line will try to detect all faces in an image either 1 or 2 or more faces
    allFaces = frontalFaceDetector(imageRGB, 0)
    
    
    print("List of all faces detected: ",len(allFaces))
    
    # List to store landmarks of all detected faces
    allFacesLandmark = []
    
    # Below loop we will use to detect all faces one by one and apply landmarks on them
    
    for k in range(0, len(allFaces)):
      # dlib rectangle class will detecting face so that landmark can apply inside of that area
      faceRectangleDlib = dlib.rectangle(int(allFaces[k].left()),int(allFaces[k].top()),
          int(allFaces[k].right()),int(allFaces[k].bottom()))
    
      # Now we are running loop on every detected face and putting landmark on that with the help of faceLandmarkDetector
      detectedLandmarks = faceLandmarkDetector(imageRGB, faceRectangleDlib)
      
      # count number of landmarks we actually detected on image
      if k==0:
        print("Total number of face landmarks detected ",len(detectedLandmarks.parts()))
    
      # Svaing the landmark one by one to the output folder
      allFacesLandmark.append(detectedLandmarks)
    
      # Now finally we drawing landmarks on face
      facePoints(img, detectedLandmarks)
    
      fileName = faceLandmarksOuput +"_"+ str(k)+ ".txt"
      print("Lanmdark is save into ", fileName)
    
      # Write landmarks to disk
      writeFaceLandmarksToLocalFile(detectedLandmarks, fileName)
    
    #Name of the output file
    outputNameofImage = "output/image.jpg"
    print("Saving output image to", outputNameofImage)
    cv2.imwrite(outputNameofImage, img)
    
    cv2.imshow("Face landmark result", img)
    
    # Pause screen to wait key from user to see result
    cv2.waitKey(0)
    cv2.destroyAllWindows()
    

    In the code below we have defined the method facePoints which is called in the Python code above.

    Now to draw landmarks on the face of the detected rectangle, we are passing the landmarks values and image to the facePoints. In the below code, we are passing landmarks and images as a parameter to a method called drawPoints which accessing the coordinates(x,y) of the ith landmarks points using the part(i).x and part(i).y. All landmarks points are saved in a NumPy array and then pass these points to in-built cv2.polylines method to draw the lines on the face using the start point and endpoint parameters.

    Here is the basic syntax of the cv2.polylines method:

    cv2.polylines(img, array of points in int32 format, bool isClosed, color, lineThickness, lineType)

    Now let's see the code,

    import cv2
    import numpy as np
    
    # This below mehtod will draw all those points which are from 0 to 67 on face one by one.
    def drawPoints(image, faceLandmarks, startpoint, endpoint, isClosed=False):
      points = []
      for i in range(startpoint, endpoint+1):
        point = [faceLandmarks.part(i).x, faceLandmarks.part(i).y]
        points.append(point)
    
      points = np.array(points, dtype=np.int32)
      cv2.polylines(image, [points], isClosed, (255, 200, 0), thickness=2, lineType=cv2.LINE_8)
    
    # Use this function for 70-points facial landmark detector model
    # we are checking if points are exactly equal to 68, then we draw all those points on face one by one
    def facePoints(image, faceLandmarks):
        assert(faceLandmarks.num_parts == 68)
        drawPoints(image, faceLandmarks, 0, 16)           # Jaw line
        drawPoints(image, faceLandmarks, 17, 21)          # Left eyebrow
        drawPoints(image, faceLandmarks, 22, 26)          # Right eyebrow
        drawPoints(image, faceLandmarks, 27, 30)          # Nose bridge
        drawPoints(image, faceLandmarks, 30, 35, True)    # Lower nose
        drawPoints(image, faceLandmarks, 36, 41, True)    # Left eye
        drawPoints(image, faceLandmarks, 42, 47, True)    # Right Eye
        drawPoints(image, faceLandmarks, 48, 59, True)    # Outer lip
        drawPoints(image, faceLandmarks, 60, 67, True)    # Inner lip
    
    # Use this function for any model other than
    # 70 points facial_landmark detector model
    def facePoints2(image, faceLandmarks, color=(0, 255, 0), radius=4):
      for p in faceLandmarks.parts():
        cv2.circle(im, (p.x, p.y), radius, color, -1)
    

    The complete code of the above post you can download from the below link:

    https://drive.google.com/file/d/1fXlpFVNdGVRszKBxGnjSM4nFLUPnmNrq/view?usp=sharing

    The 68-Dlib's point model is not included in that because of the heavy size. So that, we can download it from the below link and keep it inside that folder and you can also set the path of the model from the code.

    https://github.com/davisking/dlib-models/blob/master/shape_predictor_68_face_landmarks.dat.bz2

    Output:

    dlib 68 face landmark detection

    Conclusion

    Dlib's 68-face landmark model shows how we can access face features like eyes, eyebrows, nose, etc. But sometimes, we don't want to access all features of the face and want only some features like lips for lipstick application. So that is also possible using custom training of the Dlib's 68-landmark models and you will get details of that in the next blog.

    Frequently Asked Questions(FAQs)

    1. What is Dlib 68 points Face landmark Detection?

    Dlib 68 points Face landmark Detection is a computer vision algorithm that detects 68 facial landmarks, such as the eyes, nose, and mouth.

    2. What is OpenCV?

    OpenCV (Open Source Computer Vision Library) is a free and open-source computer vision and machine learning software library.

    3. How does Dlib's 68 points Face landmark Detection work?

    Dlib 68 points Face landmark Detection uses a machine learning algorithm to detect 68 facial landmarks by analyzing the facial features and geometry.

    4. How can I use Dlib 68 points Face landmark Detection with OpenCV and Python?

    You can use Dlib 68 points Face landmark Detection with OpenCV and Python by installing the Dlib and OpenCV libraries, loading the pre-trained model, and applying the algorithm to detect the facial landmarks in an image or video.

    You may also like:

    About the author:
    I am an Artificial Intelligence Engineer. Doing research work in Virtual Reality and Augmented Reality. I would like to write article or share my work with others apart from my professional life.
    IF YOU LIKE IT, THEN SHARE IT
     

    RELATED POSTS