New Tutorials:   NUMPY    TKINTER    KOTLIN    JAVASCRIPT    SASS/SCSS    PL/SQL    Matplotlib    C++ Programs
CLOSE
   OpenCV  Dlib  Python  
   Technology    Machine Learning

Dlib 68 points Face landmark Detection with OpenCV and Python

         
  JUNE 22, 2020   by shekharpandey

The computer engineer researching how they identify the face of a human in an image. For this, we need to identify first where the human face is located in the whole image. The face detector is the method which locates the face of a human in an image and returns as a bounding box or rectangle box values.

dlib 68 point face landmark

After getting the face position in an image and next we have to find out small features of the face like eyebrows, lips, etc. The facial landmark detection tells all the required features of a human face which we want.

Dlib's 68 Face Features:

The below image is an example of a Dlib's 68 points model. There we can see that points from 1 to 68. But sometimes we don't need all 68 feature points, then for that, we will do in the next post, how we can customize those points according to our requirements. In this post, we only going to see about 68 Dlib's points for clear understanding.

dlib 68 point face feature

Facial landmark points detection through Dlib's 68 Model:

There are mostly two steps to detect face landmarks in an image which are given below:

  1. Face detection: Face detection is the first methods which locate a human face and return a value in x,y,w,h which is a rectangle.

  2. Face landmark: After getting the location of a face in an image, then we have to through points inside of that rectangle.

There are many methods of face detector but we focus in this post only one which is Dlib's method. Like, Opencv uses methods LBP cascades and HAAR and Dlib's use methods HOG (Histogram of Oriented Gradients) and SVM (Support Vector Machine).

Dlib's 68 Facial landmark Detection in Python:

The code in python is given below and same code you can download from here

All codes are given with proper comment so that you can understand each and every line of code easily way.

Python: facial_68_landmark.py

This python code file name is facial_68_landmark.py

Dlib has already a pre-built model which can detect the face. That's why in the below python code facial_68_landmark.py line number 25, we are just accessing directly that model and creating an object faceLandmarkDetector. In the below code, we are first uploading an image than trying to face on that whole image. After getting the face position from the image, we return the rectangle value where face resides. And on that rectangle is called detection of face. Now, in code line number 54 we are using that rectangle value and image inside of the function to detect face landmarks.

#USAGE: python facial_68_Landmark.py

import dlib,cv2
import numpy as np
from facePoints import facePoints

def writeFaceLandmarksToLocalFile(faceLandmarks, fileName):
  with open(fileName, 'w') as f:
    for p in faceLandmarks.parts():
      f.write("%s %s\n" %(int(p.x),int(p.y)))

  f.close()

# location of the model (path of the model).
Model_PATH = "shape_predictor_68_face_landmarks.dat"


# now from the dlib we are extracting the method get_frontal_face_detector()
# and assign that object result to frontalFaceDetector to detect face from the image with 
# the help of the 68_face_landmarks.dat model
frontalFaceDetector = dlib.get_frontal_face_detector()


# Now the dlip shape_predictor class will take model and with the help of that, it will show 
faceLandmarkDetector = dlib.shape_predictor(Model_PATH)

# We now reading image on which we applied our face detector
image = "test.jpg"

# Now we are reading image using openCV
img= cv2.imread(image)
imageRGB = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)

# landmarks of the face image  will be stored in output/image_k.txt
faceLandmarksOuput= "output/image"

# Now this line will try to detect all faces in an image either 1 or 2 or more faces
allFaces = frontalFaceDetector(imageRGB, 0)


print("List of all faces detected: ",len(allFaces))

# List to store landmarks of all detected faces
allFacesLandmark = []

# Below loop we will use to detect all faces one by one and apply landmarks on them

for k in range(0, len(allFaces)):
  # dlib rectangle class will detecting face so that landmark can apply inside of that area
  faceRectangleDlib = dlib.rectangle(int(allFaces[k].left()),int(allFaces[k].top()),
      int(allFaces[k].right()),int(allFaces[k].bottom()))

  # Now we are running loop on every detected face and putting landmark on that with the help of faceLandmarkDetector
  detectedLandmarks = faceLandmarkDetector(imageRGB, faceRectangleDlib)
  
  # count number of landmarks we actually detected on image
  if k==0:
    print("Total number of face landmarks detected ",len(detectedLandmarks.parts()))

  # Svaing the landmark one by one to the output folder
  allFacesLandmark.append(detectedLandmarks)

  # Now finally we drawing landmarks on face
  facePoints(img, detectedLandmarks)

  fileName = faceLandmarksOuput +"_"+ str(k)+ ".txt"
  print("Lanmdark is save into ", fileName)

  # Write landmarks to disk
  writeFaceLandmarksToLocalFile(detectedLandmarks, fileName)

#Name of the output file
outputNameofImage = "output/image.jpg"
print("Saving output image to", outputNameofImage)
cv2.imwrite(outputNameofImage, img)

cv2.imshow("Face landmark result", img)

# Pause screen to wait key from user to see result
cv2.waitKey(0)
cv2.destroyAllWindows()

In the code below we have defined the method facePoints which is called in the python code above.

Now to draw landmarks on the face of the detected rectangle, we are passing the landmarks values and image to the facePoints. In the below code, we are passing landmarks and image as a parameter to a method called drawPoints which accessing the coordinates(x,y) of the ith landmarks points using the part(i).x and part(i).y. All landmarks points are saved in a numpy array and then pass these points to in-built cv2.polylines method to draw the lines on the face using the startpoint and endpoint parameters.

Here is the basic syntax of the cv2.polylines method:

cv2.polylines(img, array of points in int32 format, bool isClosed, color, lineThickness, lineType)

Now let's see the code,

import cv2
import numpy as np

# This below mehtod will draw all those points which are from 0 to 67 on face one by one.
def drawPoints(image, faceLandmarks, startpoint, endpoint, isClosed=False):
  points = []
  for i in range(startpoint, endpoint+1):
    point = [faceLandmarks.part(i).x, faceLandmarks.part(i).y]
    points.append(point)

  points = np.array(points, dtype=np.int32)
  cv2.polylines(image, [points], isClosed, (255, 200, 0), thickness=2, lineType=cv2.LINE_8)

# Use this function for 70-points facial landmark detector model
# we are checking if points are exactly equal to 68, then we draw all those points on face one by one
def facePoints(image, faceLandmarks):
    assert(faceLandmarks.num_parts == 68)
    drawPoints(image, faceLandmarks, 0, 16)           # Jaw line
    drawPoints(image, faceLandmarks, 17, 21)          # Left eyebrow
    drawPoints(image, faceLandmarks, 22, 26)          # Right eyebrow
    drawPoints(image, faceLandmarks, 27, 30)          # Nose bridge
    drawPoints(image, faceLandmarks, 30, 35, True)    # Lower nose
    drawPoints(image, faceLandmarks, 36, 41, True)    # Left eye
    drawPoints(image, faceLandmarks, 42, 47, True)    # Right Eye
    drawPoints(image, faceLandmarks, 48, 59, True)    # Outer lip
    drawPoints(image, faceLandmarks, 60, 67, True)    # Inner lip

# Use this function for any model other than
# 70 points facial_landmark detector model
def facePoints2(image, faceLandmarks, color=(0, 255, 0), radius=4):
  for p in faceLandmarks.parts():
    cv2.circle(im, (p.x, p.y), radius, color, -1)

The complete code of the above post you can download from the below link:

https://drive.google.com/file/d/1fXlpFVNdGVRszKBxGnjSM4nFLUPnmNrq/view?usp=sharing

The 68-Dlib's point model not included in that because of the heavy size. So that, we can download from the below link and keep inside of that folder and you can also set the path of the model from the code.

https://github.com/davisking/dlib-models/blob/master/shape_predictor_68_face_landmarks.dat.bz2

Output:

dlib 68 face landmark detection

Conclusion:

Dlib's 68-face landmark model shows how we can access the face features like eyes, eyebrows, nose, etc. But some times, we don't want to access all features of the face and want only some features likes, lips for lipstick application. So that is also possible using custom training of the Dlib's 68-landmark models and you will get details of that in the next blog.


RELATED POSTS



Subscribe and receive amazing posts directly in your inbox.