Written By:
shekharpandey
0 minute read
Data AugmentationOpenCVPythonComputer Vision

Horizontal And Vertical Shift Data Augmentation

Posted in Machine Learning     APRIL 6, 2021

In the previous blog, we studied data augmentation and what is the purpose of this. If you did not study the previous blog, kindly see the previous blog.

The code of this blog, you can download from the below GitHub link.

https://github.com/shekharpandey89/data_augmentation

As we know about the data augmentation and its importance. So let's discuss the different methods of this one by one. Image data augmentation is the most important part of computer vision. The following geometric transformation methods data augmentation use to re-generate the image data.

  1. Horizontal and vertical flips
  2. Horizontal and vertical shift
  3. Brightness
  4. Zoom
  5. Rotations

So we will go all the above methods in detail one by one. But before that, let's try to understand the augmentation how to use in Keras.

As we already know Keras providing a class ImageDataGenerator for this purpose. So for that first, we have to create an object of that class and then add all those transformation methods which we want to apply to the images.

So basically the most important techniques for the image are supported as follows:

  1. The argument horizontal_flip and vertical_flip to use flip the image.
  2. The argument height_shift_range and width_shift_range use to shift the image horizontal and vertical.
  3. The argument brightness_range use to change the image brightness.
  4. The argument zoom_range use to change the image zoom in and zoom out.
  5. The argument rotation_range use to rotate the image.

Let's try to understand how to create ImageDataGenerator class and how to pass the above methods arguments inside of that.

For Example:

Below is the method to create the ImageDataGenerator class object.

# create an object of the ImageDataGenerator class
imageDataGen = ImageDataGenerator()

Once we created the class object of ImageDataGenerator, then we create an iterator using the flow() method as shown below. The iterator will handle one image batch one time and returned the transformed images. In the flow() method the images already loaded into the memory but if you want to fetch images from your local directory then we have to use the flow_from_directory() function.

# creating image iterator 
iterator = imageDataGen.flow(image,..)

Below is another example to create the iterator, if we load images from the local directory.

# creating image iterator
iterator = imageDataGen.flow_from_directory(directory=/path/of/the/image/Dataset/,...)

Once we created our iterator, then we can train the neural network using the fit_generator() function. We also have to define the steps_per_epoch inside of the function as an argument value which tells how many images batch in each epoch.
For example: If we have 500 images and we want to keep the batch size of 10, then according to the formula ceil(num_samples / batch_size), we get ceil(500/10) or 50 batches. This means 50 steps per epoch and each step it covers 50 images. So in this way, we can process whole image datasets in every epoch.

# we define here model
model = ...
# now we fitting model with our newly generated dataset using fit_generator() function and we also
# define there steps_per_epoch
model.fit_generator(iterator, steps_per_epoch=50, ...)

As we know already, all datasets images not go for training to the model because ImageDataGenerator randomly chooses the transformed data and closes the copy of the original image to the training model. The data generator can also help us to define the testing and validation datasets. Now, we understand what is ImageDataGenerator. So now we are going to explore some of the methods of this technique. We will see some of these techniques one by one using code and then, at last, we combine all those methods in a single program to see the full result.

As we now understand better data augmentation only use during the training time not the test side because our motive is to increase the dataset so that machines can learn more features and at last give better results on the unseen data. So simply we can say that boost the generalization of the model.

So the first method which we going to explain is Horizontal Shift and Vertical Shift.

Also Read: Horizontal and Vertical Flip Data Augmentation

Horizontal Shift Augmentation

A shift to horizontal augmentation means pixels of the image will shift to horizontal without changing the dimension of the image. So, it means some pixels of the image will be clipped off and there will be a new area where new pixels of the image will be specified to get the same image dimension.

ImageDataGenerator has two main arguments width_shift_range and height_shift_range and based on these values the image pixels will move either to the horizontal and vertical.

These parameters which we specified width and height in the ImageDataGenerator constructor have floating-point values from 0 to 1 and on the depends upon the value the image will move horizontal and vertical. We can also specify the pixel value in a tuple to move the image pixel instead of the value 0 to 1. For example, to use the pixel we have to supply in the form of min and max value range like [-10,10] or [-0.3, 0.3].

The below example explains the horizontal shift by using the width_shift_range argument.

Python code: We save the below python program with the name horizontalShift.py. To run this program you have to keep an image (any image) in the same folder where you will keep this python file. Because it will read the image. If your image is another location then specify the full path in the load_img (/path/of/the/image).

# python program to demonstrate the horizontal shift of the image with the width_shift_range argument

# we import all our required libraries
from numpy import expand_dims
from keras.preprocessing.image import load_img
from keras.preprocessing.image import img_to_array
from keras.preprocessing.image import ImageDataGenerator
from matplotlib import pyplot

# we first load the image
image = load_img('parrot.jpg')
# we converting the image which is in PIL format into the numpy array, so that we can apply deep learning methods
dataImage = img_to_array(image)
# print(dataImage)
# expanding dimension of the load image
imageNew = expand_dims(dataImage, 0)
# print(imageNew)
# now here below we creating the object of the data augmentation class
imageDataGen = ImageDataGenerator(width_shift_range=[-200,200])
# because as we alreay load image into the memory, so we are using flow() function, to apply transformation
iterator = imageDataGen.flow(imageNew, batch_size=1)
# below we generate augmented images and plotting for visualization
for i in range(9):
	# we are below define the subplot
	pyplot.subplot(330 + 1 + i)
	# generating images of each batch
	batch = iterator.next()
	# again we convert back to the unsigned integers value of the image for viewing
	image = batch[0].astype('uint8')
	# we plot here raw pixel data
	pyplot.imshow(image)
# visualize the the figure
pyplot.show()

Line 4 to 8: We are importing our required packages to create our code.

Line 11: We load the image from our local drive and load it with the name variable image.

Line 13: In this line, we converting the PIL image format to NumPy array so that we can use that in further image processing.

Line 16: We expand our NumPy array to axis = 0 which means column side.

Line 19: We created the object (imageDataGen) for the class ImageDataGenerator and pass the argument width_shift_range.

Line 21: We created the iterator to perform the transformation on the batch.

Line 23 to 33: Then the iterator is called as per the iteration value and we got out transformed images as shown below in the result.

Output:

When we run the above code, we got the following output as shown below on the screen. So in the below output, we can see that the parrot image randomly moves horizontally by the selected pixel value range that we supplied.

Vertical Shift Augmentation

The below code will explain the vertical shift augmentation where we pass the height_shift_range to the class ImageDataGenerator. In this, we are passing the percentage value which shifts the image to the height of the image (according to the percentage value).

Python Code: We save the below python program with the name verticalShift.py. To run this program we have to keep an image (any image) in the same folder where you will keep this python file. Because it will read the image. If your image is another location then specify the full path in the load_img (/path/of/the/image).

# python program to demonstrate the vertical shift of the image with the height_shift_range argument

# we import all our required libraries
from numpy import expand_dims
from keras.preprocessing.image import load_img
from keras.preprocessing.image import img_to_array
from keras.preprocessing.image import ImageDataGenerator
from matplotlib import pyplot

# we first load the image
image = load_img('parrot.jpg')
# we converting the image which is in PIL format into the numpy array, so that we can apply deep learning methods
dataImage = img_to_array(image)
# print(dataImage)
# expanding dimension of the load image
imageNew = expand_dims(dataImage, 0)
# now here below we creating the object of the data augmentation class
imageDataGen = ImageDataGenerator(height_shift_range=0.4)
# because as we alreay load image into the memory, so we are using flow() function, to apply transformation
iterator = imageDataGen.flow(imageNew, batch_size=1)
# below we generate augmented images and plotting for visualization
for i in range(9):
	# we are below define the subplot
	pyplot.subplot(330 + 1 + i)
	# generating images of each batch
	batch = iterator.next()
	# again we convert back to the unsigned integers value of the image for viewing
	image = batch[0].astype('uint8')
	# we plot here raw pixel data
	pyplot.imshow(image)
# visualize the the figure
pyplot.show()

Output:

In the above output, we can see that the image shift 0.4 percentage of the image.

Conclusion: Till now we clearly understand different methods of image data augmentation. And how to create an object of the imageDataGenerator and iterator. After that, we studied two arguments horizontal shift and vertical shift. We can see from the above output both vertical and horizontal shift makes some new dataset but in some cases, it's gone out of the image recognition. But still, the machine can learn some features from those images to give better results on the unseen data.

In the next blog, we will be going to explain the next argument vertical flip and horizontal flip.


   IF YOU LIKE IT, THEN SHARE IT

RELATED POSTS