Curl is a command-line utility that helps in transferring data from or to a server with no user interaction. With curl, you can download or upload data using the supporting protocols which include HTTP, HTTPS, SCP, SFTP, and FTP.
Installing Curl:
The curl package is pre-installed on most Linux systems. To know whether it is installed on your system, open your terminal ? type curl ? press enter.
If installed the curl on your system, it will print
curl: try 'curl --help'
This will give you the information of the file. If not, you will find the following output.
curl command not found.
If it is not installed, you can install it from the package manager of your Linux distribution.
Install Curl on Ubuntu
You can use the following command to install curl on your system
$ sudo apt updates do apt install curlCopyCopy
Use Curl:
The syntax for the curl command is quite simple, and it is:
curl [options] [URL...]
If we do not specify the option, curl displays the that resource to the output. For example, to retrieve the studytonight.com homepage (index.php) you would run:
$ curl studytonight.com
Options using curl:
There are several options available in curl the tool to make an advanced HTTP request.
The response that we get back in the command line is all the HTML and scripts and everything that is usually rendered by the browser.
Sometimes by default, curl doesn’t follow the HTTP Location headers for example when you try to retrieve the Google.com, instead of getting the details of the source page you notice, you will be redirected to the new WWW version
To follow the redirection, use -L option. This will instruct curl to follow any redirect until it reaches the eventual destination. The command is as shown below:
$ curl -L google.com
-
Save the obtained Output to a File:
The output is saved by using either the -o or -O option. Lowercase -o saves the file with a pre-defined filename, Uppercase -O saves the file with its original filename as shown below.
$ curl -o [Filename] [URL]
$ curl -O [URL]
With Lowercase –o, the output saves with the [Filename] mentioned
-
To download multiple files at once:
To download multiple files at once, use multiple -O options, followed by the URL of the files that you want to download.
$ curl -O [URL]
$ [URL]
If suddenly your connection fails, you can resume your download with the –c - option as shown below
$ curl -C - -O HTTP://[path]
Sometimes when we download a file, the remote server may block the Curl User-Agent to return different contents depending on the visitor device and browser. In such a situation, we need to change the user agent so as they emulate a different browser for use -A option.
$ curl -A "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0" https://getfedora.org/
The example emulates Mozilla as the new browser.
Get the Specified Headers of a URL
To get the information of the specified headers, we use the following command;
$ curl -I --http2 https://www.amazon.com/
The command gives the information regarding the HTTP headers. We can also use HTTPS, Ftp instead of http2 to get its information.
To Specify a Maximum Transfer Rate in Curl
To specify a maximum limit of the data transfer rate, we use the --limit-rate option. We can express the value in bytes, kilobytes with the k suffix, megabytes with the m suffix, and gigabytes with the g suffix. This will prevent curl from consuming all the bandwidth.
The following is an example of curl downloading the Google Go information with the limit of download speed to 5 megabytes
$ curl --limit-rate 5m -O https://google.com/go
Transfer Files with Curl
To transfer files, we need to access a protected FTP server with curl.
- We use the -u option and specify the username and password to authenticate ourselves and to transfer files securely.
$ curl -u FTP_USERNAME:FTP_PASSWORD <MENTION USERNAME:PASSWORD > ftp://ftp.studytonight.com/
Once logged in, the command lists of all files and directories in the user’s home directory will be displayed.
- Uploading a file to the FTP server is done by using the
-T command that follows the name of the file you want to upload:
$ curl -T newfile.tar.gz -u FTP_USERNAME:FTP_PASSWORD ftp://ftp.studytonight.com/
Send Cookies using Curl
If you want to make an HTTP request with cookies to access a remote resource or to debug some issues, use the -b option followed by a filename containing the cookies or a string.
$ curl -L -b "oracledownload=a" -O http://download.oracle.com/otn-pub/java/jdk/10.0.2+13/19aef61b38124481863b1413dce1855f/jdk-10.0.2_linux-x64_bin.rpm
Here to download Oracle Java JDK rpm file JDK-10.0.2_linux-x64_bin.rpm we are passing a cookie named oracle download with the value a
Using Proxies with Curl
Curl supports different proxies, including HTTP, HTTPS, and SOCKS.
We use the -x (--proxy) option, followed by the proxy URL, To transfer data through a proxy server
$ curl -x 192.168.44.1:8888 http://linux.com/
Conclusion:
We use curl command to transfer the data from a server to a server using a proper set of network protocols. The post would never be enough to understand its complete use, the article gives you just an overview of the curl command, there is more in-depth to learn about it.
