The program cURL is a capable HTTP client. This means we can do a variety of HTTP queries through the command line and test our services with ease, using cURL. It’s OK for testing purposes, but sometimes you will need to parse the response and do something with it, in that case you should use something more complex… Anyway lets start:
- Requesting a page:
This executes a HTTP GET request to the given URL, simple and working.
- Requesting a page and viewing it’s response headers:
curl -s -D - http://forum-meddlings.rhcloud.com -o /dev/null
# -s (–silent) Silent or quiet mode. Don’t show progress meter or error messages.
# -D (–dump-header) <file> Writes the protocol headers to the specified file (if no file is specified to the output).
# -o (–output) <file> Writes the response to the file specified.
So we are dumping the headers to the console output and the response to the black hole of nothingness. This is helpful if we want to explore only the headers of the response, if you want to view both the header and the body of the response, just skip the -o option. Another way of doing this is:
curl -i http://forum-meddlings.rhcloud.com
# -i (–include) Include the HTTP-header in the output.
- Request with a different HTTP method. Posting data through the POST verb:
curl -i -X POST -d "author=meddle&body=BLAME" \ http://forum-meddlings.rhcloud.com/comments
Now this posts to the given URL the data. I separated the command to be on two lines for readability…
# -X (–request) Specifies the request method to use when communicating with the server.
# -d (–data) The data for the request, in the example we have POST parameters.
- Passing request headers:
curl -i -H "X-HTTP-Method-Override: PUT" -X POST -d "body=Yohohohoho" \ http://forum-meddlings.rhcloud.com/comment/1
We changed the HTTP method through the method override header.
# -H (–header) Adds a request header.
We can add as many headers as we want:
curl -i \ -H "X-HTTP-Method-Override: PUT" \ -H "Acept: application/json" \ -H "User-Agent: Mozilla/5.0 AppleWebKit/531.21.10 (KHTML, like Gecko) Mobile/7B405" \ -X POST -d "body=Now we play" \ http://forum-meddlings.rhcloud.com/comment/1
- OK, I saw the comments. Working with cookies is interesting topic for you?
Lets log in into our http://www.abv.bg account :). If you don’t have any, just sign up. Now we are going to use cURL to log in:
curl -c cookiefile i -X POST \ -d "username=<your-user-name>&password=<your-password>&host=mail01.abv.bg&service=mail" \ https://passport.abv.bg/app/profiles/servicelogin
What we’ve done so far?
– We’ve posted our credentials (and additional data, that I got from the HTML form tag of the login page) to the ‘https://passport.abv.bg/app/profiles/servicelogin’ URL and told cURL that our cookie jar is ‘cookiefile’.
# -c (–cookie-jar) Tells cURL to start its cookie engine and write cookies to the given file after the request
– Now we can use this ‘cookiefile’ to send cookies to the site.
curl -b cookiefile -L -i \ http://mail50.abv.bg/app/servlet/plogin?s=<some-id-from-the-response>
Now we should be able to view the markup of our inbox.
# -b (–cookie) Tells cURL a file to read cookies from and start its cookie engine, or if it isn’t a file it will pass on the given string. -b name=var works and so does -b cookiefile.
# -L (–location) Tells cURL to follow redirect location; The request above returns a 302 response and cURL follows the new location.
It is common to use something like this to write and read from the cookie file for the same request:
curl -b cookies -c cookies http://www.somewhere.com
So this is how you can use cURL as HTTP client, you can type ‘man curl’ to see all the other options it supports. For example cURL can be used as FTP or SFTP client too.