CURL : All I Ever Wanted Was Before My Eyes (I’m talking about HTTP client)

The program cURL is a capable HTTP client. This means we can do a variety of HTTP queries through the command line and test our services with ease, using cURL. It’s OK for testing purposes, but sometimes you will need to parse the response and do something with it, in that case you should use something more complex… Anyway lets start:

  1. Requesting a page:
    curl http://forum-meddlings.rhcloud.com

    This executes a HTTP GET request to the given URL, simple and working.

  2. Requesting a page and viewing it’s response headers:
    curl -s -D - http://forum-meddlings.rhcloud.com -o /dev/null

    -s (–silent) Silent or quiet mode. Don’t show progress meter or error messages.
    # -D (–dump-header) <file> Writes the protocol headers to the specified file (if no file is specified to the output).
    # -o (–output) <file> Writes the response to the file specified.
    So we are dumping the headers to the console output and the response to the black hole of nothingness. This is helpful if we want to explore only the headers of the response, if you want to view both the header and the body of the response, just  skip the -o option. Another way of doing this is:

    curl -i  http://forum-meddlings.rhcloud.com

    # -i (–include) Include the HTTP-header in the output.

  3. Request with a different HTTP method. Posting data through the POST verb:
    curl -i -X POST -d "author=meddle&body=BLAME" \
    http://forum-meddlings.rhcloud.com/comments
    

    Now this posts to the given URL the data. I separated the command to be on two lines for readability…
    # -X (–request) Specifies the request method to use when communicating with the server.
    # -d (–data) The data for the request, in the example we have POST parameters.

  4. Passing request headers:
    curl -i -H "X-HTTP-Method-Override: PUT" -X POST -d "body=Yohohohoho" \
        http://forum-meddlings.rhcloud.com/comment/1

    We changed the HTTP method through the method override header.
    # -H (–header) Adds a request header.

    We can add as many headers as we want:

    curl -i  \
        -H "X-HTTP-Method-Override: PUT" \
        -H "Acept: application/json" \
        -H "User-Agent: Mozilla/5.0 AppleWebKit/531.21.10 (KHTML, like Gecko) Mobile/7B405" \
        -X POST -d "body=Now we play" \
        http://forum-meddlings.rhcloud.com/comment/1
  5. OK, I saw the comments. Working with cookies is interesting topic for you?
    Lets log in into our http://www.abv.bg account :). If you don’t have any, just sign up. Now we are going to use cURL to log in:

    curl -c cookiefile i -X POST \
      -d "username=<your-user-name>&password=<your-password>&host=mail01.abv.bg&service=mail" \
      https://passport.abv.bg/app/profiles/servicelogin

    What we’ve done so far?
    – We’ve posted our credentials (and additional data, that I got from the HTML form tag of the login page) to the https://passport.abv.bg/app/profiles/servicelogin’ URL and told cURL that our cookie jar is ‘cookiefile’.
    # -c (–cookie-jar) Tells cURL to start its cookie engine and write cookies to the given file after the request
    – Now we can use this ‘cookiefile’ to send cookies to the site.
    – The response page of the request executes a javascript, replacing the location with a new URL. This is the URL we have to request to get into the home page.

    curl -b cookiefile -L -i \
      http://mail50.abv.bg/app/servlet/plogin?s=<some-id-from-the-response>

    Now we should be able to view the markup of our inbox.
    # -b (–cookie) Tells cURL a file to read cookies from and start its cookie engine, or if it isn’t a file it will pass on the given string. -b name=var works and so does -b cookiefile.
    # -L (–location) Tells cURL to follow redirect location; The request above returns a 302 response and cURL follows the new location.

    It is common to use something like this to write and read from the cookie file for the same request:

    curl -b cookies -c cookies http://www.somewhere.com

So this is how you can use cURL as HTTP client, you can type ‘man curl’ to see all the other options it supports. For example cURL can be used as FTP or SFTP client too.

Advertisements

7 thoughts on “CURL : All I Ever Wanted Was Before My Eyes (I’m talking about HTTP client)

  1. Martin Gogov (@mgogov)

    Good tips, an example with cookies would be great, indeed. It would very handy for writing curl scripts for testing web services which require authentication or just to speed up development.

    Reply
  2. pinspin

    I was recommended this blog by my cousin. I’m not sure whether this post is written
    by him as nobody else know such detailed about my trouble.
    You are amazing! Thanks!

    Reply
  3. malaysia mmm

    Greetings! I’ve been reading your web site for some time now and finally got the bravery to
    go ahead and give you a shout out from Atascocita Tx!

    Just wanted to say keep up the great work!

    Reply
  4. forever drake video

    I do not even know thhe wayy I stopped up
    right here, however I thought this put up waas good.
    I don’t understand who you’re however definitely you
    are going to a famous blogger for thse who aren’t already.

    Cheers!

    fashion plaza amazon
    fashion nova northridge

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s