Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

Introduction HTTP via cURL

•
•
Kyosuke MOROHASHI
Kyosuke MOROHASHI

curl is a command line tool used to transfer data from or to a server using various protocols. It allows sending HTTP requests and receiving HTTP responses, including reading response headers and content. curl is useful for web developers to test HTTP requests and debug communications between servers and clients. It has options to set request headers, send POST data, handle cookies, perform authentication, and more.

1 of 44
Downloaded 50 times
cURL ( ) 2011-09-12( )
curl - transfer a URL • curl is a tool to transfer data from or to a server, using one of the supported protocols (DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, TELNET and TFTP). 2011-09-12( )
why cURL is useful? • curl -O http://example.com/ wallpapers/img[01-19].jpg 2011-09-12( )
why cURL is useful for Web Developers 2011-09-12( )
HTTP $ telnet www.google.co.jp 80 Trying 74.125.153.105... Connected to www.l.google.com. Escape character is '^]'. GET / HTTP/1.1 Host: www.google.co.jp < > 2011-09-12( )
GET / HTTP/1.1 [method] [request-uri] HTTP/1.1 2011-09-12( )
2011-09-12( )
Host: www.google.co.jp Host: < > 2011-09-12( )
HTTP $ telnet www.google.co.jp 80 Trying 74.125.153.105... Connected to www.l.google.com. Escape character is '^]'. GET / HTTP/1.1 Host: www.google.co.jp < > 2011-09-12( )
curl $ curl -v http://www.google.co.jp * About to connect() to www.google.co.jp port 80 (#0) * Trying 74.125.153.106... connected* Connected to www.google.co.jp (74.125.153.106) port 80 (#0) > GET / HTTP/1.1 > User-Agent: curl/7.21.7 ... > Host: www.google.co.jp > Accept: */* > < HTTP/1.1 200 OK < Date: Thu, 08 Sep 2011 06:47:44 GMT < Expires: -1 < Cache-Control: private, max-age=0 2011-09-12( )
curl --head curl --verbose|-v (curl --trace-ascii dump) 2011-09-12( )
HTTP Header $ curl -v http://www.google.co.jp * (ry) > GET / HTTP/1.1 > User-Agent: curl/7.21.7 ... > Host: www.google.co.jp > Accept: */* > < HTTP/1.1 200 OK < Date: Thu, 08 Sep 2011 06:47:44 GMT < Expires: -1 < Cache-Control: private, max-age=0 2011-09-12( )
HTTP Request Headers Accept Accept-Language Authorization Host Referer User-Agent 2011-09-12( )
$ curl -v -H 'Accept-Language: ja' http://twitter.com | nokogrep -a content 'meta[name=description]' > GET / HTTP/1.1 > User-Agent: curl/7.21.7 > Host: twitter.com > Accept: */* > Accept-Language: ja > < HTTP/1.1 200 OK ... 2011-09-12( )
$ curl -v -H 'Accept-Language: en' http://twitter.com | nokogrep -a content 'meta[name=description]' > GET / HTTP/1.1 > User-Agent: curl/7.21.7 > Host: twitter.com > Accept: */* > Accept-Language: en > < HTTP/1.1 200 OK ... Instantly connect to what's most important to you. Follow your friends, experts, favorite celebrities, and breaking news. 2011-09-12( )
$ curl -v -H 'Accept: application/json' http://jobs.dev/jobs | ruby -rpp -rjson -e 'pp JSON.parse(STDIN.read)' > GET /jobs HTTP/1.1 > User-Agent: curl/7.21.7 > Host: jobs.dev > Accept: application/json 2011-09-12( )
< HTTP/1.1 200 OK < Content-Type: application/json; charset=utf-8 < X-UA-Compatible: IE=Edge < ETag: "1490b2d47b09a5abb92bd7eb09c4c51e" < Cache-Control: max-age=0, private, must-revalidate < X-Runtime: 0.059816 < Connection: keep-alive < Transfer-Encoding: chunked [{"created_at"=>"2011-09-07T05:56:22Z", "deadline"=>"2011-09-07", "description"=>" !!!", "id"=>1, "public"=>true, "title"=>"10 ", "updated_at"=>"2011-09-07T09:59:33Z"}] 2011-09-12( )
HTTP Response HTTP/1.1 200 OK Date: Mon, 12 Sep 2011 01:27:34 GMT Server: hi Status: 200 OK ETag: "ceedd0ac4f0339411aab820e225afcdf" Content-Type: text/html; charset=utf-8 Content-Length: 44188 Pragma: no-cache Expires: Tue, 31 Mar 1981 05:00:00 GMT Cache-Control: no-cache, no-store, must- revalidate, pre-check=0, post-check=0 X-XSS-Protection: 1; mode=block 2011-09-12( )
HTTP Response HTTP/1.1 200 OK Date: Mon, 12 Sep 2011 01:27:34 GMT Server: hi Status: 200 OK ETag: "ceedd0ac4f0339411aab820e225afcdf" Content-Type: text/html; charset=utf-8 Content-Length: 44188 Pragma: no-cache Expires: Tue, 31 Mar 1981 05:00:00 GMT Cache-Control: no-cache, no-store, must- revalidate, pre-check=0, post-check=0 X-XSS-Protection: 1; mode=block 2011-09-12( )
Status line 200 302 400 404 500 503 2011-09-12( )
HTTP Status is app UI 200 404 ? 410(Gone) 2011-09-12( )
HTTP Response Headers Location Server Content-Type Content-Length Expire, ETag, Cache-Control 2011-09-12( )
curl --data|-d 2011-09-12( )
POST via curl $ curl -v -d email=moro@example.jp -d password=hogehoge http://example.com/users POST /users HTTP/1.1 User-Agent: curl/7.21.7 Host: example.com Accept: */* Content-Length: 36 Content-Type: application/x-www-form-urlencoded 2011-09-12( )
HTTP/1.1 201 Created Content-Type: application/json Date: Mon, 12 Sep 2011 01:43:41 GMT Server: journey/0.4.0 Content-Length: 71 Connection: keep-alive {"email":"moro@example.jp","_id":"4e6d63cd981 5710100000005","profiles":[]} 2011-09-12( )
curl --user|-u 2011-09-12( )
HTTP Authentication $ curl -v -u moro@example.jp:hogehoge http://example.com/account GET /account HTTP/1.1 Authorization: Basic bW9yb0BleGFtcGxlLmpwOmhvZ2Vob2dl User-Agent: curl/7.21.7 Host: example.com Accept: */* 2011-09-12( )
HTTP Authentication HTTP/1.1 200 OK Content-Type: application/json Date: Mon, 12 Sep 2011 01:51:00 GMT Server: journey/0.4.0 Content-Length: 71 Connection: keep-alive {"email":"moro@example.jp","_id":"4e6d63cd9 815710100000005","profiles":[]} 2011-09-12( )
cookie & cURL http://flic.kr/p/5ekGdX 2011-09-12( )
cookie • HTTP {Cookie} Web http://www.studyinghttp.net/cookies 2011-09-12( )
[res] Set-Cookie [req] Cookie • Netscape Set-Cookie Cookie 2 HTTP http://www.studyinghttp.net/cookies#HeadersForCookie 2011-09-12( )
2011-09-12( )
2011-09-12( )
curl --cookie|-b 2011-09-12( )
without cookie $ curl -q http://jobs.dev | nokogrep '#header p.greeting' Who are you? Sign in 2011-09-12( )
Cookie & cURL $ curl -v -b '_jobs_session=BAh7CEkiD3Nl( )' http://jobs.dev | nokogrep '#header p.greeting' GET / HTTP/1.1 User-Agent: curl/7.21.7 Host: jobs.dev Accept: */* Cookie: _jobs_session=BAh7CEkiD3Nl( ) 2011-09-12( )
Cookie & cURL HTTP/1.1 200 OK Content-Type: text/html; charset=utf-8 X-UA-Compatible: IE=Edge ETag: "8c0c63144925e4cc29b655906c221a3f" Set-Cookie:_jobs_session=BAh7CEkiD3Nl( ) X-Runtime: 0.156343 Connection: keep-alive Transfer-Encoding: chunked Hi moro@example.jp Sign out 2011-09-12( )
2011-09-12( )
$ curl -v -d 'user[email]=moro@example.jp' -d 'user[password]=hogehoge' http://jobs.dev/users/sign_in POST /users/sign_in HTTP/1.1 User-Agent: curl/7.21.7 Host: jobs.dev Accept: */* Content-Length: 51 Content-Type: application/x-www-form-urlencoded 2011-09-12( )
HTTP/1.1 302 Moved Temporarily Location: http://jobs.dev/ Content-Type: text/html; charset=utf-8 X-UA-Compatible: IE=Edge Cache-Control: no-cache Set-Cookie: _jobs_session=BAh7CE( ) X-Runtime: 0.217055 Connection: keep-alive Transfer-Encoding: chunked <html><body>You are being <a href="http://jobs.dev/">redirected</ a>.</body></html> 2011-09-12( )
$ curl -v -b '_jobs_session=BAh7CE( )' http://jobs.dev/ | nokogrep '#header p.greeting' GET / HTTP/1.1 User-Agent: curl/7.21.7 Host: jobs.dev Accept: */* Cookie: _jobs_session=BAh7CE( ) 2011-09-12( )
HTTP/1.1 200 OK Content-Type: text/html; charset=utf-8 X-UA-Compatible: IE=Edge ETag: "98d16875bdceb8c5451e7706e6071ece" Cache-Control: max-age=0, private, must-revalidate Set-Cookie: _jobs_session=BAh7CUki( ) X-Runtime: 0.193951 Connection: keep-alive Transfer-Encoding: chunked Hi moro@example.jp Sign out 2011-09-12( )
2011-09-12( )
see also • http://www.studyinghttp.net/ • http://www.studyinghttp.net/ rfc_ja/rfc2616 • http://amazon.jp/dp/ 4774142042/morodiary05-22/ ref=nosim/ 2011-09-12( )

More Related Content

Introduction HTTP via cURL

  • 1. cURL ( ) 2011-09-12( )
  • 2. curl - transfer a URL • curl is a tool to transfer data from or to a server, using one of the supported protocols (DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, TELNET and TFTP). 2011-09-12( )
  • 3. why cURL is useful? • curl -O http://example.com/ wallpapers/img[01-19].jpg 2011-09-12( )
  • 4. why cURL is useful for Web Developers 2011-09-12( )
  • 5. HTTP $ telnet www.google.co.jp 80 Trying 74.125.153.105... Connected to www.l.google.com. Escape character is '^]'. GET / HTTP/1.1 Host: www.google.co.jp < > 2011-09-12( )
  • 6. GET / HTTP/1.1 [method] [request-uri] HTTP/1.1 2011-09-12( )
  • 8. Host: www.google.co.jp Host: < > 2011-09-12( )
  • 9. HTTP $ telnet www.google.co.jp 80 Trying 74.125.153.105... Connected to www.l.google.com. Escape character is '^]'. GET / HTTP/1.1 Host: www.google.co.jp < > 2011-09-12( )
  • 10. curl $ curl -v http://www.google.co.jp * About to connect() to www.google.co.jp port 80 (#0) * Trying 74.125.153.106... connected* Connected to www.google.co.jp (74.125.153.106) port 80 (#0) > GET / HTTP/1.1 > User-Agent: curl/7.21.7 ... > Host: www.google.co.jp > Accept: */* > < HTTP/1.1 200 OK < Date: Thu, 08 Sep 2011 06:47:44 GMT < Expires: -1 < Cache-Control: private, max-age=0 2011-09-12( )
  • 11. curl --head curl --verbose|-v (curl --trace-ascii dump) 2011-09-12( )
  • 12. HTTP Header $ curl -v http://www.google.co.jp * (ry) > GET / HTTP/1.1 > User-Agent: curl/7.21.7 ... > Host: www.google.co.jp > Accept: */* > < HTTP/1.1 200 OK < Date: Thu, 08 Sep 2011 06:47:44 GMT < Expires: -1 < Cache-Control: private, max-age=0 2011-09-12( )
  • 13. HTTP Request Headers Accept Accept-Language Authorization Host Referer User-Agent 2011-09-12( )
  • 14. $ curl -v -H 'Accept-Language: ja' http://twitter.com | nokogrep -a content 'meta[name=description]' > GET / HTTP/1.1 > User-Agent: curl/7.21.7 > Host: twitter.com > Accept: */* > Accept-Language: ja > < HTTP/1.1 200 OK ... 2011-09-12( )
  • 15. $ curl -v -H 'Accept-Language: en' http://twitter.com | nokogrep -a content 'meta[name=description]' > GET / HTTP/1.1 > User-Agent: curl/7.21.7 > Host: twitter.com > Accept: */* > Accept-Language: en > < HTTP/1.1 200 OK ... Instantly connect to what's most important to you. Follow your friends, experts, favorite celebrities, and breaking news. 2011-09-12( )
  • 16. $ curl -v -H 'Accept: application/json' http://jobs.dev/jobs | ruby -rpp -rjson -e 'pp JSON.parse(STDIN.read)' > GET /jobs HTTP/1.1 > User-Agent: curl/7.21.7 > Host: jobs.dev > Accept: application/json 2011-09-12( )
  • 17. < HTTP/1.1 200 OK < Content-Type: application/json; charset=utf-8 < X-UA-Compatible: IE=Edge < ETag: "1490b2d47b09a5abb92bd7eb09c4c51e" < Cache-Control: max-age=0, private, must-revalidate < X-Runtime: 0.059816 < Connection: keep-alive < Transfer-Encoding: chunked [{"created_at"=>"2011-09-07T05:56:22Z", "deadline"=>"2011-09-07", "description"=>" !!!", "id"=>1, "public"=>true, "title"=>"10 ", "updated_at"=>"2011-09-07T09:59:33Z"}] 2011-09-12( )
  • 18. HTTP Response HTTP/1.1 200 OK Date: Mon, 12 Sep 2011 01:27:34 GMT Server: hi Status: 200 OK ETag: "ceedd0ac4f0339411aab820e225afcdf" Content-Type: text/html; charset=utf-8 Content-Length: 44188 Pragma: no-cache Expires: Tue, 31 Mar 1981 05:00:00 GMT Cache-Control: no-cache, no-store, must- revalidate, pre-check=0, post-check=0 X-XSS-Protection: 1; mode=block 2011-09-12( )
  • 19. HTTP Response HTTP/1.1 200 OK Date: Mon, 12 Sep 2011 01:27:34 GMT Server: hi Status: 200 OK ETag: "ceedd0ac4f0339411aab820e225afcdf" Content-Type: text/html; charset=utf-8 Content-Length: 44188 Pragma: no-cache Expires: Tue, 31 Mar 1981 05:00:00 GMT Cache-Control: no-cache, no-store, must- revalidate, pre-check=0, post-check=0 X-XSS-Protection: 1; mode=block 2011-09-12( )
  • 20. Status line 200 302 400 404 500 503 2011-09-12( )
  • 21. HTTP Status is app UI 200 404 ? 410(Gone) 2011-09-12( )
  • 22. HTTP Response Headers Location Server Content-Type Content-Length Expire, ETag, Cache-Control 2011-09-12( )
  • 24. POST via curl $ curl -v -d email=moro@example.jp -d password=hogehoge http://example.com/users POST /users HTTP/1.1 User-Agent: curl/7.21.7 Host: example.com Accept: */* Content-Length: 36 Content-Type: application/x-www-form-urlencoded 2011-09-12( )
  • 25. HTTP/1.1 201 Created Content-Type: application/json Date: Mon, 12 Sep 2011 01:43:41 GMT Server: journey/0.4.0 Content-Length: 71 Connection: keep-alive {"email":"moro@example.jp","_id":"4e6d63cd981 5710100000005","profiles":[]} 2011-09-12( )
  • 27. HTTP Authentication $ curl -v -u moro@example.jp:hogehoge http://example.com/account GET /account HTTP/1.1 Authorization: Basic bW9yb0BleGFtcGxlLmpwOmhvZ2Vob2dl User-Agent: curl/7.21.7 Host: example.com Accept: */* 2011-09-12( )
  • 28. HTTP Authentication HTTP/1.1 200 OK Content-Type: application/json Date: Mon, 12 Sep 2011 01:51:00 GMT Server: journey/0.4.0 Content-Length: 71 Connection: keep-alive {"email":"moro@example.jp","_id":"4e6d63cd9 815710100000005","profiles":[]} 2011-09-12( )
  • 29. cookie & cURL http://flic.kr/p/5ekGdX 2011-09-12( )
  • 30. cookie • HTTP {Cookie} Web http://www.studyinghttp.net/cookies 2011-09-12( )
  • 31. [res] Set-Cookie [req] Cookie • Netscape Set-Cookie Cookie 2 HTTP http://www.studyinghttp.net/cookies#HeadersForCookie 2011-09-12( )
  • 35. without cookie $ curl -q http://jobs.dev | nokogrep '#header p.greeting' Who are you? Sign in 2011-09-12( )
  • 36. Cookie & cURL $ curl -v -b '_jobs_session=BAh7CEkiD3Nl( )' http://jobs.dev | nokogrep '#header p.greeting' GET / HTTP/1.1 User-Agent: curl/7.21.7 Host: jobs.dev Accept: */* Cookie: _jobs_session=BAh7CEkiD3Nl( ) 2011-09-12( )
  • 37. Cookie & cURL HTTP/1.1 200 OK Content-Type: text/html; charset=utf-8 X-UA-Compatible: IE=Edge ETag: "8c0c63144925e4cc29b655906c221a3f" Set-Cookie:_jobs_session=BAh7CEkiD3Nl( ) X-Runtime: 0.156343 Connection: keep-alive Transfer-Encoding: chunked Hi moro@example.jp Sign out 2011-09-12( )
  • 39. $ curl -v -d 'user[email]=moro@example.jp' -d 'user[password]=hogehoge' http://jobs.dev/users/sign_in POST /users/sign_in HTTP/1.1 User-Agent: curl/7.21.7 Host: jobs.dev Accept: */* Content-Length: 51 Content-Type: application/x-www-form-urlencoded 2011-09-12( )
  • 40. HTTP/1.1 302 Moved Temporarily Location: http://jobs.dev/ Content-Type: text/html; charset=utf-8 X-UA-Compatible: IE=Edge Cache-Control: no-cache Set-Cookie: _jobs_session=BAh7CE( ) X-Runtime: 0.217055 Connection: keep-alive Transfer-Encoding: chunked <html><body>You are being <a href="http://jobs.dev/">redirected</ a>.</body></html> 2011-09-12( )
  • 41. $ curl -v -b '_jobs_session=BAh7CE( )' http://jobs.dev/ | nokogrep '#header p.greeting' GET / HTTP/1.1 User-Agent: curl/7.21.7 Host: jobs.dev Accept: */* Cookie: _jobs_session=BAh7CE( ) 2011-09-12( )
  • 42. HTTP/1.1 200 OK Content-Type: text/html; charset=utf-8 X-UA-Compatible: IE=Edge ETag: "98d16875bdceb8c5451e7706e6071ece" Cache-Control: max-age=0, private, must-revalidate Set-Cookie: _jobs_session=BAh7CUki( ) X-Runtime: 0.193951 Connection: keep-alive Transfer-Encoding: chunked Hi moro@example.jp Sign out 2011-09-12( )
  • 44. see also • http://www.studyinghttp.net/ • http://www.studyinghttp.net/ rfc_ja/rfc2616 • http://amazon.jp/dp/ 4774142042/morodiary05-22/ ref=nosim/ 2011-09-12( )