Requests 庫學習筆記

01-29

1、概覽

1.1、實例引入

# 引入Requests庫nimport requestsnn# 發起GET請求nresponse = requests.get(https://www.baidu.com/)n# 查看響應類型 requests.models.Responsenprint(type(response))n# 輸出狀態碼nprint(response.status_code)n# 輸出響應內容類型 textnprint(type(response.text))n# 輸出響應內容nprint(response.text)n# 輸出cookiesnprint(response.cookies)n

1.2、各種請求方式

import requestsnn# 發起POST請求nrequests.post(http://httpbin.org/post)n# 發起PUT請求nrequests.put(http://httpbin.org/put)n# 發起DELETE請求nrequests.delete(http://httpbin.org/delete)n# 發送HEAD請求nrequests.head(http://httpbin.org/get)n# 發送OPTION請求nrequests.options(http://httpbin.org/get)n

2、請求

2.1 、基本GET請求

2.1.1、基本寫法

import requestsnnresponse = requests.get(http://httpbin.org/get)nprint(response.text)n

2.1.2、帶參數的GET請求

import requestsnnresponse = requests.get(http://httpbin.org/get?name=jyx&age=18)nprint(response.text)n

2.1.3、帶參數的GET請求(2)

import requestsnn# 分裝GET請求參數nparam = {name:jyx,age:19}n# 設置GET請求參數(Params)nresponse = requests.get(http://httpbin.org/get,params=param)nprint(response.text)n

2.1.4、解析json

import requestsnnresponse = requests.get(http://httpbin.org/get)n# 獲取響應內容nprint(type(response.text))n# 如果響應內容是json,就將其轉為jsonnprint(response.json())n# 輸出的是字典類型nprint(type(response.json()))n

2.1.5、獲取二進位數據

import requestsnnresponse = requests.get(http://github.com/favicon.ico)nn# str，bytesnprint(type(response.text),type(response.content))n# 輸出響應的文本內容nprint(response.text)n# 輸出響應的二進位內容nprint(response.content)n# 下載二進位數據到本地nwith open(favicon.ico,wb) as f:n f.write(response.content)n f.close()n

2.1.6、添加headers

import requestsnn# 設置User-Agent瀏覽器信息nheaders = {n "User-Agent": "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36"n}n# 設置請求頭信息nresponse = requests.get(https://www.zhihu.com/explore,headers=headers)nprint(response.text)n

2.2、基本POST請求

import requestsnn# 設置傳入post表單信息ndata= { name:jyx, age:18}n# 設置請求頭信息nheaders = {n "User-Agent": "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36"n}n# 設置請求頭信息和POST請求參數(data)nresponse = requests.post(http://httpbin.org/post, data=data, headers=headers)nprint(response.text)n

3、響應

3.1 response屬性

import requestsnnresponse = requests.get(http://www.jianshu.com/)n# 獲取響應狀態碼nprint(type(response.status_code),response.status_code)n# 獲取響應頭信息n print(type(response.headers),response.headers)n# 獲取響應頭中的cookiesnprint(type(response.cookies),response.cookies)n# 獲取訪問的urln print(type(response.url),response.url)n# 獲取訪問的歷史記錄n print(type(response.history),response.history)n

3.2、狀態碼判斷

import requestsnnresponse = requests.get(http://www.jianshu.com/404.html)n# 使用request內置的字母判斷狀態碼nif not response.status_code == requests.codes.ok:n print(404-1)nresponse = requests.get(http://www.jianshu.com)n# 使用狀態碼數字判斷nif not response.status_code == 200:n print(404-2)n

3.3 requests內置的狀態字元

100: (continue,),
101: (switching_protocols,),
102: (processing,),
103: (checkpoint,),
122: (uri_too_long, request_uri_too_long),
200: (ok, okay, all_ok, all_okay, all_good, o/, ?),
201: (created,),
202: (accepted,),

203: (non_authoritative_info, non_authoritative_information),
204: (no_content,),
205: (reset_content, reset),
206: (partial_content, partial),
207: (multi_status, multiple_status, multi_stati, multiple_stati),
208: (already_reported,),
226: (im_used,),
# Redirection.
300: (multiple_choices,),
301: (moved_permanently, moved, ),

302: (found,),
303: (see_other, other),
304: (not_modified,),
305: (use_proxy,),
306: (switch_proxy,),
307: (temporary_redirect, temporary_moved, temporary),
308: (permanent_redirect, resume_incomplete, resume,), # These 2 to be removed in 3.0
# Client Error.
400: (bad_request, bad),
401: (unauthorized,),

402: (payment_required, payment),
403: (forbidden,),
404: (not_found, -),
405: (method_not_allowed, not_allowed),
406: (not_acceptable,),
407: (proxy_authentication_required, proxy_auth, proxy_authentication),
408: (request_timeout, timeout),
409: (conflict,),
410: (gone,),
411: (length_required,),

412: (precondition_failed, precondition),
413: (request_entity_too_large,),
414: (request_uri_too_large,),
415: (unsupported_media_type, unsupported_media, media_type),
416: (requested_range_not_satisfiable, requested_range, range_not_satisfiable),
417: (expectation_failed,),
418: (im_a_teapot, teapot, i_am_a_teapot),
421: (misdirected_request,),
422: (unprocessable_entity, unprocessable),
423: (locked,),

424: (failed_dependency, dependency),
425: (unordered_collection, unordered),
426: (upgrade_required, upgrade),
428: (precondition_required, precondition),
429: (too_many_requests, too_many),
431: (header_fields_too_large, fields_too_large),
444: (no_response, none),
449: (retry_with, retry),
450: (blocked_by_windows_parental_controls, parental_controls),
451: (unavailable_for_legal_reasons, legal_reasons),

499: (client_closed_request,),
# Server Error.
500: (internal_server_error, server_error, /o, ?),
501: (not_implemented,),
502: (bad_gateway,),
503: (service_unavailable, unavailable),
504: (gateway_timeout,),
505: (http_version_not_supported, http_version),
506: (variant_also_negotiates,),
507: (insufficient_storage,),

509: (bandwidth_limit_exceeded, bandwidth),
510: (not_extended,),
511: (network_authentication_required, network_auth, network_authentication),

4、高級操作

4.1、文件上傳

import requestsnnfiles = {file:open(favicon.ico,rb)}n# 往POST請求頭中設置文件(files)nresponse = requests.post(http://httpbin.org/post,files=files)nprint(response.text)n

4.2、獲取cookies

import requestsnnresponse = requests.get(https://www.baidu.com)nprint(response.cookies)nfor key,value in response.cookies.items():n print(key,=====,value)n

4.3、會話維持

4.3.1、普通請求

import requestsnnrequests.get(http://httpbin.org/cookies/set/number/12456)nresponse = requests.get(http://httpbin.org/cookies)n# 本質上是兩次不同的請求，session不一致nprint(response.text)n

4.3.2、會話維持請求

import requestsnn# 從Requests中獲取sessionnsession = requests.session()n# 使用seesion去請求保證了請求是同一個sessionnsession.get(http://httpbin.org/cookies/set/number/12456)nresponse = session.get(http://httpbin.org/cookies)nprint(response.text)n

4.4、證書驗證

4.4.1、無證書訪問

import requestsnnresponse = requests.get(https://www.12306.cn)n# 在請求https時，request會進行證書的驗證，如果驗證失敗則會拋出異常nprint(response.status_code) n

4.4.2、關閉證書驗證

import requestsnn# 關閉驗證，但是仍然會報出證書警告nresponse = requests.get(https://www.12306.cn,verify=False)nprint(response.status_code)n

4.4.3、消除關閉證書驗證的警告

from requests.packages import urllib3nimport requestsnn# 關閉警告nurllib3.disable_warnings()nresponse = requests.get(https://www.12306.cn,verify=False)nprint(response.status_code)n

4.4.4、手動設置證書

import requestsnn# 設置本地證書nresponse = requests.get(https://www.12306.cn, cert=(/path/server.crt, /path/key))nprint(response.status_code) n

4.5、代理設置

4.5.1、設置普通代理

import requestsnnproxies = {n "http": "http://127.0.0.1:9743",n "https": "https://127.0.0.1:9743",n}n# 往請求中設置代理(proxiesn)nresponse = requests.get("https://www.taobao.com", proxies=proxies)nprint(response.status_code)n

4.5.2、設置帶有用戶名和密碼的代理

import requestsnnproxies = {n "http": "http://user:password@127.0.0.1:9743/",n}nresponse = requests.get("https://www.taobao.com", proxies=proxies)nprint(response.status_code)n

4.5.3、設置socks代理

pip3 install requests[socks]

import requestsnnproxies = {n http: socks5://127.0.0.1:9742,n https: socks5://127.0.0.1:9742n}nresponse = requests.get("https://www.taobao.com", proxies=proxies)nprint(response.status_code)n

4.6、超時設置

import requestsnfrom requests.exceptions import ReadTimeoutnn nntry:n # 設置必須在500ms內收到響應，不然或拋出ReadTimeout異常n response = requests.get("http://httpbin.org/get", timeout=0.5)n print(response.status_code)nexcept ReadTimeout:n print(Timeout)n

4.7、認證設置

import requestsnfrom requests.auth import HTTPBasicAuthnnr = requests.get(http://120.27.34.24:9001, auth=HTTPBasicAuth(user, 123))n# r = requests.get(http://120.27.34.24:9001, auth=(user, 123))nprint(r.status_code)n

4.8、異常處理

import requestsnfrom requests.exceptions import ReadTimeout, ConnectionError, RequestExceptionnntry:n response = requests.get("http://httpbin.org/get", timeout = 0.5)n print(response.status_code)nexcept ReadTimeout:n # 超時異常n print(Timeout)nexcept ConnectionError:n # 連接異常n print(Connection error)nexcept RequestException:n # 請求異常n print(Error)n

作者：蔣蜀黍 Python愛好者社區專欄作者授權原創發布，請勿轉載，謝謝。
出處：Requests 庫學習筆記
推薦免費視頻教程：Python3爬蟲三大案例實戰分享：貓眼電影、今日頭條街拍美圖、淘寶美食 Python3爬蟲三大案例實戰分享
公眾號：Python愛好者社區（微信ID：python_shequ）
小編個人微信：tsdatajob ，來不急回復的，可以加小編微信溝通，謝謝。