3
Lj* @ sr d Z ddlZddlZddlZddlZddlZddlZddlZddl Z ddl
m Z dddddd d
d d d
ddddddddgZ dZ
dZdZdZdZdZe jejj d d ejjj D Zd Zd Zd Zej d!j Z ej d"j Z ej d#Z!ej d$Z"d%d&d'hZ#dCd)d*Z$G d+d, d,ej%j&Z'd-d. Z(e'fd/d0Z)G d1d dej*Z+G d2d dZ,y ddl-Z-W n e.k
rl Y n X G d3d4 d4e,Z/e j0d4 G d5d de1Z2G d6d de2Z3G d7d d e2Z4G d8d de2Z5G d9d d e2Z6G d:d
d
e2Z7G d;d d e2Z8G dd de9Z;G d?d de9ZG dBd de?e=Z@e2ZAdS )Da
HTTP/1.1 client library
HTTPConnection goes through a number of "states", which define when a client
may legally make another request or fetch the response for a particular
request. This diagram details these state transitions:
(null)
|
| HTTPConnection()
v
Idle
|
| putrequest()
v
Request-started
|
| ( putheader() )* endheaders()
v
Request-sent
|\_____________________________
| | getresponse() raises
| response = getresponse() | ConnectionError
v v
Unread-response Idle
[Response-headers-read]
|\____________________
| |
| response.read() | putrequest()
v v
Idle Req-started-unread-response
______/|
/ |
response.read() | | ( putheader() )* endheaders()
v v
Request-started Req-sent-unread-response
|
| response.read()
v
Request-sent
This diagram presents the following rules:
-- a second request may not be started until {response-headers-read}
-- a response [object] cannot be retrieved until {request-sent}
-- there is no differentiation between an unread response body and a
partially read response body
Note: this enforcement is applied by the HTTPConnection class. The
HTTPResponse class does not enforce this state machine, which
implies sophisticated clients may accelerate the request/response
pipeline. Caution should be taken, though: accelerating the states
beyond the above pattern may imply knowledge of the server's
connection-close behavior for certain requests. For example, it
is impossible to tell whether the server will close the connection
UNTIL the response headers have been read; this means that further
requests cannot be placed into the pipeline until it is known that
the server will NOT be closing the connection.
Logical State __state __response
------------- ------- ----------
Idle _CS_IDLE None
Request-started _CS_REQ_STARTED None
Request-sent _CS_REQ_SENT None
Unread-response _CS_IDLE
Req-started-unread-response _CS_REQ_STARTED
Req-sent-unread-response _CS_REQ_SENT
N)urlsplit HTTPResponseHTTPConnection
HTTPException NotConnectedUnknownProtocolUnknownTransferEncodingUnimplementedFileModeIncompleteRead
InvalidURLImproperConnectionStateCannotSendRequestCannotSendHeaderResponseNotReady
BadStatusLine LineTooLongRemoteDisconnectederror responsesP i ZUNKNOWNZIdlezRequest-startedz Request-sentc C s i | ] }|j |qS )phrase).0vr r #/usr/lib64/python3.6/http/client.py
k s r i i d s [^:\s][^:\r\n]*s \n(?![ \t])|\r(?![ \t\n])z[ - ]z[ - ]ZPATCHZPOSTZPUTdatac C sf y
| j dS tk
r` } z:t|j|j|j|jd|j | |j|j |f dW Y dd}~X nX dS )z Zhstringr r r
parse_headers s
rE c s e Zd Zd@ddZdd Zdd Zd d
Zd d Z fd
dZ fddZ dd Z
dd Z dAddZ dd Z
dd Zdd Zd d Zd d Zd!d" Zd#d$ Zd%d& ZdCd(d)ZdEd*d+ZdG fd,d- Zd.d/ Zd0d1 Zd2d3 ZdHd4d5Zd6d7 Z d8d9 Z d:d; Z dd? Z Z!S )Ir r Nc C sR |j d| _|| _|| _d | _| _t| _t| _t| _ t| _
t| _ t| _ t| _
d S )Nrb)makefiler=
debuglevel_methodr> msg_UNKNOWNversionstatusreasonchunked
chunk_leftlength
will_close)r0 sockrH methodurlr r r __init__ s zHTTPResponse.__init__c C s t | jjtd d}t|tkr*td| jdkrBtdt| |sNt dy|j
d d\}}}W nF t k
r y|j
d d\}}d}W n t k
r d}Y nX Y nX |j d s| j
t|y$t|}|d
k s|d krt|W n t k
r t|Y nX |||fS ) Nr* z
iso-8859-1z status liner zreply:z-Remote end closed connection without response zHTTP/r i )strr= r: r; r, r rH printreprr split
ValueError
startswith _close_connr int)r0 r2 rL rM rN r r r _read_status s2
zHTTPResponse._read_statusc C s | j d k rd S x<| j \}}}|tkr(P t| j}| jdkrFtd| ~qW | | _| _|j | _
|dkrrd| _ n|j drd| _ nt
|t| j | _ | _| jdkrx&| j D ] }td|d | j j| qW | j jd
}|r|j d krd | _d | _nd
| _| j | _d | _| j jd}| j jd
}|rx| j rxyt|| _W n tk
rb d | _Y nX | jdk r~d | _nd | _|tks|tksd| kodk n s| jdkrd| _| j r| j r| jd krd | _d S )Nr zheaders:HTTP/1.0HTTP/0.9
zHTTP/1. zheader:r) ztransfer-encodingrO TFzcontent-lengthr HEAD)rb rc ) r> ra ZCONTINUEr? r= rH rZ coderM striprN rL r^ r rE rJ getr+ rO rP _check_closerR rQ r` r] Z
NO_CONTENTZ NOT_MODIFIEDrI )r0 rL rM rN Zskipped_headershdrZtr_encrQ r r r begin9 s\
zHTTPResponse.beginc C s | j jd}| jdkr:| j jd}|r6d|j kr6dS dS | j jdrJdS |r^d|j kr^dS | j jd}|r~d|j kr~dS dS )NZ
connectionre closeTFz
keep-alivezproxy-connection)r> rj rL r+ )r0 ZconnZpconnr r r rk s
zHTTPResponse._check_closec C s | j }d | _ |j d S )N)r= rn )r0 r= r r r r_ s zHTTPResponse._close_connc
s$ zt j W d | jr | j X d S )N)superrn r= r_ )r0 ) __class__r r rn s zHTTPResponse.closec s t j | jr| jj d S )N)ro flushr= )r0 )rp r r rq s
zHTTPResponse.flushc C s dS )zAlways returns TrueTr )r0 r r r readable s zHTTPResponse.readablec C s
| j dkS )z!True if the connection is closed.N)r= )r0 r r r isclosed s zHTTPResponse.isclosedc C s | j d krdS | jdkr$| j dS |d k rRt|}| j|}t|d | j S | jr`| j S | j d krv| j j
}n6y| j | j }W n t k
r | j Y nX d| _ | j |S d S )Nr9 rg r )
r= rI r_ bytearrayreadinto
memoryviewtobytesrO _readall_chunkedrQ read
_safe_readr
)r0 amtbr1 sr r r ry s*
zHTTPResponse.readc C s | j dkrdS | jdkr$| j dS | jr4| j|S | jdk r^t|| jkr^t|d| j }| j j|}| r~|r~| j n&| jdk r| j|8 _| js| j |S )z^Read up to len(b) bytes into bytearray b and return the number
of bytes read.
Nr rg ) r= rI r_ rO _readinto_chunkedrQ r, rv ru )r0 r| r1 r r r ru s$
zHTTPResponse.readintoc C sp | j jtd }t|tkr$td|jd}|dkrB|d | }y
t|dS tk
rj | j Y nX d S )Nr* z
chunk size ;r ) r= r: r; r, r findr` r] r_ )r0 r2 ir r r _read_next_chunk_size
s
z"HTTPResponse._read_next_chunk_sizec C s> x8| j jtd }t|tkr&td|s,P |dkrP qW d S )Nr* z trailer line
r8 r9 )r r8 r9 )r= r: r; r, r )r0 r2 r r r _read_and_discard_trailer s z&HTTPResponse._read_and_discard_trailerc C sl | j }|sh|d k r | jd y | j }W n tk
rD tdY nX |dkrb| j | j d }|| _ |S )NrW r9 r )rP rz r r] r
r r_ )r0 rP r r r _get_chunk_left( s
z HTTPResponse._get_chunk_leftc C sd g }y8x,| j }|d krP |j| j| d| _qW dj|S tk
r^ tdj|Y nX d S )Nr r9 )r r/ rz rP rA r
)r0 valuerP r r r rx @ s
z HTTPResponse._readall_chunkedc C s d}t |}yvxp| j }|d kr$|S t||krL| j|}|| | _|| S |d | }| j|}||d }||7 }d| _qW W n( tk
r tt|d| Y nX d S )Nr )rv r r, _safe_readintorP r
bytes)r0 r| total_bytesmvbrP r1 temp_mvbr r r r~ N s$
z HTTPResponse._readinto_chunkedc C sX g }xH|dkrL| j jt|t}|s4tdj|||j| |t|8 }qW dj|S )aV Read the number of bytes requested, compensating for partial reads.
Normally, we have a blocking socket, but a read() can be interrupted
by a signal (resulting in a partial read).
Note that we cannot distinguish between EOF and an interrupt when zero
bytes have been read. IncompleteRead() will be raised in this
situation.
This function should be used when bytes "should" be present for
reading. If the bytes are truly not available (due to EOF), then the
IncompleteRead exception can be used to detect the problem.
r r9 )r= ry min MAXAMOUNTr
rA r/ r, )r0 r{ r} chunkr r r rz f s
zHTTPResponse._safe_readc C s d}t |}xt|t|k rtt|k r@|dt }| jj|}n | jj|}|sjtt|d| t|||d }||7 }qW |S )z2Same as _safe_read, but for reading into a buffer.r N)rv r, r r= ru r
r )r0 r| r r r r1 r r r r } s zHTTPResponse._safe_readintor* c C s | j dks| jdkrdS | jr(| j|S | jdk rJ|dk sD|| jkrJ| j}y| j j|}W n* tk
r |dkrt | j jd}Y nX | r|r| j n | jdk r| jt|8 _|S )zvRead with at most one underlying system call. If at least one
byte is buffered, return that instead.
Nrg r9 r r i i @ ) r= rI rO _read1_chunkedrQ read1r] r_ r, )r0 r1 resultr r r r s"
zHTTPResponse.read1c C s4 | j d ks| jdkrdS | jr(| j|S | j j|S )Nrg r9 )r= rI rO
_peek_chunkedpeek)r0 r1 r r r r s
zHTTPResponse.peekc s | j d ks| jdkrdS | jr*t j|S | jd k rL|dk sF|| jkrL| j}| j j|}| rl|rl| j n | jd k r| jt|8 _|S )Nrg r9 r )r= rI rO ro r: rQ r_ r, )r0 limitr )rp r r r: s
zHTTPResponse.readlinec C sf | j }|d ks|dkr dS d| ko.|kn s8|}| jj|}| jt|8 _|sbtd|S )Nr r9 )r r= r rP r, r
)r0 r1 rP ry r r r r s zHTTPResponse._read1_chunkedc C sB y | j }W n tk
r dS X |d kr.dS | jj|d | S )Nr9 )r r
r= r )r0 r1 rP r r r r s zHTTPResponse._peek_chunkedc C s
| j j S )N)r= fileno)r0 r r r r s zHTTPResponse.filenoc C sH | j dkrt | j j|p |}t|ts6t|d r:|S dj|S dS )ax Returns the value of the header matching *name*.
If there are multiple matching headers, the values are
combined into a single string separated by commas and spaces.
If no matching header is found, returns *default* or None if
the *default* is not specified.
If the headers are unknown, raises http.client.ResponseNotReady.
N__iter__z, )r> r Zget_all
isinstancerY hasattrrA )r0 r% defaultr> r r r getheader s
zHTTPResponse.getheaderc C s | j dkrt t| j j S )z&Return list of (header, value) tuples.N)r> r listitems)r0 r r r
getheaders s
zHTTPResponse.getheadersc C s | S )Nr )r0 r r r r s zHTTPResponse.__iter__c C s | j S )aj Returns an instance of the class mimetools.Message containing
meta-information associated with the URL.
When the method is HTTP, these headers are those returned by
the server at the head of the retrieved HTML page (including
Content-Length and Content-Type).
When the method is FTP, a Content-Length header will be
present if (as is now usual) the server passed back a file
length in response to the FTP retrieval request. A
Content-Type header will be present if the MIME type can be
guessed.
When the method is local-file, returned headers will include
a Date representing the file's last-modified time, a
Content-Length giving file size, and a Content-Type
containing a guess at the file's type. See also the
description of the mimetools module.
)r> )r0 r r r info s zHTTPResponse.infoc C s | j S )aZ Return the real URL of the page.
In some cases, the HTTP server redirects a client to another
URL. The urlopen() function handles this transparently, but in
some cases the caller needs to know which URL the client was
redirected to. The geturl() method can be used to get at this
redirected URL.
)rU )r0 r r r geturl s
zHTTPResponse.geturlc C s | j S )zuReturn the HTTP status code that was sent with the response,
or None if the URL is not an HTTP URL.
)rM )r0 r r r getcode s zHTTPResponse.getcode)r NN)N)r r )r r )r )N)"r4 r5 r6 rV ra rm rk r_ rn rq rr rs ry ru r r r rx r~ rz r r r r: r r r r r r r r r
__classcell__r r )rp r r s<
!K
"
c @ s e Zd ZdZdZeZeZdZ dZ
e dd Z e dd Z
d ejd fd
d Zd0d d
Zdd Zdd Zdd Zdd Zdd Zdd Zdd Zd d Zd1d d Zd2d!d"Zd#d$ Z d%d& Z d3d d'd(d)Z d i fd d'd*d+Z d,d- Z d.d/ Z!d S )4r re zHTTP/1.1r* r c C s t | tjS )zFTest whether a file-like object is a text or a binary stream.
)r io
TextIOBase)streamr r r
_is_textIO0 s zHTTPConnection._is_textIOc C sd | dkr |j tkrdS dS t| dr*dS yt| }|jS tk
rL Y nX t| tr`t| S dS )a Get the content-length based on the body.
If the body is None, we set Content-Length: 0 for methods that expect
a body (RFC 7230, Section 3.3.2). We also set the Content-Length for
any method if the body is a str or bytes-like object and not a file.
Nr ry ) upper_METHODS_EXPECTING_BODYr rv nbytes TypeErrorr rY r, )bodyrT Zmvr r r _get_content_length6 s
z"HTTPConnection._get_content_lengthNc C s\ || _ || _d | _g | _d | _t| _d | _d | _d | _ i | _
| j ||\| _ | _
tj| _d S )N)timeoutsource_addressrS _buffer_HTTPConnection__response_CS_IDLE_HTTPConnection__staterI _tunnel_host _tunnel_port_tunnel_headers
_get_hostporthostportsocketZcreate_connection_create_connection)r0 r r r r r r r rV V s zHTTPConnection.__init__c C s< | j rtd| j||\| _| _|r.|| _n
| jj dS )aD Set up host and port for HTTP CONNECT tunnelling.
In a connection that uses HTTP CONNECT tunneling, the host passed to the
constructor is used as a proxy server that relays all communication to
the endpoint passed to `set_tunnel`. This done by sending an HTTP
CONNECT request to the proxy server when the connection is established.
This method must be called before the HTML connection has been
established.
The headers argument should be a mapping of extra HTTP headers to send
with the CONNECT request.
z.Can't set up tunnel for established connectionN)rS RuntimeErrorr r r r clear)r0 r r r> r r r
set_tunneli s zHTTPConnection.set_tunnelc C s |d kr|j d}|j d}||kryt||d d }W nH tk
r ||d d dkrh| j}ntd||d d Y nX |d | }n| j}|r|d dkr|d dkr|dd }||fS )
Nr) ]r* rX znonnumeric port: '%s'r [r r )rfindr` r] default_portr )r0 r r r jr r r r s
z HTTPConnection._get_hostportc C s
|| _ d S )N)rH )r0 levelr r r set_debuglevel s z HTTPConnection.set_debuglevelc C s d| j | jf }|jd}| j| x6| jj D ](\}}d||f }|jd}| j| q0W | jd | j| j| jd}|j \}} }
| t
j j kr| j
td| |
j f xP|jjtd } t| tkrtd | sP | dkrP | jd krtd
| j qW d S )NzCONNECT %s:%d HTTP/1.0
asciiz%s: %s
zlatin-1
)rT z Tunnel connection failed: %d %sr* z header liner8 r9 r zheader:)r r8 r9 )r r r sendr r response_classrS rI ra http
HTTPStatusZOKrn OSErrorri r= r: r; r, r rH rZ rB ) r0 Z connect_strZ
connect_bytesheaderr Z
header_strZ header_bytesresponserL rh messager2 r r r _tunnel s2
zHTTPConnection._tunnelc C sB | j | j| jf| j| j| _| jjtjtj d | j
r>| j dS )z3Connect to the host and port specified in __init__.r* N) r r r r r rS Z
setsockoptr Z IPPROTO_TCPZ TCP_NODELAYr r )r0 r r r connect s
zHTTPConnection.connectc C sB t | _z | j}|r d| _|j W d| j}|r r r r r r s zHTTPConnection.requestc C s t dd |D }i }d|kr&d|d< d|kr6d|d< | j||f| d|krd |krd
}| j||}|d kr|d k r| jd krtd | d
}| jdd q| jdt| nd
}x |j D ]\} }
| j| |
qW t|trt |d}| j
||d d S )Nc s s | ]}|j V qd S )N)r+ )r kr r r s z/HTTPConnection._send_request..r r* r zaccept-encodingr zcontent-lengthztransfer-encodingFr z Unable to determine size of %rTzTransfer-EncodingrO zContent-Lengthr )r ) frozensetr r rH rZ r rY r r r' r ) r0 rT rU r r> r Z header_namesZskipsZcontent_lengthrl r r r r r s0
z HTTPConnection._send_requestc C s | j r| j j rd| _ | jtks&| j r0t| j| jdkrR| j| j| j| jd}n| j| j| jd}yLy |j W n t
k
r | j Y nX t | _|j
r| j n|| _ |S |j Y nX dS )a) Get the response from the server.
If the HTTPConnection is in the correct state, returns an
instance of HTTPResponse or of whatever object is returned by
the response_class variable.
If a request has not been sent or if a previous response has
not be handled, ResponseNotReady is raised. If the HTTP
response indicates that the connection should be closed, then
it will be closed before the response is returned. When the
connection is closed, the underlying socket is closed.
Nr )rT )r rs r r r rH r rS rI rm ConnectionErrorrn r rR )r0 r r r r getresponse) s,
zHTTPConnection.getresponse)NN)NF)FF)N)"r4 r5 r6 r r r r HTTP_PORTr r rH staticmethodr r r _GLOBAL_DEFAULT_TIMEOUTrV r r r r r rn r r r r r r r r r r r r r r r r & s<
'
6
.c sF e Zd ZdZeZdddejdfddd fdd Z fddZ Z
S )HTTPSConnectionz(This class allows communication via SSL.N)contextcheck_hostnamec s t t| j|||| |d k s.|d k s.|d k rDdd l} | jdtd || _|| _|d krptj }|j
d k rpd|_
|j tj k}
|d kr|j
}|r|
rtd|s|r|j|| |j
d k rd|_
|| _|| _d S )Nr zTkey_file, cert_file and check_hostname are deprecated, use a custom context instead.rW TzMcheck_hostname needs a SSL context with either CERT_OPTIONAL or CERT_REQUIRED)ro r rV warningswarnDeprecationWarningkey_file cert_filesslZ _create_default_https_contextZpost_handshake_authZ verify_modeZ CERT_NONEr r] Zload_cert_chain_context_check_hostname) r0 r r r r r r r r r Z will_verify)rp r r rV t s0
zHTTPSConnection.__init__c s t j | jr| j}n| j}| jj| j|d| _| jj r| jryt j
| jj | W n. t k
r | jj
tj | jj Y nX dS )z(Connect to a host on a given (SSL) port.)server_hostnameN)ro r r r r Z wrap_socketrS r r r
Zmatch_hostnameZ getpeercert ExceptionZshutdownr Z SHUT_RDWRrn )r0 r
)rp r r r s
zHTTPSConnection.connect) r4 r5 r6 __doc__
HTTPS_PORTr r r rV r r r r )rp r r m s r c @ s e Zd ZdS )r N)r4 r5 r6 r r r r r s c @ s e Zd ZdS )r N)r4 r5 r6 r r r r r s c @ s e Zd ZdS )r N)r4 r5 r6 r r r r r s c @ s e Zd Zdd ZdS )r c C s |f| _ || _d S )N)argsrL )r0 rL r r r rV s zUnknownProtocol.__init__N)r4 r5 r6 rV r r r r r s c @ s e Zd ZdS )r N)r4 r5 r6 r r r r r s c @ s e Zd ZdS )r N)r4 r5 r6 r r r r r s c @ s& e Zd ZdddZdd Zdd ZdS ) r
Nc C s |f| _ || _|| _d S )N)r partialexpected)r0 r r r r r rV s zIncompleteRead.__init__c C s2 | j d k rd| j }nd}d| jjt| j|f S )Nz, %i more expectedrX z%s(%i bytes read%s))r rp r4 r, r )r0 er r r __repr__ s
zIncompleteRead.__repr__c C s t | S )N)r[ )r0 r r r __str__ s zIncompleteRead.__str__)N)r4 r5 r6 rV r r r r r r r
s
c @ s e Zd ZdS )r N)r4 r5 r6 r r r r r s c @ s e Zd ZdS )r
N)r4 r5 r6 r r r r r
s c @ s e Zd ZdS )r N)r4 r5 r6 r r r r r s c @ s e Zd ZdS )r N)r4 r5 r6 r r r r r s c @ s e Zd Zdd ZdS )r c C s |s t |}|f| _|| _d S )N)r[ r r2 )r0 r2 r r r rV s zBadStatusLine.__init__N)r4 r5 r6 rV r r r r r s c @ s e Zd Zdd ZdS )r c C s t j| dt|f d S )Nz&got more than %d bytes when reading %s)r rV r; )r0 Z line_typer r r rV s zLineTooLong.__init__N)r4 r5 r6 rV r r r r r s c @ s e Zd Zdd ZdS )r c O s" t j| d tj| f|| d S )NrX )r rV ConnectionResetError)r0 poskwr r r rV s zRemoteDisconnected.__init__N)r4 r5 r6 rV r r r r r s )r )Br Z email.parserrC Z
email.messager r osrer r Z urllib.parser __all__r r rK r r r globalsupdater __members__r r r r; r< compile fullmatchr r r r r r r' r ZMessager( r? rE BufferedIOBaser r r
ImportErrorr r/ r r r r r r r r
r r
r r r r r r r r r r r E s
9 F =