U e$@sddlmZddlZddlZddlZddlZddlZddlZddlZddl m Z ddl m Z ddlmZddlmZddlmZdd lmZmZmZmZmZmZmZmZdd lmZdd lmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+dd l,m-Z-dd l.m/Z/ddl0m1Z1ddl2m3Z3m4Z4ddl5m6Z6ddl7m8Z8ddl9m:Z:m;Z;mZ>m?Z?ddl=m@ZAddl=mBZBddlCmDZDejErddlFZFddlmGZGddlmHZHmIZIeJeKZLejMeZdS)@HTTPConnectionPoola Thread-safe connection pool for one host. :param host: Host used for this HTTP Connection (e.g. "localhost"), passed into :class:`http.client.HTTPConnection`. :param port: Port used for this HTTP Connection (None is equivalent to 80), passed into :class:`http.client.HTTPConnection`. :param timeout: Socket timeout in seconds for each individual connection. This can be a float or integer, which sets the timeout for the HTTP request, or an instance of :class:`urllib3.util.Timeout` which gives you more fine-grained control over request timeouts. After the constructor has been parsed, this is always a `urllib3.util.Timeout` object. :param maxsize: Number of connections to save that can be reused. More than 1 is useful in multithreaded situations. If ``block`` is set to False, more connections will be created but they will not be saved once they've been used. :param block: If set to True, no more than ``maxsize`` connections will be used at a time. When no free connections are available, the call will block until a connection has been released. This is a useful side effect for particular multithreaded situations where one does not want to use more than maxsize connections per host to prevent flooding. :param headers: Headers to include with all requests, unless other headers are given explicitly. :param retries: Retry configuration to use by default with requests in this pool. :param _proxy: Parsed proxy URL, should not be used directly, instead, see :class:`urllib3.ProxyManager` :param _proxy_headers: A dictionary with proxy headers, should not be used directly, instead, see :class:`urllib3.ProxyManager` :param \**conn_kw: Additional parameters are used to create fresh :class:`urllib3.connection.HTTPConnection`, :class:`urllib3.connection.HTTPSConnection` instances. httpz4type[BaseHTTPConnection] | type[BaseHTTPSConnection] ConnectionClsNrFr6r7_TYPE_TIMEOUT | Noneintbooltyping.Mapping[str, str] | NoneRetry | bool | int | None Url | NonezProxyConfig | None typing.Any) r9r:rmaxsizeblockheadersretries_proxy_proxy_headers _proxy_configconn_kwc Kst|||t||t|ts.t|}|dkrPool reached maximum size and no more connections are allowed.zLConnection pool is full, discarding connection: %s. Connection pool size: %s) rqrtrrVFullrOrdrr}warningr9qsizerrArArB _put_conn)s.  zHTTPConnectionPool._put_conncCsdS)U Called right before a request is made, after the socket is created. NrArrArArB_validate_connTsz!HTTPConnectionPool._validate_conncCsdSrJrArrArArB_prepare_proxyYsz!HTTPConnectionPool._prepare_proxy _TYPE_TIMEOUTr)cCs2|tkr|jSt|tr$|St|SdS)z:Helper that always returns a :class:`urllib3.util.Timeout`N)r'rclonernr)ro)r@rrArArB _get_timeout]s   zHTTPConnectionPool._get_timeoutz&BaseSSLError | OSError | SocketTimeout)errurl timeout_valuer;cCsNt|tr t||d|d|t|drJ|jtkrJt||d|d|dS)zAIs the error actually a timeout? Will raise a ReadTimeout or passRead timed out. (read timeout=rEerrnoN)rn SocketTimeoutrhasattrr_blocking_errnos)r@rrrrArArB_raise_timeoutis    z!HTTPConnectionPool._raise_timeoutTz_TYPE_BODY | Nonez Retry | Noner ) rmethodrbodyrerfrchunked response_connpreload_contentdecode_contentenforce_content_lengthr;c  Cs0|jd7_||} | t| j|_zNz||Wn:tt fk rx}z|j |||jdW5d}~XYnXWn|t t t t ttfk r}zR|}t|t tfrt|}t|t t t tfr|r|jr|jst||jj}|W5d}~XYnXz|j|||||| | | dWnVtk r.YnBt k rn}z"|jtjkr^|jtjkr^W5d}~XYnX| j}|js|dkrt||d|d||_z |}Wn:t t fk r}z|j |||dW5d}~XYnX||_| |_||_ t!|dd }t"#d |j|j$|j%||||j&|j' |S) a8 Perform a request on a given urllib connection object taken from our pool. :param conn: a connection from one of our connection pools :param method: HTTP request method (such as GET, POST, PUT, etc.) :param url: The URL to perform the request on. :param body: Data to send in the request body, either :class:`str`, :class:`bytes`, an iterable of :class:`str`/:class:`bytes`, or a file-like object. :param headers: Dictionary of custom headers to send, such as User-Agent, If-None-Match, etc. If None, pool headers are used. If provided, these headers completely replace any pool-specific headers. :param retries: Configure the number of retries to allow before raising a :class:`~urllib3.exceptions.MaxRetryError` exception. Pass ``None`` to retry until you receive a response. Pass a :class:`~urllib3.util.retry.Retry` object for fine-grained control over different types of retries. Pass an integer number to retry connection errors that many times, but no other types of errors. Pass zero to never retry. If ``False``, then retries are disabled and any exception is raised immediately. Also, instead of raising a MaxRetryError on redirects, the redirect response will be returned. :type retries: :class:`~urllib3.util.retry.Retry`, False, or an int. :param timeout: If specified, overrides the default timeout for this one request. It may be a float (in seconds) or an instance of :class:`urllib3.util.Timeout`. :param chunked: If True, urllib3 will send the body using chunked transfer encoding. Otherwise, urllib3 will send the body using the standard content-length form. Defaults to False. :param response_conn: Set this to ``None`` if you will handle releasing the connection or set the connection to have the response release it. :param preload_content: If True, the response's body will be preloaded during construction. :param decode_content: If True, will attempt to decode the body based on the 'content-encoding' header. :param enforce_content_length: Enforce content length checking. Body returned by server must match value of Content-Length header, if present. Otherwise, raise error. rrrrN)rrerrrrrrrE _http_vsn_strzHTTP/?z%s://%s:%s "%s %s %s" %s %s)(rvr start_connectr)resolve_default_timeoutrrrrr rOSErrorrrr&rrnrlhas_connected_to_proxyrr5requestr r EPROTOTYPE ECONNRESET read_timeout is_closedr getresponserf _connection_poolgetattrr}r~r9r:statuslength_remaining)r@rrrrrerfrrrrrr timeout_objenew_erresponse http_versionrArArB _make_request|sN        z HTTPConnectionPool._make_requestcCs(|jdkrdS|jd}|_t|dSrQ)rqrz)r@old_poolrArArBrO4s zHTTPConnectionPool.close)rr;cCs|drdSt|^}}}}}|p&d}|dk rr|8|YSX|8|.|t/:d||j1||||f||||| | | | | |d |S|S)a Get a connection from the pool and perform an HTTP request. This is the lowest level call for making a request, so you'll need to specify all the raw details. .. note:: More commonly, it's appropriate to use a convenience method such as :meth:`request`. .. note:: `release_conn` will only behave as expected if `preload_content=False` because we want to make `preload_content=False` the default behaviour someday soon without breaking backwards compatibility. :param method: HTTP request method (such as GET, POST, PUT, etc.) :param url: The URL to perform the request on. :param body: Data to send in the request body, either :class:`str`, :class:`bytes`, an iterable of :class:`str`/:class:`bytes`, or a file-like object. :param headers: Dictionary of custom headers to send, such as User-Agent, If-None-Match, etc. If None, pool headers are used. If provided, these headers completely replace any pool-specific headers. :param retries: Configure the number of retries to allow before raising a :class:`~urllib3.exceptions.MaxRetryError` exception. If ``None`` (default) will retry 3 times, see ``Retry.DEFAULT``. Pass a :class:`~urllib3.util.retry.Retry` object for fine-grained control over different types of retries. Pass an integer number to retry connection errors that many times, but no other types of errors. Pass zero to never retry. If ``False``, then retries are disabled and any exception is raised immediately. Also, instead of raising a MaxRetryError on redirects, the redirect response will be returned. :type retries: :class:`~urllib3.util.retry.Retry`, False, or an int. :param redirect: If True, automatically handle redirects (status codes 301, 302, 303, 307, 308). Each redirect counts as a retry. Disabling retries will disable redirect, too. :param assert_same_host: If ``True``, will make sure that the host of the pool requests is consistent else will raise HostChangedError. When ``False``, you can use the pool on an HTTP proxy and request foreign hosts. :param timeout: If specified, overrides the default timeout for this one request. It may be a float (in seconds) or an instance of :class:`urllib3.util.Timeout`. :param pool_timeout: If set and the pool is set to block=True, then this method will block for ``pool_timeout`` seconds and raise EmptyPoolError if no connection is available within the time period. :param bool preload_content: If True, the response's body will be preloaded into memory. :param bool decode_content: If True, will attempt to decode the body based on the 'content-encoding' header. :param release_conn: If False, then the urlopen call will not release the connection back into the pool once a response is received (but will release if you read the entire contents of the response such as when `preload_content=True`). This is useful if you're not preloading the response's content immediately. You will need to call ``r.release_conn()`` on the response ``r`` to return the connection back into the pool. If None, it takes the value of ``preload_content`` which defaults to ``True``. :param bool chunked: If True, urllib3 will send the body using chunked transfer encoding. Otherwise, urllib3 will send the body using the standard content-length form. Defaults to False. :param int body_pos: Position to seek to in file-like body in the event of a retry or redirect. Typically this won't need to be set because urllib3 will auto-populate the value when needed. N)rdefaultrFTrr)rrrerrfrrrzConnection aborted.)errorr _stacktracez1Retrying (%r) after connection broken by '%r': %s)rrrrrrri/GET)rrzRedirecting %s -> %s) rfrrrrrrrrrz Retry-Afterz Retry: %s)?r-r5rernr%from_intrfrrrr.r+rr"rlrmcopyupdaterrr$rOrrrrrrrr rrrrrrrrrr&rrrr incrementsysexc_infosleepr}rurlopenget_redirect_locationrrZ_prepare_for_method_changerraise_on_redirect drain_connsleep_for_retryr~r^ris_retryraise_on_status)r@rrrrerfrrrrrrrrrr parsed_urldestination_schemerrelease_this_connhttp_tunnel_requiredr clean_exitrrrrrredirect_locationhas_retry_afterrArArBrVsq                         zHTTPConnectionPool.urlopen)N)rGrRrSrTr5r r[rUr'rCrrrrrrrrrOrrrArArArBrYys^ 3 $=&+ (9 rYcseZdZUdZdZeZded<dedddddddddddddddddfdd d d d d ddd ddddddddddddddfdd Z dddddZ dddd Z d!ddfd"d# Z Z S)$HTTPSConnectionPoola Same as :class:`.HTTPConnectionPool`, but HTTPS. :class:`.HTTPSConnection` uses one of ``assert_fingerprint``, ``assert_hostname`` and ``host`` in this order to verify connections. If ``assert_hostname`` is False, no verification is done. The ``key_file``, ``cert_file``, ``cert_reqs``, ``ca_certs``, ``ca_cert_dir``, ``ssl_version``, ``key_password`` are only used if :mod:`ssl` is available and are fed into :meth:`urllib3.util.ssl_wrap_socket` to upgrade the connection socket into an SSL socket. httpsztype[BaseHTTPSConnection]r[NrFr6r7r\r]r^r_r`rar4zint | str | Nonezssl.TLSVersion | Nonezstr | Literal[False] | Nonerbr8)r9r:rrcrdrerfrgrhkey_file cert_file cert_reqs key_passwordca_certs ssl_versionssl_minimum_versionssl_maximum_versionassert_hostnameassert_fingerprint ca_cert_dirrjr;c sftj||||||||| f || |_| |_| |_| |_||_||_||_||_ ||_ ||_ ||_ dSrJ) superrCrrrrrrrrrrr)r@r9r:rrcrdrerfrgrhrrrrrrrrrrrrj __class__rArBrCs0  zHTTPSConnectionPool.__init__rrcCs@|jr|jjdkrd}nd}|j||j|j|jd|dS)z5Establishes a tunnel connection through HTTP CONNECT.rrZ)r5r9r:reN)rlr5 set_tunnelr?r:rrconnect)r@rZ tunnel_schemerArArBr sz"HTTPSConnectionPool._prepare_proxyr1rDcCs|jd7_td|j|j|jp$d|jr8|jtkr@td|j}|j}|jdk rr|jjdk rr|jj}|jj}|jf|||j j |j |j |j |j|j|j|j|j|j|j|jd|jS)zL Return a fresh :class:`urllib3.connection.HTTPConnection`. rz)Starting new HTTPS connection (%d): %s:%s443zCCan't connect to HTTPS URL because the SSL module is not available.N)r9r:rrrrrrrrrrrr)rur}r~r9r:r[r ImportErrorrlrrrrrrrrrrrrrrj)r@ actual_host actual_portrArArBrsDzHTTPSConnectionPool._new_connr0cs@t||jr||js<|js1)rr6rb)rkwr;cKs^t|^}}}}}|pd}|p(t|d}|dkrFt|fd|i|St|fd|i|SdS)a Given a url, return an :class:`.ConnectionPool` instance of its host. This is a shortcut for not having to parse out the scheme, host, and port of the url before creating an :class:`.ConnectionPool` instance. :param url: Absolute URL string that must include the scheme. Port is optional. :param \**kw: Passes additional parameters to the constructor of the appropriate :class:`.ConnectionPool`. Useful for specifying things like timeout, maxsize, headers, etc. Example:: >>> conn = connection_from_url('http://google.com/') >>> r = conn.request('GET', '/') rZPrr:N)r-rrrrY)rrr5r{r9r:rArArBconnection_from_urlZs rr8r4)r9r5r;cCsdSrJrAr9r5rArArBr,wsr,cCsdSrJrArrArArBr,|scCs2t||}|r.|dr.|dr.|dd}|S)z? Normalize hosts for comparisons and use with sockets. []r)r=rendswithrrArArBr,s  z(HTTPConnectionPool | HTTPSConnectionPool)rqpathr;cCst|j|j|j|djS)zZReturns the URL from a given connection pool. This is mainly used for testing and logging.)r5r9r:r)r*r5r9r:r)rqrrArArB_url_from_poolsrzqueue.LifoQueue[typing.Any])rqr;cCs:z|jdd}|r|qWntjk r4YnXdS)z2Drains a queue of connections and closes each one.FrN)rrOrVr)rqrrArArBrzs  rz)N)\ __future__rrloggingrVrtypingrrxsocketrrtypesrZ_base_connectionr _collectionsrZ_request_methodsr connectionr r r r rrrrr exceptionsrrrrrrrrrrrrrrr Zutil.connectionr!Z util.proxyr"Z util.requestr#r$Z util.retryr%Zutil.ssl_match_hostnamer&Z util.timeoutr'r(r)Zutil.urlr*r+r,r=r-Z util.utilr. TYPE_CHECKINGsslr/r0r1 getLoggerrGr}UnionfloatrTypeVarr2r3EAGAIN EWOULDBLOCKrrYrroverloadrrzrArArArBsj      ( <           2 V