Pool Manager - urllib3 1.26.18 documentation

Hide navigation sidebar
Hide table of contents sidebar
Support urllib3 on GitHub Sponsors
Back to top
Toggle table of contents sidebar

# Pool Manager

class urllib3.PoolManager(num_pools=10, headers=None, **connection_pool_kw)

  Bases: `RequestMethods`

  Allows for arbitrary requests while transparently keeping track of necessary connection pools for you.

  Parameters:

    - num_pools – Number of connection pools to cache before discarding the least recently used pool.

    - headers – Headers to include with all requests, unless other headers are given explicitly.

    - **connection_pool_kw – Additional parameters are used to create fresh `urllib3.connectionpool.ConnectionPool` instances.

  Example:

      >>> manager = PoolManager(num_pools=2)
      >>> r = manager.request('GET', 'http://google.com/')
      >>> r = manager.request('GET', 'http://google.com/mail')
      >>> r = manager.request('GET', 'http://yahoo.com/')
      >>> len(manager.pools)
      2

  clear()

    Empty our store of pools and direct them all to close.

    This will not affect in-flight connections, but they will not be re-used after completion.

  connection_from_context(request_context)

    Get a `urllib3.connectionpool.ConnectionPool` based on the request context.

    `request_context` must at least contain the `scheme` key and its value must be a key in `key_fn_by_scheme` instance variable.

  connection_from_host(host, port=None, scheme='http', pool_kwargs=None)

    Get a `urllib3.connectionpool.ConnectionPool` based on the host, port, and scheme.

    If `port` isn’t given, it will be derived from the `scheme` using `urllib3.connectionpool.port_by_scheme`. If `pool_kwargs` is provided, it is merged with the instance’s `connection_pool_kw` variable and used to create the new connection pool, if one is needed.

  connection_from_pool_key(pool_key, request_context=None)

    Get a `urllib3.connectionpool.ConnectionPool` based on the provided pool key.

    `pool_key` should be a namedtuple that only contains immutable objects. At a minimum it must have the `scheme`, `host`, and `port` fields.

  connection_from_url(url, pool_kwargs=None)

    Similar to `urllib3.connectionpool.connection_from_url()`.

    If `pool_kwargs` is not provided and a new pool needs to be constructed, `self.connection_pool_kw` is used to initialize the `urllib3.connectionpool.ConnectionPool`. If `pool_kwargs` is provided, it is used instead. Note that if a new pool does not need to be created for the request, the provided `pool_kwargs` are not used.

  proxy = None
  proxy_config = None
  urlopen(method, url, redirect=True, **kw)

    Same as `urllib3.HTTPConnectionPool.urlopen()` with custom cross-host redirect logic and only sends the request-uri portion of the `url`.

    The given `url` parameter must be absolute, such that an appropriate `urllib3.connectionpool.ConnectionPool` can be chosen for it.

class urllib3.ProxyManager(proxy_url, num_pools=10, headers=None, proxy_headers=None, proxy_ssl_context=None, use_forwarding_for_https=False, **connection_pool_kw)

  Bases: `PoolManager`

  Behaves just like `PoolManager`, but sends all requests through the defined proxy, using the CONNECT method for HTTPS URLs.

  Parameters:

    - proxy_url – The URL of the proxy to be used.

    - proxy_headers – A dictionary containing headers that will be sent to the proxy. In case of HTTP they are being sent with each request, while in the HTTPS/CONNECT case they are sent only once. Could be used for proxy authentication.

    - proxy_ssl_context – The proxy SSL context is used to establish the TLS connection to the proxy when using HTTPS proxies.

    - use_forwarding_for_https – (Defaults to False) If set to True will forward requests to the HTTPS proxy to be made on behalf of the client instead of creating a TLS tunnel via the CONNECT method. Enabling this flag means that request and response headers and content will be visible from the HTTPS proxy whereas tunneling keeps request and response headers and content private. IP address, target hostname, SNI, and port are always visible to an HTTPS proxy even when this flag is disabled.

  Example:

        >>> proxy = urllib3.ProxyManager('http://localhost:3128/')
        >>> r1 = proxy.request('GET', 'http://google.com/')
        >>> r2 = proxy.request('GET', 'http://httpbin.org/')
        >>> len(proxy.pools)
        1
        >>> r3 = proxy.request('GET', 'https://httpbin.org/')
        >>> r4 = proxy.request('GET', 'https://twitter.com/')
        >>> len(proxy.pools)
        3

  connection_from_host(host, port=None, scheme='http', pool_kwargs=None)

    Get a `urllib3.connectionpool.ConnectionPool` based on the host, port, and scheme.

    If `port` isn’t given, it will be derived from the `scheme` using `urllib3.connectionpool.port_by_scheme`. If `pool_kwargs` is provided, it is merged with the instance’s `connection_pool_kw` variable and used to create the new connection pool, if one is needed.

  urlopen(method, url, redirect=True, **kw)

    Same as HTTP(S)ConnectionPool.urlopen, `url` must be absolute.