RoProxy
Back to all posts
Leveraging HTTP/2 and HTTP/3 Proxies for High-Speed Web Scraping
proxy http2 http3 scraping performance

Leveraging HTTP/2 and HTTP/3 Proxies for High-Speed Web Scraping

August 1, 2026

Why HTTP/2 and HTTP/3 Matter for Scraping

Modern websites increasingly serve content over HTTP/2 or HTTP/3 (QUIC) to reduce latency and improve multiplexing. When your scraper still talks HTTP/1.1, you miss out on these gains and may even trigger anti‑bot heuristics that look for outdated protocol usage.

Using a proxy that speaks HTTP/2 or HTTP/3 lets you:

  • Multiplex many requests over a single TCP/UDP connection, cutting handshake overhead.
  • Benefit from header compression (HPACK/QPACK), which reduces bandwidth.
  • Appear more like a regular browser, because most modern clients negotiate HTTP/2/3 automatically.

If your proxy only supports HTTP/1.1, the connection will be downgraded, negating the advantages. Therefore, choosing a proxy provider that offers HTTP/2/3‑capable endpoints—or setting up your own—can give you a measurable edge.

How Proxies Fit Into HTTP/2/3

A proxy for HTTP/2/3 works similarly to an HTTP/1.1 proxy but relies on protocol‑specific mechanisms:

  • ALPN (Application‑Layer Protocol Negotiation) during the TLS handshake tells the server (or proxy) which protocol to use.
  • For HTTP/3, the underlying transport is UDP over QUIC, so the proxy must understand QUIC framing and be able to forward streams.

Many residential and datacenter proxy services now offer HTTP/2 support out of the box; HTTP/3 is still emerging but available from a growing number of providers.

Setting Up an HTTP/2 Proxy

If you prefer to run your own proxy (e.g., for testing or private infrastructure), tools like nghttpx, Envoy, or Caddy can terminate TLS and speak HTTP/2 to the origin while accepting HTTP/1.1 or HTTP/2 from clients.

Example: nghttpx Docker container

# Pull the image
docker pull nghttpx/nghttpx

# Run with a simple config that forwards to an origin
# (replace ORIGIN_HOST with your target)
docker run -d --name nghttpx-proxy \
  -p 8080:8080 \
  -v $(pwd)/nghttpx.conf:/etc/nghttpx/nghttpx.conf:ro \
  nghttpx/nghttpx

# nghttpx.conf
frontend=*:"8080"
backend=origin-host:443

# TLS settings (optional, for HTTPS)
tls-certificate=/etc/nghttpx/cert.pem
tls-private-key=/etc/nghttpx/key.pem

Once the proxy is listening on localhost:8080, any client that connects and negotiates HTTP/2 via ALPN will get HTTP/2 upstream to the origin.

Using the Proxy from Python (httpx)

import httpx

# httpx automatically negotiates HTTP/2 when the server supports it
client = httpx.Client(
    proxy="http://localhost:8080",
    http2=True,          # enable HTTP/2 support
    verify=False,        # for testing with self‑signed certs
)

response = client.get("https://example.com/api/data
print(response.status_code)
print(response.headers)

If the proxy and origin both speak HTTP/2, you’ll see a single connection handling multiple concurrent requests.

Enabling HTTP/3 (QUIC) Proxies

HTTP/3 is still less common in proxy offerings, but you can test it with proxy‑based QUIC relays like quicproxy or by using a cloud load balancer that supports QUIC (e.g., Cloudflare Spectrum).

Quick test with curl (HTTP/3)

# curl built with nghttp3 support
curl --proxy http://localhost:8080 \
     --proxy-insecure \
     --http3 \
     https://example.com/

If the proxy forwards QUIC packets correctly, you’ll see a successful response and the HTTP/3 line in the verbose output.

Python example with aioquic

import asyncio
from aioquic.asyncio import connect
from aioquic.h3.connection import H3_ALPN
from aioquic.h3.events import DataReceived, HeadersReceived

async def fetch():
    async with connect(
        "localhost",
        4433,                     # QUIC port of the proxy
        configuration=None,
        alpn_protocols=[H3_ALPN],
    ) as (reader, writer):
        # Send a simple GET request via H3
        writer.send_headers(
            {
                \):method: "GET",
                \):path: /
                \):authority: "example.com",
                \):scheme: "https",
            }
        )
        # Receive response
        while True:
            event = await reader.read()
            if isinstance(event, HeadersReceived):
                print("Headers:" , event.headers)
            elif isinstance(event, DataReceived):
                print("Body chunk:" , event.data)
            elif event.stream_ended:
                break

asyncio.run(fetch())

Note: This snippet assumes the proxy terminates QUIC and forwards to the origin over TCP/TLS. Adjust based on your proxy’s architecture.

Performance Benefits: What to Expect

When you switch from HTTP/1.1 to HTTP/2/3 via a capable proxy, you can observe:

  • Reduced connection establishment time – multiplexing removes the need for a new TCP handshake per request.
  • Lower head‑of‑line blocking – lost packets in QUIC affect only the impacted stream, not the whole connection.
  • Better bandwidth utilization – header compression cuts overhead, especially for APIs with small payloads.

A quick benchmark (using hey or wrk) against a test endpoint often shows 20‑40 % lower latency and up to 2× higher requests per second when HTTP/2 is used, with HTTP/3 providing similar latency gains and better resilience on lossy networks.

Practical Tips for Using HTTP/2/3 Proxies

  1. Verify ALPN support – Ensure your proxy advertises h2 (HTTP/2) and/or h3 (HTTP/3) in the TLS handshake. Tools like openssl s_client -alpn h2,h3 -connect proxy:443 -servername example.com can help.
  2. Keep connections alive – Reuse the same client/session object across many requests to reap multiplexing benefits.
  3. Monitor for fallback – If the proxy cannot negotiate HTTP/2/3, it will fall back to HTTP/1.1. Log the negotiated protocol (most HTTP clients expose it) to detect misconfigurations.
  4. Mind upstream limits – Even with a performant proxy, the origin server may enforce rate limits. Combine protocol upgrades with smart rotation and back‑off strategies.
  5. Select a proxy provider that offers HTTP/2/3 – Look for listings that mention "HTTP/2 support" or "QUIC/HTTP/3" in their documentation. Many premium residential networks now include these protocols as a standard feature.
  6. Test with browser‑like headers – Pair HTTP/2/3 with a realistic User‑Agent and Accept headers to avoid fingerprint‑based blocks.

Example: Scraping a Modern E‑commerce Site with HTTP/2 Proxy

Suppose you need to pull product listings from a site that only serves over HTTP/2. The following script uses httpx with a rotating pool of residential proxies from a provider like RoProxy (which offers HTTP/2‑enabled endpoints).

import asyncio
import httpx
from itertools import cycle

# List of proxy URLs that support HTTP/2
PROXIES = [
    "http://user:[email protected]:8000",
    "http://user:[email protected]:8000",
    "http://user:[email protected]:8000",
]
proxy_pool = cycle(PROXIES)

async def fetch_page(session, url):
    proxy = next(proxy_pool)
    try:
        resp = await session.get(url, proxy=proxy, http2=True, timeout=15)
        resp.raise_for_status()
        return resp.text
    except Exception as exc:
        print(f"Error via {proxy}: {exc}
        return None

async def main():
    async with httpx.AsyncClient(http2=True, follow_redirects=True) as client:
        tasks = [
            fetch_page(client, f"https://shop.example.com/products?page={i}
            for i in range(1, 6)
        ]
        pages = await asyncio.gather(*tasks)
        for i, html in enumerate(pages, start=1):
            if html:
                print(f"Page {i} length: {len(html)}" )

if __name__ == "__main__":
    asyncio.run(main())

The script rotates through three HTTP/2‑capable proxies, reuses a single AsyncClient (so connections are multiplexed), and logs any errors. If a proxy fails to negotiate HTTP/2, httpx will fall back to HTTP/1.1; you can detect this by checking resp.http_version and retry with another proxy.

Troubleshooting Common Issues

  • "Protocol h2 not supported" – The proxy either does not have HTTP/2 enabled or TLS is mis‑configured. Verify with openssl s_client -alpn h2 -connect proxy:443 -servername example.com.
  • Connection resets – Some proxies block QUIC/UDP traffic. Ensure the provider allows UDP on the QUIC port (usually 443) or fall back to HTTP/2 over TCP.
  • High latency despite HTTP/2 – Check the geographic distance between you, the proxy, and the origin. Choose proxies located close to your target audience or origin.
  • Authentication errors – Make sure proxy credentials are URL‑encoded if they contain special characters.

Conclusion

Adopting HTTP/2 and HTTP/3 proxies is a straightforward way to modernize your scraping stack, cut latency, and blend in with regular browser traffic. By verifying ALPN support, reusing connections, and rotating through high‑quality, protocol‑aware proxies (such as those offered by RoProxy), you can achieve noticeable performance gains while reducing the chance of being flagged as a bot.

Start small: test a single endpoint with an HTTP/2‑enabled proxy, measure the response time, then scale up to your full workload. The investment in protocol‑aware proxying pays off quickly, especially when dealing with high‑frequency APIs or content‑heavy sites that already prefer modern HTTP versions.

Feel free to share your own results or ask questions in the comments below.