Smart Proxy Selection: Latency & Success Rate for Low‑Latency APIs

September 8, 2026

Why Latency and Success Rate Matter

When your application talks to an API that serves data in real time, every millisecond counts. A proxy that adds 200 ms of extra round‑trip time can turn a smooth user experience into a sluggish one, while a proxy that frequently returns errors forces you to retry, further increasing latency and risking rate‑limit bans. The ideal proxy therefore balances two metrics:

Latency – the time from the moment the request leaves your process until the first byte of the response is received.
Success rate – the percentage of requests that return a successful HTTP status (2xx or 3xx) without network‑level errors such as connection resets or timeouts.

A proxy that is ultra‑fast but drops 30 % of requests is worse than a slightly slower proxy that succeeds 99 % of the time, especially for low‑latency services where retries are costly.

Measuring Proxy Performance

Latency measurement

Open a TCP connection to the target host through the proxy.
Send a minimal HTTP request (e.g., GET / with a short timeout).
Record the elapsed time from the moment the request is sent to the moment the response headers are received.

Success rate measurement

Track the HTTP status code:

Success – 2xx or 3xx codes.
Transient failure – connection timeout, DNS error, or a 4xx/5xx response.
Permanent failure – proxy authentication error (401/407) or a blocked IP.

Collect both metrics over a representative sample (e.g., 100 requests) to smooth out occasional spikes.

Building a Scoring Model

A simple, effective way to combine the two dimensions is to create a score for each proxy:

score = w_latency * latency_norm + w_success * success_norm

latency_norm is the latency divided by the best observed latency (so the fastest proxy gets 1.0).
success_norm is the success rate (successful requests / total requests) so a 100 % success proxy gets 1.0.
w_latency and w_success are weighting factors that reflect your priorities (e.g., 0.7 for latency, 0.3 for success).

This normalisation ensures that a proxy with low latency but many failures will not dominate the ranking, and vice‑versa.

Selecting the Best Proxy

Once you have scores for all available proxies, pick the one with the lowest score (fastest and most reliable). In practice you keep a priority queue (min‑heap) so that the next best proxy is always at the top.

import heapq

# Example data structure
proxy_scores = []  # heap of (score, proxy_url)

for proxy_url in proxy_list:
    latency = measure_latency(proxy_url)
    success = measure_success(proxy_url)
    score = 0.7 * (latency / best_latency) + 0.3 * success
    heapq.heappush(proxy_scores, (score, proxy_url))

best_proxy = heapq.heappop(proxy_scores)[1]

The heap guarantees O(log n) insertion and O(1) access to the best candidate, which is ideal when the proxy list changes frequently.

Adaptive Selection Loop

In a real‑world scraper or API client, you cannot pre‑measure every proxy once and forget it. Network conditions change, proxies become stale, and new ones are added. An adaptive loop does the following:

Test a batch of proxies (e.g., 10 at a time) by sending a lightweight request.
Update scores based on the latest latency and success results.
Re‑heapify the priority queue.
Select the top proxy for the actual work.
Fallback – if the chosen proxy fails consecutively (e.g., 3 errors), remove it temporarily and re‑run the loop.

import time, requests, heapq

def measure_latency(proxy):
    start = time.time()
    try:
        resp = requests.get('https://api.example.com/ping', timeout=5, proxies={'http': proxy, 'https': proxy})
        return time.time() - start
    except Exception:
        return float('inf')  # treat as timeout

def measure_success(proxy):
    try:
        resp = requests.get('https://api.example.com/ping', timeout=5, proxies={'http': proxy, 'https': proxy})
        return 1 if 200 <= resp.status_code < 300 else 0
    except Exception:
        return 0

# Initial best latency (run once)
best_latency = min(measure_latency(p) for p in proxy_list)

# Heap initialization
pq = []
for p in proxy_list:
    lat = measure_latency(p)
    suc = measure_success(p)
    score = 0.7 * (lat / best_latency) + 0.3 * suc
    heapq.heappush(pq, (score, p))

# Adaptive loop for a request
while True:
    score, chosen_proxy = heapq.heappop(pq)
    latency = measure_latency(chosen_proxy)
    suc = measure_success(chosen_proxy)
    # Update score with fresh data
    new_score = 0.7 * (latency / best_latency) + 0.3 * suc
    if lat > 5.0:  # consider high latency as a sign of trouble
        # re‑insert with a higher score to discourage immediate use
        new_score += 0.2
    heapq.heappush(pq, (new_score, chosen_proxy))
    # Try the request
    try:
        resp = requests.get('https://api.example.com/data', timeout=10, proxies={'http': chosen_proxy, 'https': chosen_proxy})
        if resp.status_code >= 200 and resp.status_code < 300:
            # success – exit loop
            break
        else:
            # treat as failure, continue to next iteration
            continue
    except requests.RequestException:
        # network error – retry with next proxy
        continue

The loop keeps trying until a successful response is received, automatically falling back to the next best proxy when needed.

Real‑World Example

Imagine a weather‑API client that must return current conditions within 300 ms. You have three proxies:

A – datacenter proxy, 80 ms latency, 95 % success.
B – residential proxy, 120 ms latency, 99 % success.
C – mobile proxy, 150 ms latency, 98 % success.

Using the scoring formula (0.7 latency, 0.3 success) and normalising latency by the best (80 ms):

A: 0.7 * (80/80) + 0.3 * 0.95 = 0.7 + 0.285 = 0.985
B: 0.7 * (120/80) + 0.3 * 0.99 = 0.7 * 1.5 + 0.297 = 1.05 + 0.297 = 1.347
C: 0.7 * (150/80) + 0.3 * 0.98 = 0.7 * 1.875 + 0.294 = 1.3125 + 0.294 = 1.6065

Proxy A has the lowest score, so it is selected. If after a few calls A starts timing out (latency spikes to 600 ms) its score rises, and the adaptive loop will promote B or C automatically, keeping the overall response time within the target.

Edge Cases and Error Handling

Timeouts – treat any request that exceeds a configurable threshold as a failure and increase the proxy’s score.
Authentication errors – if a proxy returns 401/407, mark it as permanently invalid for the current session and remove it from the pool.
Rate‑limit responses – some APIs return 429; a proxy that consistently triggers them may be flagged for temporary blacklisting.
Proxy list freshness – periodically refresh the proxy list (e.g., every 5 minutes) because IP reputation can change quickly.
Thread safety – when using async or multithreaded clients, protect the shared heap with a lock or use an async priority queue to avoid race conditions.

Best Practices for Production

Cache latency baselines – store the best observed latency per target host; reuse it for normalisation to avoid repeated measurement overhead.
Dynamic weighting – if your SLA changes (e.g., you need ultra‑low latency for a specific endpoint), adjust w_latency on the fly.
Monitor health – expose the scoring metrics to Prometheus so you can set alerts when success rate drops below a threshold.
Graceful degradation – if the entire proxy pool becomes unhealthy, fall back to a direct connection (no proxy) to keep the service alive.
Testing – run a nightly job that validates every proxy with a real request; discard any that fail more than X % of the time.

Conclusion

Smart proxy selection is not about choosing the fastest or the most anonymous proxy; it is about continuously measuring latency and success rate, turning those measurements into a lightweight score, and using that score to drive real‑time decisions. By implementing the adaptive loop shown above, you can keep your low‑latency API calls fast, reliable, and resilient to the inevitable churn of proxy providers. The result is a smoother user experience, fewer retries, and lower operational cost – exactly what any performance‑critical application needs.