Preventing Proxy IP Leaks in Web Scraping: DNS, IPv6, and WebRTC Protection

When you use a proxy for web scraping, your goal is simple: all your traffic should appear to come from the proxy IP, not your real one. But even with a correctly configured proxy, subtle leaks can expose your true identity. DNS leaks, IPv6 leaks, and WebRTC leaks are three common culprits that silently bypass your proxy setup.

In this guide, we’ll explain why each leak happens, show you how to detect them, and walk through concrete steps to block them in Python scrapers, headless browsers, and system-level configurations.

Why Proxy Leaks Happen

A proxy leak occurs when some part of your network traffic ignores the proxy and connects directly to the internet. This usually happens because:

• DNS requests are resolved by your local DNS resolver instead of the proxy’s DNS.
• IPv6 traffic bypasses the proxy if only IPv4 is configured.
• WebRTC uses STUN servers to discover your public IP, which can reveal your real address even when a proxy is active.

These leaks are especially dangerous in web scraping because anti-bot systems can detect them and immediately flag or block your real IP.

DNS Leaks: The Most Common Leak

What Is a DNS Leak?

When you make a request to example.com, your system needs to resolve the domain to an IP address. If DNS resolution happens locally instead of through the proxy, the target server (or a man-in-the-middle) can see your real IP in the DNS query.

How to Detect DNS Leaks

You can check for DNS leaks using online tools like dnsleaktest.com or by inspecting your scraper’s behavior:

import requests

# This request should go through the proxy
response = requests.get('http://httpbin.org/dns',
    proxies={'http': 'http://127.0.0.1:8080',
             'https': 'http://127.0.0.1:8080'})
print(response.text)

If the response shows your real IP or local DNS resolver, you have a DNS leak.

Blocking DNS Leaks in Python

To prevent DNS leaks, ensure that all HTTP and HTTPS traffic goes through the proxy, including DNS resolution:

import requests

proxy_ip = 'http://your-proxy-ip:port'
proxies = {
    'http': proxy_ip,
    'https': proxy_ip,
}

# Disable trust_env to prevent environment variables from overriding proxy
session = requests.Session()
session.trust_env = False
session.proxies.update(proxies)

response = session.get('http://httpbin.org/ip')
print(response.json())

Setting trust_env = False ensures that environment variables like HTTP_PROXY or HTTPS_PROXY don’t interfere with your proxy configuration.

IPv6 Leaks: The Silent Bypass

Why IPv6 Leaks Matter

Many proxy setups only handle IPv4 traffic. If your system has IPv6 connectivity and the website supports it, your request may bypass the proxy entirely through IPv6.

Detecting IPv6 Leaks

Use a service like test-ipv6.com or check your IP after enabling a proxy:

curl -x http://your-proxy-ip:port https://httpbin.org/ip

If the returned IP is your real one, you may be leaking over IPv6.

Blocking IPv6 Leaks

Option 1: Disable IPv6 System-Wide

On Linux:

sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1
sudo sysctl -w net.ipv6.conf.default.disable_ipv6=1

On Windows (PowerShell as Administrator):

Disable-NetAdapterBinding -InterfaceAlias "Ethernet" -ComponentID ms_tcpip6

Option 2: Force IPv4 in Python

Use socket.create_connection with socket.AF_INET to force IPv4:

import socket
import requests
from urllib3.util import connection

# Force IPv4
def get_ipv4_socket(*args, **kwargs):
    return socket.socket(socket.AF_INET, socket.SOCK_STREAM)

connection.create_connection = get_ipv4_socket

response = requests.get('https://httpbin.org/ip', proxies=proxies)
print(response.json())

WebRTC Leaks: The Browser-Based Threat

What Is a WebRTC Leak?

WebRTC (Web Real-Time Communication) allows browsers to establish peer-to-peer connections. During this process, STUN servers can discover your real public IP address, even if you’re using a proxy or VPN.

Detecting WebRTC Leaks

If you’re using headless browsers like Puppeteer or Selenium, check your IP with a WebRTC test page:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.add_argument('--proxy-server=http://your-proxy-ip:port')

# Disable WebRTC
options.add_argument('--disable-webrtc')
options.add_experimental_option('prefs', {
    'webrtc.ip_handling_policy': 'default_public_interface_only',
    'webrtc.multiple_routes_enabled': False,
    'webrtc.udp_port_range': '0',
})

driver = webdriver.Chrome(options=options)
driver.get('https://browserle.git.ci/demos/get-ip')
print(driver.page_source)

Blocking WebRTC Leaks in Headless Browsers

For Puppeteer:

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch({
    args: [
      '--proxy-server=http://your-proxy-ip:port',
      '--disable-webrtc',
      '--disable-features=WebRtcHideLocalIpsWithMdns'
    ]
  });

  const page = await browser.newPage();
  await page.goto('https://httpbin.org/ip');
  const content = await page.content();
  console.log(content);

  await browser.close();
})();

For Selenium with Firefox:

from selenium.webdriver import Firefox
from selenium.webdriver.common.proxy import Proxy, ProxyType

proxy = Proxy()
proxy.proxy_type = ProxyType.MANUAL
proxy.http_proxy = 'your-proxy-ip:port'
proxy.ssl_proxy = 'your-proxy-ip:port'

profile = webdriver.FirefoxProfile()
profile.set_preference('media.peerconnection.enabled', False)
profile.set_preference('media.peerconnection.ice.default_address_only', True)
profile.set_preference('media.peerconnection.ice.ipv6_address_disabled', True)

options = webdriver.FirefoxOptions()
options.profile = profile

# Apply proxy
options.proxy = proxy

driver = Firefox(options=options)

System-Level Leak Prevention

Proxy Configuration Files

Ensure your system uses the proxy for all traffic:

Linux (Environment Variables)

Add to /etc/environment:

http_proxy=http://your-proxy-ip:port
https_proxy=http://your-proxy-ip:port
ftp_proxy=http://your-proxy-ip:port

dns_proxy=http://your-proxy-ip:port

macOS (Network Settings)

1. Go to System Preferences > Network.
2. Select your active connection and click Advanced.
3. Go to the Proxies tab.
4. Enable Web Proxy (HTTP) and Secure Web Proxy (HTTPS).
5. Enter your proxy details.
6. Click “Bypass proxy settings” and ensure no IPs are listed.

Verifying Your Setup

After applying all leak protections, verify your configuration:

1. Visit ipleak.net or doileak.com.
2. Check that your public IP matches your proxy IP.
3. Ensure no DNS servers or IPv6 addresses are exposed.
4. Confirm WebRTC shows no local or public IPs.

Best Practices Summary

• Always set trust_env = False in Python requests sessions.
• Disable IPv6 if your proxy doesn’t support it.
• Turn off WebRTC in headless browsers using flags or preferences.
• Use proxy-aware DNS resolvers when available.
• Regularly test your setup with leak detection tools.

By following these steps, you can build scrapers that stay hidden behind your proxy, avoiding detection and ensuring reliable data collection.

Conclusion

Proxy leaks are silent killers of anonymity in web scraping. Whether it’s a DNS query, an IPv6 packet, or a WebRTC handshake, any unproxied traffic can expose your real IP and compromise your entire scraping operation. By understanding how each leak works and applying the right fixes—whether in code, browser configuration, or system settings—you can confidently route all your traffic through your proxy and protect your scraping infrastructure from detection.

Remember: a proxy is only as secure as your weakest configuration. Take the time to test, verify, and lock down every potential leak path.