Replace urlopen automatic redirect following with custom HTTPRedirectHandler
that raises on 3xx redirects. Each redirect hop is validated (scheme, host,
port) before the request is made using _validate_url_before_io().
Key changes:
- _StrictRedirectHandler intercepts 301/302/303/307/308 responses
- _extract_redirect_url() extracts Location header from redirect responses
- fetch_html() manually follows redirects with hop count limit (MAX_REDIRECT_HOPS=5)
- Relative redirect URLs resolved with urljoin() before validation
- All redirect targets validated against ALLOWED_HOSTS, ALLOWED_PORTS, HTTPS-only
Tests:
- test_fetch_html_redirect_to_disallowed_host_rejected (mocked redirect)
- test_fetch_html_redirect_chain_limit (exceeds MAX_REDIRECT_HOPS)
- test_extract_redirect_url_from_headers (Location/location headers)
- test_urljoin_resolves_relative_redirects (relative URL resolution)
ik
d0d208ebd7