Toolisk LogoToolisk
Web

HTTP Headers Deep Dive: The Ones You Actually Need to Know

Hundreds of HTTP headers exist. About 30 matter day-to-day. Learn the request and response headers that drive caching, security, content negotiation, and the security headers that defend against common attacks.

•13 min read

HTTP headers are how clients and servers exchange metadata about a request beyond the URL and body. The IANA registry lists hundreds. In practice, about thirty matter day-to-day — and getting them right affects security, performance, and SEO.

Request headers worth knowing

Host

Indicates the domain the request is for. Required since HTTP/1.1. Enables virtual hosting (multiple domains on the same IP). The single header that turned the modern web possible.

Accept

What media types the client wants in response. Accept: application/json tells the server to return JSON. Used in content negotiation.

Accept: text/html,application/xhtml+xml;q=0.9,*/*;q=0.8

The q=0.9 values are quality preferences. Higher = preferred.

Authorization

Credentials for accessing the resource. Common schemes:

  • Authorization: Basic dXNlcjpwYXNz — base64-encoded user:password (insecure over HTTP).
  • Authorization: Bearer eyJhbGc... — JWT or other token.

User-Agent

Identifies the client. Often abused by servers to vary content (deprecated practice — feature detection on the client is better). Mostly used for logging and analytics now.

Cookie

Sends previously stored cookies back to the server. Format: Cookie: name1=value1; name2=value2.

Origin

The origin (scheme + host + port) that initiated the request. Used for CORS decisions.

Referer

URL of the page that linked to the current request. Useful for analytics; privacy-sensitive — modern browsers send only the origin by default for cross-origin requests.

If-None-Match / If-Modified-Since

Conditional requests for caching. "Send me this resource only if it's changed since [ETag] / [date]." Server responds 304 Not Modified if cache is still valid.

Response headers worth knowing

Content-Type

What media type the response body is. Critical for browsers to render correctly:

  • text/html; charset=utf-8 — HTML pages.
  • application/json — JSON APIs.
  • image/png, image/svg+xml — images.
  • application/pdf — PDFs.
  • text/css, application/javascript — CSS/JS.

Always include charset=utf-8 for text types. Otherwise some browsers default to ISO-8859-1 and break non-ASCII content.

Content-Length

Size of the body in bytes. Required for HTTP/1.1 unless using chunked transfer encoding. Wrong values cause truncated responses.

Cache-Control

Caching directives:

  • Cache-Control: no-store — don't cache anywhere. For sensitive responses.
  • Cache-Control: no-cache — cache, but always revalidate before using.
  • Cache-Control: public, max-age=3600 — cacheable, valid for an hour.
  • Cache-Control: private, max-age=600 — only browsers cache (not CDNs); valid for 10 minutes.
  • Cache-Control: immutable — never changes; browsers can skip revalidation entirely. Critical for hashed assets.

ETag

Identifier for a specific version of a resource. Server returns; client sends back asIf-None-Match for conditional requests. Most efficient cache validation mechanism.

Set-Cookie

Tells the browser to store a cookie. See our HTTP Cookies guide for full details on attributes (HttpOnly, Secure, SameSite, etc.).

Location

Used with 3xx redirects to tell the client where to go next. Should be absolute URLs in modern HTTP (relative URLs are spec-allowed but cause issues with proxies).

The security headers

Strict-Transport-Security (HSTS)

Forces browsers to use HTTPS for this domain for the specified time:

Strict-Transport-Security: max-age=31536000; includeSubDomains; preload

Once received, the browser refuses to load HTTP versions of your site for the duration. Thepreload directive opts you into the browser's built-in HSTS list (apply at hstspreload.org).

Content-Security-Policy (CSP)

Whitelist of where resources can come from:

Content-Security-Policy: default-src 'self'; script-src 'self' https://cdn.example.com; style-src 'self' 'unsafe-inline'

The single most powerful XSS defense. Implementing CSP correctly is non-trivial — start withContent-Security-Policy-Report-Only in production for a few weeks to find violations before enforcing.

X-Content-Type-Options

X-Content-Type-Options: nosniff prevents browsers from guessing the content type. Without this, a user-uploaded file labeled as image/png but containing HTML could be rendered as HTML — XSS.

X-Frame-Options / Content-Security-Policy frame-ancestors

Prevents your page from being framed by other origins (clickjacking defense):

  • X-Frame-Options: DENY — never framed.
  • X-Frame-Options: SAMEORIGIN — only framed by your own origin.
  • Modern equivalent: Content-Security-Policy: frame-ancestors 'self'

Referrer-Policy

Controls how much of the URL is sent in the Referer header on outgoing links. Recommended:

Referrer-Policy: strict-origin-when-cross-origin

Sends full URL same-origin, just origin cross-origin, nothing on HTTPS → HTTP downgrade.

Permissions-Policy

Restricts which browser features your page (and embedded iframes) can use:

Permissions-Policy: camera=(), microphone=(), geolocation=(self)

Defense in depth

Setting all the security headers correctly is a small effort with significant payoff. Tools like Mozilla Observatory and securityheaders.com grade your site on these. A green grade isn't bulletproof — but a red grade indicates you're leaving easy wins on the table.

Content negotiation headers

  • Accept-Language: client's preferred languages. en-US,en;q=0.5
  • Accept-Encoding: compression formats supported. gzip, deflate, br
  • Vary: response header listing which request headers were used to compute this response. Critical for correct caching.

The Vary header trap

If your response varies by Origin, Cookie, Accept-Language, or any other header, you must include it in the Vary response header. Without it, caches will serve the wrong response to other clients.

Vary: Accept-Encoding, Origin, Accept-Language

Common bug: forgetting Vary: Origin when echoing CORS Origin in responses. CDN serves a cached response with the wrong allowed origin to the wrong client.

Custom headers

Convention is to prefix custom headers with X- historically (X-Request-ID, X-API-Key). Modern best practice is to skip the X- prefix (it's not standardized) and use clear vendor-specific names: Stripe-Signature, GitHub-Event.

Common header mistakes

  • Missing Content-Type charset. Causes encoding issues with non-ASCII content.
  • Wrong Cache-Control for HTML pages. Caching HTML for hours means users see stale content. Use no-cache + ETag for HTML.
  • Forgetting Vary. Cache poisoning between client populations.
  • Setting both Cache-Control and Expires. Cache-Control wins; Expires is legacy.
  • Trusting Referer for security. Easily spoofed; can be missing entirely.
  • Putting auth tokens in URL parameters. Logged everywhere. Use Authorization header.
  • No security headers in production. Free defense-in-depth wins missed.

Key Takeaways

  • About 30 HTTP headers matter day-to-day. Master Content-Type, Cache-Control, ETag, Authorization, and the security headers.
  • Security headers (CSP, HSTS, X-Content-Type-Options, Referrer-Policy, Permissions-Policy) are easy wins. Use a tool to audit yours.
  • Cache-Control: immutable is critical for hashed assets — browsers skip revalidation entirely.
  • Vary: Origin is mandatory when echoing the CORS Origin header. Without it, CDNs cache-poison.
  • Always include charset=utf-8 in Content-Type for text resources. Non-ASCII content depends on it.