Conditional GET With ETag and Last-Modified Cuts RSS Polling Costs

A feed reader asks the same question over and over. Anything new? Most of the time the answer is no, and the server sends the whole feed anyway. Conditional GET lets the server answer no in a few hundred bytes instead.

Here's a real number. The main feed at pxef.com, a WordPress site in the Syndicator network, is 99,498 bytes. Fetched with the right header, the same request came back as 304 Not Modified with an empty body. One reader polling every hour would save about 2.4 MB a day on that one feed. Spread that across every reader and aggregator polling every feed on the web and you can see why feed readers care about this.

The two validators

HTTP gives a server two ways to label a version of a file.

ETag is an opaque string, usually a hash of the content. pxef.com sends one like this:

etag: "fa888eb1f8d4f123025bfae734658655"

Last-Modified is a date:

last-modified: Wed, 28 Oct 2026 11:43:41 GMT

A client stores both with its copy of the feed. Next time, it sends them back:

If-None-Match: "fa888eb1f8d4f123025bfae734658655"
If-Modified-Since: Wed, 28 Oct 2026 11:43:41 GMT

If nothing changed, the server replies 304 Not Modified and no body. If something did, it sends a normal 200 with the new feed and new validators.

You can try it on any feed:

curl -sI https://pxef.com/feed/ | grep -i etag
curl -s -o /dev/null -w "%{http_code}\n" \
  -H 'If-None-Match: "paste-the-etag-here"' https://pxef.com/feed/

When both are present

If a client sends both headers, the ETag wins. The date is the weaker check, because one-second resolution can miss two edits in the same second, and because some servers report the time a file was generated, not when its content changed.

That second problem is common with static site generators. Rebuild the site and every file gets a fresh modification time, even files whose bytes didn't change. ETags based on content hashes don't have that problem. Cloudflare Pages, Netlify and most CDNs generate ETags from content, so a static feed gets this almost free as long as the bytes really are identical between builds.

The trap for static feeds

Identical bytes is the hard part. A lot of custom feed templates write the current time into lastBuildDate, or a generator comment with a timestamp. Then the file changes on every build, and every reader downloads the whole thing again even though no post was added.

The fix is to make the feed's dates come from the content, not from the clock. Hugo's built-in RSS template already does this, taking lastBuildDate from the newest content. If you wrote your own template, use the newest post's date or drop the element. Then a rebuild with no new posts produces byte for byte the same file, and conditional GET works.

What clients should do

If you write a feed client, store the ETag and Last-Modified per feed, send them back every time, and treat a 304 as "keep what you have". Respect Cache-Control max-age if the server sends one. Back off on feeds that rarely change.

Syndicator currently uses a simpler time-based cache: it keeps each feed body on disk for 45 minutes and skips the network inside that window. Since it rebuilds every 12 hours, that's mostly a guard against double fetching during one build. Adding conditional requests is the obvious next improvement, and with 114 feeds per run it would cut most of its transfer on quiet days.

The pattern is old and it isn't exciting. It's just the cheapest way to make RSS polling cost almost nothing.