Syndicator Blog

WebSub

How WebSub Lets RSS Feeds Push Updates Instead of Waiting to Be Polled

Feeds are pull by default. A reader asks your server every so often whether anything changed. Usually nothing did. Multiply that by every subscriber and every reader app and you get a lot of requests for very little news. It's also slow: a post can sit unseen until the next poll, which might be an hour away.

WebSub fixes the slow part and some of the waste. It became a W3C Recommendation in January 2018. Before that it was called PubSubHubbub, a name nobody could spell, and it had already been in use for close to ten years.

Three parties

WebSub adds a middleman, called a hub, between the publisher and the subscribers.

  1. The publisher has a feed, and says in that feed which hub it uses.
  2. The subscriber, say a feed reader, asks the hub to tell it when the feed changes.
  3. The hub gets a nudge from the publisher when there's something new, fetches the feed, and pushes the update to every subscriber.

The publisher only has to notify one place. Subscribers get new items in seconds instead of on the next poll.

Discovery

A feed points at its hub with two links. In an RSS feed, they go in the channel using the Atom namespace:

<atom:link rel="hub" href="https://pubsubhubbub.appspot.com/" />
<atom:link rel="self" href="https://example.com/feed/" type="application/rss+xml" />

self matters as much as hub. It's the canonical URL of the feed, which is the "topic" the subscriber will subscribe to. The same two links can be sent as HTTP Link headers instead, which works for any content type.

Subscribing

The reader sends a POST to the hub with three form fields: hub.mode=subscribe, hub.topic with the feed URL, and hub.callback with a URL of its own where it wants updates delivered.

The hub doesn't just believe it. It sends a GET to the callback with a random hub.challenge value, and the subscriber has to echo that value back. This stops anyone from subscribing a third party's server to a firehose of notifications. Subscriptions also have a lease, so readers renew them every so often, and dead subscribers fall away.

Publishing

When you publish, you tell the hub. The spec deliberately leaves the exact mechanism to each hub, but the common convention is a POST with hub.mode=publish and hub.url set to your feed URL. Then the hub fetches the feed and pushes the new content to every callback.

For WordPress there's a WebSub plugin that does the ping on every publish. For a static site, add one request to the end of your deploy:

curl -X POST https://pubsubhubbub.appspot.com/ \
  -d hub.mode=publish \
  -d hub.url=https://example.com/index.xml

Google's public hub at that address is free and has been running for years. You can also run your own. Several open source hubs exist.

Is it worth it

For a personal blog with a few hundred readers, polling works fine. Readers already back off on feeds that rarely change, and conditional GET makes each poll nearly free.

It's worth it if speed matters. News, alerts, anything time sensitive. It's also worth it if you're on the other side and run an aggregator with thousands of feeds, since push saves a lot of pointless requests.

Not every reader supports WebSub, so you never get to turn polling off. You add WebSub on top. Cheap to add, harmless if unused, and the readers that do support it get your posts right away.