How Syndicator Turns WordPress and Hugo Summaries Into Clean Plain Text Excerpts
Every card on Syndicator's front page shows a short excerpt under the headline. That line comes from each feed's <description> element, and it looks like the easiest part of the whole build. It isn't. The two systems behind almost every site Syndicator reads, WordPress and Hugo, put markup and entities into that field in different ways, and a cleaner that works for one quietly breaks the other.
All of the logic lives in lib/xml.mjs, in two functions: toPlainText and excerpt. This post walks through what each platform sends and why the code looks a little strange.
What WordPress sends
WordPress builds its RSS 2.0 feed from feed-rss2.php. The description is always the excerpt, wrapped in CDATA:
<description><![CDATA[<?php the_excerpt_rss(); ?>]]></description>
The full post goes in content:encoded, unless the site is set to show summaries in feeds. If the author didn't write an excerpt by hand, WordPress makes one: it strips the tags from the post, cuts it to 55 words and, when it had to cut, adds […]. Then the ent2ncr filter on the_excerpt_rss turns named entities into numeric ones, so a curly apostrophe arrives as ’ and the ellipsis as ….
So a typical WordPress description is plain text with numeric entities, sitting inside CDATA. Hand-written excerpts and plugins can add real tags too, and since it's CDATA, those tags arrive as literal <p> and <a>.
What Hugo sends
Hugo's built-in RSS template does this instead:
<description>{{ .Summary | transform.XMLEscape | safeHTML }}</description>
.Summary is rendered HTML, and with Hugo's default Goldmark settings the typographer turns apostrophes into ’. transform.XMLEscape then escapes the whole string with Go's xml.EscapeText. Running a sample summary through that exact function gives this:
<p>Hugo&rsquo;s summary has <em>markup</em> &amp; entities.</p>
The tags are escaped once. The entities that were already in the HTML are escaped twice. There's no CDATA. A reader has to decode this once to get HTML back, and then decode again to get readable text.
Why one pass isn't enough
The obvious cleaner strips tags, decodes entities and collapses whitespace. On WordPress input that works fine. On the Hugo string above, the strip step finds nothing to strip, because there are no real tags yet. The decode then produces <p>, <em> and the rest. If the angle brackets get removed after that, you're left with this:
p Hugo’s summary has em markup /em /p
That leftover p is exactly what used to appear at the start of Hugo cards on Syndicator. The comment above toPlainText still records it.
The fix is to strip markup on both sides of a single decode:
export function toPlainText(input) {
if (!input) return "";
return stripMarkup(decodeEntities(stripMarkup(String(input))))
.replace(/[<>]/g, " ")
.replace(/\s+/g, " ")
.trim();
}
The first stripMarkup removes WordPress's real tags. The decode turns Hugo's escaped tags into real ones. The second stripMarkup removes those. Entities are decoded only once in this function, on purpose, so a title written as Q&amp;A comes out as Q&A and not Q&A.
That still leaves Hugo's ’, which was escaped one layer deeper. It gets resolved later, in excerpt:
export function excerpt(text, maxLength) {
const clean = toPlainText(text);
// ...
}
The summary already went through toPlainText during parsing, and excerpt runs it again. That second run is the one that turns ’ into a real apostrophe. It looks like a redundant call, and the code comment warns that removing it will break Hugo cards. Each pass peels one layer of escaping.
Stray brackets and other junk
stripMarkup does a little more than remove tags:
function stripMarkup(value) {
return value
.replace(/<(?![a-zA-Z/!?])/g, " ")
.replace(/<(script|style)[\s\S]*?<\/\1\s*>/gi, " ")
.replace(/<!--[\s\S]*?-->/g, " ")
.replace(/<[^>]*>/g, " ");
}
The first line handles a < that can't start a tag. The WordPress test fixture in the repo keeps a real case: a description that starts with < The day’s main signal. Without that rule, the tag regex treated the bracket as the start of a tag and swallowed text up to the next >. Now it becomes a space. Script and style blocks go with their contents, and so do HTML comments.
Picking the field and cutting it
The parser asks for the summary in this order: description, summary, content, then content:encoded. RSS 2.0 feeds use the first, Atom feeds the second or third. A WordPress feed always has a description, so the excerpt wins over the full post in content:encoded, and Syndicator doesn't have to strip a 3,000 word article down to 200 characters.
Then excerpt cuts the clean text to 200 characters (excerptLength in config.mjs), backs up to the last space, drops trailing punctuation and adds a single …. The test suite checks that "one two, three four" cut at 9 characters becomes one two…, without a dangling comma.
What it gets wrong
The approach is tuned for the feeds Syndicator actually reads, and it has edges worth knowing about if you borrow it.
- Literal brackets in text get eaten. A WordPress excerpt that says
Use 3 < 5comes out asUse 3 5, and one that mentions a<p>tag loses the tag name. The double decode can't tell escaped markup from text about markup. - Only common named entities are known.
decodeEntitiesknows 18 of them, such as&,’,…andé. Anything else, likeïor•, passes through as literal text. WordPress feeds dodge this becauseent2ncrmakes everything numeric, and numeric entities always decode. - WordPress's marker can survive. A trimmed WordPress excerpt ends in
[…]. Fifty-five words usually run past 200 characters, so the cut normally lands before the marker. When the excerpt is short enough to fit, the card shows[…]as is.
None of these have shown up as a real problem on the front page yet, which is why they're documented here and not fixed. A real HTML parser would solve the first two and add a dependency to a build that has none.
If you publish a feed
Both platforms' defaults are valid, and a careful reader handles both. But if you control your template, the WordPress shape is the easier one to consume: plain text, numeric entities, and any real markup kept in CDATA. Every layer of escaping you add is a layer each aggregator has to peel, and not all of them peel the right number of times. Hugo sites can change what goes into <description> with their own RSS template.
For how Syndicator handles feeds that fail outright, see how it keeps one broken feed from emptying the front page.