Four sources, one story.
The same story arrives through four feeds. The newsroom shows it once — with the source count beside it, and without paying to process it four times.
What you can do with it
- Read the same story from several sources as one card, with the source count and the list behind it.
- Trust that different companies are never merged — the entity gate checks companies, symbols and commodities, and knows VW / Volkswagen / VOW3 as the same firm.
- Save paid processing: the most advanced story leads the group, the remaining sources skip classification, fact extraction and research.
- Rule out a wrong merge permanently — the marker applies to the pair and survives every later run.
- Merge across day boundaries: the comparison window reaches three days in both directions.
The thinking behind it
Without grouping you pay for four classifications, four fact extractions and four research runs for a single story. And the desk reads the same thing four times before noticing it is the same thing.
The detection deliberately works without a language model: title similarity over word sets, edit distance and the longest common substring, plus a comparison of the bodies. That costs nothing and can be reasoned about.
The expensive part is not the finding, it is the not-confusing. Two stories phrased almost identically about two different companies are not duplicates — which is why every merge is preceded by a check that the same company is even meant.
A story is a story — even with four senders.
Improvements
Grouping runs again — it was hanging off the wrong stage
It only fired if at least one full text could be fetched in the same pass. When fetching failed for weeks on end, grouping stopped running altogether: a story carried by four sources stood in the stream four times. Yet grouping does not need the full text at all — it compares headline and summary. It now runs once per pass, right after the source scan.
No more catch-all groups built from templates
Machine-generated stories following the same pattern — “Die X-Aktie legt zu”, “Why is X stock sliding today?” — ended up dozens to a group although they were about different companies. Two rules now keep them apart: stories from the same source are no longer grouped (that is not what the feature is for), and headlines whose distinctive words contradict each other do not count as the same story. In testing, the largest group fell from 47 stories to 8; every group formed now spans more than one source.
Late arrivals find their group
If a story's second source only came in after the next pass, it stayed ungrouped forever — on its own it formed no group, and it never joined an existing one. It does now.
Company detection has a say
Title similarity used to decide alone. Detected companies, tickers and commodities now feed into the decision — two stories about two different firms no longer land in the same group, however similarly they are phrased.
The group has a representative
If one member of a group is already enriched — facts extracted, checked, researched — it leads the group. The other sources are marked as duplicates and skipped by every processing stage.
Give feedback or ask a question
Missing something, bothered by something, unsure about something? Write it down here — it goes straight to the people working on this feature.