Facebook’s Breaking News Indicator valuable to publishers despite rumors of hidden risks, post cannibalization

13 min readJun 30, 2021

Three years into Facebook’s attempt to take a bite from Twitter’s breaking news empire, there exists very little good data on the social media’s giant’s most visible effort — the Breaking News Indicator.

Facebook’s Breaking News Indicator is a bright red label that certain U.S. and global publishers can apply to a News Feed post, highlighting a general sense of urgency around its topic. The Breaking News Indicator was tested in late 2017 and more broadly expanded in March 2018.

The rules of engagement around the Breaking News Indicator are that most publishers can use one per day plus a slush fund of five extra per month in cases where unexpected critical news needs to be conveyed. The label can be set depending on the expected duration of the news, displaying for 30 minutes to six hours and can be manually extended.

Most importantly, according to Facebook, the label by design has no impact on News Feed prioritization. Any engagement benefits are from the visual cue — not a thumb on the scale of Facebook’s algorithm. According to Facebook’s six-week alpha test with the Washington Post, posts with the Breaking News Indicator saw a 4% lift in click-through rate, 7% lift in likes, 4% lift in comments and an 11% lift in shares. Later studies by Facebook suggested “breaking news posts about politics, crime, disaster, and business perform best.”

However, even Facebook qualifies those results and potential upside: “We cannot guarantee that use of the Breaking News Indicator will result in a distribution boost.”

THE CASE AGAINST

One of the few studies on Facebook’s Breaking News Indicator was conducted by Echobox in 2019, and it’s often cited as a reason against full adoption. The chief takeaway is that while the label does appear to be effective in boosting engagement around a post with the Breaking News Indicator, the study suggests that the engagement boost on that individual post didn’t translate to any overall page-level performance increases.

Reporting on the study for Social Media Today, Sebastian Huempfer wrote: “The root cause is most likely that shares for which the Breaking News Label was not used performed relatively worse than they could have — it seems likely that this is at least partly a symptom of an underlying cannibalization effect.”

“In summary, while the Breaking News label evidently generates increased viewership for breaking stories, the findings show that it also decreases traffic to other posts, and therefore does not raise overall traffic from social media.”

I think there are a few problems with that cannibalization hypothesis if we assume Facebook is telling the truth that the tag has no impact on the News Feed algorithm. And Facebook assures me we should believe them.

First, due to the frequency limitations, any benefit is going to be small. If your page shares 40 posts a day and two have a Breaking News label getting the aforementioned modest engagement boost, that might not move the needle that week, let alone that month. It may be like a muon — it’s there, but that particle is small and difficult to detect.

Second, without true multivariate testing available to us — serving simultaneously the post with the label and the post without the label — it’s a degree of guesswork. What if you’re putting the label on posts that would have overperformed anyways? Lastly, we know the content of the article, its timeliness and the quality of the post matters most. A boring post to a procedural story is unlikely to overperform on engagement KPIs just because it has a red label.

So given all that, how can we assess the value and risks, if any, of the Breaking News Indicator?

KEY QUESTIONS

The per-share engagement question seems fairly straightforward to hone in on. The questions seem to be:

Does a post with a Breaking News label get more interactions than ones without a label?
Does a post with a Breaking News label reach more people and get more clicks than ones without a label?

In a subscription-focused business model, referral traffic is critical so I wanted to expand the analysis beyond just comments, likes and shares and tackle reach and link clicks.

The cannibalization question is more interesting and, honestly, harder to pin down. I’m going to approach it by trying to answer these questions:

Did we see gains in interactions on the days when we used 1 or more Breaking News labels, or were the gains in those posts offset by losses in others?
Does the use of the Breaking News label on a post negatively affect posts immediately before or after that share?

For the first two questions, since the default Breaking News indicator guideline is daily, it seems fair to incorporate that into our analysis. If the Breaking News label adds value, we would expect to have more total daily interactions vs. a day when we don’t use it — even if only by a little. And if we use the label more than once in a day, the likelihood of exceeding the average/median should further increase.

For the last question, I want to limit the scope to posts immediately before and after a Breaking News-labeled post, because it should not affect a post 18 hours later. If it’s going to have an impact, we probably should see it most acutely in that immediate time vicinity, stealing traffic from the post right before or right after the Breaking News post was shared.

For our dataset, Tribune Publishing has four markets with the Breaking News Indicator enabled — the Chicago Tribune, New York Daily News, South Florida Sun Sentinel and the Hartford Courant. And we’ll look at every post in 2020 to get a sample size of around 50,000 total posts, hopefully minimizing the effect of outlier social data. It’s also a good range of page sizes, from 143,000 likes to 3.2 million. The only wrinkle is that in 2020, Facebook loosened restrictions during COVID-19 outbreak, allowing twice the posts to be labeled Breaking News each month during the worst of the pandemic, I’m told. However, this could prove beneficial in the end by increasing our sample size a bit.

TRIBUNE PUBLISHING ENGAGEMENT DATA

To get the data to address the questions, CrowdTangle and Facebook Insights should be plenty. CrowdTangle has the useful function of logging all posts that had a Breaking News label, even after expiration. An important note, however: Posts with a Breaking News label from one account shared by another account can track falsely as having used the Breaking News Indicator by the account that reshared the post, so you have to be careful.

The usage data for 2020 was pretty easy to pull together. And we avoided a situation I was dreading. All of the 1,150 Tribune Publishing posts with a Breaking News Indicator in 2020 were on link posts, not video or photo posts. As engagement can vary by post type, we can limit our analysis to just link posts with the Breaking News Indicator and all link posts, excising photo and video posts.

Looking at the data, the Daily News (389) and the Sun Sentinel (419) more aggressively used the label, around one per day, but the Sun Sentinel did it in about half the link posts. The Tribune (212) used one every other day and the Courant (130) one every three days. The team at CrowdTangle confirmed my data.

As a percentage of total posts, the shares with the Breaking News label represented between 1.66% and 3.22% of total link posts in 2020 — so a fairly small percentage of total volume, even in the more aggressive cases.

Looking at a year’s worth of link posts from CrowdTangle, we can get average and median post data. Daily News has about 1,000 interactions per post, Tribune 500 and Sun Sentinel and Courant around 100.

Looking at just posts with the Breaking News label we see much higher average and median interactions, from 3x to 20x.

The pages with the most fans still have the most interactions, but it’s interesting to see the highest gains at the Courant, which has the fewest page likes, fewest link posts and fewest Breaking News posts.

Percentage wise, the difference between the Breaking News post and all posts is pretty gaudy — but again, there’s many factors, including obviously the quality of the story being shared and the quality of the post itself.

But while not a cause, the above data align with Facebook’s data that show increased engagement with Breaking News-labeled posts. The same largely holds true with reach and link clicks, as shown by two per-post datasets I grabbed from Facebook Insights.

I laid Chicago Tribune’s data out by month to isolate any seasonal or news-based fluctuations. We saw average per-post link clicks range from 1,200 to 2,000 and median from about 400 to nearly 1,000. Average monthly per-post reach was in the mid-40,000s for much of the year with a monthly median around 30,000.

But looking at just the posts labeled Breaking News, we saw average per-post reach was higher in 11 of the 12 months and monthly median per-post reach was higher all year. The same pattern held with average and median monthly per-post link clicks.

Again, the results fall into line with Facebook’s data that suggest an increase in link clicks as well. In our case, we saw a healthy margin of 100%, 200% and even more than 400% in two of our months.

So, whatever the cause, we’re clearly seeing per-post engagement track above the median and above average on posts with a Breaking News Indicator on a fairly decent dataset.

But that wasn’t really the core question, although it was valuable to validate. The central premise to probe is the thought that while the Breaking News label may help an individual post, it doesn’t help overall because it may hurt other posts. And that’s a tougher concept to approach.

MARKET(ING) CANNIBALIZATION

The most straightforward way to approach the first of our cannibalization questions is to look at the sum of daily interactions. Our markets have an extremely steady cadence when it comes to posting on Facebook.

Given the steadiness of the post cadence, roughly 1 an hour, every day, all year for the Tribune, we should be able to address our first question — did we see gains in interactions on the days when we used 1 or more Breaking News labels, or were the gains given by Breaking News posts offset by losses in others? In other words, did those Breaking News posts cannibalize engagement to the extent that we saw no meaningful difference between a day where we didn’t apply a Breaking News Indicator to a post and a day where we used one or more Breaking News labels.

To set up the daily data in Excel, I downloaded a CrowdTangle interaction report by day and added a merged column for the number of Breaking News Indicators on that day, with 0 representing that none were used. Then I created another new column adding text if the label was used: =IF(G289>0, “BRK TAG USED”,”NO BRK TAG”). Once I calculated the average and median interactions per day, I added two more columns checking if that interaction total was either below or above those daily totals. No day fell exactly on the average or adjusted median.

=IF(M289>$M$369, “ABOVE AVG”,”BELOW AVG”)

=IF(M289>$M$370, “ABOVE MED”,”BELOW MED”)

Then I tallied the number of days where the tag was used and if it was above or below the average or median. Here’s an example of the counters for averages.

=COUNTIFS(N2:N367,”ABOVE AVG”,P2:P367,”BRK TAG USED”)

=COUNTIFS(N2:N367,”BELOW AVG”,P2:P367,”BRK TAG USED”)

In 2020, the Tribune saw an average of 12,487.93 interactions per day and a median of 11,320 interactions per day. The Tribune applied the Breaking News label to at least one post on 170 different days in 2020. For 93 of those 170 days, or 54.7%, the Tribune had above average interactions levels. For 109 of the 170 days, 64.1%, interactions exceeded the daily median. So in both cases, average or median, the Tribune was more likely to exceed the typical number of daily interactions when the Breaking News Indicator was used on that day.

The same was true for the Sun Sentinel. In 2020, the Sun Sentinel saw an average of 4,405.12 daily interactions and a daily median of 3,797. The Sun Sentinel used the Breaking News label on at least one post on 182 different days in 2020. For 115 of those 182 days, or 63.1%, the Sun Sentinel had above average interactions levels. For 126 of the 182 days, 69.3%, interactions were higher than the total daily median.

In both cases, we do see correlation between higher daily engagement and the use of the Breaking News Indicator. While that certainly isn’t causation, if there were a hidden cannibalization risk, we would expect to see those usage days to track more toward the average or median — not roughly two-thirds of the days when it was used to perform above normal engagement levels.

The second way to explore the cannibalization theory was attempting to quantify if the use of the Breaking News label on a post negatively affects posts immediately before or after that post. Basically, does that label cannibalize engagement from other posts, in essence, using up all the oxygen in the room?

I picked the posts immediately before and after because, in theory, those should be more susceptible to influence. Maybe it cuts short a post’s reach/exposure in favor of a new post with the Breaking News Indicator, or maybe it doesn’t surface as energetically a brand new post following a post with a Breaking News label. Again, not being able to imagine a reason a post published at 5 a.m. should have any bearing on a post at 8 p.m., I started with those immediately adjacent.

To test this theory, I used the same year’s worth of data from CrowdTangle and added a column marking a post as “pre brk,” “brk”, “post brk” or “pre brk post brk” and screened the rows. And the initial results were anecdotally intriguing. There did appear to be an interaction spike around Breaking News labels.

Then I used an Excel sumproduct function to count each category.

=SUMPRODUCT(( — EXACT(“brk”,N2:N8708)))

=SUMPRODUCT(( — EXACT(“pre brk”,N2:N8708)))

=SUMPRODUCT(( — EXACT(“post brk”,N2:N8708)))

=SUMPRODUCT(( — EXACT(“pre brk post brk”,N2:N8708)))

The data show that we had those 212 Breaking News-labeled posts. However, due to timing irregularities, there weren’t 212 before and after. Sometimes we saw sequential breaking posts, or a post that both followed a Breaking News post and preceded another. In the end, we had 196 posts that preceded a Breaking News post, 196 that followed one and 3 that both preceded and followed.

Then I tallied the interactions and created averages for all four categories, looking for the labels I added in Column N and summing the total interactions in Column O.

=SUMIF($N$2:$N$8708, “pre brk”, $O$2:$O$8708)

=SUMIF($N$2:$N$8708, “brk”, $O$2:$O$8708)

=SUMIF($N$2:$N$8708, “pre brk post brk”, $O$2:$O$8708)

However, looking at average interactions in all four categories for an entire year’s worth of Breaking News Indicator data let the air out of the cannibalization balloon.

We still saw 2,139.59 interactions per Breaking News link post and average per-post interactions on all link posts at 523.05. These matched earlier data. But now we also have posts immediately before a Breaking News label averaging 536.58 interactions, posts following a Breaking News label averaging 569.60 interactions and our 3 posts serving as both before and after averaging 642.33 interactions. So there doesn’t appear to be any engagement loss in shares with adjacency to a Breaking News post. In fact, all are slightly above average.

And when I looked closer at the Excel file, there were examples of unengaging Breaking News posts and posts in their proximity which outperformed them.

In the end, I couldn’t find any conclusive data to support the cannibalization hypothesis.

FINAL VERDICT

Does Facebook’s Breaking News Indicator give an engagement boost to shares? Facebook’s tests say it does and it appears to correlate with higher performance in likes, shares, comments, reach and clicks in all of Tribune Publishing’s markets that have it.

Is there a hidden downside, where it potentially adversely affects other posts? I couldn’t find evidence of it. There are more good days than bad when the label is used. And the shares in the proximity of a Breaking News post perform right around the average.

Facebook’s Breaking News Indicator appears to function exactly as advertised. It’s a tool publishers can use to help readers quickly identify time-sensitive news and information. And given the delta over typical engagement, readers seem to recognize and respond to it. However, its usage is capped. So if you post dozens of times each day, it’s unlikely to move the engagement meter in a significant way once you start collecting and analyzing months of data.

That said, there does appear to be measurable gains without any quantifiable risk. Given that Facebook already throttles usage, publishers should use it as aggressively as the news cycle merits.