Header Bidding trial by fire: How 6 popular header bidders perform

Christopher Reid Ad Ops, Uncategorized

As publishers, we must monetize, and those of us who aren't in a monopoly position Google or Facebook are at the mercy of a fragmented and messy, but necessary, adtech ecosystem. It's how thousands of us make money and it certainly has its problems and frustrations.

If you've been in ad operations for a while, you've seen many shifts that have taken revenue and control away from you:

  1. The shift from first-party to third-party ad creatives and servers meant fewer impressions got attributed and paid for
  2. Third-party verification resulted in other organizations being the arbiters of your impressions' quality and worth
  3. The rise and fall of ad networks shifted yet more impressions into lossy waterfalls that were impossible to tune

Now the rise of header bidding is upon us and it seems, in many respects, much more publisher-friendly. It offers the first real alternative to Google, and a vast improvement over waterfalls.

While ad servers haven't changed a whole lot in the last decade, the ad tech industry continues to morph and shift at a rapid pace. It's odd, then, that ad servers, a tool originally designed to serve a publisher's direct campaigns has also become the center of how we manage programmatic inventory and relationships with multiple partners. In so many respects, header bidding represents a new shift for publishers and moves decisioning away from the hegemony of ad servers, ad networks and SSPs, and towards a more open programmatic environment focused on the publisher.

2 Reasons Header Bidding Exists

Header bidding (which we'll define officially in a moment) has become a compelling tool for two simple reasons:

1. Demand Partners want First Look

Publishers have always had many options for filling impressions, but they've also had to make tough decisions about how to prioritize partners. This prompted the need for waterfalls - a way for publishers to send ad impressions first to Partner A in hope they'd fill and then to Partner B in case they didn't, and so on and so forth.

This tiered system only serves to create friction between demand providers and publishers. Publishers wanted the highest-paying partner in the first place, and demand partners felt like they deserved that privileged position. Partners would argue that they perform better when all the impressions haven't been cherry-picked (a valid, although not always true, statement) and publishers had no good way of measuring in real-time who would pay the most for a given impression.  First-look position would simply go to the partner who previously performed best (circular logic, since the partner who previously performed best probably did so because of their position) and millions of impressions would potentially go out to the wrong partner. Header bidding gives us the data to know how inefficient the systems of old were (and yes, a fixed partner is an incorrect allocation most of the time).

2. Waterfalls Hurt Publishers

To ensure impressions get filled, publishers have historically had to waterfall their demand partners.  Jumping from your ad server to the first ad network in your waterfall creates 10-15% loss in impressions, which is bad enough to begin with.  But every time you send an impression and it doesn't get filled, it has to hop back to your ad server and then to a new partner, incurring another 15-20% loss each time. It quickly becomes hard to justify the effort.

Limited Demand Pools

Because loss creeps up so quickly, waterfalls of depths greater than 2 partners make little sense.  So right out of the gate, the depth of demand partners you can use is limited.

No Competition and No Granularity

When you send an impression to an ad network you typically find out the next day how they performed, so if you sent them 1MM impressions you might learn that you made a $2CPM and $2000 in revenue.  What you don't know is what the ad network paid for each of those million impressions, and whether someone might have bid more.  Even with DFP's dynamic allocation, you're only allowing AdX and AdSense to compete at the average CPM with a waterfall, when there are almost certainly many high-value CPM impressions that could have enticed AdX to pay more.


Slow ads aren't good for anyone - not advertisers, users or publishers - and waterfalls can negatively impact speed, as impressions bounce around between ad servers.

Punishes your Demand Partners

Giving a demand partner First Look makes sense, because then they could put their best bid forward on all the impressions.  Waterfalls mean only one partner is awarded this position and, as such, it's assumed that they and they alone are the perfect partner for all impressions (when the whole premise of a waterfall is that each partner can not be everything for all impressions). Ideally, all partners would be given First Look, so all could bid on your inventory in a unified auction.

Here Cometh Header Bidding

Demand partners and publishers are both motivated to find an alternative to the traditional waterfall. Buyers want first look access and a more level playing field, and publishers want a guarantee that impressions will be filled by the highest-paying partner.  Header bidding seems like the perfect solution to satisfy both buyers and publishers.

So what is header bidding? The name hints at its meaning.  In the HTML header you have tags from each of your demand partners. These tags don't get ads like traditional ad network tags; they go out and retrieve bids, and they do so simultaneously.  By getting bids from multiple partners you've now eliminated the need to see if an ad network will or won't serve an ad. With the highest bid in hand, you can pass it into your ad server and perhaps supersede a direct campaign, or have AdX compete one last time, before finally serving the creative.

Programmatic always offered the promise of publishers having demand partners bid on their inventory in an auction, with the highest bidder winning.  While that did exist on a micro scale in individual exchanges, on a macro scale there are many exchanges and differentiated demand sources that do not wish to come together, and the dream remained unrealized for a long time.  Header bidding offers an opportunity to change that.

A Serious Threat to Google

For the first time, there is a threat to Google's dominance - header bidding.  Google has always had the most powerful position in the market: the dominant ad server with first right of refusal to any and all ad impressions passing through it (with restrictions to ensure direct delivery, of course).  While this right to raise their hand at the last moment still exists, and is responsible for a lot of the lift that header bidding provides, the fight is much fairer now.  For the first time, a publisher can present AdX with real-time competition. The waterfall setup hamstrung demand sources to only be able to compete at the average price they could fill at the previous day, after loss and after publisher floors.  With header bidding, Google's competitors can launch their attack in real-time and with perfect price granularity   

The theory goes that with more partners all throwing their collective weight against the giant, they can together take away a significant portion of Google's market share.  At Sortable, we now see 40% of programmatic revenue generated by partners other than Google and that's an incredible shift.

The Experiment

If bidders are good for business then it stands to reason that the more pressure put on Google, the better. We ran an experiment to measure what happens when you increase the number of bidders and how that impacts CPM.

Here's what the setup looked like:

  • We tested header bidders for 6 partners
  • For every impression, header bidders were randomly turned on/off
  • We tested performance of no header bidders, all six, and every permutation in-between

Statistically, it looks like binomial distribution, but all that really means is every network in every permutation was tried and measured.

* Tribal Fusion does not have a proper header bidding solution, instead we model their performance and emulate them as a bidder in our framework.


This test was conducted on a single site and we can safely say, having seen how partners perform across over 100 sites, that the results will vary widely by site in terms of partner performance.

Secondly, this test was done during a given time period, set of users, devices, and geographies, Again, we've seen that partners performance changes both over time but also across these other dimensions.  

Thirdly, the test was run with 6 popular header bidders.  We currently work with many more bidding partners than these, and in many instances and time periods the correct set of partners varies, sometimes wildly.  

Fourthly, we run our own bidding framework that does a lot of learning, adjusting and optimizing to suss out the best performance of any given partner.  The results include this framework, and it was included consistently across all experiments. Relative results are valid, but your experience may certainly vary from ours.

The Results

Reliable Outperformance
All Bidders

The following graph shows performance across all geographies and all devices.  Along the x-axis is the number of bidders, and on the y-axis is the average CPM generated.  We're using a box plot, which is just a fancy way of showing the variation in CPM, as well as the average.  The variation is important, because it helps us understand how consistent the performance is across many scenarios.

There's a strong correlation between increasing bidders and increased average CPM performance. We saw a 58% increase in CPMs when running six bidders versus none.  It's hard to overstate how significant this increase is.  If you're a publisher running 20% net margins, and programmatic represents 50% of your revenue, then you just increased profit by 145%.  Now there are caveats, of course (see above), but in our experience across hundreds of sites one thing is true: header bidding adds lift.  

The next observation is that increasing bidders also reduces the variance in performance.  This is really useful because it speaks to the setup working well across many scenarios.

Outperformance Not Guaranteed

If we take a look at just desktop traffic, it tells a slightly different story.  One of the first things that is immediately obvious is the huge variation in CPMs, and how adding bidders tends to reduce this variation.  There is also a local minimum, where running two bidders appears to be  worse than just running one.  

But let's not digress: as you add three or more bidders, the situation becomes favorable and consistent, with six bidders providing 25% lift over zero bidders.

More Bidders Always Better?

The real answer is yes. And maybe.  In this test, the answer is a clear yes, but change the timeframe, site, demand source or one of many other variables, and you may get a different answer.  This test was for a specific site and a discrete group of users, geos, devices and page URLs, and when you change those variables, you may change the outcome.  When we're running bidders, sometimes we run none, sometimes we run five or six. We also run bidders selectively, because not every demand source is appropriate for every situation.  

The simple answer, borne out across hundreds of sites is simple: if you add bidders, they'll provide lift.  The more complex answer is to take a data-centric approach to bidders and revenue, and you'll be surprised to learn how much extra revenue is available at the margins.


At Sortablewe take a very data-focused approach to ad operations. We have ten engineers including two PhDs who spend all day focused on revenue optimization.  We started off as publishers, and we still are. With 20 of our own websites and apps, we now exclusively manage almost a billion impressions per month.  We think using big data, machine learning and automation are the best way to drive programmatic improvements, and our goal is to leverage our learnings and provide the best possible outcomes. If you don't spend your days dreaming of multi-variate testing, machine learning algorithms, mediating between 20-30 demand partners, managing creative quality across networks and other such fussy work, we'd love to talk about we can make your life easier.