Podcast Advertiser Boot Camp, Part 4: Podcast Ad Campaign Attribution and Analysis

Right Side Up’s Offline team hosted a Podcast Advertiser Boot Camp at Podcast Movement Virtual in October 2020. This deep-dive course was designed for aspiring and current podcast advertisers to help them execute campaigns effectively and efficiently. In this series of articles, we’ll capture the expert insights from the boot camp and offer practical tips to get the most out of your podcast advertising campaigns. This article tackles campaign measurement and analysis, and how to close out your campaign. Check out the other articles in this series for our insights on pre-planning, media buying and planning, and podcast ad creative.

Tracking and measuring the results of podcast advertising isn’t as straightforward as what you’ve likely encountered in other marketing channels. Podcast distribution relies on the one-way data flow of RSS, which makes it easy for creators to share content, but limits the data available for advertisers trying to track their podcast ad spots. And although it is probably one of the most challenging things about this channel, thoroughly measuring your podcast ad results, often with overlapping forms of measurement, is crucial in making podcasts work for your growth marketing mix.

Podcast Campaign Attribution: Learning from direct and indirect attribution

To measure the success of your podcast ad campaign, you need to determine how many listeners converted to customers after hearing your spot. But unlike most digital ads, a podcast ad isn’t clickable, which leads to breakage in following the customer’s journey to acquisition. This is why, despite its digital transmission, we consider podcasts an offline channel—they behave more similarly to offline channels vs. digital channels. 

There are two ways to track attribution for podcast ads:

• Direct attribution: Vanity URLs or promo codes
• Indirect attribution: Post-purchase channel surveys

It’s worth noting that there are pixel tracking capabilities that have been developed in recent years, however, they are very nascent and not the main method we recommend using to track acquisitions. It can be valid in certain instances, as with brands who can’t use an offer to triangulate direct response, or on campaigns with a high ratio of dynamic to baked-in inventory, e.g. branding campaigns not tied as closely to customer acquisition goals. The widely adopted method of tracking most used by D2C and even B2B marketers relies on the direct and indirect attribution that we describe more in detail below. 

Direct attribution only tells part of the story

Direct attribution does provide useful information, but measuring customer actions based solely on a direct basis can leave huge gaps in your data because of the breakage we mentioned. This attribution method only captures information from the 20–30% of customers dedicated enough to follow the prescribed path. As most marketers know, people rarely follow the ideal purchase path without deviating, which is why multi-touch attribution models have increased in prevalence over the last 10–15 years. 

“This is a tried-and-true method,” said Grant Durando, Director of Offline Marketing at RSU. “It’s worked for a decade and there’s no reason to say it won’t work for another decade, but it doesn’t tell the whole story.”

If you only focus on measuring your podcast campaign through direct activity, you’ll misattribute 70% of your customers. Optimizing your acquisition portfolio based on faulty or incomplete data like this can negatively impact performance and prevent you from effectively scaling offline media, not to mention giving too much credit to other paid or organic channels in your mix.

“What happens if you rely solely on promo codes and vanity URLs? The channel as a whole is going to look really inefficient,” Durando cautioned. “If you can imagine taking any other channels in your marketing mix and trying to rely on 20 to 30 percent of overall activity until you hit your goal CPA and make your boss happy, it’s probably not going to happen anywhere.”

Indirect attribution captures valuable podcast ad results

As for indirect attribution in offline marketing, using a survey-based methodology is the gold standard. The most common form is a post-conversion survey, also called HDYHAU or how did you hear about us, so-named for the question that we ask consumers. Your survey should be as close to the end of your funnel as possible to avoid impeding conversion rates and to capture a better picture of what channels contributed to conversion activity.

This allows you to capture feedback from highly qualified customers with minimal to no disruption to the purchase funnel. It also offers a timely feedback look and granular insights, which can help you fine-tune your future campaigns. This method can be used for channels besides podcasts as well, and many marketers we work with have started to use it to gut check results from paid search and paid social, especially post-iOS 14.5 changes. 

The main drawback to watch out for with survey-based methodology is that because you’re working with a small sample size (your customer base) it can take time to reach statistical significance with your data. 

“The more responses you have as a portion of your total converting audience or population, the more stable, reliable, and trustworthy your survey-based analysis is going to be,” Durando said.

Your best bet is to serve 100% of customers post-conversion—for example, with ecommerce, have it as a choice post-checkout on your “thank you” page. We’ve seen completion rates exceed 60–70% with this approach, and this allows us to paint a full picture of marketing attribution in a way not matched by many other approaches. That said, don’t let perfect be the enemy of the good here. We’ve also seen, as an example of less than optimal circumstances, statistically significant results by sending out email surveys to new customers with relatively low response rates.

How to design, implement, and interpret your attribution survey

Getting useful results from your attribution survey starts in the design stage. Follow these best practices to ensure your data is usable and reliable:

• Integrate the survey into the checkout flow immediately post-purchase to ensure qualified customer feedback and an increased the likelihood of survey completion.
• Randomize multiple choice sections to avoid survey bias; for example, alphabetization can lead to false returns of the top choice.
• Avoid marketing jargon and focus on understandable, common language so customers are confident in their choice.
• Survey choices should be a mix of paid channels and organic/earned channels and allow for a selection of one choice. Try not to exceed 10–12 total options, including Can’t remember and Other responses.
• Run the survey for a minimum of 30 days pre-channel launch, and if that’s not possible, take a look at patterns over time (especially in dark periods) to determine baseline channel response.

Once the results from your survey are in, the next step is to calculate the total data for your fully attributed audience. Here’s how to do it:

Direct Converters, e.g. via URL or promo code redemption

+ (Indirect Converters, e.g. survey respondents indicating a particular channel

- Pre-Campaign Baseline, e.g. respondents who answered that channel pre-campaign)

= Fully Attributed Audience

• Identify the total population who followed the CTA (direct activity, i.e. conversions from unique URLs, promo codes, etc.).
• Estimate the population who converted as a result of the channel you’re measuring (indirect activity, i.e. channel survey respondents), for example, if 3.2% of survey respondents selected podcast as their response, then approximately 3.2% of all conversions were driven by podcasts.

Once the fully attributable audience is determined using these inputs, the multiplier is representative of the difference between the direct acquisitions and the fully attributable acquisitions (direct and indirect). The multiplier that is calculated based on the survey should be applied to the total direct acquisitions for the full, attributed results. 

This multiplier will give you a more accurate overall CPA than relying only on the direct activity CPA. If your data shows that for every one person that converted via the vanity URL/promo code path, there were two others that converted via podcasts, but not directly, according to the survey, you should apply a multiplier of 3x. 

“The multiplier can be applied to 100 percent of the shows you run, whether you’re running on a boutique show with 5,000 downloads or you’re running on Pod Save America with a million downloads," Durando said. "The challenge with pixel-based tools is that you can’t get that type of widespread adoption and it doesn’t offer a universal solution like a HDYHAU survey.”

If you don’t use a multiplier, your ability to scale the channel into a profitable acquisition driver becomes much more challenging. It also ignores valuable acquisition drivers that, proven with multiplier methodology, are actually efficient in the channel. Additionally, you may be over-attributing to other channels in your mix, like search. 

How to Analyze Podcast Ad Campaign Results

Because podcast ad campaigns are flighted with several ad drops over the course of multiple weeks, analyzing results requires patience and a big-picture approach. 

“The analysis stage helps us identify opportunities to improve,” Durando said. “Understanding campaign results allows us to more successfully manage stakeholder expectations, and optimize every campaign to make it not just better, leaner, and more efficient, but also set up a framework to scale,” Durando said.

Unlike other marketing channels, which rely on ads that are experienced in real time, the on-demand nature of podcasts creates a unique consumption lifecycle. It can take a single episode up to three weeks to reach its booked downloads and the consumption curve for people to actually listen to a podcast they downloaded is about 19–21 days. 

Because of this unique consumer consumption pattern, there are multiple instances and touch points where listeners can interact with the podcast that basically reset specific curves. Every action, from downloading the episode, listening to it, checking out the company—via prescribed call to action, or otherwise—and making a purchase, all have downstream impact on results.

Performance should be monitored weekly, but keep an eye on your medium- to long-term goals. You should see results continually increasing toward your goal, but it often takes a full initial flight—the entire time ads are scheduled to run, usually about 3 episodes per show—to hit peak performance. Don’t worry if your CPA and spend look like a roller coaster. As you begin to convert shows to renewals and refine future tests, the curves will smooth out over time. Allow your campaigns time to age into efficiency, as spend accrues and listeners familiarize themselves with the brand.

Podcast ad campaign optimization and troubleshooting

Once you’ve gathered your results, there are a few important events and metrics to incorporate when gauging what’s influencing your outcomes:

• Did your ads drop? Many shows are host-read, which can lead to occasional unpredictability in scheduling.
• When did they drop? If you just had a big show drop for the first time and are looking at results in-week, performance will look very soft.
• Compare all-time, YTD, 12-week, and 4-week CPA. Don’t just look at one time window to make optimization or renewal decisions; look at the full picture.
• Identify intrinsic trends (download numbers, placement) and extrinsic trends (political noise, news, sports events) that may impact performance.
• Ensure your creative was delivered accurately and in a compelling fashion, ideally with some personalization or customization by the host(s).

Aside from those factors, there are many internal elements that can affect your campaign, such as:

• Offer is not competitive
• Business seasonality
• Code leakage
• Bad/unclear messaging
• Landing page optimization
• No indirect attribution methodology or attribution breakage

If the results don't match your expectations, consider your options for optimizing both your media planning and buying, as well as your creative.

Podcast ads rely heavily on host reads and other factors out of your control. On the one hand, this is positive, as it leads to compelling creative chock full of ad-libs from influential hosts that often make spots punch above their weight, performance-wise. However, this also means mistakes and oversights can happen. You could have grounds for a “makegood,” or added value (think extra spot, social post, etc.), from the show or the network, if any of the following occur:

• Wrong or incomplete CTA
• Not hitting download numbers
• Violating competitive clauses
• Off-brand comments

To request a makegood:

• Establish and maintain a good relationship with shows before bad things happen
• Be polite, firm, and fair
• Suggest solutions that meet your needs
• Be as transparent as is legally possible
• Get everything in writing

Specific makegood criteria is baked into the terms and conditions for podcast advertising that we use for all Right Side Up clients, which has led to hundreds of thousands of dollars in makegoods while eliminating nearly all of the back and forth that can happen if expectations aren’t set with publishers ahead of a flight. 

What to Do After Your Podcast Ad Campaign Ends

Give yourself a pat on the back, you did it! Pre-planning, media planning and buying, creative development, tracking, and analyzing your campaign was no small task. Now it’s time to take what you learned and use that to scale podcast advertising for your brand. 

You likely tested several shows, but that doesn’t mean every show deserves a renewal. 

“We don’t need or even want every show to be a winner,” Durando said. “We don’t necessarily want to run a campaign of 15 shows and have every single one meet our goal because then we don’t have any definitive direction for iterative testing and scale.” 

If results are looking more all-or-nothing (either the campaign “worked” or didn’t) it could mean your test was too small to drive significant results, or that you need to reassess some of the aforementioned business-specific elements (e.g. proper goal setting or attribution). 

For the shows you do choose to renew, be sure to:

• Book proven media out as long as partners will allow in order to to lock rates, block competitors, and secure ideal flighting.
• Be sure to negotiate an industry standard 30-day out clause or better, as performance attrition can happen over time, usually faster for smaller shows.
• Identify opportunities and trends with lookalike audiences.

We recommend allocating at least 30% of your second flight’s budget to testing. Use this to drill deeper into successful verticals and test failed verticals repeatedly.

Now that you have your podcast program established, you can continue to optimize over time and find the right balance to drive continued success in the channel and unlock offline marketing more broadly. Most podcast programs can scale to $500,000 to $1M+ in media spend monthly, profitably. And, success in podcast advertising is often indicative of strong possibilities for influencer marketing, affiliate, and/or other offline channels, like radio, satellite radio, terrestrial streaming, etc. where you can find even more scale.

Are you struggling to sort out attribution for your podcast ads or looking to scale your program after initial tests? Give us a shout at hello@rightsideup.co—we’d love to chat about your podcast advertising strategy, no strings attached.