What We Learned About Creative From Analyzing $3M in Podcast Media

Podcasts Pose Challenges For True Ad Creative Testing

In a landscape where creative undergoes continuous A/B testing in digital environments, like we see with Meta and Google, growth teams frequently ask about testing creative within podcasts. Seems like a simple way to answer what makes one type of spot or copy more successful than another, right? Not exactly. Unlike traditional digital channels, there are specific hurdles within podcast advertising that make it a challenging channel to successfully A/B test in, including:

• Different lengths: All placements in a media plan are not necessarily all the same length (e.g. :15s, :30s, and :60s).
• Different placement types: Placements can be embedded or dynamically inserted.
• Different dates: Episodes can drop on dates that may or may not overlap with on-site promotions or initiatives, or other messages in-market like TV campaigns.
• Time-shifted media: The delayed consumption curve of podcast listenership also time-shifts results, unlike the more seemingly instantaneous effects of digital channels like social.
• Host-read creative: If you’re doing host-read ads (which we hope you are), you relinquish a lot of control over how your creative is interpreted and announced by the hosts. Some hosts may emphasize certain value props that others ignore. Others may record over two minutes of content for your ad, despite being contracted to a :60 spot.

You could, in theory, try running Creative A to one group of shows and Creative B to another group of shows. But then you’re still left with a big question. Was it the creative or the show (and its audience) that drove results? Making decisions without robust data to support your choices feels like a leap of faith, especially for new advertisers in the space. In order to jump into the channel, it may be tempting to ignore these inputs altogether, especially as new technology creates easy avenues to purchase spots.

After working with over 120 clients on podcast advertising campaigns, we set out to extract insights by analyzing data through a new lens. Rather than focusing solely on key performance indicators tied to shows, networks, and verticals, we developed an analytics tool to explore quantitative attributes like placement type and ad length, as well as qualitative creative attributes such as personal endorsement and ad copy prompts/scripts. Do creative choices matter for this channel? Or should marketers find the easy path and just get their brand on air? Let’s explore the answer.

Navigating the Evolving Podcast Media Buying Landscape

Self-service tools, like those offered by Spotify and distributors making bets on programmatic, attempt to democratize podcast media buying by making it simple for brands to be in the channel. While this approach is tempting because it is lower-lift compared to the typical process, it treats podcasts as a digital channel (spoiler alert—it’s not), which poses challenges for advertisers who may lack clear direction on what works or how to optimize campaigns, especially when it comes to creative. 

But with article after article touting investments in podcast content and ad-serving technologies, is this the direction us growth marketers should be going in? Will we get the same results in podcast campaigns if we just press the buttons to book the media, or does it require the ability to pull qualitative levers like personalization, testimonials, and good old-fashioned relationships? 

Methodology for Analyzing Podcast Ad Creative

For this analysis, we leveraged data from seven advertisers spanning diverse industries (including fintech, retail, food/beverage, and health/wellness) during Q2 2023, representing a total media spend of $3 million. We selected these clients as a cohort sub-segment of our broader buying due to the variety of industries they represent, the maturity of their podcast advertising initiatives, and the cleanliness of their data.

Utilizing Magellan AI for episode-level data and transcripts, we introduced a new analytics schema combining spend-level, performance, and creative data. This allowed us to explore variables previously inaccessible, such as placement location (pre-roll, mid-roll, post-roll), ad length, placement type (dynamic, embedded), and creative attributes (ad copy used / creative topic, use of personal endorsement).

Our metrics of success across the board were tied to CPA (cost per acquisition), relative to each client’s goals. Client-specific CPAs tend to vary based on price point, so we noted whether the CPA was higher or lower than their specific goal based on the variable we were assessing:

• Placement location refers to where in the episode a spot ran. RSU defines pre-roll as being within the first 10% of an episode, post-roll in the last 10%, and mid-rolls as between 11-89%.
• Contracted length in spot buying, refers to the length of the ad you have purchased. Typically this is :15 or :30 for a pre-roll, :30-:90 for a mid-roll (with :60 being the most common length), and :30 or :60 for a post-roll.
• Placement type refers to how the ad is inserted into the episode. Embedded, or baked-in ads, are native placements that exist within the podcast forever (or for an agreed-upon period like 60, 90, or 120 days before being removed) and are bought on a flat rate cost structure. Dynamic, or DAI placements, leverage ad servers for impression-based delivery. They’re pre-recorded and then stitched into the episode or catalog, and are bought on a CPM cost structure. In this analysis, we only took into account host-read placements (both embedded and DAI). We did not analyze any branded spots delivered via DAI.
• Ad copy is talking points or loose scripts shared with the show hosts. At RSU, our creative best practice is to share 3–4 “intro ideas” or prompts that tee up a problem and then pose the brand as the solution to that problem. We encourage show hosts to ad-lib, riff, and go off script so the reads sound natural and conversational, but usually they tie back to one of the ideas we include as inspiration.‍
• ‍Personal endorsement (PE) is a testimonial made by the host about the brand hinging on their experience with the product itself. Depending on the product, PE can be mandatory or not (e.g. when promoting medications, supplements, or financial products like credit cards or loans, PE is not always required).

Findings on What Works for Podcast Ad Creative

Placement matters for podcast ad success

When analyzing placement location, we discovered that for 100% of advertisers we analyzed, mid-rolls outperformed pre-rolls or other placements (like post-rolls). While those other placements sometimes come with a major price reduction, you’re getting what you pay for. They’re more likely to be skipped over or listenership abandoned, making your eCPM for actual listens much higher than what you contracted. That isn’t to say other placements don’t have a place on your media plans, but we can confidently say mid-rolls will give you the most bang for your buck.

Ad length enhances efficiency 

Typically, podcast ads are purchased as :30s or :60s, or rarely :90s. The majority of our buys tend to be contracted as :60s, but one upside of buying host-read mid-rolls is that they tend to sometimes go longer than the contracted length. This is beneficial to advertisers, because, generally speaking, longer ads tend to perform better. For all advertisers in our data set, ads over one minute drove more efficient results. This is indicative of the inherent value added when hosts “go long.”

When you think about your CPMs being tied to the :60 length, that CPM will go down exponentially for every second the host goes over the contracted amount. While we saw a couple of advertisers benefit from reads over three minutes in length, the sweet spot seems to be between 1–3 minutes. It’s long enough for the host to make a convincing testimonial, but not so long that listeners may be tempted to skip.

Placement type is tied to value 

At our first look into the data surrounding placement type, we were surprised to see a mixed bag: Three out of seven advertisers saw DAI perform better, three out of seven saw embedded perform better, and one advertiser had no real difference in performance. Our hypothesis was always that embedded will outperform DAI due to the delayed consumption curve and inherent added value that embedded spots provide over time. In cases where DAI outperformed embedded ads, the correlation to lower CPMs for DAI placements suggests that DAI can work efficiently if priced accordingly.

To script or not to script?

Perhaps the biggest question we get from growth teams is around the thematics of creative copy. Should we utilize in-house copywriters to write podcast scripts? Should we let hosts riff off our copy points? RSU’s approach to creative is to provide 3–4 intro ideas or prompts, and then let the hosts riff off of them and make them their own. This helps us to adjust for different variables like seasonality, while allowing podcast hosts to speak in a way that’s authentic and convincing. The data revealed a mixed scenario: Four out of seven advertisers saw success with prescribed intros, meaning we provided guidance and the hosts took it, then made it their own or read it word for word.

On the flip side, three advertisers saw better results when hosts went completely off-script and came up with their own intros. So does it make sense to provide hosts with guidelines and scripts? We think so—at least to start. We always recommend testing the channel with a problem-solution format to podcast copy. Set up the problem, then have your product or service be the solution to that problem.

Don’t be afraid to get personal

Perhaps the not-so-secret sauce in podcast creative is the use of testimonial-style personal endorsements. In six out of seven advertisers, we saw reads with personal endorsement outperform ads without PE. The exception to this was an advertiser in the health/supplement industry that did not require PE from hosts. Personal experience/endorsement is one of the biggest levers we can pull from an optimization perspective and the data shows it really does make a big difference. Host didn’t include it? Make sure they have tried the product and push for inclusion on the next read. PE was minimal/not really personal (“I love this product”)? Ask them to re-do it for next time and provide guidance on how to make it better (i.e. What was in your shipment? What was your experience ordering on the website? Where did you use the product you were gifted? etc.).

With this qualitative lever being the most valuable to the majority of the advertisers we studied, we can confidently say that it behooves brands to leverage personalization via endorsements and testimonials. They should embrace creative elements that promote personalization, regardless of the tactics they use to place the media.

Conclusion

Podcasts as a channel doesn’t lend itself well to traditional A/B testing, so we consider these findings to be directional insights rather than definitive rules. As a practice, we advocate for initially investing in what is known to perform well, and then test outwards from there. Use these findings as a way to optimize your campaign the best you can, and when you find your best practices, test the opposite to make sure they hold true. Podcasts often defy expectations, and strategies that go against the findings may still yield success. Despite these caveats, we hold firm on this takeaway: Success for growth marketers and unlocking the podcast channel hinges on personalization. 

Having strong relationships with network partners enables optimization levers that don’t exist in a purely programmatic landscape, and can lead to meaningful changes in results. 

From a buying perspective, this can be achieved through tactics like host-read, long-form mid-rolls. When onboarding shows, sharing samples, or gifting flagship, best-selling products with hosts will help ensure they are able to properly experience your brand and share their endorsement in an authentic way. Having kick-off calls with hosts or producers and building an onboarding document for hosts can also help push personalization into a new realm as it helps create excitement about the partnership (and not just the monetary transaction of giving them money to talk about your brand). Ensuring personal experience/endorsements are included in the reads or encouraging hosts to go a little longer than your 60-second contracted spot can sometimes make or break the performance of that particular show.

A fully dynamic/programmatic marketplace for podcasts may seem like a low-risk way to test into the channel. And as growth marketers who have been in-house and at agencies, we understand the allure here, but as channel experts with data to back it up, we know that if this is the sole method you’re using to test into the channel, odds are you’re setting yourself up for failure. True success in podcast advertising lies in personalized approaches, moving beyond the allure of the cheapest or easiest options to focus on what truly moves the needle.

Are you ready to try podcast advertising? Connect with Caroline and her team.