April Data Insights – Facebook

Lessons from a deep dive into 1,396 Facebook ads

We’ve categorized all of the Facebook ads we currently have in our database of campaigns related to the IRA & clean energy (total spend of $1.3 million, from three partner organizations). We statistically analyzed the tone, message type, and call to action (CTA) to identify those associated with the highest click-through rates (CTRs) for four key audiences — BIPOC communities, left-leaning groups, influencer audiences, and broad populations.

Click through rate benchmarks by audience

Click-through rates (CTRs) are crucial metrics in measuring audience engagement with ads. We’ve pulled the averages from 1,396 Facebook ads as benchmark CTRs that serve as valuable points of reference for assessing the performance of your advertising campaigns.

Speaking to these audiences – key takeaways:

1. BIPOC Audiences

This audience covers Latinx, Native American, Black, Asian and Pacific Islander audiences. When speaking to these groups, the highest performing ads had active and negative tones (as opposed to positive tones), with messaging that instills a sense of urgency.

• Use your messaging to draw attention and motivate — Our analysis has found that  active tones (urgent or surprising) and negative tones (confrontational, fear-inducing, or suspicious) perform best with BIPOC audiences, with higher CTRs than positive tones (such as celebratory or friendly). Try using messaging that creates an emotional response in your audience by presenting surprising truths or by identifying and calling out actors causing harm.

• Keep it local — Across ads targeting BIPOC audiences, CTRs are higher for ads with messaging that focuses on the impacts and benefits for people and communities rather than the broader economic benefits or energy independence. Instead of spotlighting big numbers, try pointing to the change your audience and their family will see in their day-to-day lives: things like cleaner air, energy bill savings, healthier neighborhoods and more jobs in their local community.

• Environmental themes cut through — Protecting the environment is a powerful motivator for BIPOC audiences- ads highlighting the damage fossil fuels inflict on nature delivered higher CTRs than ads focused on economic benefits or job growth. In your messaging, when speaking about how climate change affects us all, try focusing on messaging around the impacts on our natural environment, rather than the wider economy or national jobs

2. Left-leaning coalition Audiences

This audience includes groups who may be more supportive of the IRA and the transition to clean energy: climate activists, Democrats, young people, labor union members, and people with an interest in climate more broadly. When speaking to these left-leaning audiences, our analysis shows that ads have higher CTRs when they have messaging about specific clean energy initiatives like EVs, and more detail on clean energy policies like the IRA, as compared to messaging that encourages energy independence and highlights the positive impact it has on energy reliability and financial security.

• Talk up the wins, not energy independence — Rather than energy independence, try to use messaging that focuses on policy change, new tax laws, and initiatives like EV production that will spotlight the tangible steps being taken to improve clean energy uptake:

3. Broad Audiences

This category covers wide public audiences without specific interest-based targeting. This audience is either nationwide or state-wide, including gender targeting and ranging from suburban women to job seekers and rural audiences. For these groups, we’ve seen that CTRs are higher when policy and clean energy messaging is used, rather than economic messaging that highlights job growth and bigger-picture economic impacts. 

• Ground your messaging in the policy and clean energy initiatives… — from new wind and solar developments to legislation supporting the energy transition. These can be state-specific or national policies and initiatives.

• …and don’t get carried away with mythbusting – This messaging with a policy and clean energy focus also performs better than copy focused on addressing misinformation. Unless you have a strong strategic reason to confront misinformation head on, try emphasizing real policy developments (positive or negative), clean energy initiatives, and the impacts of EVs, rather than calling out the “misleading content” about a piece of legislation or initiative.

4. Influencer Audiences
This audience includes policymakers, state governments, and other influential individuals. An invested and knowledgeable group, our preliminary data finds that higher CTRs occur in ads clearly identifying issues in a negative tone, rather than an informative tone.

• Problematize the issue —  As decision makers, these groups may be more knowledgeable about climate, so they don’t necessarily need to be enticed with positive framing and are ready for the bad news as well. Feel free to set out the issues and what they can do about them. They responded better to a negative tone (vs. an informative tone), which makes clear that their work relates to a problem in need of a solution. The negative-toned copy below performs better than more bland, informative statements such as “Today’s farmers are looking to the past for solutions to new climate challenges.”

A recap of our data-driven recommendations:

• ✊🏾 BIPOC audiences:
  • 😡📢 Negative and active tones: Use confrontational and urgent tones to draw attention to the issues at hand.
  • 👪 People & community messaging: Highlight the change your audience and those around them will see in their day-to-day lives.
• 🔋 Policy & clean energy messaging:
  • Talking about specific policies and clean energy initiatives generated higher CTRs than certain other messaging types for two audience groups:
    • 🌎 Left-leaning audiences: it outperforms energy independence messaging.
    • 🇺🇲 Broad population: it outperforms economic messaging and trying to address misinformation.
• 🏛️ Influencer audiences:
  • 😡 Negative tone: This group includes many decision makers and those in positions of power — don’t be afraid to draw their attention to the problems.

More data from you will help us firm up some tentative insights

Our analysis uncovered many tantalizing hints for other audiences – like Workers, Moderates and Republicans –  but we don’t yet have enough data to draw strong conclusions for them.  For example, how does more personal messaging perform with Moderate audiences?

In order to uncover these and other insights, we need to analyze larger volumes of data – so we need your help!

Click here to learn about the name generator – a simple, easy-to-use tool that can help us transform your data into valuable insights.

Coming soon 

In a future update, we’ll take a look at community organizing and on-the-ground advocacy. While our first updates have been ads-based, we’d love to include learnings from your experience working in and organizing your communities.

Please get in touch to share your story!

About the methodology

To evaluate the best kinds of ad copy for each audience, the 89up team first developed tag typologies for different types of tone, messaging type, CTA, and audience, and then tagged these characteristics for each of 1,396 Facebook ads shared with us by partners. To test for statistical significance, 89up calculated pairwise t-tests between ads sharing a tag, for each variable (tone, message type,  CTA), within each audience, while controlling for the family-wise error-rates with Bonferroni’s correction. See more in the Methodology section below.

April Data Insights – Snapchat

 

We analyzed data from $180,000 of Snapchat ads. This is what we found!

We examined 70 ads on Snapchat shared with us by Climate Power, examining their click-through rates (CTRs) and “cost per click” (CPCs) to identify characteristics shared by the most successful ads.

This is what the top creatives had in common

• Local and personal: the top performing ads (by CTR) had copy and images that spoke to the audience’s interest and location.
  • For example, one top performing ad featured a Milwaukeean who spoke about the harsh winters in Wisconsin.

• People, straight to camera: The top performing group of ads tended to have people looking directly at the camera in both static and video creatives, while the ads with CTRs below the average tended to have people looking away from the camera, or using word or picture graphics rather than images of real people.

Open configuration optionsOpen configuration options

Methodology Insights

Below you will find details on the methodology of the manual tagging project that was undertaken by 89up to achieve the results that were published in the April edition of the IRA Knowledge Product. Below we have outlined:

• Details on sample
• The tagging methodology 
• Definitions of the manual tagging terms
• Detail on the outcome of interest
• The analysis methodology
• A summary table of the results
• Our rationale for testing for click-through rates
• Limitations of the data analysis 

Details on sample 

• 1,396 Facebook ads were analyzed from three partner organizations, representing a total spend of $1.3 million.

Tagging methodology

• We first developed tag typologies for priority variables (different types of tone, messaging type, call to action, and audience), and then manually tagged these characteristics for each of 1,396 Facebook ads shared with us by partners. The data coding was conducted by two people, and a third person double-checked the results.

Manual tagging term definitions

To enable data analysis, we needed to structure the data by labeling it, coding each ad with terms corresponding to audience, tone, message type and call to action, from a list of possible options.

With limited data available, we grouped/categorized terms under broader headings to increase our chances of finding statistically significant results in the data. 

Below we have outlined the concepts that make up the broader categories which were used in the analysis results:

Audience	Definition
BIPOC	Black, Indigenous, Latinx, Asian American, Pacific Islander
Left-leaning coalitions	Climate activists, democrats, young people, climate interest, labor union workers
Broad audiences	Men, women, suburban, rural, nationwide, statewide, countywide, multistate, job seekers
Influencer audiences	Policy elites, decision makers
Moderate/consumers*	Politically moderate or persuadable
Republican audiences*	Republican audiences
Tone
Positive	Hopeful, empowering, celebratory, friendly
Negative	Disappointed, confrontational, suspicious, fear inducing, accusatory
Informative	Factual – complex, factual – simple, mythbusting
Active	Urgent, surprising
Messaging Type
People and Community	Climate change impacts (people), clean air benefits , community benefits, safety (positive and negative), protecting the future for next generations
Energy Independence	Security and insecurity (financial security, energy security)
Environmental	Climate change impacts (environment), environmental benefits, fossil fuel impacts, climate change initiatives
Economic	Economic benefits (e.g. energy bill savings), job growth
Policy & clean energy	Clean energy initiatives, EVs, IRA explainers, policy change (positive), policy change (negative)
Misinformation	Counter-misinformation
CTA Type
Action	‘Register’, ‘Apply now’, ‘Enter here’, ‘Follow our page’
	‘Call’, ‘Tell Congress’, ‘Follow the link’, ‘Survey’, ‘Fill out’
Awareness	‘Learn more’, ‘Learn how’, ‘Read more’, ‘Read about’, ‘Here’s how’, ‘Here are X things to know’, ‘Listen here’
Engagement	‘Check it out’, ‘Tune in’, ‘Watch’, ‘Swipe up’, ‘Check out’, ‘Follow us’
No CTA	No CTA used in ad copy

Outcome of interest: click-through rate (CTR)

• The clickthrough rate for each ad was calculated as clicks per impressions. This was then weighted by the cost of the campaign (which was rescaled by min-max-normalization).

Analysis methodology

• The unit of observation was the individual ad and the associated click-through rate. The unit of analysis was those groups of ads that shared a common tag (a given tone, message, type, or CTA), as compared against ads with different tags.
• To test for statistical significance, we first performed Analysis of Variance (ANOVA) and Kruskal-Wallis tests to evaluate the overall pattern of results. To identify which specific groups of ads differed from each other, we followed up with pairwise t-tests between ads sharing a tag, for each variable (tone, message type, CTA), within each audience, while controlling for the family-wise error-rates with Bonferroni’s correction.

Why choosing click-through rate (CTR) over cost per click (CPC):

• We have been using click through rates (clicks/impressions) as our metric as it is a clear indicator of ad performance.
• CTR can effectively and consistently assess audience resonance and engagement, indicating actions taken and ad quality.
• CTR can assess the ad impact and audience response, facilitating comparisons between creatives and copy.
  • Example: If we want to know the best way to engage with a BIPOC audience, by analyzing the tone, messaging and CTA you can establish trends based on CTRs. This makes it invaluable for refining targeting strategies and drawing benchmarks. 
• On the other hand, CPC is very platform- and and bidding strategy-sensitive, and subject to fluctuations based on elements that are harder to control such as audience size, competing ads, and seasonality.
• CPC serves as a metric for monitoring ad performance and controlling budget during live campaigns. It can help to decide whether to pause or increase spending on an ad during a campaign. For example, if an ad’s CPC exceeds the benchmark, consider pausing the campaign. Conversely, if the CPC is below the benchmark, consider increasing ad spend.

Limitations of data analysis

• This analysis is based on aggregating ad data from hundreds of campaigns run by several partner organizations. As such, the patterns we have identified (such as the performance of one message vs. another) represent observed correlations after the fact, rather than results from carefully planned comparisons as would be done in randomized controlled trials (RCTs) and the like that allow stronger inferences about causation. That is, additional factors may be behind the patterns in our results other than those that we have identified, limiting the confidence with which we can make statements such as “X message caused the CTR to be higher”.
• In addition to the results we reported, many of the comparisons between ad types we examined were not statistically significant. This lack of significance does not necessarily imply that there is no true difference between those types of ad content, as it may be due to a lack of data or the limits of our statistical methods.

Reach out if you would like to see more detailed data and results!

Related Resources

 

Last chance: IRA grants closing in April

The Energy Efficiency and Conservation Block Grant Program welcomes applications from municipalities, utilities, and other local entities interested in implementing energy-efficient lighting upgrades, weatherization projects, and renewable energy installations.

Applications close April 30, 2024 for local governments and May 31, 2025 for Tribes. Click here to read more.

The Green and Resilient Retrofit Program: developers, building contractors, and housing authorities can apply for funding for projects such as energy-efficient building envelope upgrades, installation of renewable energy systems, and implementation of green infrastructure solutions. 

Final round applications close April 30, 2024. Click here to read more.

The Federal Funding Primer on Climate Justice

This Federal Funding Primer on Climate Justice provides an invaluable roadmap for navigating the maze of funding opportunities provided by Justice40, the Inflation Reduction Act (IRA), and Bipartisan Infrastructure Law (BIL). The Justice40 Initiative is a requirement of Executive Order 14008 and mandates that 40 percent of the benefits from environmental and climate justice related federal investments go to communities that are disadvantaged, underserved, or overburdened by pollution.Open configuration options

Image

How’s IRA doing?

Is the Inflation Reduction Act, passed nearly two years ago, doing what it set out to do? In this episode, Trevor Houser of the Rhodium Group compares the predictions of pre-IRA energy-sector models to the real-world data on clean-energy investment since its passage.Open configuration options

Image

Help Center

How Do I Get Involved?

The data and insights in this Hub gets better with more participation, and if you are here, the chances are pretty good that you’ve got something to contribute. There are a variety of ways you can get involved, from contributing ad data, to sharing reports you are creating with your own polling or focus groups. Let’s have a conversation about how you can contribute to our effort. Reach us by email here, and we’ll get the ball rolling.

Data FAQs

Wow, that’s a lot of data. You are probably curious about how we’re handling it and how we’re keeping it secure. We’ve got a document for you! Check out our data FAQ here.

Name Generator

We have created this name generator that can be used by those running ads to create file names for the ad data they are sending to us; doing so will provide consistent naming conventions that will convey to us useful information on the digital ads, enhancing in turn the insights we can offer back to you all. 

• Why? Following discussions with others working on the IRA, there has been a demand for more in-depth analysis of ad performance based on multiple factors (like audience and message type). We’ve pinpointed the priority factors to measure, and this name generator will enable us to analyze them across all ads.
• Who is this relevant for? So we can collect the most extensive, useful and accurate insights, we would love for any organizations running ads (with any size budgets) to use the generator to name the files they send us. 
• How to get involved? Simply duplicate this doc and follow the instructions in the guide and start generating your file names! If you’ve got questions, you can reach us here.

We want to hear from you!

Have any questions or comments about the data? Have datapoints you think we should consider? Contact us here and let us know.

Quarterly Deep Dives

Session One (April 3)

Video Recording

Slide Presentation

A special thank you

We would like to extend a special thank you to 89up. 89up is an impact agency, based in the UK, who will be our partners as we build and make the Hub a more robust and useful tool for everyone who touches IRA work. They’re dealing with the mountain of backend data that we’ll be sorting through, and providing critical thought partnership in our analysis. Check them out at the link.