Note: this series is about ideas. It is not a step-by-step tutorial on how to navigate the ever-changing PPC advertising interfaces. Screen shots are employed selectively to elucidate the subject matter, but figuring out the precise order of implementation steps…that’s up to you.
What more important topic is there to discuss under the heading of “The Science of PPC” than the design and conception of experiments?
In this case, I don’t use the term “experiments” loosely, in the sense of just tinkering with this and that.
A better way would be to split campaign traffic in real time into two (unbiased) streams – a control group and an experiment group. That’s exactly what Google Ads Campaign Drafts and Experiments can do.
It’s a fascinating feature of Google Ads. Or perhaps “architectural framework” is a better term than “feature.”
I’m not much for the Drafts part, so let’s focus on the Experiments.
Getting Started with Google Ads Campaign Drafts and Experiments
Telling people that a fancy architectural framework (like a laboratory with unlimited beakers, powerful data interpretation capacity, sound methodological underpinnings such as isolating the impact of one major change (or set of changes) from contaminating influences, intervening variables, spurious correlation, however you want to put it) exists, and “go nuts,” is a little like setting up an Open Forum Panel at a digital marketing conference in Sweden starting at 8:00 a.m. after the late-night party. “Really! The experts are here. You can ask them anything. Anything at all. Anyone? How about you sir in the very back next to the door?”
Some people leave.
After coffee and handfuls of candy, the day brightens further. People converse more and more.
Like the sleepy-eyed and buttoned-down Swedes at the awkward Q&A, it seems like unstructured “go aheads” need to be lubricated with a little prompting. And some people just like to take notes.
That’s why, next week, I’ll provide a few specific examples of fruitful experiments you may want to set up in well-established PPC campaigns.
For now, the Campaign Experiments methodology typically works as follows:
- Take a campaign that you have some serious questions about. How would it perform if we took a markedly different approach to “X”? X could mean a number of things: bidding, types of ads, adherence or disdain for grandiose theories about campaign structure or match types, etc. (hence my plan here, to provide you with several ideas for what X might actually be). Notice how instrumental this Experiments architecture could be in taking assertions, arguments, questions, subjective biases, “personal, quirky, heroic” methodologies that individuals proudly wear like tattoos in an attempt to get noticed (“best practices or admitting that there might be a few ways to get to the same result are for sheep!”), etc., into the realm of Proof with a capital P. Sweeping assertions give way to empirical research with a sound, verifiable methodology. In a climate of full transparency.
- Start by stating for all involved your hypothesis and the type of theory you intend to test. For example, “I believe that replacing every single traditional broad match keyword in this large campaign with broad match modifier will lead to higher conversion rates, better CTR’s, slightly lower volume, and overall better ROI. The end result will be more conversions for fewer dollars, with the added benefit of reducing the time burden associated with regularly adding negative keywords.”
- Begin using the Campaign Drafts & Experiments architecture by initiating a campaign draft, following the steps such as establishing the duration (depending on volume, 4-8 weeks is a good time frame – any longer can be a burden, any shorter may leave you wondering if you should have let statistical confidence grow stronger), specifying the % of split traffic that will go to the experiment vs. the control (50-50 is most intuitive), and so on.
- Make all the detailed changes to the campaign you have to make to activate a test of your theory from point 2 above, doing so of course in the Drafts section of the architecture for this campaign.
- Save the draft and initiate it as an Experiment (it’s a little tricky for newcomers and even experienced people, and sometimes you want to ask yourself “where am I?,” but you get used to it).
- You’re off to the races! You’ll need to wait a day or two to see the initial horse race results in the Experiments dashboard. Remember, Experiments reside at the Campaign level. Govern your thinking and scope appropriately. (Tiny changes to a couple of ad groups, and claims about whether or not to add a certain type of campaign at all, for example, wouldn’t make good candidates for a Campaign Experiment.)
- Huge caveat here! (Huge, which is why I ended the last sentence with an exclamation point instead of a colon!!!) Potential ad impressions for your business aren’t exclusive to campaigns. In other words, the keywords and other targeting methods that trigger ad impressions can overlap across campaigns, and queries can be associated with these keywords and targeting methods in different parts of your account depending on whichever one of the competing keywords or methods in your account garners the highest Ad Rank on a given user query. The same thing happens at a micro level if you have a variety of keywords and match types with potentially overlapping audiences coexisting in the same ad group. Bid one keyword up, another’s potential audience may decrease. The shorthand term for this is cannibalization. You’d better hope your experiment design doesn’t just lead to one or the other half of the experiment showing a marked increase in how many queries it pulls away from other existing parts of your account – especially if those parts are low-hanging fruit or parts that, for one reason or another, you’d prefer to bid less on. (Cannibalization and cherry-picking sound like a potential feast for a starving person, but they won’t help your financial performance.) Nope, the methodology here isn’t airtight. It can be very good at getting us further down the road of advanced testing, but you have to spot-check a few things to ensure it isn’t a form of cheating.
- Don’t touch anything on either side of the Experiment for the duration of the Experiment. Google is working on a feature so that changes to one part will be mirrored in both parts so it won’t matter if you wandered into the lab like a bull in a beaker shop, but hey, let’s keep it real.
- Modify columns in the Experiments dashboard if you’d like to see a different set of KPI’s in the reporting.
- Enjoy the fantastic dashboard feature that reports on the statistical confidence level that any particular KPI associated with the Experiment group diverged (or “beat,” or “lost to”) the same in your Control group. The reporting is user-friendly, even for a relative novice.
- Decide whether your theory was proven or disproven.
- Either accept the experimental version of your campaign as the new, reigning version of your campaign, or revert fully to the original, having conceded that the null hypothesis – the theory didn’t pan out as expected or hoped – must be respected.
- Resume normal optimization efforts in this campaign.
This experiment did not pan out. A match type strategy produced a marked dropoff in conversions, so we reverted to the original campaign. Hover over the blue asterisk and Google Ads provides even more detail about the statistical significance associated with any given performance metric in the Experiment group.
Five ideas for experiments: Coming next week
By doing so using the Campaign Experiments framework, you can be open to productive change, rather than stubbornly sticking to the status quo or capriciously adopting trendy strategies second-hand. (Certain crazy theories in any industry have a tendency to go viral. And, as I suggested above, some individual campaign managers love to adopt novel strategies so they can get credit for originality. Performance, though, is all that should matter. And you should be able to isolate exactly what significant change should be credited with improved performance in real time, as opposed to “before-and-after” studies that may carry on as many moving parts are, well, moving.)
Here’s to science. Tailored, first-hand experimental science.
I’ll dig in with comments on some specific examples of Campaign Experiments next week.Read Part 9: 5 Suggested PPC Campaign Experiments