Recently while looking at some AdWords conversion data a friend and I were discussing how to present acquisition costs to a client. We knew that PPC was contributing more than just what was showing in the basic Google Analytics reports, but weren’t sure what the best way to present the value assisted conversions to the client. We had seen what was in the Multi-Channel Funnels area in GA and knew that plenty of assisting going on, but struggled with how to assign a value to it.

The question boiled down to: “how much is an assisted conversion worth?”

Consider the following example:
Your site sold a $50 shoe.  The user had three interactions with your site before they purchased, represented by this conversion path:

How do you split up that $50 credit?
The default Google Analytics report would give all $50 to Organic.
Custom tracking on your site (or many software packages other than GA) based on referral would give it all to Direct.
A Google Adwords conversion tracking tag would give it all to Paid Search.

Common sense seems to say that they are all wrong and that the credit should be split. After all any source of traffic has some cost associated with it, even if you aren’t importing it into GA all traffic costs you something (though of course with Organic that can be particular hard to place a price on).

Unfortunately there’s a lot of moving parts that make this splitting of credit very hard question to answer. This is one of those times that having the additional data seems like a great thing, but the analysis of it is hard enough that it just doesn’t seem worth it and we stick with our old ways.

So why is splitting the credit hard? First & foremost is that any value you assign to an assist inherently has to take value from another step, but it’s not clear how much. You can’t credit that $50 to Organic but then also credit part of it to Paid; you still only made $50 on the sale. Maybe that seems obvious, but it is less obvious when the metric you’re looking at is cost-per-acquisition, not total sales value. CPA is really the important metric here for most cases, and it not entirely intuitive that adding tracking of assists would then potentially raise the CPA of some channels and lower others.

To understand all the parts I think it’s helpful to start back at how we got to state of tracking that we are at now.

The Revolution That Hasn’t Been

Long ago in the dark ages of web analytics, all tracking was based on the “last-touch”; the last click that came through to our site before conversion.

A user might have first been referred via organic search last week and then come back via paid search immediately before conversion, and the paid ad would take all the credit.

Then came multi-touch tracking, and we finally could start seeing all the touch points a customer had with our site before they converted.

The Google Analytics version of this is mostly under Multi-Channel Funnels, launched more than three years ago with a comprehensive look at all the different interactions a user might have with your site before they convert. So, problem solved, right? Out of the dark ages into a “real” representation of these interactions over time?

Well maybe, but it sure doesn’t seem like it to me. Most reports still default to last-touch and most reporting packages that I’ve seen still assume last-touch for most everything.

If you’re already using multi-touch attribution, congrats — nice work! But how come some many of rest of us are so far behind the curve on this if the data has been available for so long?

I can think of two main reasons why this has mostly not caught on:
#1 – Defaults.
Google Analytics relegates MCF to one area and defaults everything else to last-touch (well, technically last non-direct touch). As long as that’s the case; that’s how most people will pull reports. I don’t blame Google here though, it’s the default because of:

#2 – Lack of definitive model.
What attribution model should we use??? This is the core question.  There’s a lot of options, even just starting with the defaults in GA. Unfortunately the defaults are pretty underwhelming in sophistication and “building your own” is a daunting option for most people that can end up being even more wrong.

This is really just one reason — that we don’t know what attribution model to use. If there was one model that was generally accepted then maybe it could be considered for more default reporting. Even if there were a few more sophisticated pre-built models available easily in GA that users could pick from it might get more people using & trusting the insight that this data can provide.

Why isn’t there a “sensible default” model for everyone’s basic usage?

Pass the Click-ball

To understand that I want to look more at what “assisted” conversion means, oddly using some sports metaphors (I’m not that much of a sports fan).

Google’s video introduction of Multi-Channel Funnels (MCF) compares an assisted conversion to basketball assist. A player passes the ball to another player who scores; the first player gets an assist. It’s a fine metaphor for what Google means by an assisted conversion. It’s kind of thin though, so let’s consider where it falls down.

1. In basketball, each field goal (i.e. basket) can only have one assist. In Google MCF tracking you can have a any number of assist touches before the goal completion. The further back you draw your conversion window (default is 30 days but can be extended back to 90), the more assists you may see.

This of course makes sense, since the GA tracking cookies last a little longer than a 24 second shot clock (thank goodness). It makes giving a value to an assist much harder though, because in GA it could be the 3rd touch out of 12 total, or the 1st out of 2. I might seem obvious that the 1st out of 2 is worth far more towards a conversion than the 3rd of 12, but it’s hard to quantify!

2. In basketball, assists are totaled separately from points. By that I mean there’s no “offensive efficiency” metric that shows up in a game recap to measure a player by. Say you want to know the value of a particular conversion channel in basketball (some people might call that a “player”) — you can look at their total points scored, but that’s nowhere near to the full value of that player.

Think about this in terms of web analytics. If a score is our analytics goal (e.g. signup or checkout), then the player who made that score is the last-touch channel (e.g. PPC or email, etc.). Applying last-touch attribution alone is the same as just looking at the “points” total to evaluate the worth of a player, and completely ignoring the assists number. Of course teams, agents & statisticians that evaluate players don’t just look at points totals, and there are several different formulas for overall player efficiency.

John Stockton is concerned about your web analytics practices.

To continue the sports metaphor a little further, I think that hockey is a better analogy to start with for MCF. In NHL hockey, each goal can have 2 assists (not just 1 like in basketball), and one of the top-level stats  is “points”, which is goals + assists. These two differences are closer to the direction of showing the full offensive impact of a player, and hopefully puts us in a mindset of some more comparisons.

Models, Models, Models

Ok, so let’s finally get down to some models. All of this is under the wonderful Model Comparison Tool under Conversions > Attribution in GA. The URL of this report calls it the “ROI calculator”, and in some ways I think that’s actually a more descriptive name of what it does. We should think of the goal of calculating ROI, not as A/B testing a bunch of different models.

Google Analytics Default Models

GA has a number of default models for comparison, as you can see on the left here. I won’t go into great detail about all of them, if you want more explanation I’ll send you to Avinash Kaushik’s really excellent article on the subject (an inspiration for a lot of what I’m covering in this post). To keep it short, most of the defaults are not generally that useful, here’s why:

1. Last Interaction, Last Non-Direct, Last AdWords — meh, that’s the last-touch tracking we want to get away from.

2. First Interaction, Linear — well, they aren’t last-touch, but they sure aren’t going to be representative for most people, so definitely pass (no pun intended).

3. Time Decay — closer, and potentially pretty useful. In this model the further in time you get away from the actual conversion the less value the touch has. As Avinash says, this makes quite a bit of sense intuitively. What I don’t like about it though is that the first touch can easily fade to nothing. Just because a user first found your site 14 days ago instead of 7 shouldn’t half the value of that touch in my opinion, and I don’t think that time is a good a base measure as number of times a user has interacted with your site.

4. Position Based — this is the one we want, but the defaults rules are not so great. Position-based means adjusting the amount of credit based on the order of interactions with your site; e.g.:
40% credit for the first touch, 20% for the 2nd touch, 40% for the last touch (that’s the default, the 20% is split amongst as many “middle” touches as there are).

So, here’s where it starts getting tricky. We have a general vicinity we want our model to be in, but we need to adjust it to add some more intelligence. Google has taken this approach of giving simple starting part models because anything beyond that starts applying a lot of assumptions, and Google would like us to make these assumptions and the risk in being wrong that goes along with them. They are giving us a starting place & the powerful tools to make our own decisions, and as we all know that can be a dangerous place to be.

This difficultly is all because there’s no set formula we can apply to figure out the real value. Adjustment to these models are essentially based on opinion, not empirical testing. Certainly empirical testing to determine the marginal value of each adjustment is possible, but that is a massive undertaking I know I won’t be trying anytime soon at least. Testing & modeling all of this thoroughly sounds great, but is beyond the reach of most small or mid-size sites from a resources perspective.

So we’re stuck with opinion. Hopefully well-informed opinion rooted in real experience, but opinion and “feel” nonetheless. For example, we may think that the 2nd click of 3 is worth 20% of the overall goal credit, but unless we’ve actually tested that it’s really just a guess.

Plus your site’s experience isn’t going to be the same as my site’s. That’s why even if someone did do a rigorous experiment on these incremental contributions to the sale and then publicly shared that info it isn’t something that we can all use with confidence.

With these caveats Avinash recommended the following model:

Market Motive Mindblowing Model

I think this is a solid model. It is, of course, based on Avinash’s experience & opinions. And not everyone will agree with all of it. For example, I don’t agree with giving quite such a small amount to first interaction, and I don’t see the point in not doing 90 days if it is available to us.

I got curious what other people’s opinions were, so I looked through all 107 items in the Google Analytics Solutions Gallery that matched for “Attribution Model”.

You can be glad I did this and you don’t have to, because there is a lot of junk in there. Spam, items that aren’t actually models, tests, etc…. before you even get to the models that are just frankly badly done. I went through these, dumped the junk, and found the following:

1. The plurality of the non-junk is of the baseline type “Position”, which is what the sample above is. This is good, since that validates our supposition that it is the most “sensible”.

2. After all this filtering, there is unfortunately only a small handful of “real” models. I suppose this shows that there really aren’t a lot of people doing this model comparison, and/or they are keeping their models to themselves.

3. There is some commonality among the models, not enough to be rigorous, but maybe we can try to make a model based a mashup of the models in the gallery?

General Agreements:
-Baseline model: Position.
-Page adjustment: page depth.
-Lookback: > 30 day default.

Most Variation:
Custom rules (duh)
First/Middle/Last Interaction shares breakdown.

As I said, unfortunately there’s not enough reasonable models to pull averages this with any sort of statistical significance (yet)… Maybe someday, but for now I’m just kind of winging it based on the ~15 that I imported.

Combining these agreements along with a weighted average for what they don’t agree on (based on the number of people that have imported the model for usage) and some of my own opinions I suggest the following:

Quantable Model

Not all that much different than Avinash’s model really. Let’s go over the settings:
– Position-Based, split 30/35/35. This is just the (rounded) weighted average of the splits I found. It gives more credit to the first interaction, taking from the middle mostly and a little from the last.

-Lookback window of 90 days. The average would have been lower, but I don’t really see a lot of benefit of not looking back as far as you can for credit, so that’s the max. Maybe if 180 days was an option I would not choose that since at some point a user may forget completely about a site and “refind” it totally, but 90 days still seems reasonable to me.

-Adjust credit based on engagement: page depth. This means the more pages a user had in that session, the more credit that session gets in the total.

-Custom credits: decrease impression & rich media interactions. This means we are discounting sessions where the user only saw ads and wasn’t actually on the site. We give a little more credit to rich media since that could have more substantive interaction. This is pretty close to what Avinash’s model does, he gives a 40% boost to clicks, I took away either 50% or 30% to things that weren’t clicks (or direct visits).

-Custom credits: decrease last position direct. This is a 50% penalty if the last touch was a direct visit. This was used in a couple examples (including one Google provided). To me this makes sense because “direct” is the one source of traffic we can’t really assign a cost to since we don’t actually know what it is, so let’s shift those credits to other steps. This is some of the same logic why GA defaults to last non-direct touch for the default reports.

Results?

After this what does this all look like? Here’s a quick example of just Paid Search:

Model Comparison Tool Output

Either model shows a significant increase in CPA for Paid Search, the Quantable model even more so. Of course this means that other channels had their CPAs increased, but overall this is hopefully much more representative and can help answer our initial question of how much value was derived from assisted conversions (almost 10% in my model).

How does this differ from Avinash’s model in output? In the couple of sites I tested it on it shows:
– More credit for Paid (as seen above).
– Less credit for Direct (since it explicitly depresses that).
– More credit for Organic.
– Less credit for Display (due to giving less credit for last touch, thus depressing retargeting).

Is this a good outcome? Hard to say.. for the examples I looked at I felt it was more representative, but that’s my bias. I don’t put this model up as a suggested replacement for Avinash’s or as our mythical “sensible default”, simply as another suggested starting point to hopefully get more people on the attribution modeling bandwagon.

My example model is available in the GA solutions gallery here, have fun with it and please share any feedback you have!

Photo by martinak15

Thanks to Mark Subel at Two Wheels Marketing for the idea for this post as well as discussion & help.

Share this post: on Twitter on Facebook on LinkedIn