Data‑driven vs data informed

On Monday 3rd March 2021, the Austrian government suspended a batch of the Oxford/Astrazeneca Covid-19 vaccine following reports of blood clots in some recipients. Days later, Denmark became the first country to temporarily suspend the distribution of the jab entirely, quoting the same reasons.

By Tuesday 17th March, the EU’s 4 largest populations (Germany, France, Italy, Spain) alongside Austria, Bulgaria, The Democratic Republic of the Congo (DRC), Cyprus, Iceland, Indonesia, The Republic of Ireland (ROI), Latvia, Luxembourg, the Netherlands, Norway, Portugal, Romania, Slovenia, Sweden and Thailand had all followed suit.

In contrast, throughout the same two-week period leading scientists and advisory groups publicly assured politicians and citizens alike that there was no correlation between the reported blood clots and the Oxford/Astrazeneca jab.

As early as 11th March, the European Medicines Agency publicly stated that there was no need to suspend the vaccine’s worldwide rollout because the data didn’t point to any increased risk. Nonetheless, less than a week later the EU’s biggest players had all followed one another in temporarily suspending the rollout.

(*Since then, and whilst writing this article, the vaccine has been suspended again - it should be made clear, however, that we are considering only what was known during the period March 3rd and March 17th 2021.)

Data plays a key role in this story. On the one hand leading scientists and researchers used it as evidence of the vaccine’s safety. Whilst simultaneously, government after government declared they had taken the data into consideration and subsequently decided to temporarily suspend the rollout.

To be clear, we’re not interested in whether the decision to suspend the rollout was right or wrong. We’re only interested in how the decision was reached. How, with data telling them it was safe, did so many governments reach the decision to temporarily suspend the Oxford/Astrazeneca vaccine?

And what does any of this have to do with e-commerce?

Over recent years data has grown in significance. And as it continues to do so, understanding what to do with it is increasingly important. It doesn’t just affect world health research and international politicians - everything, from business to sports to YouTube algorithms, is now optimised by data in some way.

E-commerce is no different. It’s fertile ground for data analysis, and those that master it have great potential for optimising conversion and growth and leveraging a serious advantage over their competition.

The vaccine rollout example shows that data is used in different ways. The data itself only ever delivers facts, unequivocal and without vested interest. But it is up to those involved to decide what course of action to take. Do the facts alone drive the decision, or must other factors also come into play?

This is the difference between being data-driven and data-informed. Briefly, the former goes only where the data leads. No questions asked. The latter takes data into consideration, whilst still allowing room for opinion, instinct, gut feeling, human emotion, and so on.

If you can understand the difference between data-driven and data-informed, and understand that there’s an appropriate time to be either one or the other, you can drastically increase your business’ bottom line.

A data-driven decision is a scientific approach to a specific problem.

“Scientific?”, you ask. Forget the white lab coats and bubbling test tubes. Scientific simply means something based on the methods and principles of science. Approaching a problem scientifically means doing so in a structured, controlled and formal way that others could repeat and replicate if given precisely the same conditions.

In any scientific investigation there’s only ever a single variable that changes. This variable is decided upon in advance and all others are deliberately kept at a constant so as not to interfere with the results.

Decisions in government and international politics very rarely, if ever, have only a single variable to consider, making it very difficult to take data-driven decisions. And you may think e-commerce is similar, with all number of opinions and outcomes to consider when making any decision. However there are far more instances than you might realise where it's very easy, and even necessary, to be data-driven.

Take an example: does the position of the ‘add to cart’ button affect conversion? It's not difficult to imagine a scenario where a button's position is the subject of intense scrutiny and debate. The designer wants it here, the manager wants it there. You get the picture.

But if they were to establish an A/B test, keeping every feature of their webpage exactly the same except the position of the button over a set period of time, one of the two positions will give statistically better conversion rates. This is raw data, collected in a scientific manner.

It's the next step that’s crucial in defining a data-driven approach. If you’re data-driven, you follow the data. Leave everything else at the door - your ego, your opinion, your boss’ opinion, all of it - and simply go where the data leads. If done right, it’s actually very easy to do because by the time you have the data the decision will have already been made.

How so? Well in our example above, the team will agree before testing that they'll ultimately place the button in whichever position gives the best conversion results. Once the testing is over all they have to do is look at the data and go where it leads.

Sometimes it’s simply impossible to leave opinions at the door and in these situations you cannot be data-driven.

The governments that suspended the Oxford/Astrazeneca vaccine would argue they were in this position. They were informed by the data, but ultimately swayed by more than just the science. As reported in the New York Times, the director of Italy’s Medicine’s Agency, Nicola Magrini, described the move as a “political choice”. Especially once Germany - the EU’s largest population and richest nation - had suspended the jab, other governments simply felt that they had to do likewise to keep public opinion on side.

Had they not, the headlines would have arguably increased in volume. Some recipients of the vaccine had died from blood clots, and whether related to the vaccine or not, people want reassurances that their government is as cautious as the next. Right now, the public is nervous and in politics their opinion ultimately counts for more than science.

Whilst far removed from the seriousness of life and death, there are also times in e-commerce when opinion and data are inseparable and you must ultimately go with the former. The trick is to know when.

Next, we’ll separate e-commerce into two key stages and explain why one requires opinion and the other solely data.

The first is the exploratory stage. This is a point, usually very early on, when you’re exploring a number of different avenues and possibilities simultaneously. It’s a creative moment in time when nothing is set and you’re building ideas.

When sketching initial website designs, for example, there are endless possibilities and combinations to process. Using our previous example, you aren’t only deciding where your ‘add to cart’ button should go, you’re simultaneously deciding its size, shape, colour, background colour, and so on. Data is of limited help here. You may have some data on what the best practices are, or what your competitors are doing, but ultimately it’s your own opinion and preferences that make the decision.

We’ve heard it said that those who are data-driven “lose sight of the bigger picture”. Or that the data-driven approach is ineffective because it tries to replace the irreplaceable human instinct. “If people designed websites solely through data,” they protest, “then every website would look identical.”

These are criticisms of attempts to force data into the exploratory stage. Because here the human instinct certainly is irreplaceable. Its individuality is crucial to designing beautiful websites - crucial to designing anything beautiful, for that matter. Using data in design will have you lost in the details because there’s simply too much information to process, too many options to test. To date nothing creates as wonderfully as the human mind, because to date nothing can process so much at the same time. (Whether AI one day catches up is another matter.)

However, once you reach a certain point, the human mind can be a massive hindrance. This is the second stage, the optimisation stage.

You have a website up and running and now you want to make specific tweaks. This stage is about high-level performance boosting. Fine-tuning your e-commerce machine to gain a competitive advantage.

With a foundation already set you can identify individual areas to modify. You already know how to do it - by keeping all factors deliberately the same except one, you generate data that tells you the answer. The key is to know in advance what you’re testing, how you’ll test it, and what the measure of success is. Follow this step-by-step guide and you can’t go wrong:

1) We believe that… Start with a simple premise, for example, “We believe that position A for the ‘add to cart’ button is best for conversion, and that the button should be placed here”. Notice that included in this premise is the consequence of success. If the data proves this premise correct, your decision is already made for you.

2) And to prove it we will… This is the test you will carry out. “And to prove it we will conduct a test for a set period of time between position A and position B”.

3) And measure… You must only measure 1 variable per test. “And measure the page’s conversion rate.”

4) We are right if… This should link back to the first premise so that even before beginning the test you know what the measure of success is. “We are right if the page delivers better conversion rates when the button is in position A.” We already know from the first premise that if this takes place the button moves to A. No questions asked.

Once you have the results, the most important thing is to follow the data. Because in controlled situations such as this the data simply doesn’t lie. There’s no room for opinion. It doesn’t matter what you might think will give a better result because you have scientific proof of what actually gives better results.

This is where the human mind can hinder proceedings. We’ve seen it happen before - tests are delivered but the results don’t match with a boss’ opinion (read: ego). They think they know best and ignore the data, opting for what they prefer. The result? No competitive advantage whatsoever.

At the optimisation stage everyone involved must be prepared to let the data take the lead. If this cannot be agreed upon, don’t waste your time conducting the tests in the first place.

To recap: there’s room for both data-informed and data-driven approaches within e-commerce. Sometimes it’s better to be one, and sometimes the other. Don’t let anyone tell you that you should never be data-driven, or that you should only ever be data-driven, because it isn’t true. The reality is that there’s a time and a place for data to lead your decision making. If you find it you'll soon reap the rewards.

The fact is that right now most people are data-informed but few are data-driven. People understand that data is a tool they should be using, but in truth they don’t know how. They’ll most likely have data in the room when making decisions, but without ever really referring to it in the proper way and at the proper time.

This is a great opportunity for brands that want to get ahead. Start to use data effectively, know when to leave politics and egos at the door to simply focus on what the metrics tell them, and they’ll quickly see themselves pulling ahead of their competition.

Here at Ask Phill we have a dedicated data team helping our clients make the decisions that get them ahead of their competitors. We use algorithms to track every module on every page of their website, and identify those that positively affect conversion.

For instance, we have previously identified that customers in specific international markets respond negatively to reading blog posts (the irony is not lost on us). Customers from these regions that click on and view our client’s blog are less likely to then go on and make a purchase.

Our client reads the data on their custom data-dashboard and subsequently makes the positive, data-driven decision to remove all blog posts from their site for customers in a specific market. This decision would likely never have been made if it weren’t for our data analysis, but it has already been proven to have a positive impact on conversion rates.

We've also identified the actions that high-spending customers are more likely to make. Our clients can subsequently customise their store to encourage more customers to behave in the same way. Again, these techniques have been proven to boost sales, make more money for our clients and get them ahead of the competition.

If you’re ready to make data-driven decisions and bring your e-commerce brand to the next level, get in touch with us for a consultation.