Back to Insights

A Reckless Approach to Optimisation

 

‘We do CRO!’ said every agency in the UK…

Don’t get me wrong, companies and agencies focusing more on optimisation is a good thing. Done well, it leads to a better experience for all; we all save time, we’re all happy, we make money, the sun comes out and all is good in the world.

But there was a key phrase in the last sentence, ‘Done well’.

Optimisation is one of the most power tools a business can have, but in the wrong hands it can be a very dangerous thing and sadly, the wrong hands aren’t that hard to find.

But why is this the case? Can you tell a good CRO agency from the bad? What should we be looking for and what questions should we be asking?


Well in the spirit of optimisation, let’s do some analysis.


Over the past few years there has been an astronomic rise of “Optimisation Tools” from click map and scroll map software to A-B testing; Hotjar, VWO and Optimal-sort to name but a few. These tools are great, and they have made my life a lot easier allowing me to do new types of analysis I could only dream of until recently. But that is all they are, tools and just because I own a drill does not mean I am good at DIY (this is also painfully accurate).

Unfortunately, these tools make it easy to look the part. I mean, look at this:

 

Reckless Clickmap

 

Looks cool right? This person clearly must know what they’re doing… maybe.

 

 

So what is optimisation?

Optimisation is all about encouraging a shift in behaviour.

There’s a misconception that it’s all to do with optimising conversion rate, commonly the checkout. Yes, this may be important but true optimisation is about focusing on your objective and discovering what are the most important actions and behaviours we need to influence to change that.

Take the following example:

 

conversion funnel

 

In this example we can see over 90% of users are not converting from the homepage to a product page whereas conversion through the basket is performing well. So this instance we’d be better focused on trying to identify what friction is causing customers to leave on the homepage, resolve this and try to encourage more users down the funnel to a checkout which already performs well.

But we need to think bigger.

Optimisation isn’t just about a website we need to consider the whole user journey; pre-sale, post-sale offline and on.

Do you know what one of the biggest influences on hotels room bookings is?

Travel infrastructure

If a plane or train costs suddenly rise or your hotel is difficult to get to then it doesn’t matter what colour your CTA button is, people will go elsewhere. Sometimes there is nothing that can be done about these external factors but sometimes there is; could you offer a shuttle service? Adjust your prices in-line with travel costs? Offer a price guarantee if the customer needs to come on a different date? Only by having this mindset allows you to adapt to these changes and this is what real optimisation is about.

 

 

What you should look for in a CRO agency?

When looking for an CRO agency it can be a bit of a minefield.  We’ve already established it’s easy to look the part so how can you tell an imitator from the real thing and not just find a good agency but a great one.

Well here is our guide on some the things to look out for and questions to ask…

 

1. Do they have a process?

You need a process. Without one you’re just fumbling in the dark, your optimisation plan may succeed, or it may fail, but you won’t know why. Was it that you didn’t do market research this time? Did you forget to do proper QA and there was a bug in your experiment? Who knows.

A structured process allows for accountability, measurement and refinement so it’s critical we establish what this is.

Any agency who takes CRO seriously should have a process defined, all will be different but share similar themes, this is ours:

cro process

 

  • Configure
    Here we look at your data landscape. Are you tracking everything you can be and of what you are tracking is tracking correctly? Bad data leads to bad decisions, so we need to ensure what your data landscape is configured correctly.

 

  • Heuristic Review
    We then review your site from a CRO, UI and UX perspective to identify issues and opportunities. Each issue or opportunity is driven by evidence and experience. If you ask us why and we can’t answer we haven’t done our job correctly.

 

  • Analyse and Prioritise
    Each recommendation we put forward we will then prioritise in order of importance. To do this we use and adaption of the Nielsen severity scale. This measures each issue against task abandonment/impact, the percentage of users which are likely to face this issue, business goals and time.nielson severity scaleOnce ranked we then place the task into one of 3 buckets:

    • Test it – we have a sound hypothesis but are unsure of the outcome and the change is of high risk
    • Do it – outcome is known and low risk e.g. bug on the website
    • Research it – not enough evidence to justify change, more research is needed.

 

  • QA
    Each recommendation we test goes through a vigorous QA process. Tests such as an A-B test can take a number of weeks to bare results, a single bug can invalidate the whole test so it is pivotal that each test we run is checked and checked again (and then checked one more time).

 

  • Experiment
    The experiment is configured and run. The type of experiment differs depending on the issue or problem we’re trying to solve, for example if we’re trying to solve a navigation issue a ‘Tree Test’ may be more appropriate than an A-B test.

 

  • Learn
    Did it work? If so why and if not what did we learn from it. Not all experiments work but that doesn’t mean they are a failure, learning what not to do can be far more important.

 

  • Rinse, wash, refine and repeat.

 

2. Do they listen?

Have you ever been in one of those meetings where you realise you haven’t spoke for an hour? Well this may be a warning sign.

CRO can be a world of smoke and mirrors and it can be easy to spout jargon and make it sound impressive…

“We use a Bayesian approach to data modelling, the frequentest approach does not yield results that we have confidence in for our multi-variate bandit testing’

 

confused math

 

The best agencies work in collaboration, you after all are the expert of your brand. The agency should pry, ask questions but more than anything should absorbing everything you have to say. Getting to full picture is the only way to create a meaningful optimisation plan which is right for your objectives.

 

 

3. Do they teach?

Optimisation is not a secret, or at least it shouldn’t be.

Some agencies have the tendency to keep any optimisation methods under lock and key in fear that the client will run off and try and do it themselves. Well we say if a client has a passion to learn then the agency should not only embrace that, they should be encouraging it.

At Reckless we are a strong advocate of knowledge sharing, it allows us to collaborate and create a strong optimisation team with our clients. For example, we regularly teach our clients Google Analytics and Measurement. This is a huge help for us as it not only adds an extra pair of eyes to help find opportunities but helps create efficiencies in the account allowing us to move on to new exciting areas of optimisation.

Optimisation is for everyone and we should all be sharing knowledge more.

 

4. Do they know their numbers?

This may sound like a basic one but it’s surprising how often you come across a distinct lack of knowledge when it comes to the numbers behind an optimisation decision.

One of my favourite conversations I’ve had over my career involved the phase:

‘Don’t worry about the ROI the product speaks for itself’

 

mib

 

I’m still envious of that person to this day, unfortunately not all of us are that lucky and the numbers do matter.

Numbers and research alleviate risk and help prioritise what optimisation decisions are going to have the biggest impact. Without this we’re just guessing, it may be an educated guess but a guess nonetheless and if you run a multi-million pound business I wouldn’t be making decisions without it being backed by some heavy-weight research.

I’m not saying you have to be a statistician (though it helps), but you have to have an understanding of the qualitative and quantitative data behind a decision.

 

5. Do they have experience?

You wouldn’t trust a pilot who’s only learnt to fly through YouTube tutorials…

So why would you trust an agency who’s never run an optimisation program?

Optimisation is can be hard, really hard. It’s full of pitfalls, the known and the unknown and you need to someone who’s waded through these waters before. Especially if you have an agency who’s especially confident in what they say, this could be a warning sign and you could be experiencing the Dunning-Kruger Effect…

Dunning-Kruger Effect

So do your research, ask for case studies and ask for references because without an experienced guide you can waste a lot of time and money very quickly.

 

6. Is it in the culture of the wider team?

Do they live and breathe optimisation? Optimisation isn’t just restricted to one team it should span across every team member.

Is the team PPC rotating ad types? Are SEO testing the best keywords for commercial intent? Does the project management team try a new method of delivery to help efficiencies?

As mentioned, optimisation is bigger than just an A-B test sometimes it means focusing on a different channel all together. If your site drives very little traffic then the best way to optimise this would be to do some channel optimisation and focus of PPC, SEO or even Social. If that’s the case you need a team who understands the value of CRO. This isn’t just to ensure that they’re testing their methods but it means they should have the true business objectives in mind. For example if you have an event is a week and attendance numbers are low the team needs to be aligned that PPC and Social are a better solution to drive attendee awareness than SEO. There’s the tendency to sometimes throw the whole suite of channels at a solution but having an underlying culture of optimisation stops teams prioritising their own work over what is best for the client.

 

So what have we learnt…

  • The rise of optimisation is a great thing, but with anything that grows in popularity everyone will try and claim their piece. So it can be hard to identify the masters from the mimics.
  • Keep an eye out for the following when looking for a CRO agency:
    • Don’t be swayed by complex jargon and interesting visuals
    • Look for a Process
    • Do they listen
    • Will they teach
    • Ensure they know their numbers
    • Check for Experience
    • Is Optimisation embedded in their culture

And finally, always question, don’t be afraid to challenge and keep on optimising.


Is there anything you think is important when looking for a CRO agency? What do you think about the current state of Optimisation? Anything we missed?

Have your say and let us know by emailing Matt: matt.tilling@reckless.agency

You can also connect with Matt on Linkedin

Matt Tilling | Optimisation Lead