2 weeks ago I joined the Optimizely AB-testing conference in San Francisco and hearing the passion, the quality of analysis and most important the impact of the results it strengthened the idea that website optimization and testing should be at the core of any company.
I have often seen that website testing and optimizing is done by a small part of the web development team by a few junior guys that want to move up or out as soon as possible. Next to that within the company these guys are seen as the “button color department”.
Which has a disastrous effect on optimizing and testing impact:
First of all the quality of hypotheses that are being tested: The quality of tests is influenced by anyone in the organization. It becomes a modern day “idea box” (one of those overhyped 80’s initiatives). Anybody can submit a test and more important, can get it tested, so the queue for new tests is typically a few months. The guys are trying hard to make everyone happy and working to get rid of the queue, where they already have given up on.
Which has a disastrous effect on doing impactful tests.
For example: Junior analyst Benny who is still ignorant to office politics suggests to optimize the check-out process as it doesn’t work on a mobile phone and you have to fill 30 fields in 4 steps. Boss tells him that that system is far too complex too change: “It would take a consulting firm at least 1 year to do it.” Boss suggests to do a button color test, as he read on a blog that it improved web performance for company X with 20% and it’s far easier to do. Benny agrees.
I think this is a typical situation where the initial direction is great. Though “forces” keep the testing team from doing the tests that really matter. A testing team should have te power and state of mind to pick up those difficult challenges and therefore the top of organizations need to be involved.
In real successful organizations testing is valued and the team would rather be called “ the core performance team” as is it does more right to the impact it can and will have on the business.
So far for my negative raving on the current testing status quo.
My message is actually a positive one. Effectively this is a great opportunity for you to blow your competition away because there are not too many companies out there who are great at this. If you want to be a successful digital company: Invest in a strong testing team, work with a big part of the team on high impact tests and get the top of the organization on board to make it all happen.