The distracter test
I hear the word engagement a lot. In the world of e-commerce, the key performance indicators are pretty clear: sell more. Return on investment is measurable; what is the overall spend against the income. OK, it is quite a bit more complicated than this but the overall concept is pretty clear! Now take a look at websites that are not selling anything. How do we know if they are successful? Even more tricky than this, how about selling to a client who is not the end user. We have to convince the client that the end user is engaged with the product. We have to prove that our product is better than the next. Analytics can tell us where people are clicking but are they taking it all in? My role is to improve the product at design stage so that when released it is the best product out there. With a website, this is about how the user interacts. We have to improve the end user’s engagement and there are tools to do this.
User testing is one of the core components of user centred design and indeed the whole UX (user experience) process. It really should not be called user testing as you are not testing the user- the user is testing the product for you. A really simple explanation of the process is this…
– Find someone who represents one of your users
– Get them in a room with your product
– Have tasks for them to do using your product
– Record how they use it noting any pain points for you to improve on
This process highlights issues that may have been overlooked and it is very powerful. When you are so close to a project it is easy to miss something that is glaringly obvious. The only downside to user testing is that it is staged- an unfamiliar room with no distractions. In real life the opposition to engagement may be many distractions!
At this point you may be wondering why there is a big picture of big bird and friends at Sesame street… there is a reason.
The creation and maintenance of Sesame street was very heavily user tested, even for today’s standards. The challenge was to engage children and teach them in a way that hadn’t really existed before. It is common knowledge that good teaching is carried out by understanding the person whom you are teaching, responding to the things that they like and dislike, taking the stimulus from them and adapting. A TV programme cannot do any of this, it talks at the person and cannot adapt. The people behind Sesame Street wanted to challenge this and establish some rules to maximise the level of engagement.
They came up with a great way of user testing that measures engagement in a way that is bringing in distractions that may be present in real life. They would play the show at the same time as showing images on another screen, alternating every seven and a half seconds. They would leave children in a room and watch them watch the TV, calling the images on the other screen “the distracter”. If a child was engaged, then the distracter may as well have not been there as they were glued to the TV show. When not engaged, the change of image caused the child to look away from the show and at the images. By mapping this they could really see areas of the show that engage and other areas where the children switch off. If the child was left in the room without the distracter then it would be harder to see the true stickiness of the product as they may well have looked at the TV without really being engaged at all.
I am not suggesting that a distracter like this should be used in our user testing of websites. What I am saying is that considering everyday distractions could add another dimension to user testing. Current user testing aims to minimise distractions so that you can understand the response to a product. Maybe increasing distractions could highlight some issues more relevant to real world use. I see the potential of a room filled with different media displaying the same information and seeing what people engage with. Different websites on a few screens could add a real time element to A B testing. Distractions could be placed in the room in the form of newspapers, food, games (things not even related to the product being tested) and users could be left without instruction.
There are many possibilities…
Book: The Tipping Point, Malcolm Gladwell, 2000