I knew diary study was going to come up eventually. It’s kind of been the purple elephant I’ve ignored when talking about the various UX testing methodologies that are popular and unique.
The problem is, I really don’t like the diary study model, for a multitude of reasons, and I think by the end of this, you may agree with me, but then, if you’re not bothered by the uphill climb of using this, then you may actually be enthralled with it. We’ll see which is which.
What it Is:
Well, it’s exactly what it sounds like it’d be. Test subjects will keep a diary or journal of their use of a program. They will discuss what they did each time, how their experience went, how they felt about it, and any problems or ways they see it could improve.
Then, these journals are studied by the designers, and this gives them a fix on a number of aspects of usability. For one, it lets them see what kind of mental and emotional journey they are having, and if it matches what you intend.
It shows how customers behave and what they try to do when confronted with the interface. It also gives you an inside view of things they’d like you to do that maybe you never thought of trying. Also, you know the things you do that annoys them, because sometimes features we design annoy customers when we expected them to help them.
The Problems With This:
It’s a tedious process. There’s no technological way at the moment to streamline this process and reduce man hours and abstractions to a less atrocious level.
Well, people have to write their diaries and be consistent, which is the first obstacle. But, after that, people have to pour through them, and study the different experiences, record them as metrics, and create reports and retrospectives out of the sum total, for study by the team.
This takes forever, the data’s kind of fuzzy and soft, and it’s just not an efficient way to handle all of this. But, the unique insight it provides does make it kind of tantalizing at times, doesn’t it?
Maybe Solving the Problem:
It’s that lack of automation and stricter metrics which is this concept’s downfall and why I don’t like it. Maybe refining the journal to a form per entry, with comments as a field, and then some smart regular expression parsing for tracking trends in the comments could actually save this idea.
But, this is not proven, as I’ve not seen it attempted. I’d like to see it, but honestly, I think for now, testing for usability is still best handled by classic models that just do heuristic analysis and the like.
Maybe in the future, that technology will open this up as a tremendously powerful way of handling it, and it’ll be the gamification to usability’s training. It’s due for a shake up like that, so why not?
If any developers are reading this, get on that. Otherwise, diary study is never going to be as viable as it could be, and we’ll be stuck with old, tedious practices.