How to run a focus group on website usabilty - ideas wanted

By admin on 18/05/2007

I've been tasked with planning a real (as in get a load of people together) forum focus group (edit: I think focus group is a better term than forum) for assessing the usability of a website. I've always wanted to do such an excercise but to date none of the projects i've worked on have either had the budget or appreciation for the value of such a task.

I've a few ideas and some textbook definitions of what such a forum should consist of but I thought I could come up with something far better if I abuse leverage the full potential of the MXNA audience.

My current plan is as follows:

Sessions

We need to do four sessions, i'm thinking about 5 attendees per session and about an hour per session. Any longer and I think we'll struggle to get people to attend (even with a token financial incentive).

Will hire a private room with no distractions.

Setup

Am going to setup 'caves' (XP speak for private single PC desks) with remote monitoring software so I (or the facilitator) can watch but not interfere with user operation. This is so we can replicate how they would likely use the site if they were using the site for real.

Monitoring

Each attendee will be given 5 minutes to familiarise themselves with the site with no other instruction than play around and see what you find. Then 10 minutes to complete a set of tasks.

I intend to have a list of objectives I can assess each user on by giving a score out of 4 on their competence in using the site (using 4 not 5 so there's not fence sitting, it's either Very Bad, Bad, Good or Very Good) also note any other relevant observations.

I'll also probably record the screens and maybe the user themselves for the option of further review later.

Discussion

Once all 5 have used the site we will move to a 'commons' area where the group can have an open discussion about their experience in using the site. I'll have some leading questions for if the conversation needs any steering or kick-starting but I'd like to keep it as much led by the attendees as possible, whilst I can make claims of being the usability guru, they are the users and will have a far more valuable collective opinion than my own blinkered, tech-led view.

Your Help

I'm interested in any comments you might have on the rough approach outlined about, any suggestions you can make to add to, remove or improve.

Ideas on what to look out for when monitoring users.

The biggest unknown to me is where to get the attendees from, we have some ideas but it's not something we've been tasked with before, if anyone knows of any London-based companies who can arrange this type of thing then i'd be interested to know.

Also any ideas on how to summarise findings and make recommendation, I'm goign to try to record and note as much as possible from the sessions so any decent ideas on how to best summarise and present findings would be much appreciated.

And finally if anyone has any useful references for this type of activity that would also be very useful.

If anyone can offer comment I'd be extremely grateful and I'm intending to blog the progress and findings as we go.