I was interested to learn about betterdesktop.org on /. If you have never seen usability testing before, it is well worth going and downloading the videos.
The basic idea behind usability testing is simple, without any guidance from you or anyone else, ask someone to perform a task with the software being tested, and record it on video, along with asking them to provide a running verbal commentary about what they are doing or trying to do.
Most software developers, in this situation, have a very strong urge to say “no, not like that, like this!” when watching such a test – seeing failures as the fault of the subject, rather than that of the UI design. This emphasises why it is so important that the subject receives no guidance, since typical users won’t have a developer standing over their shoulder to show them how to use the software.
Most open source software is notoriously difficult to use. Some apologists claim that it isn’t any more difficult than commercial software like Windows, its just that people are used to Windows. I am afraid that such apologists are kidding themselves.
The BetterDesktop project is a laudable effort to address this with Linux desktop software. Unfortunately it took a well funded company like Novel to pay for this, it would be nice if usability testing could be performed in the same distributed voluntary spirit as open source.
One possible solution to this would be an open source app which allowed people to easily set up usability tests remotely. This app might record a video of the user’s screen, while also recording what the user is saying. The app could support simple instant messaging functionality so that the tester could interact with the subject in a limited way. I guess I will put this on my todo list, which means I will probably implement it somewhere around 2125AD.