GNOME Desktop Usability Study Report Feedback

Received through the gnome-sun-hci@eng.sun.com email alias
Original list compiled by Dave Engen - August 8th, 2001 Last updated by Calum Benson - January 7th, 2002


Page Jump Menu
    Kudos: Positive feedback and thanks we have received for conducting and publishing the GNOME Desktop Usability Study Report to the community
    Criticism: Some constructive and some not so constructive and unqualified negative feedback about the recommendations set forth in the report
    Future Studies: Suggestions for issues to investigate in future studies
    Design Suggestions: Proposals for discrete design fixes that still may require thought and/or testing
    Process Feedback: Suggestions for methods of improving the way in which we conduct usability studies in general here at Sun
    Corrections: Criticism directed specifically at individual recommendations set out in the original report
    Questions: Feedback regarding uncertainties about the report, the findings, or user behavior


Kudos:

  • "This report is amazing, fascinating, and a definite step in the right direction. I'm excited to see what happens because of it, and looking forward to the next one down the line, when all the issues you've raised are addressed. Thank you for doing this!"

  • "Bravo for a job well done. I am a veteran UNIX programmer and to me GNOME is an easy challenge, but I have watched other users (e. g. my wife) struggle with "GNOME mysteries" and I believe you've caught many of them."

  • "Congratulations again on doing such a thorough job of analyzing and documenting user interface issues. You've nailed them over the head."

  • "Congratulations! This is one of Sun's essential contributions to the open-source movement ..."

  • "Note the reactions of the users as they discovered the new-to-them functionality of Nautilus. Confusion at first (because they were expecting more limited functioanlity), then comments like "pretty cool". Just because it did not work like windows or the Mac does not make it wrong."

  • "Still reading through it but have done so enough to say "Thanks". This is great info for the open-source community. You guys have done a great service!"

  • "I really enjoy reading the above GNOME user study you conducted, thank you for making it publicly available. ... I sent the article's link out to my engineering department providing them more of a feel for the kind of scrutiny that's possible to give about UI design, and also encouraging feedback to me in particular."

  • "I just wanted to say thanks for a wonderful usability study, with top notch suggestions. This felt like a solid, critical look with good suggestions, much like the good ol' MacOS UI analyses."

  • "Thanks, P9, you're my hero. :)"

  • "This study highlights a lot of things which need to be changed, and may provide an incentive for people to finally agree on what to do about them. (Many of the issues brought up aren't new, but people had gotten used to them.) So I'd like to thank the team involved for the work done."

  • "Your Usability Study Report is tremendous! That is just the kind of thing that the Linux desktops and applications need to become more competitive and acceptable. Thanks for your good work. Unfortunately, I am not a developer -- just a user - but I hope that the developors will incorporate your comments and suggestions into future releases. I hope that your input will be taken to heart in the KDE arena as well."

  • "I work at PDI/DreamWorks, the company that made the recent film Shrek. I am in our software development group, where I am responsible for user inteface issues. We have recently started using Linux machines on our desktop, running Gnome, after a long history of using SGI machines. All I can say about your Gnome usability study is: OH GOD, THANK YOU, THANK YOU."

  • "Just wanted to drop you a quick note: Thanks for doing this study. It's the sort of information we need. I have both my father and mother working on Linux desktops (using Ximian Gnome) and a number of the concerns raised by your study I have heard from my parents. Thanks, and I hope to see another study when Gnome 2.0 is released :)"

  • "I must say again this is a WONDERFUL document and I greatly appreciate all the work you put into it. It makes me very excited to even think that there is even the smallest shred of a chance that someone out there might work to fix up all the crufty interface problems that Gnome exhibits."

  • "As a developer of GUI applications for GNU/Linux and Solaris systems, I read with considerable interest your GNOME Usability Study Report. Fascinating stuff, and great work! You guys have an impressive team."

  • "I read some of the Gnome usability results. Great presentation. Pretty good laughs. Next time you should have them try and install a piece of software that's not in the Gnome kit. Fun fun fun. :) "

  • "I think that was an extremely worthwhile and well-executed project. It will be good for GNOME and good for GNOME's users if the issues listed are addressed."

  • "I read the report and was very interested. Some of the points raised are vital if Gnome, GNU/Linux or any UNIX is to be more accessible."

  • "Great report. Almost all of the comments apply to KDE and other popular desktops as well."

  • "I just wanted to say that it's great to see the results of your HCI work on the GNOME UI. GNOME has a lot of capabilities, but could use some real tuning to be competitive with Windows and the Mac. Thanks!"

  • "You're on good ground. All the issues covered in your study are worth exploring with care."

  • "Thanks very much for this. It's a really nice study, and from what I've seen of Linux and UNIX window managers and GUIs, input like this is REALLY needed if these OSs are ever to approach an audience outside of the hard core open source development community."

  • "I've just had the chance to read over your usability report. Having not done any UI design as a result of my new job I found it to be an excellent read and I was suprised at how easily we (GNOME Users) can overlook issues that are blindingly obvious. I was shocked at the discussion of the three locations required to change the Font settings. Things that I'm accustomed to doing and learned by exploration."

  • "I was reading with interest the report Sun recently produced on Gnome usability. I was pleased to notice that the Sun Gnome HCI team picked up on many of the same usability problems as did MIT the Information Systems' internal tests."

  • "Thanks for a great read! Even though I am very familiar with the Gnome desktop I still find some of the things that were mentioned annoying and indeed it was difficult at first even though I'm very experienced in using computers and technically oriented."

  • This was an excellent study, well constructed, with much valuable feedback from users, showing the value of working with even a small number of subjects. Most of the offered recommendations should be heeded by every interface designer.



Criticism:

  • "I do not like to see everything "dumbed down" for all the beginners at the expense of losing more advanced/enhanced functionality for more advanced/technical users. I do think the UI should be clear and concise and present all the neccessary information without distractions." The suggested fix is to design an idiom for presenting users with 'advanced' options such as "Halt" and "Reboot" in the Logout dialog box.

  • "If you were to follow all the recommendations, that accommodate what the user knows , you will end up with Windows!!! It seems to me that you need to consider several different users: new users need learnability, experienced users need efficiency, medium level users and occasional user need memorability. So just one type of user can't define usability. A good system also need two things that users will not notice: a small possibility of errors (and no fatal errors) and subjective satisfaction. Your recommendations should reflect all these things."

  • "The test was extremely biased due to the fact they [the participants] have used other operating systems and have been accustomed to Windows specific or Mac specific idiosyncrasies. It was also intellectually biased due to the fact that these people have limited vocabularies. If these people would consider synonyms the test would have a different outcome. The test subjects were probably over the age of 35, are set in their ways and have a set routine for their common everyday computer tasks hence invalidating this test. If the subject people would read what is presented rather than quickly saying, "Oh, no! Too many options!" or "I have limited intellect, please take away my options and give me the one size fits all version" they would not be intimidated or feel stupid. I hope people like that stay away from linux forever! "

  • "I'm surprised that the report didn't think it mandatory to have login and password fields on the same screen. It's most annoying in CDE today to type in a login name, wait for the screen to redraw and prompt for the password, and then type in the password. It makes the overall system feel really slow; it takes at least a few more seconds than a quick login name tab password enter sequence. This is a real problem especially on the web when the login and password fields are presented on different pages. Thankfully most web sites have gotten wise by now, which in turn implies people are probably used to typing in the login name and password on the same screen."

  • "There are some places where the suggestions are overly constrained by what is fashionable and customary rather than by what is most effective and efficient. In other places, the given advice seems to me to be incorrect, incomplete, or misleading. A critique of GNOME could go a lot further and be even more helpful as a guide to interface developers."



Future Studies

  • There is a "great need for a 'Welcome to Unix' short primer on the Unix/Xwindow conventions a new user can expect to see." The concepts are not difficult to grasp but are new and should be presented early on for orientation purposes.

  • "I'm curious as to whether any studies of the value of rollover hilighting in GNOME have been conducted. I've heard that rollover highlighting is distracting, that it provides more information, etc, but not of studies on it, so far. Is a look at it in the works? .. Also, I've been following the improvements in the mouse-cursor-tracking code for menus. Currently, as a mouse moves down a menu, submenus pop open *immediately*, rather than with a slight delay (as on the MacOS or Windows). Also, the range that a user is contrained to move a mouse within when moving from a submenu heading (the item that brings up the submenu) to the submenu body is rather small. The last work I've seen done in this area is over a year old. Is this still in progress?"

  • "Why do you think menu titles are a good thing? Surely the simplest method is to do away with them entirely?"

  • "Rumor is that as this UI was being defined, new studies showed that multiple button, single- lick was better than single button, multiple click that Apple chose. But the "one button" ads were already in the works. Jef Raskin advocated single button AND single click. While we probably do want to provide some constraints or defaults to limit the infinite desktop UI configuration choices, I don't think we should limit it quite as much as Jef Raskin advocates. Consistency to an individual user is very important but consistency between users should not override intuitiveness, which is very individual. That's why a set of user profiles would be a very important contribution to the GNOME project."

  • "As for Sun themselves, I would have a set of Sun Icons and backgrounds. I would have these designed to a specific standard. I would not have them done necessarily in house, perhaps a Gnome competition. List the icons you want in sets of say six icons per set (10 sets) and let people design icons for these sets. Give a prize for each set you select. It wouldn't cost that much ( a few sun blade 100s), but would be good PR, get good results ('cos even the loosing icons would be used by someone) and the Sun icons should still be available, but if they incorporate the Sun logo, or Sun colours, so much the better. "

  • "I have only recently installed Linux and GNOME on a new laptop, moving from a Windows PC. I would agree with much that has been said in the report on the usability issues. As a new user of GNOME, I have come across many of the same problems that the participants in your study did. The thing that I find most frustrating using GNOME and UNIX, coming from a Windows background, is the way that you are expected to know what to do and how the system works in order to do relatively simple things. Currently I'm trying to figure out how to change desktop themes (specifically how to install a theme I have downloaded), with little success."

  • "Another problem I find frustrating is knowing where I can get software, and once I have located it, how do I install it? This seems to depend upon the particular website representing the software - some have great documentation, some has only documentation that is applicable to the developers of that software. It would be nice to know there is a portal that is "idiot proof" that has rpms of software with really good instructions on how to install it - maybe Sun could host such a site? I realise that Sun is keen on getting users to adopt Solaris, but I think it would be a very good idea to have a standardised GUI (GNOME) that users of UNIX OSs can use no matter what OS they are using (Linux, Solaris, BSD etc.) - and so I think Sun is in a position to define a standard GUI and to distribute it - much the same way that Sun distributes an "official" version of NetBeans (Forte for Java) - users could choose either to get GNOME from GNOME.org or go to Sun for an idiot-proof distribution."

  • "A simple "tour" that gave users a very quick introduction to the desktop sounds like a good idea - regardless of the design decisions chosen. This could, for example, point out how the right mouse butten is important. I seem to remember the old OS/2 workplace shell needing this because it was used extensively."

  • I just wanted to put my two cents in on GNOME usability. I applaud your honest study of the issues facing GNOME right now, and would also like to see some studies on the following issues: Fitt'sLaw testing: right now, two corners of the screen are used by default for panel hide buttons. Time how long it takes the users to hit the menu button in GNOME, where a large hide button is in the corner, and in Windows, where the start menu is *almost* in the left corner. For best results, also time against Macintosh users going for the Apple menu. Also, in non-Ximian GNOME, the menu panel items are offset from the top left and top right by one or two pixels, preventing the users from throwing their mouse to the corner and clicking. Have half of the users test the Ximian version, and the other half another (such as GNOME from source or Red Hat's version). Do the users who test the non-Ximian version start to adapt by slowing their mouse down to hit the menus? In addition, the default Ximian panel layout (as selected by the doorman) is very small. Time how long it takes users to hit the buttons at each panel size. I would predict that the regular size is an order of magnitude faster to hit than the smallest size, and that the largest size buttons don't offer much advantage over the regular size. At what size do the users start complaining about it taking up too much screen space? Duplicity: are the users confused or annoyed by the many duplicate ways to open a programs menu? This was touched upon somewhat in the useability study. Are there better uses for the four corners of the screen than duplcating other functions? Basic needed desktop tasks. Do the users know how to unpack an archive of any format? Do they know how to play an audio CD? (Try this one on Ximian GNOME and regular GNOME. The Ximian menus removed the regular GNOME CD player and replaced it with Grip. Test Grip against the regular CD player, GTCD.) Can they install and remove software? (Once again, try with both Ximian GNOME and Red Hat's GNOME. Do they 'get' that they can use Red Carpet on Ximian GNOME? What happens when they double-click on an RPM in Nautilus? On Ximian GNOME, will they be confused when their program doesn't appear in the Ximian menus?) Where do they go for email? Put together a 'customized' system that addresses these issues without doing too many source changes. Re-organize the menus and rename the Control Center items (as well as reorganize them). Do your original batch of users find the new system easier to use? What about a new batch of users?



Design Suggestions:

  • "There is tremendous duplication between the file navigation facilities offered by GMC, Nautilus and the built-in "Open" menu in Star Office. Other applications may have their own quirks in that regard. ... Your interface style guide should have guidelines about how applications that save and open files could/should handle tasks related to file management."

  • Issues that need more design thought

    • Those who are used to single-user systems may have trouble understanding how UNIX supports multiple users on the same machine

    • Logging into another computer across a network may also be a new concept.

    • Getting used to the uses of the various mouse buttons.

    • The idea of each file having permission levels.

  • "My stable of newbies can't stand to go back to windows now because they quickly became addicted to the general linux idea of being able to configure everything to work the way THEY want it to, rather than trying to figure out how windows will force them to work. They express a feeling of liberation. The most common complaint is that they cannot find enough documentation on how to configure and what to configure. This is a valid complaint - and a problem for many open source projects."

  • "We should NOT be trying to dumb the system down to the level of windows. We should continue to build on and highlight the advantages offered - flexibility and power. Try taking a user unfamiliar with Windows and have them perform the same tasks. They will have a learning curve also. Same for the Mac. Part of the confusion faced by your testers is their misconception of the system, much brought about by the limitations learned under windows. Their mental model of the system does not match reality. It is impossible to completely remove the learning phase, but it can most certainly be made easier by using more easily understood icons, by making help more easily available, and providing clear and appropriate feedback. We should not remove the power and flexibility just because they don't use it all immediately. Users can and will learn to make the system work for them, and will appreciate the power and flexibility. Many of them have put considerable effort into figuring out how to work in windows to get things done, and will be much happier once they see that they can do things their way instead of the one "blessed" way (that may or may not work...)."

  • "When I start this thing (Nautilus) it starts all this other stuff that I don't want. It sometimes complains about something missing that I don't care about. It's big and slow, just like windows." (Note - they really dislike KDE for the same reasons - much amplified)."

  • "I'd also like to throw in one more UI idea, in the hopes that someone up there might be interested: Windows hasn't really used "barber poles" (aka infinite progress bars) to indicate that a task of unknown duration is in progress -- instead, it uses a totally nonstandardized range of animated icons. At the very least, this places the burden on the each programmer of producing a good chunk of appropriate animated art. It also takes up a significant amount of screen space. The barber pole solution has been well recieved on the MacOS, and in various Windows products, like Navigator. I would like to encourage the creation and use of a barber pole widget."

  • "'Really do such-and-such?' can be slightly opaque in meaning to a naive user, but more importantly, is suggests that the computer thinks the user's command ought NOT to be executed. I recommend: It's more tactful to ask 'Are you sure?', which suggests that the computer's attitude is one of concerned help and moderate skepticism rather than outright disbelief."

  • "I agree that the relationship could be made more distinct - I suggest having "Colour" at the top of the dialog, with the Wallpaper controls below that in a groupbox where the "Wallpaper" label has been replaced by a check box labelled "Add wallpaper". "

  • "GNOME should allow for users to switch between novice, intermediate and expert levels. All GNOME applications could be designed to accommodate these levels. Applications that are designed only for advanced users could present an alert when launched in beginner mode warning of the possible skill discrepancy."

  • "Entries on the login screen, such as 'marge-hci' should have popup descriptions to aid new users."

  • "Overall, it seems like many of the applications would benefit from an "Advanced Mode" and a "Basic Mode" much like Apple(TM) used in MacOS 9. This would help in the configuration menus and in things like the search function in Nautalis, where the average user has no use for such features as "regular expression" searching, even if they are better explained, as you suggest."

  • It might be useful to have a GNOME tutorial that provides brief training sessions for novice users transitioning from a non-Unix system. This would eliminate 'hidden features' and a more comfortable learning curve.

  • "Use 'Restart' instead of 'Reboot'."

  • Instead of using Yes/No dialogs, the option to proceed should be replaced with the appropriate action verb, and the option to keep from proceeding should be replaced with 'Cancel'. Thus, "Are you sure you want to cancel the missile launch? [Yes] [No]" , becomes "Are you sure you want to cancel the missile launch? [Cancel] [Launch]"

  • You said: "We can't let the development process, of componentized, modularized pieces from different origins, permeate up to the user's level." Yes we can and should. We should also provide simplified views for beginners. Frustration at too much complexity when first learning is bad. Frustration at too many limits after the initial learning is also bad. We need to provide simplified and consolidated interfaces for beginners, but we should not remove the ability to get at the underlying systems easily. More power and flexibility takes longer to learn, but it is worth it - according to my users. We need to make it as easy to start and learn as possible, but we should not take away the power."

  • "Users NEED to learn about that right mouse button. And the MIDDLE button. I know many windows users who have as many as 7 buttons on their mice. They did not know what to do with all of them "intuitively". Some learning is required. We need to make that learning as pain-free as possible. We should not remove access to the real power of the system just to make the first few minutes of using it without reading the manual easier."

  • "The GNOME Usability Study Report at http://developer.gnome.org/projects/gup/ut1_report/logging_out.html recommends that the logout dialog Halt "checkbox label reads 'Suspend (Halt - stops the processor)'". I belive it should say 'Halt (stop for poweroff)'. This better describes what happens when the checkbox is chosen. (Usability Principle: Match between system and the real world.)"

  • "I think the Sun people mistook that Halt to mean the old-fashioned halt, which was not a nice thing to do (it did stop the processor)... in most systems with more recent versions of sysvinit, halt just runs shutdown. It would be best if it just said "Shut Down" I think. :)"

  • "People _are_ used to having to powercycle to recover. It's some folks default response! So I'd go for the clarity of 'Turn off computer'. It contrasts plainly with 'Restart computer'."

  • "I think the Sun people mistook that Halt to mean the old-fashioned halt, which was not a nice thing to do (it did stop the processor)... in most systems with more recent versions of sysvinit, halt just runs shutdown. It would be best if it just said "Shut Down" I think. :) It already does [I think]. I know it's been applied to the gnome-session, and I expect to gdm as well. "

  • "The Login / Username and Password should not appear on the same screen because it is easy for the text box focus not to change. This means that a user could enter their password into the username (login) box by mistake. Since text in this box is visible this would compromise security. I would add a section on the password screen indicating that the text will not be visible (I know most users of any system know this, but it doesn't hurt) The help system - while I agree that the extra items - HTML, man etc .. are distracting, I would hestitate at using Gnome Help, since the help system is for more than Gnome, it is for the whole OS. I think "Get Help" would be better. Then the help browser / Application could be customised to look for help on what you want."

  • "On the whole, the Gnome system is getting better, though I do think that some icons could be better and their placement more ordered. Having multiple ways to complete tasks is not a problem either, in fact I would encourage it. Everyone does things differently. I would want a more consitent approach though so that these multiple ways worked for each task."

  • "Make every legend =ABWinLux=BB or else, quite plain, for example: [ Username (login): ], [ Machine Name: _name_ ]. ... By all means, bring all comments, instructions, help text, etc., over to standard english. In Unix/Linux too many of them read like sourcecode written in =ABC=BB with all coherence stripped (pun intended).

  • Be prepared to write parallel =ABman=BB files (pages) devoid of jargon. (This was not addressed in your study due, I presume, to its introductory nature inasmuch as there is no such thing in Windows). These should not be mere transliterations. It's not the words that count, it's their meaning in sequence, that is, the ideas they jointly convey. The Windows culture and language must be addressed on its own terms before they can be translated usefully. Interpret precisely and explain in standard English everything that may be unique to Linux. This helps Linux, and Linux is the Future. Put together clear PS or PDF user-oriented help files. (Not user-friendly, please, too often, user-friendly becomes an excuse for repetitive dumb-user babble.)"

  • 'The problem with the addition of clock to the panel would be replaced cleanly with a panel configuration dialog. Putting settings into menus is misusing menus. They're not good for it, because an user wants to configure the system in one run, not clicking again and again thru the same menu levels. There could be a 'Select a clock' dialog, which would consist a list with a scrollbar, and a preview what the clock looks like."

  • " 'Terminal Emulation Program' should be 'Command Prompt', and 'Integrated Help Sytem (Info, man, HTML)' should be 'Gnome User Guide'."

  • "You recommended "username" instead of "login"; why not "User Name:" or "Please type your assigned 'user name' here." (so long as you clearly call it "your user name" when assigning it). The principle here is not to force yourself to use single words or to use internal spelling conventions (such as "username" without spaces. You might even instruct "Please type your assigned 'user name' here and then tap the Return key." Why not? Experts won't even bother to read it so it won't slow them down, and it helps the novice."

  • "Another suggestion: the "OK" button is an invitation to use the mouse to close the box. We know that the users have just been typing, why then ever ask them to move their hands from keyboard to mouse? It wastes over a second (on average) and adds to the motions that cause RSI."

  • "Another suggestion involves too much user overhead: "Not knowing what 'marge-hci' was did not hinder users from successfully completing the task. However, novice GNOME users' confusion did highlight an opportunity for us to provide more assistance. To support new GNOME users we suggest including a Help button or menu in the login dialog. Selecting this button or menu should nvoke Help with information about the elements and terminology found in the dialog..." There is plenty of room in the box to say "'marge-hci' is the name of this computer. The system will use the name when it is talking about this computer." or some such. Why bother the user with accessing a help system when you can just put the information where it's needed?

  • "A number of suggestions for a better logout icon are given, along with a recommendation for testing. What would certainly have worked (guaranteed, without testing) was button had containing not an icon but the words "Click here to log out." The idea of not using an icon on this button does not seem to have occurred to the designers, caught up in standard GUI methodology. If this suggestions offends the graphic eye, then put the design in the background with the words overlaid clearly. The relevant design principle is not "Match between system and the real world" but "Don't use pictures where words work better." I guess you could say that this is a better match to the user's real (cognitive) world. This same suggestion applies to all the other icons that gave the test subjects trouble. They also suggested improving the tooltip, which would have been unnecessary if the buttons had contained text. Please note that tooltips are words used to explain an icon; they are never icons used to explain words (though now that I've said this, somebody will deliberately build a counterexample). Even if there are a handful of exceptions, words should be the first choice. "

  • "A good design principle is to eliminate user preferences wherever possible. This is another example where open-source development leads to poor interfaces; everybody wants it to work the way they are accustomed to an interface working. This leads to clutter, and to usability and maintenance difficulties"

  • "Seeing the successful deployment of MIT's analogs to your own usability testing in this large scale way, I invite the GNOME developers to consider adopting some or all of our solutions as implementations of the guidelines raised. We might end up saving you folks some work. (And yes, it would save us changing our look and feel when your improved versions come along.)" [The MIT study can be found at http://web.mit.edu/aui/notebook/charter.html]

  • "I agree with alot of what you said, except where you talk about removing choices and customizability from users. Nautilus has a drop down on preferences that lets you select beg. inter, or advanced. This should be integrated with the control panel and should apply to both. It will help solve much of what is shown here. Second: I agree that the three places to change a font is freaking crazy. The theme manager could be shown as a subset of the window manager and be the only place provided to change the font. Since theme managers have the ability to select a default font (a feature that advanced users like me like) this should be clearly explained in the panel and the current override provided. Good stuff about the "Font" wording though."

  • 'I have to think that open source people working on gnome make gnome for themselves as much as anyone else... and that they must be comforable with the GUI or they would not use it. If we take away options or programs to 'Dumb it down' for newbies-- we are not doing anyone a favor. I hope the advanced- intermediate - beg. selection idea can solve this without removing any functionality that I currently use. I already had to quit using KDE because of its dorked-ace window manager. I don't want windoze or Smac. I like GNOME's power. I like the window manager and I like having all the gnome programs and applets and keybindings. And I like the name applets. I don't think we should change the terminology of everything to make Joe Six pac happy. If you do you will piss off Mary UNIX Lady, and you don't want her wrath. I think a distinction needs to be made between easy to learn and easy to use-- indoze is easy to learn and it sucks to use. Unix is hard to learn and a delight to use. The user levels setting of Beg, Inter, Advanced can give us both without sacrificing anything-- Just pick the setting you can handle. And please don't make everything right-click---- three buttons three times the power-- eliminate confusion new users have through tutorials and documentation, not by emulating a dumb-ace OS. It is no wonder one of your test people who had problems with right-like was a MAC user. Narrow minded GUI's make for narroow minded users-- one button on mouse for 15 years-- cannot comprehend three buttons-- brain overloading....AHHH. Must use cnt-openapple-shift-mouseclick- umlock to access feature, what is right click? '

  • 'What is needed to fix confusion is a good users manual, like you suggested. The ones so far are quite abit better than they used to be, but need some integration and organization. I would set up a seperate project leader to integrate all GNOME documentation for consistent usability and feel. This would apply to GNOME programs as well, such as GNUMERIC and the GIMP. This person would not initially write the documents, but edit and compile them-- a tech writer.'

  • 'You recommended 'username' instead of 'login'; why not 'User Name:' or 'Please type your assigned 'user name' here.' (so long as you clearly call it 'your user name' when assigning it). The principle here is not to force yourself to use single words or to use internal spelling conventions (such as 'username' without spaces. You might even instruct 'Please type your assigned 'user name' here and then tap the Return key.' Why not? Experts won't even bother to read it so it won't slow them down, and it helps the novice.'

  • "You quite correctly suggest that 'developers [should] involve a technical writer in the review of any terminology, error messages, or other on-screen text'. Might as well suggest that an interface designer be involved, too."

  • "For panel buttons that are important but that are difficult to translate into a graphic icon could be replaced with text-only labels. This same suggestion applies to all the other icons that gave the test subjects trouble. They also suggested improving the tooltip, which would have been unnecessary if the buttons had contained text."

  • "I agree with the group's suggestion that terms such as 'regexp' be replaced by the words 'regular expression'. This suggestion needs to be used more widely. Especially in drop down menus, which are transient, we can afford a lot more words, in those frequent cases where they would be helpful."

  • "Another recommendation was, 'We recommend providing more visual indication that the user is in viewer mode and cannot edit.' As has been shown repeatedly, no amount of visual indication will always work, but if you cannot get rid of the mode, then changing a border or the cursor shape is not nearly as satisfactory as (1) giving a readable message (as was done!) and carrying the grey background or some other design element across the entire text. Text that is non-editable, for whatever reason, even if it is part of a menu, should have a uniform appearance, different from that of editable text. (See The Humane Interface, p 41 for more discussion)."

  • "The log out dialog that you recommend is inefficient. While correctly separating the Halt option and renaming more clearly (good moves) you did not simplify the entire box by having only three buttons,each of which did the action listed on it, and eliminating the silly 'Yes' and 'No' buttons at the bottom. That would substitute three large buttons (easier to use) for three tiny ones (perhaps you can click on the words, giving a larger target, but there is no way to know that from the box design) and two large ones. As we know, due to habituation, choose-and-confirm is no more secure than simply giving the choise."

  • "'Match between system and the real world' is a poor heuristic; we all have our own worlds, and I'll bet that some of the bad design choices were caused by a programmer matching the system to his or her real world. It is also a vague standard."

  • "Another [recommendation] states that 'Dialogs should not contain information which is irrelevant or rarely needed.' but the researchers did not follow this advice in some of their own redesigns. The principle, The claim that 'The system should always support multiple ways of accessing functionality' is wrong, perhaps even backwards. In fact, in places, the authors correctly tell us to do just the opposite."

  • The tree sidebar in Nautilus requires some reorganization: "Take a look at: http://www.eps.ufsc.br/~juliano/nautilus/tree.png Right now, the sidebar shows the tree as it is. It shows the root directory, and when expanded shows home, usr, tmp, and so on. My suggestion is to mask things a little, making navigation easier. In the base of the tree is better to have the user home, the machine (/), and removable media (when mounted). Maybe we can put here the magic desktop URIs from RedHat Labs too. I think Windows, KDE, Mac work like that. ... I would be glad to reorganize the tree view but I think a bit of thought overall needs to be put into how Nautilus presents the filesystem and what common shortcuts there are. I'll post my thoughts soon."

  • "I would really like to change query-drags to be based on the middle button instead of the right button, so the right-click menu can come up faster. Does anyone object to doing this? I know Pavel (who did a lot of the DnD work) objected before, but he no longer works on the project."

  • The change zoom widget on the Nautilus toolbar needs to be rethought as well: "Take a look at: http://www.eps.ufsc.br/~juliano/nautilus/zoom.png Again this is just to make navigation faster. For example, I'm in the default view (100%) and I want to zoom to 400%. I need to pass to 150%, then to 200% and finally to 400%. With the widget change I can pass directly from 100 to 400%."

  • "I decided to start working on adding the ability to make components editable and have a reasonable UI for saving and such. Since one of the top usability concerns with Nautilus according to Sun's report is the fact that the text view does not allow editing, I figure the best solution is to make it allow editing."

  • "Regarding the proposed change to how "Halt" is portrayed in the logout box: it doesn't go far enough. "Suspend" sounds to me at least like something temporary and easily undone. I think the information given there should include an indication that "halt" means that the machine will be shut down."

  • "It's too easy to remove things from panels... One way to fix this problem would be to have a "foolproofing" option, probably in the Control Center under Panel (or under Advanced), which would be enabled by default. If foolproofing were turned on, GNOME would ask "Do you really want to add/delete to/from the Panel?" This would cut down on novice users' frustration when they accidentally delete something from the panel and don't know how to get it back, as well as providing feedback that the item is being added to or removed from the panel. Allowing users to turn off foolproofing would enable power users to modify the panel without being harassed by dialogue boxes."

  • "The tasklist and pager are at least as important to the panel as a logout button or screen-lock button, if not more so. They should be on the "Add to panel" menu, instead of being hidden under "Applets"."

  • "Another problem users have noticed here is visibility of important items on the panel. For example, I found the pager as currently implemented very difficult to find on the panel. I thought it wasn't there until someone pointed it out to me. The default theme should have the pager be a different color than the rest of the panel, and perhaps have default labels for the different desktops."

  • Users don't necessarily know that they are using GNOME. Some of them have asked me for help, and I ask them "Which desktop are you using?" Many of them aren't sure. They just know that they are using the default desktop for their machine. Some are confused by the "KDE Programs" choice in the foot menu, and think they are running KDE. It would be helpful to have the login screen tell users what desktop they are logging into, and perhaps to have the word "GNOME" on the foot button."



Process Feedback

  • "Keep reports up-to-date with new studies for each release. Some of the issues are ones that have already been broched, but still have not been addressed."

  • "I read the GNOME usability study report and I am very excited to see more work being done in this area. I do have one question though: what was the hardware setup for the desktop machine(s)? Some users commented that the system responded quickly, and considering the test was done using Nautilus 1.0 (which performs very poorly, especially compared to the latest version) I am curious about the hardware specifications. It might be useful to add this information to the procedural notes appendix."

  • "I understand why you required participants to have at least 3 years GUI experience. At the same time, I think you could learn a lot about making the GUI intuitive by studying users with _no_ GUI experience."

  • "They were not told to "log off" or "log out". They were simply told to leave their machine so that the coworker could access his files from it, but would be unable to access the user's files. Some users, especially those with minimal UNIX experience, did not understand the task. ' This is poor instructions on your part. You cannot expect people to sit down at a computer and just 'work' if they have never been exposed to a computer before, or have never used the OS on that machine. Even windows comes with instructions and there are dozons of training guides on how to use windows effectively. Your 'failure' is due on part to your instructions. Try giving people a little better instructions next time. Also try giving a training class on GNOME and see if people can follow what you are talking about. ... It is good to see that someone is doing a useability study, but I think it is bad that they are not supplying much instruction."

  • "It's great to see that some time and effort have been spent in rating the useability of the current state-of-the-art of GNOME. However, I'd love to see a little more care taken with the formatting of the webpages that report these results. First off, the width of the page appears to vary depending on which page you're viewing. Secondly, the graphics accompanying the report are variously in .png and .gif format. I'm accessing this page on a Windows 98 computer with IE 5.5. I can't see the .png files without saving them individually to a folder and viewing them separately. Consistency is a value to be cherished in this regard. I know that MS IE is not a politically correct browser for this purpose, but it's what I've got at the moment, and I'm disappointed that I can't access all the report's graphics, maybe as disappointed with Microsoft as with anyone. Finally, I've got the usual bits about spelling errors and poor grammar in places--minor stuff compared to the formatting issues, really. What really comes out of this report is the need to make the GNOME desktop more coherent and consistent, and more aimed at the general user, if it's ever going to be a real contender against the M$ monopoly."

  • "Another states that "Dialogs should not contain information which is irrelevant or rarely needed." but the researchers did not follow this advice in some of their own redesigns."


Corrections

  • "Although Halt is not clear, Suspend is worse because it implies that the system will remain sedentary till next contact. Some systems use "Suspend" as a power-saving mode, so that term is also loaded. Rebooting also rarely serves a purpose on networked machines. Possible fix would be to provide three options: Logout (geometrically separate from other options), Halt system, and Reboot system."

  • "'Suspend' will be familiar, especially to laptop users, it will not do what they are most likely to expect. 'Halt' stops the processor and is not a part of the normal logout on a multi-user system. Possible fix is to use "Logout/End Session - prepare computer for shutdown" (the default option) , 'Halt - stop processor'."

  • "Having the username text entry field be locked into focus precludes a user being able to select a style or language for the session before logging in."

  • "'Matches glob' is over the head of even seasoned users - just leave it out and put the functionality under 'matches Regular Expression'.

  • "Your recommend term "Suspend" is not a solution. 'Suspend' does not convey the idea of shutting down the computer so that the power automatically goes off (on most machines) and a complete reboot sequence is required to start using the computer again. In fact "suspend" is used in various of the Microsoft operating systems (though inconsistently, and with variation in meaning between the different OSes) to refer to putting the computer into a low-power or power-off state from which the machine can resume quickly at the point it left off without a reboot of the OS--i.e. a variety of "sleep" mode. For this reason, 'suspend' should not be used to refer to a processor halt."

  • "The tooltip for the 'terminal emulator' should be changed to something more like "Command Prompt". Although you mention it in the body of the study, it is not in the final design recommendations. This seems like an accidental omission."

  • The wallpaper and the background color settings are not mutually exclusive because an image that is smaller than the set screen-size will have a border of that predetermined color.

  • Suspend means low power mode, and doesn't imply the machine will be turned off.

  • "I disagree with some of the given "Usability Principles". "Match between system and the real world" is a poor heuristic; we all have our own worlds, and I'll bet that some of the bad design choices were caused by a programmer matching the system to his or her real world. It is also a vague standard."

  • "This was an excellent study, well-constructed, with much valuable feedback from users, showing the value of working with even a small number of subjects. Most of the offered recommendations should be heeded by every interface designer. But, as I have noted, there are some places where the suggestions are overly constrained by what is fashionable and customary rather than by what is most effective and efficient. In other places, the given advice seems to me to be incorrect, incomplete, or misleading. A critique of GNOME could go a lot further and be even more helpful as a guide to interface developers."



Questions:

  • " I'm wondering why you recommended "Suspend" over the standard term "Shutdown" or "Shut Down" as used on Windows or Mac OS. Suspend puts me in mind of putting the computer to sleep (ie, into power-saving mode, as is commonly done with laptops). If the desire to keep the standard Unix term "Halt" is strong, then the working Shut Down (Halt) would seem to be ideal."

  • "Yes, in fact users do need a lot of options. Making the chioces clear, and providing a way to get good descriptions about the choices is neccessary, though. They will appreciate the choices if they understand them."

  • "If we had designed the system for the lowest common denominator, many of the "more advanced" users would not have been able to achieve the improvements they did."

  • "At least half of the PNG images in the Gnome usability study appear completely black under Microsoft Internet Explorer version 5 or 6, for example on this page: http://developer.gnome.org/projects/gup/ut1_report/file_management.html."

  • "PNG is not a universal browser Image format - use GIF,JPG, this article is not viewable under IE5 The first two pages correctly resize left to right, some pages have a default width of 1000 pixels and do not reformat text."

  • "Take a look at [http://developer.gnome.org/projects/gup/ut1_report/logging_in.html]. It's the first one, and yet, half the images won't load (they appear all black). This could be for several reasons; I think QuickTime handles the PNG's on in Internet Explorer on my [Windows 2000] system, I believe ... are the files using some non-standard PNG format? Otherwise, I'd say the report is pretty good. A few usability guidelines for the formatting of the report on gnome.org wouldn't hurt, either, but ... maybe I'm asking for too much. :)"

  • "It took me forever to figure out how to just change my clock time, and I am no computer newbie and even had some previous experience with Unix. Every clock needs a link to reset the clock. Also, I made a mistake telling the setup my clock was set to GMT time and I spent an hour trying to figure out how to fix that. This stuff should be an easy fix from any clock menu. I know this is ROOT proveledge stuff, but don't forget about us newbies running single computer Linux."

  • "My question is [whether utilizing all three mouse buttons is] bad usability design? Or is this a paradign shift that new users need to be made aware of? In Unix, it is (roughtly) the case that the left button is for dragging and activating. The right button is for options and control. The middle is for extreamly specific operations or cut and paste (a specific operation)."

  • "I like the information that you have highlighted in the Usability report, however I was wondering if there was a way that the report could be obtained in a format that could be printed and read offline, or if the html version will be all thats available."


Users · Developers · ViewCVS · FTP · Bugzilla · Software Map · Contact