Social design feedback: evaluations with users in online ad-hoc groups

Social design feedback is a novel approach to usability evaluation where user participants are asked to comment on designs asynchronously in online ad-hoc groups. Two key features of this approach are that (1) it supports interaction between user participants and development team representatives and (2) user participants can see and respond to other participants’ comments. Two design cases, involving 250 user participants, were studied to explore the output of social design feedback and investigate the effect of the two key features of this approach. Of all the design feedback, 17% was rated highly useful, and 21% contained change suggestions. The presence of an active moderator, representing the development team and interacting with the user participants, increased the usefulness of the design feedback. The opportunity to see and respond to others’ design feedback had a minor effect on the kind of design feedback provided, but no effect on usefulness. Based on the findings, we offer advice on how to implement social design feedback and suggest future research