Go-Lab Deliverable D3.3 Formative usability and user experience evaluation report on the initial Go-Lab portal
Abstract
The key objective of WP3 is to identify, update and integrate, on an ongoing basis,
requirements for developing the Go-Lab system as highly beneficial and usable platform for
use in science teaching. Methodologically, WP3 adopts a participatory design (PD) approach
grounded in the established User-centred Design (UCD) frameworks, which also underpin
Usability and User Experience (UX) methodologies. In Year 3, WP3 is responsible to collect
data on usability and UX of the Go-Lab design artefacts, ranging from individual scaffolds to
an entire online lab to inform the refinement of the pedagogical specifications (WP1) for such
artefacts and the improvement on their design and development (WP4, WP5).
Specifically, we provide teachers and students with access to prototypes of parts of or the
whole system in increasing levels of fidelity, some iteratively in different levels of refinement,
in order to elicit their feedback on usefulness, enjoyability, learnability, memorability,
effectiveness, and other pragmatic as well as hedonic qualities. Several complementary HCI
approaches have been adopted and adapted for PD data collection and analysis, including:
1) Face-to-face PD workshops with a range of engaging activities that encourage
students or teachers to provide feedback on the artefacts under scrutiny. Different
instruments were deployed, including traditional paper-and-pen and its software-
supported counterpart PDot for capturing feedback, surveys, audience response
system, interviews, and observations;
2) Remote studies involving 21 “Core Group” teachers from nine countries, who are
highly motivated in following the development of Go-Lab and are willing to provide
prompt feedback to specific questions posed to them on a regular basis;
3) Analytical Evaluations conducted by usability researchers without involving end-
users, to investigate a few artefacts; the results have directly been fed to the
development team for improving the prototype before testing it with users.
From November 2014 to October 2015, the work package has designed, conducted,
analysed and reported 46 face-to-face PD events with teachers and/or students (and has
received, analysed and reported on usability data from 7 others), 13 remote events involving
Core Group teachers and 4 analytical studies involving usability specialists. Altogether these
PD activities involved over 550 students and over 450 teachers from 18 countries. The face-
to-face PD events took place at schools, research/training centres, conference venues, and
other venues such as the Go-Lab Summer School. The data are predominantly subjective
self-reports of teachers and students on their experiences and opinions after using specific
Go-Lab artefacts for specified tasks, together with some performance measures such as
timings, number of errors, etc.
Overall, the participating teachers and students are excited, enthused and positive about the
potential benefits of Go-Lab, to a significantly greater extent than in Year 2. However many
possible areas for improvement of usability or user experience have also been reported. The
findings from the many studies have provided a steady flow of information back to the project
leadership and thereby influenced system development direction and priorities. This
document reports the year’s findings, particularly areas of possible improvement. For all but
the most recent studies, the findings have been communicated to the development cluster
(WP4 & WP5), and their responses and progress are also documented herein.
Origin : Explicit agreement for this submission
Loading...