Go-Lab Deliverable D3.3 Formative usability and user experience evaluation report on the initial Go-Lab portal

Abstract : The key objective of WP3 is to identify, update and integrate, on an ongoing basis, requirements for developing the Go-Lab system as highly beneficial and usable platform for use in science teaching. Methodologically, WP3 adopts a participatory design (PD) approach grounded in the established User-centred Design (UCD) frameworks, which also underpin Usability and User Experience (UX) methodologies. In Year 3, WP3 is responsible to collect data on usability and UX of the Go-Lab design artefacts, ranging from individual scaffolds to an entire online lab to inform the refinement of the pedagogical specifications (WP1) for such artefacts and the improvement on their design and development (WP4, WP5). Specifically, we provide teachers and students with access to prototypes of parts of or the whole system in increasing levels of fidelity, some iteratively in different levels of refinement, in order to elicit their feedback on usefulness, enjoyability, learnability, memorability, effectiveness, and other pragmatic as well as hedonic qualities. Several complementary HCI approaches have been adopted and adapted for PD data collection and analysis, including: 1) Face-to-face PD workshops with a range of engaging activities that encourage students or teachers to provide feedback on the artefacts under scrutiny. Different instruments were deployed, including traditional paper-and-pen and its software- supported counterpart PDot for capturing feedback, surveys, audience response system, interviews, and observations; 2) Remote studies involving 21 “Core Group” teachers from nine countries, who are highly motivated in following the development of Go-Lab and are willing to provide prompt feedback to specific questions posed to them on a regular basis; 3) Analytical Evaluations conducted by usability researchers without involving end- users, to investigate a few artefacts; the results have directly been fed to the development team for improving the prototype before testing it with users. From November 2014 to October 2015, the work package has designed, conducted, analysed and reported 46 face-to-face PD events with teachers and/or students (and has received, analysed and reported on usability data from 7 others), 13 remote events involving Core Group teachers and 4 analytical studies involving usability specialists. Altogether these PD activities involved over 550 students and over 450 teachers from 18 countries. The face- to-face PD events took place at schools, research/training centres, conference venues, and other venues such as the Go-Lab Summer School. The data are predominantly subjective self-reports of teachers and students on their experiences and opinions after using specific Go-Lab artefacts for specified tasks, together with some performance measures such as timings, number of errors, etc. Overall, the participating teachers and students are excited, enthused and positive about the potential benefits of Go-Lab, to a significantly greater extent than in Year 2. However many possible areas for improvement of usability or user experience have also been reported. The findings from the many studies have provided a steady flow of information back to the project leadership and thereby influenced system development direction and priorities. This document reports the year’s findings, particularly areas of possible improvement. For all but the most recent studies, the findings have been communicated to the development cluster (WP4 & WP5), and their responses and progress are also documented herein.
Document type :
Reports
Complete list of metadatas

Cited literature [55 references]  Display  Hide  Download

https://telearn.archives-ouvertes.fr/hal-01274930
Contributor : Go-Lab Project <>
Submitted on : Tuesday, February 16, 2016 - 2:33:05 PM
Last modification on : Wednesday, September 18, 2019 - 1:29:00 AM
Long-term archiving on : Tuesday, May 17, 2016 - 5:26:09 PM

File

Go-Lab_D3.3.pdf
Explicit agreement for this submission

Identifiers

  • HAL Id : hal-01274930, version 1

Collections

Citation

Effie Law. Go-Lab Deliverable D3.3 Formative usability and user experience evaluation report on the initial Go-Lab portal. [Research Report] Go-Lab Project. 2015. ⟨hal-01274930⟩

Share

Metrics

Record views

200

Files downloads

247