Question 14 of 14: Laboratory Usability-Testing Protocol for the High-Fidelity Prototype

Question 14: Laboratory Usability-Testing Protocol for the High-Fidelity Prototype (10 marks)

Question text not reproduced: the examination questions are © Engineers and Geoscientists BC. Open the official past paper (linked at the top of this page) to read the question, then follow the worked solution below.

Goals. Determine whether representative pharmacists can build and publish a correct drug library, using the high-fidelity prototype, with an acceptable error rate, in an acceptable time, and with acceptable subjective confidence — and identify the specific interaction points where errors or hesitation occur, so they can be fixed before further development.

Questions to be answered. Can participants build a complete, correct drug library for a given unit without assistance? Where (if anywhere) do participants enter an incorrect dosing limit, and do they notice and self-correct it before publishing? How long does the task take relative to the target, and does the confirmation/upload step give participants confidence the library was actually distributed? Which specific screens or controls generate hesitation, wrong turns, or expressions of confusion?

Evaluation approach and methods. A moderated, task-based laboratory study (Question 7) using the high-fidelity vertical prototype: each participant is given a realistic scenario (build and publish a library for a specified unit, including at least one deliberately edge-case dose limit) and asked to think aloud while completing it; the moderator observes and takes notes but does not assist unless the participant is fully blocked, and sessions are screen-and-audio recorded for later analysis. Both quantitative measures (task completion rate, time on task, number and type of errors, number of clicks/screens) and qualitative measures (think-aloud comments, a post-task satisfaction questionnaire, e.g. SUS) are collected, so the results can be triangulated (Question 8).

Practical issues. Participants: 6–8 pharmacists recruited across at least two different hospital units/personas (per Question 6), representative of typical proficiency and infrequent-use patterns, not power users. Evaluators: one moderator running the session plus one note-taker/observer, both trained on the protocol and on staying neutral (not leading the participant). Equipment: a quiet room, the high-fidelity prototype running on representative hardware, screen/audio recording software, and printed consent/questionnaire materials. Tasks: 3–4 realistic scenarios of increasing complexity (routine library edit, new-unit library build, an edge-case dose limit requiring an explicit override). Budget: covers participant incentives, moderator/observer time, room and recording equipment rental, and analysis time. Schedule: one week of sessions (roughly 60–75 minutes each, including consent, task, and debrief/questionnaire), followed by one week for analysis and a findings report before the next design iteration begins.

Ethical issues. Informed consent: each participant signs a consent form explaining the study's purpose, what will be recorded, and that they may stop at any time without consequence to their employment. Privacy: recordings and notes are anonymised (participant IDs, not names) in any report, stored securely, and used only for this evaluation; no real patient data is used in any task scenario, only fictional/synthetic drug-library content. Participants' rights: participation is voluntary, participants are told they are testing the software (not being tested themselves) to reduce performance anxiety, and they retain the right to withdraw and to have their data excluded up to the point of analysis.