Global Expert Review and Usability Testing
(Desktop & Mobile)
THE PROJECT
This global user research project focused on the website and mobile application for a popular job search company in the United States and Germany. It took place at the Bentley User Experience Center. The client came with the challenge of identifying the most significant pain points and unmet needs in the current user experience, and my team helped them to distinguish the biggest opportunities for improvement in the design of the website and mobile application.
The project involved:
Expert review of 100+ screens from the website and the mobile application in United States
Usability testing on the website and the mobile application in United States
Usability testing on the website in Germany
THE PROCESS
To begin our project, we started off with a kickoff meeting. We aligned with the client on the core goals of our study and then conducted background research to prepare. My team for this project consisted of myself, another UX associate, and one senior UX consultant. The specified goals of this project were to:
Identify the most significant pain points and unmet needs in the current user experience
Distinguish the biggest opportunities for improvement in the design of the website and mobile application
Understand the relationship between searching and applying for jobs on the mobile application in conjunction with the website
Expert Review:
The first phase of this project involved a thorough expert review. We carefully analyzed over 100 screens between the candidate and employer sections of the company website and the the mobile application using Nielsen’s heuristics and accessibility best practices. To analyze all of these screens, I worked alongside another UX Design and Research Associate. We independently identified and rated usability problems, putting ourselves in the shoes of the users and embodying their potential perspectives. We then compared our problem discoveries to gather more comprehensive global and detailed findings and to combat the evaluator effect. We collaborated to appropriately label and classify the recognized usability problems.
Sample of our expert review global findings.
Usability Test:
The second phase of this project involved usability testing in the United States and in Germany. We started by creating a screener to recruit job seekers for the usability tests in both countries. The client requested a complex recruit, which was composed of “active” and “passive” job seekers and “professional” and “others.” Our team worked alongside the client to define these terms and attain the desired recruit. Once our screener was ready, we coordinated with a recruiting agency to gather appropriate participants in both countries.
Diagram of complex recruit requested by client.
The next step of this project was to create versions of the moderator’s guides for both the website and the mobile application. I developed the initial draft of the mobile application version while my colleague created the initial draft of the website version, and then we worked together to finalize each of them. When creating the moderator’s guide for the mobile application, I focused on designing a comprehensive set of tasks based on the needs of the users and the goals of the study.
In the United States, there were a total of twelve usability test sessions, and I moderated three of the sessions. While the sessions were occurring, we looked for widespread usability issues and kept track of these key themes on large posters. We then analyzed our findings in greater detail and decided to organize them by first focusing on the broad, global issues, and then narrowing in to focus on specific, detailed issues.
OUTCOME
We created a report in PowerPoint to share our findings and recommendations from the United States expert review and the United States and Germany usability tests. We coordinated with the German agency and incorporated their findings into the report, presenting our conjoined findings to the client. During the presentation by our project manager, my colleague and I contributed with comments and answered questions from the client.
The expert review and usability testing gave us valuable insight into the company’s website, mobile application, and the relationship between the two entities. We identified significant issues involving lack of feedback and clarity, fragmented parts of the system needing to be combined, mismatches between the users’ mental models and internal inconsistencies, and expected (but missing) features. These findings allowed us to distinguish the biggest opportunities for development in the design of the website and mobile application and provide actionable recommendations to the client for improving the user experience in both the United States and Germany.
RETROSPECTIVE
Evolving Website: The internal UX team at the global job search site was simultaneously updating their website design with next iterations while our project was in progress at the User Experience Center. We were frequently surprised to see unannounced changes had been made to pages on the website and received new screens from the client for the mobile application several times throughout the venture while preparing for and conducting our expert review and usability tests. These unexpected changes required creative thinking during test sessions and adjusting the previous work completed, exemplifying that studies do no always occur as expected and it can be important to adapt as necessary.
Language Barrier: To conduct preliminary research and compare the German website to the United States website, we used Google Translate. There appeared to be minor differences between the two versions from our perspective, but it was a different story when we checked with our German partner agency. By working with this agency, we identified major areas of the website with language discrepancies that needed to be revised with local translation to improve the user experience for German website visitors. This international experience proved how important it is to do proper due diligence and find knowledgeable cultural experts or credible partners with when conducting user research in another country.