Handshake Usability Study
Role
Duration
UX Researcher
4 Months
Project Vision & Scope
Handshake is a job-hunting and career-building space with a mission to democratize access to opportunity: to help every student find the right job, no matter where they're from or who they know. It creates a centralized information hub for job opportunities, information sessions, seminars, and career fairs for students across the US.
​
I chose this website because the company’s core value resonates with me: putting students first and empathize deeply with students to increase career opportunities for all. With a focus on the student/recent graduate experience, I performed a usability study to get insights from job seekers' perspective with a goal to identify issues and improvement opportunities toward a better user experience.
Research Goal
In general, the research goal is to identify any usability problems with the Handshake interface by collecting qualitative data and assess user experience with the tools and features offered by Handshake to identify if users can query job-related information as expected. Specifically:
-
Discover where in the process people get stuck, feel confused, or need additional instruction.
-
Learn what factors cause such confusions.
-
Find opportunities to improve the website in order to streamline the user experience.
Research Questions
To achieve the research goal, I formulated several research questions:
-
Do students know how to use the filters and sort methods on the search page to help them narrow down the search range?
-
Are the current filters and sorting functions enough for users?
-
Is there any missing information on the job postings that is important to the student when applying for the position?
-
Is the event finding function intuitive to use?
-
Is it straightforward and intuitive to manage account preferences?
Methodology
To answer these research questions toward the ultimate research goal, I designed and implemented a remotely moderated usability study using video conferences. This study was performed in the US from November 21 - 25, 2022. Each individual testing session lasts about 40 minutes and participants were compensated with gift cards.
​
Each usability testing session starts with an introduction of the research goal and general instruction, a verbal statement to obtain participant consent, and a set of pre-test questions to document people's prior experience with online job hunting platforms (including but not limited to Handshake). I designed 4 tasks for participants during the usability test, each of which was followed by a post-task question to record users' perspective, comments, and suggestions. The usability testing concludes with a post-test questionnaire (System Usability Scale).
Tasks
Task 1
Please find full-time jobs of your interest that support international students and locate in your city. After that, try to find the job with the most recent posting date.
​
Task 2
Please try to apply a job of your interest that does not require you to apply externally (i.e., you can complete the application within Handshake). After that, try to send a message to the hiring manager.
​
Task 3
Please find job preparation-related seminars (virtual only) for your major that happens in this month.
​
Task 4
Finally, try to upload a cover letter and setting your transcript to be visible to the public. You can use the prepared dummy cover letter and transcript files if you want.
​
Recruitment and Participants
Participant Selection Criteria
-
Age: 18 to 30 years old
-
Gender: three males, three females
-
Roles: students studying in universities within the US; alumni recently graduated from US universities;
-
Having experience on finding jobs online
-
Having prior experience with Handshake
For participant recruitment, I chose the “personal contacts” method and used an online screener and identify 6 participants from my network who not only satisfy all five criteria listed above but also represent users from a diverse geological location, educational and professional backgrounds. For example, the participants include university students and recent graduates from multiple majors (Information Science, Mechanical Engineering, Computer Science, Biomedical Engineering) in 6 universities (University of Wisconsin-Madison, Carnegie Mellon University, University of Pittsburgh, University of Washington, Georgia Institute of Technology, Rochester Institute of Technology) who are located across multiple states (annotated map below).
Key Findings
​4 participants were confused by the ‘Distance control bar’ and spent a long time on the distance filter section. In the end, they all completed this task.
I don’t know if I should type the location first or drag the distance bar first. And I had no idea where this distance is based on.
3 participants spent more than 10 seconds finding jobs that support international students. They did not know how to set this condition at first, then they tried to look around and click through all filters. In the end, they found the ‘work authorization’ filter to
complete this task.
I think more filters should be displayed on this filter bar which has empty space now.
3 participants spent more than 30 seconds finding a job that does not require applying externally (i.e., completing the application within Handshake). They had to click jobs one by one to find a job that can be applied for within Handshake. In the end, they all completed this task.
There should be a filter for ‘apply internally / externally.”
4 participants spent more than 1 minute finding job preparation-related seminars. 2 participants failed to find job preparation-related seminars: they
tried to find information in the ‘More filter’ but failed. Finally, four participants found correct information using the ‘Category’ filter.
The label is not clear, so it is hard to find the correct information.
The ‘Category’ filter should be included in the ‘More filters’.
6 participants spent more than 10 seconds finding the ‘virtual/ In-person’ filter. They all felt ‘Medium’ is confusing, but they all finished this section of the task
despite the confusing label.
Medium is a confusing and poor label; the label should be more understandable.
4 participants spent more time finding the return button after uploading the cover letter. They had to click ‘My profile’ to return to the last page. In the end, they all completed this section of the task.
Where is the return button? The return button is missing.
2 participants felt confused by two repetitive ‘Add New Document’ buttons.
I don’t think two same buttons are necessary.
Design Implications
-
Remove the distance control bar to make the location filter easier for users to understand.
-
More filters can be displayed on the filter bar instead of using a drop-down window.
-
Offer a filter for ‘apply internally/externally’ to help users quickly access the information they want.
-
Create a way to encourage employers to offer hiring manager contacts, e.g., telling the employers why offering contacts could help them attract more qualified applicants.
-
The ‘Category’ filter can be included in the ‘More filters’, which fits users’ habits better.
-
Change the ‘Medium’ label to ‘Virtual/ In-person’ to make the label more understandable for users.
-
Offer a return button on the document submission page.
-
Remove the redundant ‘Add New Document’ button.
Takeaways
Impact
​
The proposed modifications have the potential to improve the user experience of all Handshake users.
What I learned
​
In this project, I learned how to design user research, conduct usability testing, and to analyze research data and turn them into actionable insights to guide subsequent design iteration.
Next Steps
Moving forward, the current Handshake interface can be redesigned based on the design implications outlined above.​