Optimizing the Screening Process
How we helped McGill University identify top students using Machine Learning and I/O Science
Redefine Talent Screening
McGill University’s Case League turned to talent screening technology firm nugget.ai to examine how the platform could be leveraged to screen candidates more effectively and efficiently.
The new program at McGill University is in its second year of operation, and provides university students the opportunity to develop inaugural case writting skills focused around developing various skills of business leadership. Last year, Case League selected program registrants after manually reviewing short answer questions and an in-person interview. This year, they challenged conventional norms by leveraging the power of Artifical Intelligence to automate the screening process.
Nugget was helpful in weeding out applicants, this saved us a lot of time as it's one of our most time consuming steps in the recruitment process.
Student Lead, McGill University Case League
In August 2018, scoping work began between McGill’s team and Nugget.
Together, the teams co-created an open-ended challenge for students to complete. The challenge was structured similarly to classical business cases and inspired heavily by real organizational problems. Days later, the assessment was ready for launch to 120 student applicants.
In September 2018, Nugget’s team sent the challenge to 120 students who would later compete for 1 of 40 seats in this year’s Case League program based on their performance. Over 126 features were tracked in the model to assess candidate performance, from recognizing patterns in writing to quality assurance.
Developing the Model
Each candidate is labelled green, yellow, or red by the Case League team after careful judgement suggesting their decision to keep or disqualify the applicant - yellow represented an undecided decision. Afterwards, Nugget’s model was trained to predict the top performer (green) and furthermore to examine characteristics essential for top performance. We honed in their characteristics through a rigorous process such that the predictions and inferences are generalizable and replicable.
Analyzing the Results
Lots of experimentation went into identifying the best-fitted model to serve the right purpose. For McGill it was essential to deliver a model on accurate results - choosing the wrong candidate can have drastic effects on the program. Nugget’s model is measured according to three aspects: performance, diversity, and bias.
Numbers that drive results
Nugget was able to deliver a model that instantly performs. Data-driven decision making helps Nugget understand the leading indicators of success. This is a model built for performance, closes the gap on diversity, and eliminates bias.
accuracy predicting candidate category
"We identified three types of top performers, all scored according to five key dimensions"
unique groups of top performers
“Measured against McGill’s results, we were able to reach a high level of accuracy predicting candidate to group”
17 : 20
top males : top females
“We reviewed the ratio of male to female applicants to top performers and found a consistent report in the outputs”
The Feature Space
Colors denote the identified label and the shapes represent different overall types of performance
The Five Dimensions of Performance
Every data set contains its own characteristics. Specific to McGill’s students assessed, Nugget built a model that represents students’ DNA through five key dimensions
Promptitude and Reflectiveness
You act quickly but you’re also quick to make revisions and have great attention to detail to ensure accuracy
Planning and Organization
You’re a planner and careful writer. You use your mind before exercising judgement. You chose your language carefully
Clarity and Reasoning
You use transition words to summarize information and prefer to make meaningful connections to communicate your thought process
Neutrality and Tactfulness
You maintain neutrality and tone and prefer to take your time and think critically before communicating
Meticulousness and Assertiveness
Your thoughts carry with you for long periods of time as you have strong opinions on things. You’re confident in your ability and strong in tone.
Telling a Story with Data
The Predictive Model
Model construction with features of human behaviors requires adjustment for the measurement errors, attention to the predictive goals, and consideration of the ethical responsibilities.
For McGill, the predictive goal weighed heavy both for identifying the right candidates and against selecting a wrong candidate. More importantly, for Nugget, we understand that our machine intelligence ultimately applies on matters concerning human beings. So Nugget aimed to select a model that promoted equality and diversity.
The Exploratory Model
Apart from constructing a goal- and outcome- oriented predictive model, Nugget also provided explanations for the predictions and additional insights from the data.
Nugget identified key characteristics that could explain why a candidate would be more suitable than others. From this, our model revealed insights about what make a good candidate that may have not been apparent in human judgments. Our model was also able to group candidates based on the similarities and dissimilarities between candidates. Further analyses was able to show how different groups can relate to a candidate’s suitability for the program.