Optimizing the Screening Process

How McGill University identified top student talent using Machine Learning and I/O Psychology

nugget.ai
McGill University
mus

Redefine Talent Screening

McGill University’s Case League turned to talent screening technology firm nugget.ai to examine how the platform could be leveraged to screen candidates more effectively and efficiently.

The new program at McGill University is in its second year of operation, and provides university students the opportunity to develop inaugural case writting skills focused around developing various skills of business leadership. Last year, Case League selected program registrants after manually reviewing short answer questions and an in-person interview. This year, they challenged conventional norms by leveraging the power of Artifical Intelligence to automate the screening process.
McGill university quote

The Process

In August 2018, scoping work began between McGill’s team and Nugget.

Together, the teams co-created an open-ended challenge for students to complete. The challenge was structured similarly to classical business cases and inspired heavily by real organizational problems. Days later, the assessment was ready for launch to 120 student applicants.
McGill process
McGill process

15,120

DATAPOINTS

57

MALE

63

FEMALE
McGill process

01

Collecting Data

In September 2018, Nugget’s team sent the challenge to 120 students who would later compete for 1 of 40 seats in this year’s Case League program based on their performance. Over 126 features were tracked in the model to assess candidate performance, from recognizing patterns in writing to quality assurance.

02

Developing the Model

Each candidate is labelled green, yellow, or red by the Case League team after careful judgement suggesting their decision to keep or disqualify the applicant - yellow represented an undecided decision. Afterwards, Nugget’s model was trained to predict the top performer (green) and furthermore to examine characteristics essential for top performance. We honed in their characteristics through a rigorous process such that the predictions and inferences are generalizable and replicable.

03

Analyzing the Results

Lots of experimentation went into identifying the best-fitted model to serve the right purpose. For McGill it was essential to deliver a model on accurate results - choosing the wrong candidate can have drastic effects on the program. Nugget’s model is measured according to three aspects: performance, diversity, and bias.
McGill process

Numbers that drive results

Nugget was able to deliver a model that instantly performs. Data-driven decision making helps Nugget understand the leading indicators of success. This is a model built for performance, closes the gap on diversity, and eliminates bias.

91%

accuracy predicting candidate category

"We identified three types of top performers, all scored according to five key dimensions"

3

unique groups of top performers

“Measured against McGill’s results, we were able to reach a high level of accuracy predicting candidate to group”

17 : 20

top males : top females

“We reviewed the ratio of male to female applicants to top performers and found a consistent report in the outputs. Our model did not favor any sex”

The Feature Space

Colors denote the identified label and the shapes represent different overal types of performance

McGill student results

Colors denote the identified label and the shapes represent different overal types of performance

The Five Dimensions of Performance

Every data set contains its own characteristics. Specific to McGill’s students assessed, Nugget built a model that represents students’ DNA through five key dimensions

Promptitude and Reflectiveness

You act quickly but you’re also quick to make revisions and have great attention to detail to ensure accuracy

Clarity and Reasoning

You use transitionnwords to summarize information and prefer to make meaningful connections to communicate your thought process

Planning and Organization

You’re a planner and careful writer. You use your mind before excercising judgement. You chose your language carefully

Neutrality and Tactfulness

You maintain neutrality and tone and prefer to take your time and think critically before communicating

Meticulousness and Assertiveness

Your thoughts carry with you for long periods of time as you have strong opinions on things. You’re confident in your abilility and strong in tone.

Telling a Story with Data

The Predictive Model

Model construction with features of human behaviors requires adjustment for the measurement errors, attention to the predictive goals, and consideration of the ethical responsibilities.

For McGill, the predictive goal weighed heavy both for identifying the right candidates and against selecting a wrong candidate. More importantly, for Nugget, we understand that our machine intelligence ultimately applies on matters concerning human beings. So Nugget aimed to select a model that promoted equality and diversity.

The Exploratory Model

Apart from constructing a goal- and outcome- oriented predictive model, Nugget also provided explanations for the predictions and additional insights from the data.

Nugget identified key characteristics that could explain why a candidate would be more suitable than others. From this, our model revealed insights about what make a good candidate that may have not been apparent in human judgments. Our model was also able to group candidates based on the similarities and dissimilarities between candidates. Further analyses was able to show how different groups can relate to a candidate’s suitability for the program.

Building a Continuous Feedback Loop

Nugget can enhance the model’s performance by combining human decision-making with Machine Learning.

This implementation can have significant effects to improve the model’s performance along dimensions of performance and diversity. Over time, Nugget will be able to vizualize a continuous model that adapts to human behavior.

“We’re belivers that human and machine interaction result in better outcomes.”
McGill university quote

About nugget.ai

We help companies discover nuggets of information about their employees and candidates. We use AI and I/O Psychology to drive powerful insights about talent. Our cloud-based system integrates directly into human capital management and applicant tracking systems to enable companies to transform their data strategies.

185 Spadina Avenue, Toronto, ON, CA, M5T 2C6

Visit us at nugget.ai or email us at hello@nugget.ai

nugget.ai logo