How Machine Learning Can Create a More Meritocratic, Less Biased Job Market Machine learning-based hiring corrects a traditional process that has prioritized status over skills and background over ability.

The traditional hiring process is failing candidates, companies and recruiters. It revolves around human interpretation of complex data that is too susceptible to prejudices and mental shortcuts.

Related: Artificial Intelligence Is Likely to Make a Career in Finance, Medicine or Law a Lot Less Lucrative

A hiring process assisted by machine learning could eliminate systemic biases that put social status over skill. However, in recent news, algorithms have failed at that task. A resume screening algorithm developed at Amazon became biased against female applicants. Another such algorithm favored candidates named Jared who played lacrosse.

If we apply machine learning to resumes -- the same data source used by humans in the hiring process today -- we will perpetuate biases. To reduce the biases in hiring, we must apply machine learning to new sources of data that are more objective and too complex for a human being to review.

Skills tests could provide that new source of data. In the first step of hiring, a machine learning model could analyze the skills test instead of a resume to make more objective hiring decisions.

I'd like to discuss why we all should prefer that machine learning-based approach to the current system.

A failing process

Here in Silicon Valley, hiring managers face two interrelated problems that make hiring engineers hard. The first problem is that demand for talented people with technical skills far exceeds the supply. The second issue is the barrage of applications to each open job.

Hiring managers often receive hundreds of applications for one open slot. To filter out noise, they rely on shortcuts like prioritizing the applicants from the most famous colleges and companies. However, Stanford, MIT and the Ivy League colleges won't triple their class sizes to meet this demand. They're highly selective institutions, not businesses.

Related: Artificial Intelligence Can Help Leaders Make Better Decisions Faster

Despite the talent crunch, hiring managers ignore unconventional candidates who might be more skilled than those from elite backgrounds. They know some of these candidates will be good, but it takes too much time and money to find them when humans must do all the screening.

The demand for engineers continues to increase as "software eats the world," and every company must learn how to build its own software to stay competitive. Industrial firms, consumer goods brands and financial institutions are all hiring software engineers. Meanwhile, the established technology giants like Google, Facebook and Amazon continue to expand their ambitions and build new products that increases their demand for technical talent still further.

The result: Traditional hiring is failing. Humans reviewing resumes simply can't identify technically skilled candidates fast enough to meet the demand. However, a hiring process based on machine learning could address the problems I've presented. By using skills tests in which machine learning algorithms analyze scores and match candidates to open jobs, companies can identify and hire skilled candidates quickly. For several reasons, machine learning-based hiring offers a massive improvement over traditional hiring.

1. Reading resumes is inefficient. If you need to review 100 resumes, you have no choice but to race through them using basic heuristics like looking for particular colleges or companies. Likewise, if you tried having humans administer and grade skills tests, you'd be busy reviewing that data while your competitors were making offers.

Related: Big Data Combined With Machine Learning Helps Businesses Make Much Smarter Decisions

2. Human hiring is biased. Numerous studies have shown systemic bias in hiring. Interviewers consciously and unconsciously discriminate based on gender, race, emotion, weight and even the sound of a person's voice. Reviewers filter out skilled candidates based on "cultural fit," a concept that, while important for successful hiring, is rarely thought through well or articulated as a hiring criteria. A hiring process guided by machine learning analysis of skills data can eliminate many prejudices, at least until the final in-person interviews.

3. Humans judge the wrong things. If an engineer sounds odd during a phone interview, who cares? Engineers are hired to build products, not sell them. Unfortunately, the context of the interview -- speaking on the phone -- draws the interviewers away from what should be assessed: analytical and problem-solving skills. Machine learning counterbalances these misjudgments because it doesn't have any social and cultural awareness (which is an attribute in this case).

4. Anyone can participate. Firms invest their hiring efforts in top colleges because a broader scope is too expensive if human beings do the work. Machine learning can screen candidates at any scale. If you want the world's best data scientists, invite the world to apply. Machine learning welcomes applicants from Africa, Asia and the Middle East to join a hiring process that otherwise favors elites from a few schools.

5. Recruiters have more valuable things to do. Recruiters represent the culture, values and missions of their firms. Reading resumes is not what makes the job rewarding. The best recruiters I know enjoy building relationships with candidates and giving people the opportunities of their dreams. Assisted by machine learning, a recruiter can focus more on this humanistic side -- the side that machine learning cannot do or appreciate.

Related: This AI Tool Helps Companies Eliminate Hiring Bias in the Name of Productivity

Justice in hiring

Machine learning-based hiring corrects a traditional process that has prioritized status over skills and background over ability. But, not all candidates will benefit from fairer hiring.

People who coasted through an Ivy League education and expected to have the red carpet rolled out are going to suffer. Universities that treat their students like guests at a luxury resort -- and optimize for "user satisfaction" -- will see a dip in their hiring statistics. Colleges will have to provide educations, not just marketable status symbols (aka diplomas).

On the other hand, unconventional candidates -- the self-taught coders, online students and go-getters committed to changing their lives -- will have their moment. Rather than dehumanize hiring, as many people fear, machine learning will help businesses live up to their values and actually hire the best people.