Talent leaders should approach AI-powered hiring tools with skepticism, warns one author

William Shakespheare once wrote “all the world’s a stage,” and for recruiters today, artificial intelligence seems to be at the center of that stage.

In The Algorithm: How AI Decides Who Gets Hired, Monitored, Promoted, and Fired and Why We Need to Fight Back Now, Hilke Schellmann, an author and New York University journalism professor, explores how AI has impacted hiring decisions and why talent leaders should be more skeptical when using the tech.

Schellmann told HR Brew that she hopes her book will help HR pros and company leaders make smarter decisions when using AI in hiring.

This interview has been edited for length and clarity.

What inspired you to write this book?

I guess I got interested in looking at how AI is used in the hiring process years ago, so it was in the fall of 2017. I was at a conference in Washington, DC, that actually had nothing to do with AI…I needed a ride from the conference to Union Station to take a train back to New York, and I talked to [the] Lyft driver…and I asked them, “How was your day?” They said, “Yeah, it was kind of weird…I had to interview with a robot for a job.” They kept calling it a robot, but [it was] some sort of pre-recorded voice phone call that they received asking them three questions [about] why they wanted the job [and] their strengths and weaknesses. This was in late 2017, and I had never heard of this.

A few months later, I was at an AI ethics conference, and someone who had just left the Equal Employment Opportunity Commission was giving a talk, and they said that they've seen very basic predictive analytics AI tools, those who impose emails and calendars, figure out how many days or hours folks are absent, and that they’re afraid this could be discriminatory to mothers and people with disabilities. People need to look into this. At that point, I remembered that the friendly Lyft driver had an interview with a robot, and I started looking into this.

I was utterly fascinated, and I could tell that at the conference and then following conferences I went to that this is really changing HR, and a lot of companies are buying these technologies. I didn’t hear a lot about this in the space…and seems to be developing really fast…A job helps me put food on the table, a roof over my head and my families’ heads. For a lot of us, it’s tied to our identity, so I think it does matter [what] tools companies use for work.

What do you hope people leaders learn from your book?

What I’ve uncovered is that [AI] saves a lot of money, saves a lot of labor, for a lot of companies, and it filters down the [talent] pool to fewer people than maybe the thousands that apply. But I have not seen much evidence that these tools actually pick the most qualified [candidates,] so I think what I want folks to take away from it is asking really skeptical questions about the accuracy of these tools, how these tools are trained, [and] what’s under the hood.

I found out from the whistleblowers, who shared with me, who get called in when large companies want to use a new tool, they have sort of a pilot phase, and they bring in lawyers and outside folks to do their due diligence and vet these tools, so a couple of folks I’ve spoken to share it with me that, for example, an online résumé screeners [use] discriminatory keywords. One was like when folks who had baseball on their résumé get more points and folks who had softball on their résumé get less points. In the United States, most of the time, men have a preference for baseball, and maybe put that on the résumé, and a lot of women play softball, in general, so if that was on their résumé, [women] got downgraded and men got upgraded. But it had nothing to do with the job right. This wasn’t formed [for] specifically a baseball coach. The job was just the job that has nothing to do with sports. But predictive AI tools use quantitative analysis to figure out what are the statistical patterns, and these showed up as statistical patterns.

[Be] skeptical, just using basic testing methods to look at these tools and asking vendors to open the black box and show them like what are actually the keywords that an online résumé screener predicts upon.

How do you recommend HR approach AI in hiring?

I do feel that AI technology is here to stay, and I’m actually totally not against it. I’m just trying to say like we have biases that we have in society in our previous hiring practices, human hiring practices, [and] we don’t want that bias to seep back into [how] we are digitizing the hiring process.

[Companies] want to have the tool that actually works and picks the most qualified candidates, because, otherwise, you spent millions of dollars, and you have an elaborate random number generator that you can have for free, it’s not discriminatory, and anyone can do that. You don’t need a very expensive tool for that.