In This Episode
- At HubSpot, an AI tool ranks applicants against the skills the recruiter sets, but Steph McDonald still reviews every resume and often starts with the lowest-ranked ones.
- Hard requirements, like experience building payments products, can be set up as knockout questions that reject an application automatically.
- If you use ChatGPT on an application answer, paste only the answer. Steph can tell when the chatbot's follow-up question gets pasted in too.
- Auto-apply services often send candidates to the wrong roles, like product managers applying for design jobs, so Steph's advice is to skip them.
- Steph's take on AI in your applications: write the answer yourself first, then use AI to tighten it.
Episode Summary
Can a UX recruiter tell when you’ve used ChatGPT on your application? HubSpot recruiter Steph McDonald says yes, especially when the chatbot’s follow-up question gets pasted in too.
Steph is a lead recruiter at HubSpot with 25 years in recruiting, finding and vetting candidates in product management. HubSpot is an AI first company, so she uses it too. An AI notetaker sits in her interviews and saves her at least 30 minutes per candidate per day. A scoring tool ranks applicants against the skills she sets for each role. Every resume still gets a human look, and she starts with the lowest ranked ones to see if the bot got it right. Sometimes it didn’t.
Then there’s the pile from candidates. Auto apply services sending software engineers and product managers to design roles. A resume where a product manager became a lead designer and the job description showed up as the work experience. Four resumes in a row with the same unusual formatting.
Her rule for your own writing is short. Write it yourself first, then use ChatGPT to tighten it. She also says AI shouldn’t be the one deciding which roles you apply for. HubSpot wants people who are excited to use AI and build with it.
If AI is part of your UX job search right now, this UX recruiter interview is the view from her side of the screen.
This episode is a segment of my full interview with Steph. Listen to the full conversation here.
Frequently Asked Questions
01How do recruiters use AI to screen UX job applications?
02Can recruiters tell if you used ChatGPT on a job application?
03Should I use an auto-apply service for my job search?
04Why was my job application rejected automatically?
Show Notes & Links
- Episode 168: UX Hiring Insights: Steph McDonald, UX Design Recruiter at HubSpot, on AI in Hiring, Portfolios & What Gets You Hired
- Episode 116: How to optimize your resume for the ATS so you can get more UX job interviews
- Episode 143: UX Hiring Insights: Why It Can Take 12 Weeks to Fill UX Roles (and What to Do About It)
Episode Transcript
Steph McDonald [00:00:00]: So how I actually do it is I look at the candidates that are low ranked first, which I know sounds crazy, but I can typically like see why the bot, you know, ranked them there, or sometimes I disagree with the bot and move them forward. We know you’re using ChatGPT, but just be careful what you’re pasting into the box. Because sometimes it’ll be like, you know how ChatGPT like says, do you want me to like adjust this for this? And people will paste the entire thing. And then it’s like, OK, we can tell you didn’t even audit the response from AI.
Sarah Doody [00:01:10]: So today on the podcast, we have Steph McDonald joining us. Steph is a lead recruiter at HubSpot where she finds and vets amazing candidates in product management. And Steph has worked in recruiting for 25 years. So she’s seen a lot. She’s seen shifts in the job market, the ebbs and flows that happen all the time. And Steph is really passionate about making people on all sides of the hiring equation feel comfortable in scene. Maybe that’s a good segue to the whole topic of AI because I know it’s a big one that people want to hear about. So maybe we can
Steph McDonald [00:01:48]: Yeah.
Sarah Doody [00:01:49]: kind of like put a fork in our conversation and we could talk about the role that AI plays in you finding and vetting candidates and then maybe mistakes or benefits that candidates can see from using AI. So.
Steph McDonald [00:02:05]: Yeah, absolutely.
Sarah Doody [00:02:06]: How does AI impact your vetting and hiring process right now?
Steph McDonald [00:02:11]: Sure. So I’ll start by grounding everybody in the fact that we are an AI first company from a product perspective. So the expectation that our executive leadership has for us as an entire organization is that we’re using AI as part of our role. And I’m very lucky that the VP over the talent acquisition organization is very passionate about giving us tools to be able to use AI. as a support system for the recruiters to really focus us on high value work. So specifically, a couple of examples. When I conduct an interview with a candidate, there’s an AI companion in there with me taking notes for me. This is like one of those moments in someone’s career where it’s like the dream has come true. I’ve always had this vision of someone sitting next to me taking notes so I can just talk. And now I have that, but it’s a thought. And it’s not making any decisions. It’s just like taking the information that you share and putting it into a scorecard for me. That saves me at least 30 minutes per candidate per day. And so that’s a significant amount of time. And then when it comes to, I know everybody’s wondering about like application review and things like that. The only thing that we’re using right now for application review is a tool that helps us score the candidates based on information in their resume. Now that’s not to say that we’re not looking at every resume, we’re still reviewing those. But it allows me to say, okay, this is the criteria that I have for this particular role. These are the skills that I’m looking for, and it will rank those candidates for me. So how I actually do it is I look at the candidates that are low ranked first, which I know sounds crazy, but I can typically like see why the bot, you know, ranked them there, or sometimes I disagree with the bot and move them forward. So there’s still certainly that human element in the loop. And then spend more time on the candidates that are scoring higher. So while I am not a person who believes that you should put white labeled keywords all over your resume to try to get the bot to score you, because there’s still a human absolutely making the decision about your application at HubSpot. Will that change over time? I have no idea. I hope not. My least favorite part of my job, honestly, is looking at resumes. But it’s the part that I feel like I need to control to be able to keep the quality high to just be careful. There’s just parts of the process that I would not be super excited. I think there would need to be a lot more proof that it would do a better job than I could. So, yeah.
Sarah Doody [00:04:36]: one question that often comes up is the notion of, you know, candidates applying and then receiving automatic rejection, which based on all the conversations I’ve had in research I’ve done, that generally happens because there are certain criteria that are just deal breakers like state,
Steph McDonald [00:04:54]: Yes.
Sarah Doody [00:04:55]: right? So does that happen in whatever software HubSpot is using?
Steph McDonald [00:05:00]: Yeah, yes, it can. So we tend to use auto reject questions when there is like a hard requirement. Like say, for example, I’m hiring a product designer, a senior product designer, and I need them to have built, you know, payments products.
Sarah Doody [00:05:15]: Mm-hmm.
Steph McDonald [00:05:15]: I can ask that as a question and use it as a knockout. If someone says no, then they’re not going to be qualified for that position. We still have them in our database, which I love because we do use AI tools to find you again in our database, which is delightful. But you know, the the trying to, you know, there’s that constant battle of encouraging people to apply because we want to take a look at your resume. We want to be open and, you know, have a diverse candidate pool. But sometimes there are like hard requirements that need to be met. One thing that we don’t tend to do, I haven’t seen any job postings. I don’t want to say that, you know, emphatically across all roles that I’m confident, but I’m pretty confident is that we’re not using years of experience as a measure. Now, I will say if I’m looking for a senior designer and someone is right out of school, that’s a years of experience and kind of a filter that I can use, but that’s me making the decision, not AI or a bot or anything.
Sarah Doody [00:06:05]: Right, right. And then concerning how candidates are or maybe are not using AI, like what are your observations, you know, in the last two-ish years, right? What stands
Steph McDonald [00:06:18]: Yeah.
Sarah Doody [00:06:18]: out is in terms of do’s and don’ts maybe.
Steph McDonald [00:06:22]: Yeah, think so couple things come up. One, we ask application questions where there may be like a text based answer. We know you’re using ChatGPT, but just be careful what you’re pasting into the box. Because sometimes it’ll be like, you know how ChatGPT like says, do you want me to like adjust this for this? And people will paste the entire thing. And then it’s like, OK, we can tell you didn’t even audit the response from AI. And then the other trend that we’re seeing is there are services out there that will apply on your behalf. And on paper, that sounds like a great thing, right? Like I can just like set it and forget it. I’m going to suddenly have my calendar filled with interview interviews and recruiter calls. And that sounds perfect. But the problem is, is that it’s not doing a good job for you and it’s not curating your applications like you would do it yourself. So for example, I can tell when it’s a bot applying for a position because of the way that they answer the text-based questions that we put in there. That’s not a reason why we do that. It helps us screen in and out candidates faster, but it certainly is kind of a dead giveaway. And it very often is applying for completely the wrong discipline. So I’ll get software engineers applying for design roles and… product managers applying for design roles. And it just, that doesn’t serve you when it’s your brand that you’re trying to curate. And so I would encourage people to not use those tools. I’m sure I’ll probably get some messages from those companies, but at this point, the proof is not out there that this has been a successful use case for AI for me.
Sarah Doody [00:07:52]: Well, and learning about your hiring and vetting process at HubSpot, the idea of using AI to apply, it makes me think that it increases the chance that a candidate will not score well. Because if you’re hiring for that designer that has experience doing payment stuff and that candidate didn’t make sure that one of their bullet points or multiple resume bullet points mentioned payments because that’s what the job description said, then I’m guessing that’s impacting the score they’re getting. Is that a safe guess?
Steph McDonald [00:08:33]: Yeah, yeah. And there’s multiple ways within the product that we use to set the criteria. So for example, I can tell it that I’m looking for someone with these particular skills and it will rank based on that. Or I can tell it to look for keywords, whatever the criteria that the recruiter sets is what the scoring agent is going to do for us. And you’re right. not doing a good job of making sure that your resume is a strong match for the job that you’re applying for, which a bot isn’t going to do for you, or it’ll completely hallucinate and create a resume that doesn’t make any sense. I had one of those recently where, again, it was a product manager and it changed the resume to say lead designer because the bot was trying to apply that product manager to it. And so, they basically took the job description and put it as the… the work experience history. And it made so it made no sense. they’ve it was it was just literally nonsensical. So
Sarah Doody [00:09:27]: Yeah.
Steph McDonald [00:09:28]: yeah, having having a bot do the things that really you should own as a candidate or that I should own as a recruiter. You know, there’s a big risk when you start going outside of those boundaries of what AI is really capable and good at doing.
Sarah Doody [00:09:40]: Yeah. Okay. This will maybe be my last question about AI, but I’m curious when it comes to these candidates that you come across who have clearly used AI to like answer an open-ended question or something. What goes through your head? Like how does that impact the first impression you have about that candidate? Is it kind of like swipe left, they use AI, so I’m not even going to consider
Steph McDonald [00:10:03]: Yeah.
Sarah Doody [00:10:05]: them? Like what’s the threshold there?
Steph McDonald [00:10:08]: Yeah, I think that’s a discretionary thing. for example, if I were applying and just needed to, I had an idea of how I wanted to answer question and I put it through ChatGPT to kind of tighten it up a little bit, maybe make it a little shorter, because I tend to go on, that’s totally fair play. I think, again, it’s about AI being a tool, not being the candidate, right? So my preference, my advice is to, write these things yourself. And then if you feel like it needs some polish and some, you know, just tightening up, throw it through to chat GPT. I don’t mind that. But just be really careful when you’re pasting it in that you’re only pasting in the answer and not like the bot talk that happens behind
Sarah Doody [00:10:47]: Yeah.
Steph McDonald [00:10:47]: that. So yeah, I mean, you know, we want people that embrace AI that are excited to use these tools that are excited to build these tools. So yeah, that’s for us. There’s we don’t we don’t discourage people. We know people are going to use chat GPT or other tools to be able to really make their resume polished or whatever. That’s great. We don’t mind that at all. But it shouldn’t replace you in the process and your decision making process about what roles you wanna apply for.
Sarah Doody [00:11:13]: Right. Yeah. I can only imagine the weird applications you get from some bot just randomly applying and mixing up a software engineer with a designer or something like that.
Steph McDonald [00:11:26]: Yeah, another weird one is I literally had like four back-to-back resumes that I was reviewing. They may have all been fake, I’m not sure, but they had literally the same exact formatting and it was a very unique formatting until it wasn’t, you know? And so it just made me feel like either these are all the same person and they’re fake or again, bad bot behavior. So don’t do it. It’s not ready for that use case yet.
Sarah Doody [00:11:48]: Or is the bot tool like regenerating people’s resumes all using the same template or something? wow.
Steph McDonald [00:11:56]: Yes, I think so. Yeah. Yeah.
Sarah Doody [00:11:58]: Feels shady to me. Okay. Yes.
Steph McDonald [00:12:00]: And now, and people are paying a lot of money for this. It’s not working. it’s, it’s so, that’s why I’m such an advocate for sharing. Like, what is it like on my side to experience reviewing your resume when it’s not really maybe your resume? So.
Sarah Doody [00:12:13]: Yeah. Yeah. I’m, I’m often approached to promote, you know, resume tools and stuff like that. And one, a couple of years ago, it had a bunch of resume templates that had like skills graphs and tons of icons and graphics and like, multiple columns, which we know can trip up some of the applicant tracking systems. And so
Steph McDonald [00:12:35]: Yep.
Sarah Doody [00:12:35]: I told the person like, I can’t promote this and Also, why don’t you know this to begin with? Yeah.
Steph McDonald [00:12:41]: Right. Yeah, you’re supposed to be coming to this as experts, right? So I always wonder, do the companies that are making these products actually talk to recruiters?
Sarah Doody [00:12:49]: That would be
Steph McDonald [00:12:50]: Hahaha
Sarah Doody [00:12:50]: an interesting question. Yeah, I don’t know. That’s a podcast for another day.
