Building Your Team #5: How to Make Reference Checks Less Terrible (Classic)

What's up, everyone. This is Alex Lieberman, co-founder and Executive Chairman of Morning Brew. Welcome back to Founder’s Journal, my personal audio diary, where I give you the business builder, the tools you need to think better in order to build better, whether that's building a business, a team, or a new product. It is time for our second ever miniseries of Founder’s Journal. Here's how it works. This week, we are focused on one crucial topic in business that is important to builders, founders, and all professionals, and we'll be serving you double the content to make the topic stick. The topic of this miniseries is Building Your Team.And today's episode is all about How to Make Reference Checks—yes, reference checks—actually useful. Let's hop into it. 

The problems with traditional reference checks

So in life and in the world of business, there are a lot of things that we do just because they've always been done in a certain way. We don't question it. We just do it on autopilot. And it just seems like the type of thing that you should do, because it's always been done. It's exactly why Morning Brew was originally created, because people felt like they just should read the Wall Street Journal. They've been told to read the Wall Street Journal. There isn't an alternative. And so they don't question it, they just do it. And in business, nothing feels more applicable to this category of just doing things for the sake of doing things than reference checks. I promise you, this is worth your time. Reference checks seem unsexy, but they are so incredibly important in business. Basically what happens is you have someone late in an interview process for a role. They're one of your top two candidates, and you want to talk to past coworkers or bosses or direct reports of theirs to feel settled in the decision you're making. But the issue that I have found is that these conversations are historically useless and they're flawed from the get-go.

This is typically how things go down. A candidate makes it to reference checks, meaning they're one of your last three candidates left in an interview process. You ask this candidate for references. The candidate gives you three to four references, and then you set up 20 to 30 minute calls with those three to four references. And a hundred percent of the time you end up speaking to a self-selected group of people that the potential employee knows will speak super positively about the relationship with them. And so how it goes is you usually get onto the call with a reference check. You ask the individual on the call the same type of question of, how was it working with so-and-so? How was it working for them? How is it working with them? What was their biggest challenge? Why do you think they're a good fit for the role? And what ends up happening is you leave these conversations with nothing but a script by a self-selected cheerleader of the candidate. 

And it makes sense when you step back and think about it, life is all about incentives. As we've talked about countless times on Founder’s Journal, it is the incentive of a candidate to select references that will make them sound as great as humanly possible, given their incentive is to get the job that they want to get. This way there's no question or concern in a new hiring manager's mind. So what ends up happening is all of these references answer your questions with some version of XYZ person was great and couldn't have been better in their job.

They were a great leader. They were communicative. They were easy to work with. They were a diligent worker. Give them the space to succeed and they'll be exceptional. And I would hire so-and-so anywhere I go in my career and the list goes on. And so what ends up happening is all of us that are interviewing a candidate and end up having these reference calls, we leave the conversation feeling like we felt after the last reference conversation, which is you hang up the phone and you're like, I learned absolutely nothing about this person, nothing more than what I knew about them 20 minutes prior. And then you start asking yourselves, if you kind of look introspectively, you're like, hmm, I've had four 30-minute conversations. I've spent two hours talking to people that taught me nothing about a candidate I'm looking to hire. Why did I just do that? 

3 intelligent ways to make this process valuable

So what I thought about was, how do you make the reference check process better? And also it's worth asking ourselves, is it even worth it just because this thing has always been done, and why should we do it? My short answer is:I do believe reference checks are worth it. I think it is worth understanding from the right people and through the right questions if someone you're thinking about hiring is a great person to work for or with. And so I think what needs to happen is the reference check needs to be reimagined both in terms of who you talk to and the questions you choose to ask.

So let's start with, first, who do you talk to? My view is you need to stack the deck in your favor and be more intentional about who you ask the candidate to speak with. So for example, let's say you're hiring someone that will be in some sort of managerial role. They'll have direct reports. You better damn make sure in your reference checks that you're asking to speak to at least one, but ideally two direct reports of this candidate in the past. You want to talk to the people who were managed on a weekly basis by this person who hopefully were developed as professionals by this person. If you neglect to do that, how are you going to know if they're a good manager?

The second thing you should do is be intentional about when this candidate worked with their reference. My rule of thumb is you should never talk to references that worked with a candidate more than three years ago. Three years ago in the world we live in, that candidate could have been a totally different professional doing a totally different job. And I've had a number of interviews where I asked a candidate for references, and they literally gave someone that they worked with when they were a junior employee seven years ago. That does absolutely nothing. And it's just a waste of your 20 to 30 minutes and the reference’s. The third thing I would think about when asking a candidate for references in the beginning, is after the candidate provides their three to four top references that you're inevitably going to talk to, you should follow up with another email by asking for a reference that is not in their typical top five, just to get a fresh perspective. 

What you're looking for by doing this is someone who doesn't have the go-to talk track about a candidate that they've used in several past reference calls, but it's also a great test for depth. What I mean by that is depth of how many people in this candidate’s career feel super positive about them, and is it more than just three or four people? So that's on the who side, who should you talk to to have the most effective reference check. Now, I want to talk about what you should actually ask in these conversations, but first, a quick break.

Alex’s best questions for reference checks

Let's hop back into it. Now on the what side, I think you need to be really intentional about what you ask in these conversations now that you're talking to the right people. So I've got a few go-to questions that I've found to be really difficult for a reference call to answer, which to me means that there's always going to be interesting contexts and reactions that you can glean insight from. The first question that I ask in every single reference call is the following. Let's say, you've worked with 200 people in your career. If you visualize those 200 people up on a wall and they're lined up from best person you've ever worked with to worst person you've ever worked with, where would you put this candidate in the lineup of 200?

And I think this is such a valuable question, because what you've basically done is take the conversation out of the silo of just talking about a single candidate and basically played on a reference’s own mortality, where they now need to compare this candidate to other people. And they want to speak truthfully about other people that they feel really positive about in their career. What I've found to be so far, a great predictor of quality of professional or quality of person we hire is where someone says the candidate ends up in a lineup. Ten times out of 10, when references say that a candidate was top three in this list of 200 employees they've ever worked with, it's a very good sign.

Everyone who has received answers like that has gone on to be a great employee at our company. When references ended up saying, you know, so-and-so is top 10 in my career, it has gone the exact opposite direction. And obviously a lot of variables at play, but I've just found this really interesting relationship in where people put a candidate in their lineup. The second question, which actually was recommended by Brian Sugar, who's another founder and he shared this question on Twitter, is if so-and-so gets fired from the business six months from now, what was the reason? And what I also do, I don't just ask this question, but I make sure that it's clear to the reference that they can't just talk about something the company did. So what I say to them is let's assume here that it wasn't the company's fault. What was it about this candidate that potentially just didn't make them a fit? And I force them to talk about potential areas of development or challenges in this specific hire that the reference has experienced in their own career. I don't want people coming back to me and saying, if you micromanage them and you didn't give them the autonomy to succeed, that is why they failed in the job. I don't want to learn what the company could have done, I want to learn what the candidate failed to do if they join the company.

Listen for this potential red flag

Finally, I'm going to mention one trend that I've seen in references similar to the first lineup question that I shared with you. I believe that if you don't receive like 10 out of 10 reviews from all references, it's a flag.

All of our best hires receive 10 out of 10 reviews from every single reference. And then some of our not-so-great hires who ended up not being at the company for more than six or nine months, they got what I would call A-minus reviews in the reference checks, reviews where the person said so-and-so is great, I really liked them, but you could tell the passion in their voice wasn't like, this is a top-three person that I've ever worked with in my career, and I will work with them wherever I go. Remember, these are supposed to be the three or four people that are the biggest champions of a potential hire. If they're not all giving A-plus reviews, you have to start to question why that is. And with that, I want to just share one last takeaway from this entire experience of thinking about reference checks.

We always, in life and in business, need to question why we do things and we need to make sure that we are reimagining the traditional ways or status quo of how things have done in the past. And we should never feel like we’re just doing things for the sake of doing things and just going through the motions because other people have gone through the motions. My hope from this episode is not just that you have an elevated way of thinking about reference checks, but also that you can see that reference checks are just one small example in business of just doing things because that's how they've always been done. 

And now I want to hear from you, how do you get value out of reference checks? Or do you share the view that I talked about earlier, where reference checks are broken and we shouldn't be doing them at all? I would love to hear from you. So send an email to alex@morningbrew.com or DM me on Twitter @businessbarista with any thoughts or questions you have about this reference checks topic. And finally, just a reminder, this is the second ever miniseries for Founder’s Journal, six episodes—two from Monday, two from Wednesday, and two today—all about how to build an A-plus team. Make sure to check out the second episode today and check out the previous episodes if you haven't listened to the whole series yet. Thanks so much for listening and I'll catch you next episode.