I think you may be mis-representing what I am saying. You can generally sense if someone could be a possibly good fit in 20 minutes of conversation. The ROI of an additional 500 minutes of conversation doesn't change the error bars. They could be great leetcoders, but not a good colleague. Hopefully the first 20 minutes gave you a sense of their soft skills, but even those can be mis-represented in small bursts that are un-representative of how the individual typically communicates on the job.
You're saying I am saying we should randomly give jobs. I am not. I am saying it should be, at max, a 1 hour phone call. Background checks can verify past employment and job titles. Just talk to them about work matters. Ask a fizzbuzz if you must. No additional data will meaningfully tell you much more anyways.
> You can generally sense if someone could be a possibly good fit in 20 minutes of conversation.
The last position I hired for had 700 applicants. Paper filtered down to 75. Interview screened down to 30. Detailed interviews with 5.
How would you propose this work? 30 people seemed like a possibly good fit after 20 minutes. Should we randomly hire one? Should we hire all and have them fight it out madmax thunder dome style?
I think if the position has low startup time and is assembly line work or something we could hire all 30 and fire the poor performers. Outside of tech support, I’m not aware of any positions like this.
I’ve worked with managers who thought programmers were like this and would hire 10 or 20 at a time to see who stuck. It was very frustrating as an existing employee because they all had to be trained, reviewed, etc. After a year of arguing, I (and every other senior, over about two years) left that company.
I think I need to update my mental models around the tech job market here, and what the interview process is trying to achieve. To take this example further: of the 700 initial applicants, how many would be _adequate_ for the position? You had the opportunity to have quite detailed interviews from the pick of the bunch - is the goal to find the very best of the lot because you can afford to be picky; or there are genuinely only 3-4 in the 700 who could _adequately_ do the job?
My preexisting mental model is that from the 700 applicants, 30 could have filled the position adequately, with perhaps a ~5% gain across them. Each additional step in the interview process yields increasingly diminishing returns but is still worth it because ... you can choose to be picky in this market. The risks associated with a mishire are also presumably greatly reduced with each diminishingly selective step. Or is it that there genuinely exists only 2-3 people in that pool who are a good fit for the role, and you have little choice but to engage in kissing a whole load of frogs to find your prince?
I’m not sure how many could adequately do the job. Generally because the phone screen doesn’t give a good sense of this so the 30 I talked with maybe 5-10 of them could potentially do the job adequately.
We only extended an offer to one person and maybe a second if the first declined. If both of those two declined then we wouldn’t have offered to the other 3 detail interviewees.
I’d certainly like a better way that uses less time. But it does seem that I need to kiss a lot of frogs.
Also, the goal isn’t to get an “adequate” person but to get a really good or great person. I think that a good person can do 5x more than an ok person, maybe even more. So for this position, I’d rather keep looking than just get a body.
Again, if I just needed assembly line workers or warm bodies that could bill in a consultant sweatshop, that’s a different story. But I don’t want to be in such a hiring position.
>I think that a good person can do 5x more than an ok person, maybe even more
This is really surprising to me, and I'm not sure how this can be the case. I would imagine that this would be true of a high level position at the top of their game, able to define their own schedules & goals. For a typical worker with clearly defined short & long term objectives defined by the organization, how does one become 5X more productive? Clearly my heuristics about the tech industry are way out of step with reality.
However, I could see something like this happening in my own field: academia. As a bioinformatics postdoc, the bread & butter work most of my colleagues do is routine, so they can hit their goals fairly predictably. My project, in comparison, is mostly ad-hoc, struggling to parse a novel dataset right on the edge of what is technically possible. My productivity is in the toilet, and I can imagine a different researcher in my position being 5-10x more productive. This is not how I imagined the tech industry working, though.
I used to do a lot of programming and the multiples are really high, especially since bad programmers can have negative productivity (kind of like Hammerstein-Equord’s stupid and industrious quadrant).
I remember a McKinsey study from the 80s or 90s that talked about 10x productivity but I can’t find it so maybe I’m misremembering. I don’t think people who actually try to measure this succeed well. I run away from anyone trying to precisely measure programmer productivity because it usually means some point haired boss will try to optimize on lines of code or something stupid.
So for me, it’s a bit of a hunch but a strong hunch. I’ve worked in shops of 100 devs where a single dev wrote the entire authentication stack that teams couldn’t handle. And I have lots more stories like this.
Nowadays I do “strategy” and I’d say the multiple is more than 10x in that there’s usually some magic mix to a good strategy that can’t be accomplished with giant committees and tons of hours. When it comes to creative tasks, I think the productivity leap from “ok” to “good” and from “good” to “super” is really high. Not every job is like this, but I think these are the most fun and so try to go towards them and away from commodity work.
I have limited experience in academia but have seen authors quickly crank out really useful papers where teams have been working for months. I suspect that’s more about just having good alignment of capability and need instead of some magic or intrinsic productivity power.
Personally my productivity is pretty sucky so I’m in a situation where I don’t meet my criteria but am lucky enough to get to work on cool stuff.
Hmm. The way I see benchwork scientists being 'productive' is an endless slog - your cultures grow at their own schedule, and don't respect the boundaries of the work-life balance. Of course, this isn't true for everybody, but was one of the reasons pushing me away from benchwork towards bioinformatics. Our work also tends to be much more insular, gnawing away for years on our own little piece of the problem. I suppose I'm personally at a fairly "creative" point in my career - I have absolute free reign in my study, and my PI encourages it.
If the additional interviewing time doesn't produce real improvement in outcomes over random hiring, then yes, don't waste the interviewees’ time and time the form is paying you for doing it.
My problem is that I don’t know of a good way to determine what improvement. I think it’s hard to design a study and measure.
And I think there’s a deontological vs. utilitarian aspect as I think if applicants learned I was randomly picking 1 of 30 pretty goods it would discourage high quality applicants.
For me, I think the additional 100 hours interviewing helps and is worth it.
I’d be interested in hearing of hiring managers and orgs who just hire randomly from minimally qualified applicants. I hear from lots of applicants who claim they are minimally qualified that this is a successful strategy. But, naturally, they are a bit biased and not very useful for deciding how to hire people.
In this and your previous post, you greatly under-appreciate the cost and disruption of a bad hire. Not to mention the time lost bringing a new hire up-to-speed on the company's way of doing things. A bad hire can be very expensive and disruptive.
If a few more hours of interviews can avoid a bad hire, I think almost all experienced managers would opt for the extra interviews.
If your work culture allowed you to remove bad hires, then the company would not experience the 'cost and disruption'. The problem isn't the time invested getting employees up to speed: you'll do that with good or bad employees. The problem is, bad employees stay, and often for long periods of time. Have you experienced the bureaucratic nightmare of formally dismissing someone? It's something we all would like to avoid, but clearly today's filtering is ineffective if our organizations are filled with dead weight.
I've uncovered craziness in candidates late in the process when they are no longer guarded about what they're saying. So, for myself, it has indeed produced objectively better results. I expect that most seasoned managers have had similar experiences.
You're saying I am saying we should randomly give jobs. I am not. I am saying it should be, at max, a 1 hour phone call. Background checks can verify past employment and job titles. Just talk to them about work matters. Ask a fizzbuzz if you must. No additional data will meaningfully tell you much more anyways.