A monkey in nurse's scrubs reads Polya's 'How to Solve It' while a bear and pirate look on.

The other day Gábor Szabó asked an insightful question on Twitter. Here’s his actual phrasing:

Gábor thinks about this a lot. He runs a webinar series on recruiting from the company point of view.

This is a great question and it got some great answers. My particular favourite was by Salve Nilsen:

These are great answers, but they can use some expansion. Here are my own expansions, along with some other points I think are important:

(I don’t know Mr Nilsen. No guarantee he agrees with me on all this. I just liked his answer.)

But First: Role Mismatch

Before getting into his excellent points, I’d like to talk about role mismatch. One easy answer is to say “companies want to hire experienced senior engineers, but a lot of the folks having trouble are junior.” You could also say, “in many cases, companies are trying to hire for the hot new technology and the people having trouble don’t know it.”

That’s not wrong. It’s just not a full answer.

Why don’t companies want junior engineers? They turn into senior engineers over time. It can even happen pretty quickly with good training and mentoring. And some companies do this, but most don’t.

Why don’t companies hire good generalists in the hope of training them in the new hot technology? Most of these technologies aren’t that hard to learn, and an experienced generalist has often picked up ten, fifteen or more technologies. What’s one more? And again, some companies do this, but not most.

Technology Monocultures

Salve points out that companies often fixate hard on a language (e.g. Java) or a toolset (e.g. Rails) or stack (e.g. NodeJS and Mongo) and really only look for developers who specialise in that technology stack.

This is true. A lot of it is because technology advertising is organised around them. There are good web sites to get in touch with Java programmers or MongoDB programmers or NodeJS programmers because that’s a lot of how programmers gain new skills and ask questions. Even cross-cutting web sites (Reddit, StackOverflow) tend to organise into little sub-communities by technology (/r/ruby, MongoDB tag on SO).

In many cases it’s easy to pick up a technology if you know a similar one — Python and Ruby have a lot in common, for instance, as do Java and C#. But we have a strong tendency to organise along language lines, which makes it hard to find good near-matches, and companies are often very picky about candidates knowing exactly their technical stack.

Narrow Recruiters Networks

Related to the above: a given recruiter is probably only looking in certain places for candidates. And a recruiter may well think a specific hiring manager will only look at certain candidates — most frequently that means candidates from a small number of top-tier computer science schools.

Even if a given candidate is highly capable, it may be hard to prove that fact. And requiring only certain specific universities for their education makes it especially hard.

Recruiters also tend to operate by web-scraping and spamming. So they focus very hard on people who join job networks like LinkedIn, and who answer their spam. This too is a narrow lens.

Excellent Retention and Work Culture in Competing Companies

There’s a strong perception that the best engineers aren’t actively looking for a job. Mostly this is true.

Most engineers hate job hunting and don’t especially mind their current job. There’s a possibility they could get a better one, but it’s likely that any given attempt to get a new job wouldn’t work out — the company wouldn’t be interested, or wouldn’t be a good fit, or the new job would be the-same-or-worse than their current one.

The better the competing companies are, the more true this is. If you’re competing against Netflix then you know their engineers feel well-paid. It’s unlikely a given company will be paying more than Netflix — and if they do, they should definitely advertise that fact. A company competing against Google knows the competition’s perks are first-rate and include an extensive selection of free gourmet food.

One reason that a given engineer might not fill a great role for them is that they may already have a good-enough role at a competitor and enough other benefits to not want to move.

(As a software developer myself, I’m not crying about this one. But it’s a factor and it’s rougher if you’re a hiring manager.)

Weak Community Awareness

This could mean either “weak awareness of the community” or “weak awareness by the community.” Both are true and important.

For instance, companies tend to recruit at Meetups. That’s very convenient and it’s not a bad thing. But not everybody goes to Meetups. Sometimes companies recruit on language- or community-specific internet forums or mailing lists. That’s also not a bad thing when it’s allowed. But very few established users of a language hang out on their internet forums or (mostly) mailing lists.

And so there are absolutely job vacancies going unfilled because the company doesn’t necessarily have a way to reach out to the people who would be good for them.

The flip-side of this is weak awareness by the community. Even if a company advertises successfully in some places, they still may not reach people. And if they do reach them, they may not be sufficiently well-known to get the trust they deserve. A candidate has to believe that a company would plausibly treat them well, and most candidates default to “I don’t know this company and so I’d rather not risk it.”

Which brings us to…

Widespread Risk Aversion and Impostor Syndrome Among Techies

By and large, a random software developer is insecure. Whether you call it Impostor Syndrome or something else, the nature of our work makes us doubt ourselves constantly, especially if we’re any good at it. The compiler and debugger are merciless adversaries.

As well, it turns out that freelancers (including consultants and contractors, plus sometimes founders and similar self-employers) tend to make far more money in the same position than a full-time engineer. The cost they accept is risk and instability. So a software developer with no fear of risk should be a freelancer, not a full-timer.

If you’re offering a position that might possibly be better to a risk-averse bunch, most of them won’t take you up on it.

So that’s even more unfilled positions without people to fill them.

Here are a few more reasons of my own:

Unwillingness to Train

Companies want employees who will “hit the ground running” — that’s the phrase that was common early in my career. And that means they want employees who already know their tech stack, not employees who could quickly learn their tech stack.

Which means they ignore lots of employees who could quickly learn their tech stack and would be great in the role, because the company doesn’t think they could train somebody.

Alas, a company who thinks that is probably right.

It’s one reason that “willing to train junior engineers” and “willing to take a chance on engineers from another tech stack” can both be huge advantages for a company.

The problem, of course, is that they’re advantages that require more from the company. And a company often doesn’t want that, for good reasons.

And so there are more vacancies left unfilled, as good engineers have trouble finding jobs.

Difficulty in Managing

There are a few companies so large and so data-driven (cough Google cough) that they’re repeatedly hiring for the exact same position. As a result, they find that the best way to do that is to look at who already does that, figure out what they actually do on the job, and make that the list of job requirements.

Doesn’t that sound dreamy? I’ve never worked anywhere so large and so consistent that that approach would work at all, but it sounds very nice.

Anywhere smaller or less data-driven than Google is going to need somebody to speculate about what the actual job requirements are and then put those requirements into a neatly-written package, which we often call a “job posting” or “job description.”

They are, by and large, garbage.

Humans are bad at predicting the future, so we don’t know what the job will actually entail. We’re also bad at judging other humans, so we are often wrong about what the important parts of an existing person’s job are.

And that’s how we wind up with job postings containing unreasonable requirements (Junior Dev with five years of Java experience), irrelevant requirements (deep knowledge of data structures and algorithms), fig leaves over blatant discrimination (“good person to have a beer with”) and outright toxic rubbish (“work hard, take no prisoners.”)

All of this comes from difficulty in managing.

That makes sense. Managing is hard. Most managers aren’t actually good at it. And just as tech companies don’t want to train up junior developers, they don’t usually believe in training managers to improve them.

Stereotype Overfit, aka “Culture Fit”

Managers, engineers and recruiters are all fallible human beings. We wish they judged candidates entirely by how well they would do the job. But realistically they do it by pattern-matching — by examining existing developers and asking, “what do they have in common?”

Who do they tend to favour?

One answer is “developers who went to a top-five computer science university and immediately went into a career in software.” Companies select very hard for that, harder than they’re usually willing to admit. That’s part of why recruiters do — they get paid/promoted by finding candidates who come to work for the company. If managers are only hiring from top-five universities, recruiters will supply more of those and few (or no) other candidates.

Another answer is “developers who eat, sleep and breathe software and have for many years.” Developers who have been doing this since they were thirteen, constantly, tend to have a really easy time getting hired. Developers who work on code in their spare time are in much higher demand. They’re not always better in a straight-up employee sense. Sometimes it’s hard to get them to work on actual business problems. But, speaking from personal experience, if you can project this aura then companies will hire you.

A partial answer is “developers who have other ‘normal’ signs of being a hardcore geek.” I’m thinking of Star Trek, XKCD, Doctor Who and so on. When developers say they’re looking for passionate people with obsessive hobbies (and they do say that) they mean a specific set of obsessive hobbies. They might mean the Society for Creative Anachronism, but they definitely mean logic puzzles, cryptography and juggling. They essentially never mean knitting, voter activism, technical theatre or immunology.

No individual section above causes the things I just wrote. There’s no reason somebody who wears an XKCD t-shirt should have an easier or harder time with narrow recruiter networks. And it doesn’t change what university they went to. So what causes this?

“Culture fit” is a way of saying, “we want somebody who makes our existing people feel comfortable.” This is a combination of “don’t recruite people who are mean, combative or unpleasant” (though often those people are hired anyway) with “don’t hire people who don’t feel socially similar to our current employees.”

Do I need to tell you that this weeds out a lot of skilled programmers who, for one reason or another, aren’t big on the existing geek hobbies? And that that means companies are cutting themselves off from a major source of skilled labour that they desperately need?

Free-market advocates say this is impossible, because good companies will accept expensive labour at a discount price, always. There’s a mountain of research disagreeing. But that’s not why I believe it — I believe this happens because I have watched it happen many times. So if you tell me “that doesn’t happen,” I will not believe you.

But it means there’s a particular group of people who have a really easy time getting a tech job, and if you’re not already one of them you’re going to have to work a lot harder. I have a friend who’s trying this right now. It’d be really eye-opening to talk to her about her journey if I hadn’t already heard the same ugly story many times.

The flip side of the candidate experience is that the company feels like there aren’t nearly enough developers to fill their job vacancies, when they’re overlooking people for (in effect) not being a Star-Wars-watching young man with (if possible) hygeine difficulties. The fit is even tighter than it seems. Dan Luu, a brilliant Asian software dev with perfectly ordinary techi interests still has a nearly-impossible time succeeding in modern-style job interviews, which is not a good sign vis-a-vis admitting all the right candidates.

He’s not alone. Or anywhere close. This is a really common story.

But Does It Matter?

Clearly tech companies are doing okay overall. Does any of this matter?

Only statistically. Which is to say, yes.

It means that companies are recruiting from a smaller pool, which means they’re scraping really hard on the bottom of the small barrel they will recruit from, which means they’re often hiring people that just aren’t that amazing.

Imagine companies threw away 70% of the resumes they received, at random, and hired the same number of people. That means fewer really good people for the same number of positions. If companies are even a little bit good at finding good candidates, they get lower-quality employees out of the deal, because they threw away 70% of the amazing candidates.

It also means a lot of excellent candidates can’t get jobs they’re fully qualified for.

On the flip side, it’s highly lucrative for folks like me who are in the sweet spot. We can charge far more money as software developers because over half our competition is disqualified before they even start the race.

Sadly, we lose over half of our excellent colleagues before we ever start working with them. We don’t get to learn from them. Even by being “just okay,” we can look like we’re amazing without having to do much work. But we also never get as good, because a lot of the best ideas go undiscovered. We’re never fully challenged. We’re never as good as we could have been.

Which developers do you think love that, and which do you think hate it?