The philosophy, ideals, and behaviors that guide everything from our product strategy to our own processes for hiring software engineers

At Qualified we help our customers assess and hire software engineers, giving them the tools and process required to turn hiring developers into a strategic advantage. As a software company ourselves, we face the same challenges that our customers do—we need to identify and hire great software engineers too!

Having to live our customers’ challenges first hand results in nearly constant introspection—is the product that we’re building really the best way to assess engineers? Are the hiring practices that we advocate for ones that we follow ourselves? What’s guiding our own behaviors both as a product organization and as an employer?

While every company is unique, we thought it was important that we put our own ideas, beliefs, and philosophies on hiring software engineers down on paper—this document is inwardly focused on us, for us, and by us—we simply ask that you hold us accountable to these ideals. If you derive some benefit from these concepts, even better—these are the common denominators that guide our own behaviors and that we openly advocate for.

Without further adieu, we proudly present The Qualified Manifesto on Hiring Software Developers!

Philosophy

The all-too-common tension between talent acquisition and engineering leaders is well documented—overcoming that tension is at the very core of the philosophy that guides our product development and hiring practices. Our guiding philosophy is best stated as follows:

Identifying great developer talent requires a deep understanding of how developers work and live, which leads to a mutual respect between companies and developers. We build products that we would want to use ourselves as developer candidates, because these tools let us show off our best work when we use them.

What we believe and how we behave

The following ideas are central tenets that showcase our own beliefs and drive our own actions and behaviors.

Never put a developer on the spot

Don’t stare at developers with judging eyes as you watch them think though a problem—seriously, that’s just creepy.

White board or pair-programming interviews where a developer is asked to write code while others look on in real-time is simply a bad practice. Pair-programming sessions can work great if you’re working with a candidate collaboratively to get a sense of their working style, or if you’re asking them to take you through code that they’ve already written. But putting them on the spot and asking them to write code on-demand? Don’t do it.

All you’re doing is cranking up the candidate’s anxiety and limiting their ability to showcase what they can do.

Work samples are the best predictor of job performance

While coding tests often get a bad rap—much like standardized tests—that sentiment is most often fueled by a feeling that the test simply isn’t relevant to the work to be done in the real world. So how do you make assessments more useful?

From a psychometric standpoint, relevant work samples have been shown to be one of the best predictors of on-the-job performance. Forget the generic coding tests that probe only for algorithmic skills in favor of an assessment that’s truly optimized for its stage in the hiring funnel and the role that you’re hiring for. Assessments should be designed to test actual on-the-job tasks and the technology stacks and frameworks developers will be asked to work with once hired.

A fair hiring process is built on evidence and merit

Your engineering team may have referred candidates to your company or you may be tempted to hire a developer based on their experience at a Facebook or an Uber. Don’t fall into these traps—in our experience the best developers often aren’t the ones with the shiny resumes and big company experience.

Good assessments and a sound hiring process judge engineers solely on merit and applied skills, removing all other forms of bias possible. That’s why we’ve built features like “Blind Reviews” into our assessment platform, removing any identifying information from code submissions and in turn forcing your team to make evidence based hires solely on the basis of demonstrated skills and the quality of the work sample submitted.

Let developers work in a comfortable development environment

You wouldn’t have asked Picasso to deliver his next masterpiece using the worn out pack of Crayola’s jammed into the back of your junk drawer, so why would you ask a developer to complete a coding test in a rigid coding environment that’s foreign to them?

The entire process of interviewing software developers should focus on a singular objective—surfacing developers that have the skills to effectively contribute to your code base in a real world setting. You want to give them every opportunity to showcase their best work, and that starts with letting them perform in a development environment that’s comfortable for them.

We’re committed to offering a highly flexible and configurable web IDE for developers to work in—or better yet, developers can complete Qualified assessments from the comfort of their own IDE.

Limit assessments to 30 minutes or 3 hours

Don’t ask job candidates to spend eight hours writing software as part of a job interview—that’s really messed up and shows no respect for the applicant’s time.

As a general rule of thumb, limit pre-screening coding assessments delivered at the top of the hiring funnel to 30 minutes. It makes sense to set a maximum time of 45-60 minutes for these assessments so candidates don’t feel like the challenge is a ticking time bomb, but design pre-screening assessments so that they’re completed in an average of 30 minutes.

Later stage coding projects that are more detailed and involved should be capped at about 3 hours. These are simply rough benchmarks that we believe show respect for a job candidate’s time, while still giving your company a sizeable work sample on which to assess a developer’s skills.

Best yet, by capping the amount of time a candidate spends on a coding exercise you can see how much progress each developer was able to make in the time allotted and can use interactive sessions to openly discuss where they’re at and what they might do next if they were to continue with the project.

Wield considerate pre-screening practices when hiring senior engineers

Senior developers are often turned off when they’re asked to complete a pre-screening coding assessment… “I’m a senior engineer with tons of experience, why are you asking me to waste time on this simple coding task?”

The sentiment is completely reasonable, but we also empathize with the people on the hiring side of the table. We’ve experienced the following issues first hand when hiring senior engineers:

1. Senior level engineering roles get tons of applications from junior engineers.
2. Lots of developers who look great on paper couldn’t write decent looking code for even basic tasks.
3. Looking through pre-existing code samples (often suggested as a solution when hiring senior engineers) is hugely time consuming, inconsistent, and it’s tough to verify that the candidate actually wrote the code.

As a result, we advocate for using a pre-screening assessment with senior engineers—but “senior” doesn’t mean that the assessment should be a really in-depth, difficult, or lengthy coding assessment. Instead, use a short pre-screen simply to verify that applicants can perform fairly basic coding tasks—this is a useful first cut in the hiring funnel, and should be something that a truly senior engineer can knock out in minutes. That’s fair to both sides.

As for more junior engineers? The best candidates will even appreciate your pre-screening exercises as an opportunity for them to showcase what they can do and distance themselves from other applicants.

Finally, don’t lose sight of your business objectives—if you’re hiring for a mission critical development role and have a low number of applicants, you can always skip the pre-screen to increase applicant flow and defer assessing technical competency until later in the hiring funnel.

Use project based coding challenges to assess mid to senior level engineers at later stages of the hiring funnel

While pre-screens are a useful tool in helping you separate the pretenders from the contenders, once a basic level of coding proficiency is demonstrated you should focus the rest of the hiring process on assessing the skills that you’d expect senior level developers to possess beyond coding ability.

Create a project based coding assessment that directly reflects the work the developer would be expected to deliver if hired, making sure to provide some open-ended flexibility that allows candidates to showcase their strengths. This might require the developer to digest an existing code base, gather requirements, communicate to your team a push request they’d like to make, or provide a detailed description of code that they’ve submitted before merging into production. These activities move your assessment process beyond the code, revealing insights into what a candidate would actually be like to work with.

Don't rely solely on automated scoring of code assessments—a sound code review process delivers 3x value

When evaluating project based coding assessments, don’t simply rely on automated scoring—your developers review code all the time and a proper review process should take only a few minutes (versus an hour or more spent in an interview). Use automated scoring to handle the parts of the review process that are most time consuming, allowing reviewers to focus only on the parts of the code that really deserve their attention.

Automated scoring can help you answer the question “Can the candidate do the work?”, but a short code review is what will truly provide insight into whether or not the candidate can provide quality work. Also, don’t miss out on the opportunity to discuss the work that was completed with the candidate to talk through different approaches, techniques, and things like how the solution would need to scale if new requirements were added. This is where pair-programming sessions, code playback tools, and peer code reviews can add value.

Chances are, your engineering team performs peer code reviews once code is submitted for merging into production. Why not do the same for work samples submitted during the hiring process? You can also watch short code playback clips or jump on a pair-programming session with the candidate to discuss their submission. Whichever route you choose, this process should take 5 to 15 minutes per review instead of an hour or more spent watching someone program live then writing up feedback afterwards. Your engineering team will appreciate having their afternoon back!

A three pronged review approach like this is the key to surfacing senior level talent that automated scoring in isolation simply can’t deliver.

Candidates deserve to know where they stand in your hiring process

If you’ve asked an engineer to invest significant time applying for a role at your company, they deserve timely feedback on where they stand in your hiring process. Nothing is worse than being “ghosted!”

Make no mistake about it, how you treat applicants directly reflects your company's culture. Progressing candidates swiftly through the hiring process shows that you're running an efficient process, and letting candidates that you won't be proceeding with know their status in a timely fashion shows respect and consideration of their time. Not only is it the right thing to do, but keeping candidates in the loop will help you from leaving them with a bad perception of your company.

We’re building the world that we want to live in

These guidelines may sound simple, but used collectively can help your company open up your candidate pipeline, build credibility with developers, and make a great impression throughout the hiring process—these concepts represent our north star. In short, we’re building the future of the developer hiring experience that we’d want to participate in ourselves as software engineers. It’s a pursuit that gets our team out of bed each day, and a future that we’re excited to help shape.

Ready to give Qualified a try? Sign-up for a free trial.