Engineering Interview

Any professional software engineer has been through it, standing in front of a whiteboard reciting some algorithm you are supposed to have memorized that you have never once used for anything in your career. Performing well in traditional software interviews is clearly a different skill than software engineering itself. Talented engineers get rejected for silly reasons. Obviously, that isn’t a perfect test of ability, so how do we make this better?

There is a lot more to a bad hire than just poor technical skill. Any employee with a bad attitude, poor communication, or an inability to adapt is far more of a risk to a small team than an engineer that is a below-average programmer. Poor personality and communication negatively affect other engineers’ morale, lowers team output, and makes their jobs more difficult and stressful. An inability to adapt leads to dated ideas, stale products, and eventually your technology falling behind the competition. Quickly summarizing; a good candidate must not only be technically skilled, but also adaptable, communicative, and personable. Many companies do a good job at recruiting for this, but just as many ignore some or all of these factors.

AlphaFlow tries to be different. We think that the traditional interview doesn’t adequately measure those traits, so we take steps to make our interview process both realistic and to test for a wide range of skills. Firstly, we tried to tackle the problem of untrained and inconsistent interviewers. We always use the same set of problems for each available position and spend a lot of time shadowing and ensuring new hires are engaged with the interview process. Secondly, we try to make sure the interview is a collaboration, not a challenge. Every step of the process gives the candidate the opportunity to explain their thoughts and collaborate in different ways with the interviewer. Thirdly, we make strides to make sure the candidate is comfortable and knows what to expect when they arrive on site. We want the candidate to be comfortable, and provide them with a preview of what to expect before the interview starts. Lastly, one of AlphaFlow’s core values is continuous learning and feedback. We are always looking for ways to improve, and we give the candidate a chance to debrief after any result to help with their search as well as receive feedback on how we can iterate on our process.

Having been on both sides of the interview table many times, I think I’ve identified “Must Do’s”, “Could Do’s”, and “Must Not Do’s” in an interview that will make them more fair and reliable. Obviously, this isn’t a perfect system, but I do think that following these guidelines will lead to a better experience for both candidates and interviewers.

Must Do’s

Pre-Briefing and Debriefing – Both the candidate and the interview should be well prepared and know what to expect from each other. The candidate should know both who is interviewing and what the content of the interview will be. Any setup should be given well before the interview so it does not interfere. The interviewer should know what they are testing for and be prepared and well trained. The interviewer should feel comfortable delivering feedback that is constructive and can be shared with the candidate after the end of the interview.

Relevant Technical Skills – The candidate should write code, preferably on their own computer, using any resources they would normally have. I don’t think you can interview an engineer without asking them to code at least a little bit, but the problem should be simple, straightforward, and relatively easy to manage in the time allotted for the interview (or very clearly not expected to be finished). In this part of the interview, correctness and completion are important, but it’s much more important that they know how to use the tools at their disposal, how to use common data structures, statements and expressions, and how to consume information from the internet.

Adaptability – The candidate could show they can pick up a project and understand what it is doing. This may be part of a well designed technical skill question(s), or simply through discussing some topic both the interviewer and candidate is familiar with. The candidate should be able to show they can learn and understand something new and internalize the information well.

Communication – In detail, the candidate should be able to explain the design of a project they have built. They must be able to explain the design decisions they have made. They should also be able to reason about a system they have not built, and explain their decisions and trade offs along the way. Generally this is done either within or in conjunction with some kind of system design interview, although it doesn’t have to be. Here, the interviewer should be careful to guide the interview into a discussion and not just expect a formulaic answer. This interview is about communication, not the choices they made. The candidate’s ability to explain themselves rationally is far more important than if the interviewer agrees with their design decisions.

Personality – The candidate must show they are easy to get along with. They must be personable, driven, and empathetic. The candidate must interview with the team they will be working with, and meet his future manager, PM, and peers. You should never have to introduce a new hire to his new team, it’s just far too risky that they won’t get along. The fact that companies do this regularly baffles me.

Could Do’s

Specific Technology – Tl:dr, If you are interviewing for a DBA, ask a database question. For most candidates in generalist roles, this shouldn’t be necessary, as most engineers are smart enough to pick things up on the fly. I would suggest only doing this if the role requires daily, advanced use of whatever specific technology this role is, such as a DBA or a Devops role. I’ve seen UI interviews given for engineers who will never once touch the frontend, clearly that doesn’t make sense. I’ve also seen SQL interviews for a React JS frontend only candidate, same story.

This should also be made very clear before bringing candidates in, otherwise, you are wasting both your time and the candidate’s time. The job description and hiring screens are critical here.

Public Presence / Evangelism – Ask about a public GitHub profile, or open-source contributions. Maybe blog posts they have written. Not every engineer is in a position they can do these things, but if it exists it’s a great resource to the interviewer. It lets a candidate show his communication skills, his passions, and his philosophy. Particularly for senior developers – this helps show who they are.

Must Not Do’s

Irrelevant Technical Skills – One of my interviews in my last job search was a bunch of math and physics brain teasers. This is completely not relevant to the software engineering job at hand. I was offered to move forward, but I decided I did not want to work there due to that interview alone. It’s extremely important not to waste time vetting a candidate on something that does not matter for their job. As an interviewer, make sure you are actually gathering useful information in every section of the interview. Wasted time on irrelevant skills really does not help anyone.

Large Homework projects – Do you want an engineer because they spent more time on the homework project than the next guy? If you want to give a project as homework, it should be both timed, and less than 2 or 3 hours. A supervised or partially-supervised project is also a lot more engaging and respectful to the candidate. You do not want to bias against people who do not have time to spend 30 hours on a mock website for you. It will probably scare the real talent away anyways.

Tricks – If your problem can be found in Cracking the Coding Interview it’s probably not the best problem to use. The book is a great resource for candidates and is good for practice, but this probably means it’s not tailored well to the role. If it requires an algorithm the candidate learned 10 years ago, it’s likely not a good question either. Nothing you ask should have a “trick” or an “aha” moment required to figure out the solution. By doing this all you do is add randomness to the process and bias towards people who have spent time studying for interviews, and biasing away from people who actually spend their time developing software. 

Veto – Ideally you should not be in a situation where one engineer can veto the hiring decision in an onsite interview. This is to protect the interviewer and the candidate from their own emotions. I’ve seen an interviewer ready to say no before they even meet the candidate. Everyone has had a bad day here and there and that can affect both the interviewer’s mindset and the candidate’s experience. If you see an interviewer doing this regularly, it may be time to take them out of the interview panel and address it.

Single Answer Questions – Do not ask an abstract question, perhaps about a product decision, then fail the candidate because they did not answer how you would have. Separate ways of thinking are healthy and should be encouraged, not shunned.

Conclusion

I’ve seen that many companies, and in particular larger companies, fail to do a single thing in the “Must Do” list. It’s very easy to get lazy and forget what the point of an interview is, and it’s also very easy to use unskilled and untrained interviewers as the company grows. It may not even be obvious there is a problem, but it’s important to pay attention and continuously try to improve. It’s important to train the interviewer well, using well-defined standards for your company. It’s essential to keep the interview process predictable, consistent, and relevant. Do you do everything in the “Must Do” list?

As for the candidates out there, just remember that the process is a little broken, very inconsistent, and there is no way to get every job you apply for. There are so many variables in the day-to-day work that it will not always be a good fit, even if you are excited for the company. Practice, try your best, think positively, and don’t get too sour when you come across a bad interview. There are still more CS jobs out there than candidates to fill them, so there should be a place for you.

 

References and additional reading.

https://github.com/poteto/hiring-without-whiteboards

https://sockpuppet.org/blog/2015/03/06/the-hiring-post/

https://techcrunch.com/2015/03/21/the-terrible-technical-interview/

https://salesforce.quip.com/2UKIAhvKyZf0

https://triplebyte.com/blog/how-to-pass-a-programming-interview

https://triplebyte.com/blog/how-to-interview-engineers

http://www.crackingthecodinginterview.com/

https://www.joelonsoftware.com/2006/10/25/the-guerrilla-guide-to-interviewing-version-30/

About the author:

James Wonsever James Wonsever is a Staff Software Engineer for AlphaFlow, having been on the team for about 3 years. Previously, James was a Lead Engineer at Salesforce on the Salesforce Community Cloud team.

James has a BS in Electrical Engineering and Computer Science from the University of California at Berkeley.

back to top