IT Employment

Technical challenges to use in hiring a programmer

What are some ways to measure a job candidate's technical acuity? Justin James gives some advice for those who are hiring programmers.

I've heard from many IT managers who need some advice on what kind of technical questions they should be asking job candidates. From time to time, I'll ask someone in the field to recommend technical challenges according to specific disciplines.

First up: Programmer. Justin James, himself a programmer and a blogger for our Software Engineer blog gives his advice for technical testing:

When creating a technical challenge for potential developer candidates, you should customize for the specific job. For example, if you are hiring a C# developer, handing someone a piece of Ruby code and telling them to "find the problems" just is not going to work well! That said, there are some challenges that can test the kinds of things that all developers should know. One of the most popular ones in recent years is FizzBuzz. The problem with FizzBuzz is that it has gotten so much attention in recent years, that even people who would probably fail it now understand it (it really is not that hard anyways). That said, I believe that FizzBuzz style challenges are a great way to perform an initial capabilities filter.

Let's look at the FizzBuzz-like challenge. The idea is to ask the developer to write an application that performs a task that covers the basics of development. The application can be written in any appropriate language, or even pseudo code. There are no tricks involved, but the challenge should require the same kinds of thinking that most business logic requires: conditions, loops, and filtering. Here are some examples (they all start with, "Write an application that..."):

  • ... takes an array of numbers representing credit scores and prints the loan interest rate that they should be offered based on a chart. If the score is under the minimum value, display "No loan allowed."
  • ... calculates the factorial of a number.
  • ... examines a two-dimensional array of data. If the data at each coordinate is even, then display "EVEN" otherwise do nothing.
  • ... has a string of text, and a list of strings. For each string in the list, it displays the number of times it appears in the single string of text.

As you can see, none of these should be very hard. But unfortunately, we've seen such a decline in the quality of developers, that far too many of them struggle with what should be trivial work. And the way the candidate solves these problems can teach you a lot about their style of code. How complicated do they make their solution? Is it "elegant" to the point that it cannot be maintained?

About

Toni Bowers is Managing Editor of TechRepublic and is the award-winning blogger of the Career Management blog. She has edited newsletters, books, and web sites pertaining to software, IT career, and IT management issues.

23 comments
Charles Bundy
Charles Bundy

The folks interviewing me wanted to know if I could do X language with Y revision control system. One company was so specific they wanted me to enumerate option switches in tar! Sad thing is one time I asked what revision control was used and the folks in the panel interview DID NOT KNOW. I would love to have been handed a capabilities type technical question which was geared towards candidate strengths. Bet a lot of metainfo on the candidate, including problem solving methods would come out.

jeff
jeff

I wrote the final exam using less code then the instructor's example. It was more concise than his but being original Basic and since I had no math skills, it was still straight forward. He gave me a D because I used Functions (or whatever they were called) that he didn't teach but that was because I read the entire text book and not just the chapters he taught. In time, I had gone on to lead a fast growing and successful tech company. Now, still having very little math skills but a strong understanding of accounting and business processes, I write and own my own solution. My math flat-out stinks but the other skills I have make up for it. Forums and other groups are plentiful so when I have a math problem, I can get help easily. Someone earlier mentioned this: I write slightly longer code than most but I document it less because the code is easier to read. The problem that I have sometimes is trying to outsource some math problems but it OFTEN ends up that the person with the good math skills can't comprehend the depth of the business problem well enough to formulate the solution. One size doesn't fit all when it comes to software. There is a place for many different skill sets and I think the interviewer should be smart enough to know if the person across the desk has the skills to augment the development team. Sometimes, one good mathematician is enough. Some teams are loaded with great mathematicians but they can never clearly understand the user's needs nor build an effective user interface resulting in their great code landing in the digital scrap pile.

SnoopDoug
SnoopDoug

I also have a suggestion. Use the factorial example, but ask them to do it both recursively and non-recursively. It's trivial for anyone with more than a year of experience and likely weeds out the truly incompetent.

jkameleon
jkameleon

http://www.codinghorror.com/blog/2006/07/separating-programming-sheep-from-non-programming-goats.html Atwood is right. You either can program, or you can't. There is no way one can learn it. All teachers of programming find that their results display a 'double hump'. It is as if there are two populations: those who can [program], and those who cannot [program], each with its own independent bell curve. Almost all research into programming teaching and learning have concentrated on teaching: change the language, change the application area, use an IDE and work on motivation. None of it works, and the double hump persists. We have a test which picks out the population that can program, before the course begins. We can pick apart the double hump. You probably don't believe this, but you will after you hear the talk. We don't know exactly how/why it works, but we have some good theories. Despite the enormous changes which have taken place since electronic computing was invented in the 1950s, some things remain stubbornly the same. In particular, most people can't learn to program: between 30% and 60% of every university computer science department's intake fail the first programming course. Experienced teachers are weary but never oblivious of this fact; brighteyed beginners who believe that the old ones must have been doing it wrong learn the truth from bitter experience; and so it has been for almost two generations, ever since the subject began in the 1960s.

jarzola
jarzola

I never believed that a developer should be judged by the way they code or what language do they code in. To me the most important questions to ask are: if you need help where do you go to find it and how do you go about implementing. For example. Create this app but please keep in mind reusability, security, scalability and performance. I dont care how you develop it, if you follow these steps in that order most will come up with the same type of code which to me is acceptable. If they use Google to find the answer, I really don't care as long as they follow the step that I provided. Loops and array are a waste of time for me, because I can find 10 reasons why not to use arrays and the same goes with finding 10 different ways to use arrays for that situation. So to me a candidate must understand logic of code and give me explanation on how this is reusable, secure, scalable, and fast. If they can give me a logical answer to these then they are good enough. Who cares where they find the answer. There is always room for learning and I give all developers the benefit of the doubt that they can improve it in the future. Hell I can improve in the future as well.

Tony Hopkinson
Tony Hopkinson

is you get book answers, about the only good thing about them is if you can introduce some potential ambiguities Even numbers , Int, Double etc... See if the developer asks, and of course a basic sanity check that the dev reads the questions and can come with an algorithm that will work. Factorial, would you penalise for a recursive algorithm or not? What I look for is the whys and whens. Given them mini apps, that exhibit bugs, majot and minor design flaws, see if they can debug analyse etc. Basic performance improvement, if they had to enhance it to do X, how would they do it and why. See if they can apply their ability , not just spew the answer to a question an incompetent could google and memorise. The thing is most of the work most of us we do isn't that technical most of the time, what we should look for is people who understand that change is a given, and how to cope with that. Course don't test a junior for that, they don't teach it in academia.

gharlow
gharlow

Some are good at it most are not. Bottom line if you ask 10 competent developers to build an app you will get 10 different sets of code. If you ask the same developer to write the same app 10 times you will get 10 similar sets of code. Bottom line like many things in life this cannot easily be quantified. There is nothing wrong with simple questions like when would you use an Interface and why? The problem with this approach is that you will end up with booksmart developers who may not have any idea how to go about organizing a project and determining the best and most efficient way to code it. They might know how to get the job done but what about security? What about performance? Do they understand how to avoid SQL injection? Is all their code packed into one huge shambles or nicely tucked away in appropriate classes? Will they go Class mad and produce nasty code as I have run into in the past using delegates for no good reason except to show off something new they learned? In interview situations I am a LOT more interested in talking about previous projects, (in generic terms) difficulties that emerged during the execution of the project and how those difficulties were solved. How would they go about performance tuning an application or database etc...

Tony Hopkinson
Tony Hopkinson

you can staple togther in one go with an ACME 34/5P superstapler as well? Morons.

Justin James
Justin James

Reminds me of some jobs that I interviewed for that used online "tests" to determine "capabilities" when a significant portion of the questions tested my knowledge of keyboard shortcuts in the IDE... like not knowing that F5 is "Debug and Run" will make someone a bad developer... J.Ja

gechurch
gechurch

For most jobs I'd agree with you (including the job I do), but not for programming. A good programmer will come up with a far better solution to a problem than an average programmer that Google's (or otherwise gets help). Good programmers know how to approach a problem at a high level, then go about implementing it. They might hit Google to find out the exact syntax of a command, or maybe to look for a third-party component that already does 90% of the work. An average programmer will implement half a solution, then go Googling for the other half. The problem is it's really hard to Google for fundamental, high-level approaches to problems. And once you have half a solution, you are likely to have trouble solving the other half... because the correct answer is not to solve the second half, but to approach the problem in a completely different way. You see this all the time on forums, where people ask a technical question wanting a function to work in a different way or to take different parameters. If you ask enough questions, you normally find that they have already made a fundamental mistake in the part of the problem they have already "solved". Oh, did I mention the really good programmer will create the code many, many times faster than the guy that has to hit Google, and it will be both more elegant and more maintainable.

apotheon
apotheon

The point isn't to tax someone's capabilities. It's to see if a very simple test that shouldn't tax any developer's ability actually taxes your applicant's capabilities -- and, if so, don't hire the guy. To detect further depth of ability, have a conversation with the applicant. It's far too easy to get false negatives in your technical testing process if you get too clever. The last thing you want to do is reject the best candidate because of a stroke of bad luck.

apotheon
apotheon

You still need something like a fizzbuzz test. Just don't fall into the trap of using tests that will disqualify actually good developers who just weren't prepared for the kind of test you offered. Such tests should never be clever or difficult; the point is to quickly weed out the people who cannot code their way out of a wet paper back, and not to weed out the people who get really nervous in job interviews.

Charles Bundy
Charles Bundy

"of high quality or standard" With regards to writing code one would hope maintenance would figure in, but I've seen many a design (not just software) that sacrifices maintainability for performance. I've really cursed some automotive and aerospace engineers back in the day...

Justin James
Justin James

... is what others would call "too clever by half". :( J.Ja

branchman67
branchman67

I think in these cases, 'elegant' is often used interchangeably with 'concise'. Some people assume that fewest lines of code is 'elegant' when sometimes being slightly more verbose in your coding can help make the code self-documenting without sacrificing performance.

Tony Hopkinson
Tony Hopkinson

Was a delphi test with Help, intellisense and internet disabled. First question was how do you get a the hint of component under the mouse to display in a status box. Something in fact I'd never done, in five years... Test got worse, name the properties of component X that are involved in function Y... Whether I knew what I was doing was debatable, them the issue was beyond all doubt...

Tony Hopkinson
Tony Hopkinson

Most breeze through it (fibonacci, or factorial are our usuals), but there have been some significant fails, not so much because they couldn't code, or they locked up (who hasn't on occasion), but because they didn't read the question, and then worse still spewed a big pile of BS when challenged on it. After that it all depends on what level of developer you are aiming for. A really good junior would know that a recursive algorithm is a generally bad idea, a senior who didn't, isn't. Beyond junior, I want the candidate to prove they can and will do all my thinking for me. :D

Justin James
Justin James

Exactly! When I first started interviewing people, I had these real brain-buster questions. You know what feedback I got? That no one wanted to take the job because they thought they were too dumb for it, even though I was just trying to place someone on the experiences/intermediate/junior level scale. One candidate actually ended up crying (literally) to the recruiter. Granted, I thought that someone coming in for a *DBA* position should know what a "correlated sub-query", but apparently I was wrong... Granted, it also reinforced my belief that too many in this business lack an understanding of the fundamentals. They are like mechanics who know how to change a fuel pump, but are ignorant of the fact that it is the ignition of a compressed mixture of air and fuel that makes a motor run... J.Ja

apotheon
apotheon

That seems to be how you used the word "elegant".

apotheon
apotheon

People do use "elegant" that way. The problem is that this manner of using the term does not fit the definition of the term -- and trying to duplicate the same error others make in this regard robs us of a term that describes what "elegant" should really describe.

Tony Hopkinson
Tony Hopkinson

A dba should know what correlated subquery is, might have to find out what he calls it (complex thingy that can use up most of your server's ram), but if they don't they aren't one. False positives to me are getting syntax reversed, or not knowing what Professor Wilhelm called it in their original paper in german... This is a correlated subquery, then a a bit of when and why, as opposed to write an example of one though.

Justin James
Justin James

... is my point. I see some folks put out something that they brag about being "elegant" because they found some syntax loophole to save themselves a line of code, at the expense of totally obscuring the purpose. A lot of "declarative programming" is like this, because they have replaced simple step-by-step code with instructions to a DSL parser that you have no insight into. LINQ in .NET is like this. For simple stuff it's a nifty trick, but what happens is that the purpose... the logic underlying the logic... is totally lost in many cases. J.Ja

Editor's Picks