The Joel Test in Real Life and How I Try to Get 12 Points

When I was assigned as a CTO, I was very excited. I think it’s time for me to brought the joy and delight of coding. So I wrote my plan ahead and I have a commitment to applied the Joel Test. It’s not an easy road though and in my one and a half year as a CTO, I will share how I have achieved more than a half of them.

It’s pretty easy to achieve all of them, 12 points, when you work for a company like Google, Facebook, Yahoo or even Microsoft (yes, Joel said that Microsoft always run at 12 full-time. Incredible!). When I was at Yahoo, people were really supportive and we always try to achieve perfection in building and crafting a software. It would be different if you work for a company where not all people understand about Software Craftmanship. It’s more difficult when you work for a company (or clients) who has a type of “yesterday is the deadline.” Those kind of companies (and probably clients?) didn’t understand about planning, priority and design. The Joel Test will help you, my friend, and me to be not only an ordinary developer, but also an outstanding developer who always ship the best crafting code ever written.

Let’s get started.

1. Source Control
At first, the team were already familiar with SVN. That’s good. But the downside of SVN still remain and I tried to change the habit a little by introducing Git. It wasn’t all romantic. Wrong commits, forgot to add files, confused between clone and checkout as well as commit and push (remember, there’s no push in SVN; everything is a commit), and hardly learn about .gitignore.

Once everybody’s familiar with it, I also try to introduce Git flow concept. Today, everyone can do push on master, only one person can merge it into develop and only one person, too, responsible merging it into release. The result? We hardly find a bug on our production.

2. Build in one step
I asked to the team how they do deployment from their local development to production before? The answers vary and all of them surprised me. The answer ranging from “no, we don’t do deployment. We directly code into the server” to “Using FTP software such as FileZilla or Cyberduck”. I got it. So I introduce the thing called “build automation.” At first, the team was uncomfortable because they never do that using any language other than PHP (I suggested that they use bash or Python). So I built one sample script using bash to auto-deploy one of our branches and sent to them to start from there. Today, we’re thankful that deployment only takes less than five minutes without human error. I repeat, without any human error.

3. Daily Builds
Before, it was not daily nor once every two days. We could have more than 5 deployment in a day or just only one deploy for three or four days. You can call it “on-demand deployment” but that’s not good for the speed of the team. So I propose to set the deployment schedule to twice a day. Like every change, everyone refused at first because they couldn’t predict how much time the would spent on one or more bugs and features. So I encourage them to learn how to predict and ship on time.

Here’s our schedule:

11.30 Deployment to staging
11.30 – 11.45 Testing on staging
11.45 – 12.00 Bug fixing followed by deployment to production

16.30 Deployment to staging
16.30 – 16.45 Testing on staging
16.45 – 17.00 Bug fixing followed by deployment to production

With this schedule, we’re able to plan our day ahead and finish everything on time. We also able to leave the office on time (to those who don’t want to stay late at the office) without leaving a bug on production.

Warning: there’s always be a person or two in your team, whether from management or other divisions who have a “yesterday or today is the deadline” mentality. Beware as this kind of sense-of-urgency-mentality seems good but it does actually nothing except nervous which affect your team’s ability to code well.

4. Bugs database
We use Excel at first but as the list grow larger and larger, we hardly maintain those list. So I decided to move those bugs and features list to Bitbucket. No issues while moving all of them to Bitbucket but we faced one challenge: getting everybody logged in to Bitbucket and always update the issue list. But that’s not a problem.

5. Fix bugs before writing new code
It’s hard to explain to those who don’t understand programming the differences between bugs and features (to some people, they’re look the same: as a coding activity). It also hard to explain, when your team busy coding, why the features someone just proposed five minutes ago never go to production. Well, to some extend, I hardly implement this because some people who pretend to have power doesn’t understand that bugs usually have higher priority than features, except the engineering manager wants their team to have a huge technical debt.

Some people sometimes need to push their power by ordering a technical team to just forget what they’re doing right now for a sec and rushing to the new features proposed five minutes ago to be deployed one minute ago. This kind of people usually doesn’t understand about software development and the only one they have is power. Too bad, I’ve seen a lot of technical team frightened and do what he/she says. The result is predictable. The features weren’t build very well and those bugs were getting worse and worse. Technical debt is real. Realize it before it’s too late.

6. Up-to-date schedule
No, we don’t always have an up-to-date schedule because sometimes, requirements change without notice. Yes, we try to keep the schedule up-to-date by setting a meeting for tech team everyday (just like a 5-mins stand-up, but with different discussion). Keeping a schedule up-to-date is as hard as hunting a bug sourced from days or months of coding, but it worth the shot. I don’t think I’m successfully doing this but I always do my best.

7. Writing a spec
We hardly follow this rule because, as Joel said, many programmers hate to write a spec (even though they realize it’s a good thing). We do write specs but not in regular time. Sometimes we do, sometimes we don’t. So we still miss this point.

8. Quiet Working Conditions
Yes, we do. In fact, our company is some kind of “traditional” companies which doesn’t encourage employee to play the music out loud. If you want to play music out loud, use your headphones. Of course, the downside of this, if I might tell, is that you can’t do party on your office. You should go out and find some proper place to do the party.

9. Use the best tools money can buy

10. Testers
No, not yet. There is one tester who doesn’t know about coding and how to do an automatic test but he’s still very good at spotting the bugs. He also not a dedicate guy for the test. We picked him from a tech support team and taught him how to do bug report. The result was very good. Still, we need to get at least one dedicated person to do that.

11. New candidate write code during the interview
Yes, if they applied as a programmer. I asked them how to do the merge-sort in any languages they want or just using pseudo-code. I asked them how they connect to more than one database (say it three or four). I asked them to demonstrate on their laptop connected to projector (so I can see how they code) on how to solve a word puzzle (just a very simple puzzle, trivia to some programmer). It doesn’t matter whether the answer were right or wrong. The important thing is that they can code on their own (no copy-paste from Google’s search) and solve the problem one step at a time. Oh, also, I allowed them to read the documentation, but not to search the answer on Google. Most of the time, good programmers could solve the problem but forgot what function they should use (because of anxiety, maybe?). Documentation is a programmer’s best friend so I never prohibited them to take a look while in the interview.

12. Hallway usability testing
Yes, yes and yes. We do have many volunteers to try and brake our software. They also voluntarily report the defects to us.


The result is 8 out of 12. Of course, we still need to improve that 8 to be in a better state and chase the other 4 to get the perfect 12 in score (from the original score 2 out of 12). It’s important to get 12 out of 12 but the most important thing is that to make sure everyone in the team experience this kind of challenge and be happy with it so they understand the before-and-after conditions hence they will value their job (if they called it a job because I called it a hobby which, in side effect, get paid) more than 9-to-5 activity.

“Most good programmers do programming not because they expect to get paid or get adulation by the public, but because it is fun to program.” – Linus Torvalds.

Also read...


  1. You’re doing git-flow the wrong way round, everyone should commit to develop and only a few people (or a process around) should merge to master. It sounds from your writing that you’re merging to develop only when master is stable and from develop to release.

    “We consider origin/master to be the main branch where the source code of HEAD always reflects a production-ready state.”

    Great blog post though, very interesting.

    • Thanks for your correction. We’re still in the early stage adapting git-flow where not every team member has the same speed. In fact a few members still learning about git. But you’re right. We’re still in the learning process and most of them only familiar with master. Will follow the full methodology once everybody’s familiar with all branches!

      Thanks, Andy!

  2. Your description of how you assess programming skills by asking candidates to write code in interviews is laughable.

    “I asked them how to do the merge-sort in any languages they want or just using pseudo-code”

    Why would you expect everyone to know this and those who don’t to code it under the pressure of an interview? I have written merge sorts and its not the sort of thing you can do in an interview unless you’ve memorized it and can regurgitate it on commands (I’m not talking about a general description of the algorithm but producing the actual code). All you’ve done here is select a random problem out of the almost infinite number of problems one could choose and elevating it as some kind of litmus test of programming ability. All you’re doing is selecting for people who have had the experience of solving that one particular problem and have committed the solution to memory.

    ” I asked them how they connect to more than one database (say it three or four).I asked them to demonstrate on their laptop connected to projector (so I can see how they code)”

    What does this prove? That someone may have used a couple of DB APIs before or can look them up and cobble together some trivial code to make a connection? That’s going to tell you “how they code”? Really?

    “how to solve a word puzzle (just a very simple puzzle, trivia to some programmer). It doesn’t matter whether the answer were right or wrong. The important thing is that they can code on their own (no copy-paste from Google’s search)”

    So let me understand. Under the pressure of an interview you ask a programmer to solve a problem in real time with you watching over their shoulder but you don’t care if the answer is right or wrong. Do you have even the slightest clue how unnerving this is for a candidate? This is not the job. The job is going off and working the problem which may require some trial and error (even for seemingly trivial problems). Most people do not react well to having people watch them work especially when they’re being evaluated at the same time. I’ll give you points for letting folks use the documentation but overall your approach has serious issues. Candidates are going to feel pressure to get the right solution (even if you don’t care – and if you don’t why are you subjecting them to the exercise? Just to see them type some code that may or may not compile without a few tries?) and this will affect their performance.

    All you’re doing is using the same stylized hazing ritual that unfortunately has come to be all too common in technical interviews. The focus should be on a discussion of concepts related to the tools that will be used in the job and of previous projects. Candidates who can do well in these areas are going to be able to do the job. Producing actual working code in the setting of a job interview is a gross distortion of what the actual job is going to be. I never ask candidates to code in interviews and I have always been able to tell the winners from the losers. Perhaps your lack of confidence in being able to do this is a reflection of your own technical shortcomings.

    • About the merge sort, it’s just an example. Not necessarily questioned in every interview. That would be a total lack of creativity to ask the same question for every person in the interview. I might come with a FizzBuzz, or Fibonacci numbers, or maybe the first 500 prime numbers.

      About the test to connect to more than one database, again, it’s just an example. If someone was familiar with PHP, I want to know how they do that (with or without the framework). If someone was more expert in Python, or Java, I also want to know how they handle that. From that point, I might want to open a discussion about how the languages interact with databases and what’s the best practice known to them. No, I’m not going to prove that I’m more clever than them. Otherwise, I want to know whether their curiosity really challenged them and explain the result to me as a story on how they achieve that.

      Solve a word puzzle, even it’s a trivia to some programmers. What matters to me is whether they asked a question (if they got stuck, for example). I believe many programmers were curious on how to solve a problem (including a simple word puzzle). What question they’re gonna ask was a simple lead to a more deep discussion on how to solve it. I’m really aware of the pressure and that’s one of my goal, to make them handle the pressure and win against it. Many great programmers love to discuss about almost anything as long as it involves a problem to solve. By discussing the problem, hopefully they can get along with it and who knows, they solved the puzzle! (but if they don’t, and this is common, they already got points on starting a discussion about that).

  3. Point #8 (quiet working conditions) isn’t meant to be about music, it’s meant to be about interruption. If you work in a big room together and people are in the habit of asking questions and having conversation around and across other peoples’ working spaces, then you don’t have quiet working conditions. Joel advocates private offices for each developer, but he put “quiet working conditions” instead of “private offices”, I assume, because many companies just can’t manage private offices for each developer. But what he really means is “no interruptions”. Do you have that?

    • Yes, we do have that, but still, like every other companies, this is hard to achieve a fully distraction-free environment. We have our own rules where every requests (features/bugs/spec changes) must go to one person (who also act as a tester in my post above) but there’s always somebody who just want to make a shortcut and directly hit developers with their requests. But, this is incidental and not always happen. To get a clear picture, let’s say it’s 1 in a 20-30 (for every 20-30 requests, there’s one request which coming in not following the rules).

      And for another distractions, e.g. phone calls, colleagues asking something, and another type of interruptions, we’re aware of that and always try to minimize that kind of situations. We don’t have a “private offices” but we try to put the “quite working conditions” as our priority.


Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.