Justin on Joel on Software

I just started reading the book Joel on Software and found myself mentally commenting on, replying to, and summarizing many of the chapters. I thought these might be worth documenting, so I decided to write them down in a new blog post. I was hesitant about the title Justin on Joel on Software since Joel is a much more accomplished and experienced programmer and I'm a noob, but the parallelism was too attractive.

Before I get into it, some main takeaways I got from the book:

  • Know your space, know your product, know your customers! Understand why what you're making has value and who benefits and would pay for that value
  • Always focus on maximizing value. That was an important lesson for me this summer especially since sometimes I get stuck on trying to fix or do something that's not that important
  • A way to do that is by economics, viewing your decisions in terms of economics (i.e. customers reached & pleased) to drive decision making

The chapter by chapter

Back to Basics
A fine argument for understanding lower levels of abstraction. There is a big benefit to understanding the foundation because when shit goes wrong on a higher level of abstraction the solution often lies in the more basic levels. Abstractions will fail someday and when they do, you better know what is being abstracted! I particularly liked his description of Schlemiel the painter algorithms:

Shlemiel gets a job as a street painter, painting the dotted lines down the middle of the road. On the first day he takes a can of paint out to the road and finishes 300 yards of the road. "That's pretty good!" says his boss, "you're a fast worker!" and pays him a kopeck.

The next day Shlemiel only gets 150 yards done. "Well, that's not nearly as good as yesterday, but you're still a fast worker. 150 yards is respectable," and pays him a kopeck.

The next day Shlemiel paints 30 yards of the road. "Only 30!" shouts his boss. "That's unacceptable! On the first day you did ten times that much work! What's going on?" "I can't help it," says Shlemiel. "Every day I get farther and farther away from the paint can!"

The Joel Test: 12 Steps to Better Code
Gives a cool litmus test for what good looks like, a quick 12 yes/no to see how your team is doing. One of his tests is "does your team fix bugs immediately?" I'm not too sure if a yes is always good, isn't it sometimes ok to have small bugs (especially if they're documented) to build out important features? I think there is a complex push/pull between debugging and developing new stuff that doesn't lie completely on the debugging side.

The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)
I have really nothing to add except you should read it & learn it! Provides some interesting history on ASCII, Unicode, and encodings. Some common myths:

  • ASCII is not plain text
  • Unicode is not just 16 bits
  • There is no such thing as plain text!!!!

Painless Functional Specifications
I actually love writing specs; I like writing and I make shitty versions of specs for stuff I'm working on anyways. I find it helpful in figuring out what I want to do and the scope. Joel provides some fine arguments for writing specs, and delegates the task to PMs or to programmers. It is a really similar idea to RFCs at Riot.

Even though I like specs, I am still a little iffy on them. I agree that it's helpful to write them and have plans laid out but it seems like such a huge investment in time and effort. I wrote 4 RFCs this summer and it took me a few days with a lot of help. and we did them AFTER we worked on the stuff for a few sprints. I will say though that by writing the RFCs I learned stuff about the direction and higher level plan for our project that I haven't really thought about when I was working on it. Two points I found interesting:

  • 1 author for each spec, 1 point person
  • No (or bare) template- lowers the barrier to writing a spec (ugh so many categories to fill out). I kinda disagree cause I feel like a barrier is also "ugh what goes into a spec" and starting on a blank page is tough.

Evidence Based Scheduling
Schedules are hard. One of the weaknesses I profess in interviews is estimating (I actually talked about this at my Riot interviews). It seems like I am always off, and one of my troubles with storyboarding and making agile work is having good estimates of work. He provides a few suggestions, including short scheduling (nothing longer than 16 hours to do) and careful scheduling, to think about the smaller pieces that need to be done. I do try to do both, but one big issue I always run into is that I don't know what tasks are involved and stuff always just pops up.

But to resolve this, he makes a super interesting suggestion for another type of scheduling: evidence based scheduling. You track elapsed time on each story you take and your initial estimate, then record this past historical data to estimate future with a Monte Carlo simulation. The system assumes that the more consistent & accurate your estimates have been in the past, the more consistent & accurate they will be in the future, since distractions and mistakes happen in every software cycle with some probability (taken into account by the simulation). I am really excited to give this a try soon!!

Daily Builds Are Your Friend
Introduces the idea of the REP (read eval print) loop (modernized to an edit compile test loop) and highlights an eng-verb I learned from Kyle this summer:

The tighter the loop, the faster the programmer

The time saved is exponential, because the more you can hold in your brain, the faster you can work, and the less frustrated you will be. Waiting for stuff to run and compile is equivalent to the rage of waiting for a web page to load. While I get the benefits of automatic daily and complete builds, I think the technology has advanced and people are now investing time and effort into continuous integration, tightening the loop even further. You can read more about Docker/Jenkins at Riot on our tech blog!

Hard Assed Bug Fixin'
I hate shipping shitty code, I love cleaning stuff up, and I love having neat code, so sometimes I have to fight the urge to debug and refactor. The problem with this is that it takes ages, is often unnecessary work, and psychologically ties you to your work (which is never good). Instead of fixing bugs immediately, he suggests instead that your team document non critical bugs and evaluate what to fix based on user value (he says economic, but there are other similar metrics). Here is a quote I like from the article:
"Fixing bugs is only important when the value of having the bug fixed exceeds the cost of the fixing it."

Five Worlds:
The five worlds are: shrinkwrap, internal, embedded, games, and throwaway. These different types of software require different solutions, and so you should tailor advice and approach keeping in mind the different type of products you are making. This is a more general way of saying think about your users and your product space.

Don't Let Architecture Astronauts Scare You
Architecture astronauts are what Joel calls people on such a high level of abstraction that they run out of air to breathe. The emphasis is on usability and what the user wants, not just a lofty abstract ideal that doesn't solve a useful problem. Architectures and new toys are only useful because they solve problems!!

This is a cool explanation of what excellence looks like in software. The last 1% might take 500% of effort, so Joel defines craftsmanship is software written robustly. Often the effort is not worth it except for shrinkwrap software with lots of users. This was my experience too; often the last bit of work is the hardest (spinning up the shell of a vaguely functional product is not as hard).

Three Wrong Ideas from CS
Interesting ideas but not sure if they're still really relevant. I've never really thought about these things (local resources vs resources on the network, anti aliased text, and searching).

Guerrilla Guide to Interviewing
Parts I agree with:

  • A bad hire is worse than losing a good hire
  • Good hires are the most important thing to growth and excellence
  • Don't hire maybes
  • Look for people who are smart & gets things done
  • Reduce bias before interviews (don't talk to anyone before, try to not form opinions from their resume)
  • Give an opinion about the interviewee immediately
  • Ask open ended questions
  • Try to hire generalists, especially since tech changes so much

Parts I disagree with:

  • I don't think the type of question asked at interviews is good; it forces studying something irrelevant to the job. I get that understanding people's ability to think problems through is helpful but I just hate balancing a binary tree and have never done it outside an interview
  • What is "smart?" I think aptitude is probably more important, the ability to learn and adapt

Incentive Pay Considered Harmful
Super good article, lots that I agree with in here. Traditional reviews are not very helpful, and they don't accurately capture real skills and ignore other valuable ones. They are also very demotivating:
"For them, a positive review makes them feel like they are doing good work in order to get the positive review… as if they were Pavlovian dogs working for a treat, instead of professionals who actually care about the quality of the work that they do."
Most people end up being disappointed by their reviews, so reviews lower morale (which is horrible).

I think this can be improved by:

  • Better criteria to capture range of skills 
  • Review performance more often (i.e. 1:1s) so quarterly performance reviews are not surprises
  • Performance review to improve and identify strengths

Most of these ideas are explained in a Riot tech blog post by Mike Seavers :-)

Top Five (Wrong) Reasons You Don't Have Testers
Goooooood piece. Unfortunately I can't say I have much experience with testers, I've never worked with any. The reasons he provides are:

  • There will always be bugs
  • It is cheaper to catch them before
  • Tough to fix bugs after shipping
  • Damaging to let customers be your testers

He also suggests some interesting ways to get good testers.

Human Task Switches Considered Harmful
Gives a cool graph of task switching for CPUs to make an analogy with humans. Assigning 2 jobs severely slows someone down, so it is better to work sequentially. The worst thing to do for productivity is to task switch, and it gets worse the bigger the cost of task switching.
My takeaways:

  • Hiring new people is brutal; it's a big time sink to not only train new people but also to switch from your main task to help. Sorry team.
  • 1 owner per 1 story is good- a one person team doesn't have to task switch

Things You Should Never Do, Part 1
The thing you should never do is don't scrape old code! Often a rewrite suffers from the same mistakes, plus the added negative of the time it takes to reach feature parity. It is easy to want to raze it to the ground and restart, especially since as programmers you often trust yourself more. This is a fine argument for the careful refactoring of league instead of completely scrap it, especially because the time spent redesigning and rewriting is time we aren't bringing new stuff for the players.

The Iceberg Secret, Revealed
Most people focus on UI, even though UI is maybe 10% of the work. This is an interesting problem, and I've actually noticed it at work too. UI improvements are much easier to tangibly see than 2 weeks of solid coding and improvement under the hood. One sprint, for our demo, we rushed to make some UI improvements in a day and made our demo look much more impressive despite the 2 weeks of work we put into adding functionality. This brings up the interesting question of how to properly capture progress and show progress. Perhaps a way is to measure by numbers (story items completed)? 

I think it is also noteworthy that the problem is mitigated greatly when those you report to and demo to are technically inclined.

The Law of Leaky Abstractions
The article is pretty similar to Back to Basics. Abstractions leak, and when they do, to fix them, you have to understand the underlying, complexity being abstracted. Abstractions and tools save us time working, but they don't save us time learning. This is a fine argument for understanding pointers. Another good example is git. I know the commands and I can commit stuff but when things goes wrong I have no idea how to fix it because I don't really have any idea how git works.

Lord Palmerston on Programming
Leaky abstractions mean that we live with a hockey stick learning curve: you can learn 90% of what you use day by day with a week of learning, but the other 10% might take you a couple of years catching up. This sets the difference between really experienced programmers and novices- the base of knowledge to draw upon and build upon. Because of this, Joel suggests that every team have at least 1 domain expert. It is true that new tools pop up all the time, but when I was struggling with some javascript and unfamiliar APIs this summer, my coworkers (equally inexperienced) knew what to look for and had an intuition for learning. Perhaps that experience and knowledge is what most separates a senior engineer from a new grad? The difference feels like doing surgery by sawing everything open and learning as you go, vs using a scalpel (traipsing through the chrome sources is sawing the patient open to inspect the innards).

Strict metrics to measure success often fail, and people will work to those metrics instead actual progress. I think that instead of just applying strict and formal measurements, they can be accompanied by direct supervision to really understand the value that each person is bringing to the team and to the company. Then measurements become a way of understanding what good looks like instead of a rigid unsupervised bonus tool. The downside is, of course, you'll need a lot of managers.

What is the Work of Dogs in this Country
This is an article about the idea of eating your own dog food, or trying your own code as a user. Before a demo I did this summer my dev loops were so tight they weren't following the workflow the way a real user would use the app, so when I demo'd, the workflow was buggy. It is super embarrassing to test comprehensively, go to a demo, and have it fail immediately all because you never tested creating a new page and instead edited an old one... eat your own dog food.

Getting Things Done When You're Only a Grunt  
You can improve process even when you're a grunt!!! Things can always be made better by you and good process can be demonstrated by you to influence your team. This is important on bad teams, but also important on good teams (although you will have more impact on a good team). Process is very important and should be contributed to by all members of the team. Also a good lesson on how demonstrating by example is much stronger than speaking.

Two Stories
Joel shares two stories from two different software companies and highlights the importance of ownership and responsibility. Hire smart people and then empower them to make the right decisions! Even as an intern I had a lot of decision-making power and it was very motivating. I think management works the best as the ultimate support, and the less micromanaged programmers the more effective they are.

Big Macs versus the Naked Chef
Joel uses restaurants as an analogy for scalability problems in software companies. Big Macs are easy to make everywhere, so McDonalds pop up like weeds. On the other hand, high end restaurants don't scale well. It is a pretty good piece, a good explanation of why scaling is difficult and why there are so many useless methodologies out there. Highlights the importance of hiring the best and being careful- hiring programmers to scale your company is tough! 

Nothing is as Simple as It Seems
Always try to reduce risk, and one of the biggest risks is scheduling risk. Design will help you figure out the schedule and figure out the things that are not as simple as they seem. Don't rely on your gut or your first impressions unless you're very experienced and even then design will help you understand what you need to do and in what pieces. Of course there is the danger of over engineering and over designing wasting a lot of time, but I think in general incremental and careful design will always help you write better code faster.

In Defense of Not Invented Here Syndrome
Joel argues here that it is not always the best to look for third-party solutions and sometimes it is OK to have not invented here syndrome and try to build stuff in house. I agree that for certain things you have to build them in house and that includes your core product, especially since some solutions require custom work with things that do not exist open source or third-party and it would be a lot of work to adapt and be a lot less flexible and powerful. However I am skeptical that there is no skeleton or base on what you can build and contribute and often times I find that people default to building it themselves instead of seeing what has been done already.

Strategy Letter 1: Ben & Jerry's vs Amazon
He compares two different company strategies, Amazon and Ben & Jerry's. Ben & Jerry's is slower, has more competition, less risk, weaker network effect, and culture is important. On the other hand, Amazon is fast, has no competition, burns through cash, more risk, and cannot maintain culture. This article is great insight again into understanding your space. This is super important to start a company but also important to keep in mind when you're joining a new company! Riot is big but definitely feels more like a Ben & Jerry's- we care a lot about culture and there is heavy competition in genre and in gaming.

Strategy Letter II: Chicken and Egg Problem
I love these articles, they are so insightful. He presents the chicken and egg problem of merchants and customers, software and users for platforms. It is hard to drive users to new platforms without software written for the platform, but it is hard to drive programmers to write software for the platforms without users. His solution is backward compatibility to artificially create the chickens/eggs (or as Joel calls it, bring your own truck of chickens and eggs).

Strategy Letter III: Let Me Go Back!
Another great one. Addresses barriers of entry to customers, which need to be reduced and addressed them to encourage switching. He focuses on a surprising barrier of entry in particular: the stealth lock in.
It is a bad idea to try to lock in customers early, because they are non-existing. Instead, lock-in just locks out potential customers. An important part of driving growth is to make it easy to switch by making it easy to go back. Worry about retaining customers once you have them.

Strategy Letter IV: Bloatware and the 80/20 Myth
The big idea: big programs aren't that bad, space is cheap, and you shouldn't trim stuff in exchange for "bloat." Not sure about this one though, performance and memory usage is pretty important to League so various potatoes can play it, and for many mobile apps a slim app is pretty important to many users. I think maybe the approach is a happy medium where you treat size as another feature and don't let it dominate necessary features. For example, notepad is smaller than sublime but I sure as hell won't program in notepad.

Strategy Letter V: The Economics of Open Source
This article is mostly about open source economics, and why companies support open source. The underlying economic principle is: when complement prices go down, demand goes up and companies make more profit. Open source commodifies the complement, driving prices down and thereby increasing the demand of their actual product. As an example, the complement of league skins is esports and the game itself. They are free to watch and play, so demand for in game content goes up. Open source is interesting to think about in terms more than just idealistic open source is awesome and we should free the software. There are also strong economic incentives I haven't thought of before, such as working from a base already built by others, building a brand and attracting talent by contributing to the software community, and getting "free help."

How Microsoft Lost the API War
This is the kind of stuff I'd love to read about in a book. The article details what the Windows API is, why it's so important to Microsoft, how it lost the API war, and the consequences. Super interesting material, and ends with a convincing argument that the new API is HTML.

Perhaps the best support for his arguments is that as a new programmer, my last brush with rich client applications with Windows was at the hedge fund I worked at in my freshman year. Since then I've mainly worked with web based APIs and to be honest kinda see VB and .NET as outdated (I don't personally know any programmers who are experts in those areas).