“If you know the way broadly, you will see it in everything” - Miyamoto Musashi
“If you only have a hammer, every problem is a nail” - some jerk
Many of my previous posts have looked at a very zoomed out picture of economic growth. How does a whole economy grow? What systems and organizations produce nation-wide production? These are important questions, but in a free society, there is no lever to make everything better all at once. The levers are wielded by a large group of managers, who themselves are not cohesive. Talking about “managers” does not make sense because most managers do not share goals. This book by Will Larson delves into the particular challenges of engineering management. He reflects on his time at the company Stripe as well as Digg, as well as what he has learned from other engineering managers. While Larson’s book does not solve all problems broadly, he does solve many narrow problems. Given the track record of people who solve all problems all at once, I applaud Larson’s efforts. Stripe is growing quickly at the time of this writing (2/5/2021), so they have overcome some of the initial problems with inertia and power struggles that plague other organizations.
No Problem
Larson defines a sort of axis of problem-solving by effort. Problems that can be solved quickly are solved by a process. Problems that need to be solved permanently are solved by cultural evolution. Organizational design lies in the middle. I usually think of culture as fairly ephemeral and frothy, not brooking of any macro-level changes, while processes are more ossified. The difference here is that my perspective was too zoomed out. Changing the culture of the world would be insanely difficult. Any message you have would compete against the latest song, video, or article that was written by any of the other billions of people with something to say. On the other hand, I think of processes from a customer's point of view when I am keenly aware that I am being processed. The DMV has a process to give you a license. Internet providers have a process to prevent you from leaving your contract. The difference is that, from an engineering manager’s perspective, both culture, design, and processes are malleable. Culture can be changed by modeling the correct behavior to a small group of people. Design can be planned and executed. Processes can be iteratively made and improved quickly. Managers do not have unlimited agency, but the massive ship of culture of the world becomes much more manageable at a smaller level.
In building a team, Will specifies that innovation and maintenance should stay together. In engineering, this means that there are two distinct classes of tasks solved by a single team, making new stuff and preventing the old stuff from breaking. From personal experience, the old stuff can break fairly fast sometimes, which makes innovation difficult. Despite this challenge, putting these two tasks on the same team makes sense. If you have an all-innovation team, they have weaker incentives to make sure that their code lasts, while the all-maintenance team would feel burdened by drudgery. This reminds me of a Richard Feynman situation where he rejected an offer to stop teaching and go do pure science at the Institute for Advanced Study. He said: “Nothing happens because there's not enough real activity and challenge: You're not in contact with the experimental guys. You don't have to think how to answer questions from the students. Nothing!” (Surely You’re Joking Mr. Feynman). The common problem is that innovation is valuable. Since we like innovation, we should try to get more of it. Therefore, we should have our smartest people just innovating all the time, and we will get a ton of innovation! While this sounds great, singular obligations to innovation seem to not work so well, without even considering the pride of the maintenance team. Hopefully, maintenance is even good for innovation. Maintaining a particularly painful feature will teach you a lot about the importance of not building things badly.
The Ability to Change
The hard part about being alive is things change. In management, change presents a particular challenge because the system they handle is so particularly chaotic, yet must be mastered. Larson defined efficient change as being governed by systems thinking, metrics, and vision. Systems thinking roughly means observing the links between events, and works when you want to understand all causes of an event, rather than looking at the direct causes of an event itself. I might be assuming too much, but systems thinking seems very difficult in a fast-paced environment. Code will tell you the full trace of where you went wrong, and you can reproduce certain errors in order to find the proximate cause of your problem. Organizations, on the other hand, usually do not reproduce issues. If you need to solve a problem quickly and correctly, this might end up being very difficult because the more subtle causes of an event might not be immediately obvious. I think the counter to this is that keeping track of metrics will make sure that anyone can see the causes of a failure, but this in itself might be difficult.
Goodhart’s Law would seem to ensure that any metrics that are tracked will eventually cease to be useful. The more common version is ‘When a measure becomes a target, it ceases to be a good measure’. However, the original text seems to more closely get at the point here: “Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes”. This is a problem because the point of tracking metrics is to control outcomes, but any person who can see what is being tracked will probably begin to massage those metrics to look good. Systems thinking seems essential to understand complex organizations, but complex organizations are able to observe your actions and change themselves to what they think you want to see. This could be avoided in several ways. First, you track metrics without telling the organization what you are tracking. This might run into ethical issues, as you might be spying on people at that point. You could also work on regularly changing your metrics so Goodharting the metrics becomes troublesome. The most entertaining outcomes would involve employees constantly changing what they do in order to maximize the metric of the week. Finally, creating a vision would be more realistic, intelligent, and hopefully avoids offering perverse incentives to employees.
Larson defines vision as ‘aspirational documents that enable individuals who don’t work closely together to make decisions that fit together cleanly’. As broader alignment, this works well to make sure that you avoid the worst perverse incentives. The tradeoff is that vision documents are not going to be as clear as a metric. “Make a product people love” is harder to follow than “Increase average user engagement by 5 seconds this month”. The good part about the previous contrast is that the vision will prevent much of the dark arts of software design, where you might try to capture or confuse users in order to make them watch an advertisement. That would satisfy the metric, but not the vision so it would not be done. Still, creating a good vision would be very hard in practice! Stating what success looks like is awkward, and holds you to more ambitious goals than what would normally make most people comfortable. The resolution is deciding that you will accomplish your goals anyway, but hey, I’m just a blog.
Who Narrates Your Career?
We hear it all the time. “Make sure you are developing your career” or “Try to get that promotion”. Many people ask their managers about how to manage their careers, but managers do not always have the tools to give good advice. Particularly, the simplest advice is to go from engineer -> engineering lead -> senior engineer -> staff engineer -> magical wizard. This is a somewhat difficult path to follow, because if everyone on your team sees the same advice, the competition is fierce. Don’t get me started on learning the spells to go from staff engineer to magical wizard! But how do we advance then? Larson gives good advice, “An important part of setting your goals is developing areas you’re less experienced in to maximize your global success, but it’s equally important to succeed locally within your current environment by prioritizing doing what you do well”. Even if the advice is legible, it isn’t easy. How would an engineer do this? Larson thinks that you can figure out the details yourself if you can write out your career goals. I hope this is true since very few people have access to really good and fine-grained career advice.
Counterfactuals in careers are hard. What if you did just a little better on that interview, or your boss was in a better mood on that day where he decided whether or not to promote you? As a vision of where to go, Larson has interesting advice, “Go somewhere disproportionately valuable to you because of who you are and what you want”. Go be a baker! If not that, then perhaps we should be figuring out which parts of work we actually enjoy and try to maximize the amount of time that we spend exclusively doing that. Since humans have more than one thing they like, you might even be able to find a sufficiently weird position that exists just waiting to be filled by you. This does not exactly absolve you of responsibility to get better at your craft, but at least you can get better at things that not everyone is doing. Prepare for roles that sound cool, rather than the linear next step above you. Time will tell if that works for me, but I’m prepared to follow his advice. Set a reminder for 40 years from now to see if my career worked out.
Conclusion
I did not exhaust the material in this book. I don’t think I am the right person to comment on whether or not Larson does a good job on conducting a good interview process for example. Nonetheless, it was informative to see how the process works on the business side. Most directly, this gives me a way to find out more about my manager, instead of groping in the dark to figure out what it is that managers do with their days. Even better, if some managers took the advice in this book, I think I would have a much better model of what they are trying to accomplish, instead of wondering why metric X was chosen over metric Y. Apart from that, I think I will need to revisit this book if I ever become a manager, and we will see how well I can follow advice.