Hi folks,

This is the weekly digest for my blog, Irrational Exuberance. Reach out with thoughts on Twitter at @lethain, or reply to this email.

Posts from this week:

- Mailbag: Evolving your engineer career beyond the career level.
- Interviewing senior engineering leaders.

Mailbag: Evolving your engineer career beyond the career level.

Recently I got an email asking about evolving your engineering career after you’ve hit the career level, but before you feel like you’ve accomplished what’s important to you. Here's the lightly edited email:

I'm a mid-career engineer with experience in the technical, management, and leadership tracks. I find it challenging to continue leveling up. I'm working on it, reading articles and books, but it's hard, sometimes, to know what to do next. My network in the Bay Area is small because I moved from Mexico, how do you recommend finding people that can act as role models for Staff/Principal engineering roles, and do you have any frameworks/books/resources you recommend for building technical roadmaps?

This email came at a good time for me, because I've been spending more time time reflecting on the lack of support and structure for folks in what I'm calling "staff-plus" engineering roles. There are books like The Manager's Path that explain the engineering manager's career, but few high-quality resources for engineers.

There are a few different topics in the note, that I'll approach in order. First, what are resources for continued development? Second, how do you find (and ideally meet) staff engineer role models? Finally, any resources for developing technical roadmaps in particular?

Resources for continued development

Each job is going to develop your career in different ways, sometimes with deep technical learning, sometimes by developing your network, sometimes by accumulating the universal lubricant known as "prestige." The most effective way to develop yourself is to align your current role with the dimension you're looking to build on.

If your focus is on developing your technical and technical leadership capabilities, then your best bet is to find a company which needs this sort of work, typically a company that doesn't yet have senior engineering leadership in place or whose rapid growth is creating frequent new gaps to address.

As each of those gaps widen, they turn into "staff projects." These are projects that are considered complex and important enough that the person who completes them should be considered a staff engineer. Most folks become staff engineers through the completion of at least one staff project. The best way to continue developing yourself is to successfully deliver staff projects, this is true even if you're part of the project rather than the project leader.

The other angle of developing yourself through doing is to take on broad responsibilies. This might be designing and leading an architecture group, defining the organization's development process, or formalizing mentorship for engineers earlier in their career at your company.

Continue to take on staff projects and broad responsibilities, get them to a steady state over a one to two year period, and then transition them to someone else who can learn from them. This will foster development across the organization, and allow you to move on to learn new things yourself. (I don't think you can move on sooner than 1-2 years, because maintaining and operating the thing you built is a key part of the learning.)

I do believe it helps to supplement this learning-through-operating with learning from the outside, and some recommended readings are:

• Thriving on the Technical Leadership Path by Keavy McMinn
• A Philosophy of Software Design by John Ousterhout
• Building Evolutionary Architectures by Ford, Parsons and Kua
• Accelerate by Forsgren, Humble and Kim
• Software Design X-Rays by Adam Tomhill
• Reclaim unreasonable software
• You can't reason about big balls of mud

For more, check out my favorite books and papers.

Finding role models

The easiest way to meet these folks is joining a company that's large enough to have an existing community of staff-plus engineers. You don't have to work there forever, but working at an Airbnb, Facebook, Google, Netflix or Stripe will leave you with a permanent network of these folks. Similarly, it takes a certain scale to routinely generate staff-level projects (all companies have some staff-level projects), which means that the rate of staff engineer creation is also higher at these sorts of large companies. This makes you more likely to meet folks making the same transitions that you're working on.

If joining such a company is practical, don't dispair, you can still organically develop your network. I've previously written up my approach to meeting people, and I've found that it basically works if you put in the required work.

In terms of being a bit more concrete, I brainstormed a list of some folks who I think are great role models for this sort of work. The best known folks are generally the least productive to reach out to, because they're already getting a bunch of inbound requests, but hopefully they'll serve as inspiration for the sort of roles and folks you might want to reach out to:

• Cindy Sridharan works at Apple so I have no clue what her official title is, but has written some amazing stuff, writing thorough explorations of broad technical topics like: Testing in Production, the safe way, A decade in review in tech
• Julia Evans was a staff engineer at Stripe before she started focusing on Wizard Zines full-time, and has mastered her own approach to educating folks on traditionally complex topics
• Keavy McMinn is a Sr Principal Engineer at Fastly, and is writing the best content on staff engineering right now: Thriving on the Technical Leadership Path, Where to Start, Technical Research and Preparation
• Keith Adams, who was Slack's Chief Architect, is isn't quite as prolific of a public writer, but among others wrote Taking PHP Seriously, a great post on not fast-following technology trends
• Tanya Reilly, Principal Engineer at Squarespace, and her Nobody could have predict this, Nine Takeaways from the DevOps Report, and Surviving the Organisational Side Quest

There are, without a doubt, bunch more amazing folks out there, these are just a few examples!

Developing technical roadmaps

The final question was around developing technical roadmaps, which I find a somewhat challenging question to answer because "technical roadmap" means such different things across companies. I've rarely seen any company write an explicit technical roadmap, which is a shame because I think they'd be very helpful.

My best advice is to write an engineering strategy (Magnitudes of exploration is one example), combine it with a protected project budget reserved technical investment (talk, blog), and build your roadmap by resourcing the projects identified by your strategy with that protected budget.

This is definitely a topic that folks frequently ask about, so I will brainstorm if I have something more useful to say.

Interviewing senior engineering leaders.

A friend recently reached out for advice on interviewing and hiring senior engineering leaders. I’ve spent a good deal of time on this topic over the last couple years, starting with partnering with Laura Hilton to design Stripe’s interview loops for engineering leadership, and more recently going through a search for my own role. Leadership hiring is particularly interesting as a window into an organization’s psychology: for the highest stake decisions, do they turn to structure or to intuition?

For many topics I can write the algorithm that I’m confident will lead to a pretty good outcome, but this definitely isn’t one of them. Rather these are just the notes of what I’ve seen so far.

For clarity’s sake, I’m defining “senior” as managing an engineering organization of 30+ folks, likely with a Director-plus title at a pre-IPO company or the VPE/CTO at a smaller startup. These ideas will apply to folks outside of these narrow lanes, but will require some adaptation.

Why this is hard

While hiring can be difficult in general, I’ve found hiring at this level to be challenging in a few particular ways. First and foremost, companies want so much from these hires: someone who has managed a team of 300+, has experience in a particular niche, and has worked at a trendy company. These intersecting requirements empty out the candidate pool, leaving behind a small number of folks that are highly sought after.

Further, you’re hiring for a skillset you’re missing, which makes it hard to know exactly how to evaluate the candidates. Worse, while these folks will fit many aspects of your culture to some extent, you’re also hiring them to bring a new influence into your culture, which may generate mixed results from a overly rigid approach to assessing culture fit.

Neither can you rely on hiring volume to teach you what you’re looking for, as you’ll hopefully only hire one such candidate at any given stage of company size and complexity. By the time you hire another CTO, you’ll be looking for a very different set of skills.

Finally, senior folks who invest in preparation tend to interview really, really well. Well enough that it’s very difficult to pierce the veil and get an accurate read on who they are. This is especially true if you rely on unstructured and intuitive evaluation.

The Standard Process

Well, there isn’t really one standard process, but the majority of what I’ve seen and heard of compresses into something fairly uniform. The interviews that you’ll generally see in these processes are:

• Topgrading where they dig into your accomplishments and trajectory over the course of your career. These look for career acceleration across each role. Note that most places won’t call these topgrading, and may not even be familiar their interview is influenced by topgrading.
• Deep-dives to ask about your experience in relevant topics, such as building an engineering brand, hiring, structuring organizations and so on.
• Group presentation to execs, peers or team about a relevant topic. One frequent topic is around your first ninety plan.
• Group Q&A to evaluate your ability to speak extemporaneously, as well as to get the room excited about working with you.
• Working session with would-be manager or peers on a relevant problem.
• Fit conversations with critical stakeholder, including the folks you’ll manage, to learn about you and understand your approach, beliefs and style.
• References that you provide to learn about you from your coworkers.
• Backchannel references to learn about you, ensuring they don’t only hear from folks particularly positively predisposed towards you.

The majority of processes I’m aware of are some combination of those, over the course of six to twelve distinct interviews, emails, phone calls and texting threads.

My sense is that this approach does a sufficiently good job of assessing folks against the hiring company’s current needs, but often struggles to assess “around the corners” of role evolution over the next year or two. The general lack of structure also, in my opinion anyway, generally does a better job of identifying gaps rather than strengths. I often struggle to reconcile interview performance with someone’s considerable successes, and I’m certain at least a fraction of that is due to this process extracting insufficient signal.

Are backchannels unethical?

As an aside, when I’ve chatted with folks about the standard process, the most contentious area is around the acceptability of backchannel references. I’ve been a long-time rejector of backchannel references, and can recite all the arguments against them.

Backchannels disadvantage the very folks who already have structural inequalities hindering them at each step. The sort of folks who are called intimidating whether others might be described as bold; threatening where others might be described as boisterous. They can also be little more than gossip if you chat with folks who don’t work closely with the individually being back-referenced. They can expose the candidate’s job search in a way that creates considerable undue grief. Further, they represent a distrust in your ability to evaluate folks well – shouldn’t you trust in your process more?

I hear all of that criticism against the backchannel, and I agree with it: backchannels introduce considerable bias along with all the other issues.

However, I still think they’re necessary when hiring for executives. Rather than skipping backchannels, I think your obligation is to do them well. Listen to feedback very carefully and knowing full-well that it’s packed with bias. Ensure you ask folks who worked with the individual closely. Let the candidate know that you’re planning to do backchannel references and try to incorporate their guidance. (That said, as a candidate I’ve always accepted that this will happen without folks letting me know. It’s just “how it works” in my experience.)

Getting to the heart of things, I think backchannels are essential because executives are given immense opportunity to impact the lives of folks in an organization. Executive candidates are, with some exception, far more privileged than the folks they’ll soon be responsible for, and you owe it to the team to ensure that the person you’re entrusting them to hasn’t left a trail of ethical concerns behind them. Inconveniencing one relatively privileged individual to prioritize the needs of dozens or hundreds of relatively less privileged folks strikes me as totally reasonable behavior.

That said, I’d love to find better ways to acquire backchannel signal with fewer downsides. If folks have found better approaches, please send them over.

Demonstrating expertise

The approach that we took in Stripe’s most recent senior engineering interview refresh was to focus on folks demonstrating their skills rather than describing their skills. We brainstormed the activities that experienced managers were spending their time on, and designed interviews to emulate those activities. This approach is used to some extent in “the standard process” presentation interview, but we wanted to go much further.

Much of senior leadership is editing and providing advice to folks on your team, so we tested an interview where folks had to give feedback on a poorly written set of team goals. Another common activity is reviewing an organizational health survey, something along the lines of CultureAmp, so we synthesized an organizational health dataset and asked them to analyze it, identify themes and propose how they’d address each of those themes.

We also retained several roleplay interviews, which have long been a mainstay in Stripe’s manager interviews. These roleplays focused heavily on remaining empathetic while giving clear, direct feedback.

I found this approach gave a very clear sense of how directly and seriously leaders had engaged with these specific areas that we considered integral to successful senior leadership. They’re particularly good at extracting depth (relative to asking folks to describe a time they did something along those lines) and they get signal even if the candidate is an unstructured communicator (while still capturing the signal that they are an unstructured communicator).

This demonstration of expertise approach is certainly not the only sort of interview you should do, but personally I think it works quite well and I hope to shift more of my future interviews that direction.

What else?

It feels like there must be better techniques out there, so I asked, and got some interesting perspectives:

• Yvette Pasqua proposed more topics for demonstrating expertise like translating business strategy into technology strategy, navigating your P&L statement, mediating complex conflict
• Rushabh Doshi shared his post on Hiring Engineering Leaders, which links to Martin Casado’s Hire a VP of Engineering, and in particular pushes back on backchannel references
• TechTello suggested evaluating against the role a year out (not the role today), and digging into their willingness to take unpopular opinions, something negative they’ve learned about your company and how they’d try to change it, dig into your company values, design a layoff, and their questions around the company and role
• Paul Robinson referenced a post he wrote about The Problems of the CTO Role, which includes a short section on hiring focused on understanding what sort of CTO/VPE you wish to hire

I’m sure there are other good approaches out there as well.

Recommendation

Pulling all of this together, my general recommendation for interviewing senior candidates is to identify the tasks they’ll need to perform, design a loop to have them demonstrate mastery of the related skills, check for alignment on company values, and keep doing backchannel references.

That's all for now! Hope to hear your thoughts on Twitter at @lethain!