How we built our software engineering career framework
Nick Moore, Chris Pine, Felix Becker, Patrick Dubroy
Sourcegraph provides universal code search: a navigation engine for understanding your code. We’re also an all-remote company with a global team. See our careers page for current openings.
Engineering managers felt empowered but unconfident. It was the summer of 2020 and the company was seven years old and about 50 people strong (about half of whom were engineers). In the year to come, we would grow, rapidly hiring more engineers.
But scale revealed problems with the way we facilitated career growth. Before 2021, growth happened through individual conversations and high-impact work. When the engineering team numbered in the 20s, this worked. As we added engineers, however, engineering managers felt empowered to promote individual contributors (ICs) but unconfident they were calibrated with other engineering managers.
Individually, engineering managers and ICs cared about career growth but the conversation around growth was unclear and at times uneven. We didn’t have a common language and without a shared understanding of growth, misunderstanding and bias would eventually be inevitable.
The solution? A framework that would align the entire engineering organization and create consistent, fair, and scalalabe conversations about growth. This is the deep dive, behind the scenes story of how we collaborated to form an engineering career development framework.
If you want to skip to the results, you can read the handbook page or read the summary in this post. But if you want the story, and to see how you can apply our learnings at your company, read on.
Engineering managers, assemble!
To create the career development framework, our VP of Engineering Nick Snyder formed a working group. We took on the DACI framework, meaning one person would drive the project, one person would approve the project’s results, a select few would contribute to the project, and everyone else would be informed.
The working group consisted of four people:
- Chris Pine, engineering manager, was the driver.
- Nick Snyder, VP of Engineering, was the approver.
- Felix Becker and Patrick Dubroy, both engineering managers, were contributors.
Everyone else in engineering was to be informed of the project’s progress and results.
The team, sans Nick, met once a week for half an hour each. We did most of the work asynchronously and reported, weekly, to engineering leadership.
In accordance with our transparency value, we worked in the open. All proposals were in Google Docs that we shared with the entire company. We also asked engineering managers to bring it up in their one-one-ones. If people weren’t comfortable with that directness, we also created an anonymous feedback form.
Even candidates got a peek at the framework:
We united around a passion for career growth. We often spent time talking about the specific, minute wordings of different sentences and bullet points. We considered every sentence and bullet point from multiple angles, asking questions like “What if someone takes this too literally?” and “How do we word this in a way that we don’t encourage the wrong sort of behavior?”
Sourcegraph prior to the framework
Prior to the framework, engineering managers weren't confident they were talking about career development in a way that was consistent with other engineering managers. Without a career framework, engineering managers and their reports were left without a shared language.
This worked when we were small but threatened to become dysfunctional as we scaled. As we added teams, we worried engineering managers were having, or would start having, completely different conversations about what growth looked like.
It was clear these challenges were only going to increase over time. If we were to continue scaling, and we certainly wanted to scale, then we needed to start solving these problems for the future. But first, we had to identify them.
Some engineers (and engineering managers) felt lost
Some engineers didn’t know what their options were or what our expectations were. More and more engineering managers were reporting this confusion, finding that there were engineers who didn’t understand how career progression worked at Sourcegraph.
Engineering managers wanted to level incoming hires fairly
Without a framework, engineering managers had to spend time leveling candidates on a case-by-case basis. As we scaled and made more and more offers, we found we needed the ability to evaluate an ever wider spectrum of hires. We needed to know we were consistently making fair offers.
Fairness was hard to ensure. How, for instance, could we know that hiring manager A was making an offer that’s consistent with what hiring manager B would make? How, too, do we level someone who’s coming in if we don’t have a good sense of who the IC2s or IC3s are? The need for a framework only became more apparent.
Bias was inevitable without objectivity
Before the career development framework, career growth was unclear, and more often than not, unclearness tends to benefit the overrepresented.
There was a lack of built-in fairness in the previous system. The system was not set up to be fair and at some point, someone was going to be treated unfairly.
A framework forces managers to compare their assessment of how someone is doing and compare that to an objective list of expectations. Cultural differences, for example, can produce different assessments.
Our working group, based on their own experiences, noted that what’s considered a positive statement in Germany, for example, might be different from what’s considered a positive statement in the U.S. (Patrick, who lives in Germany, pointed out this particular dynamic). In other words, what might feel like the straight reporting of facts in one culture might feel like bragging in another culture.
The framework provided concreteness, which shifted the evaluation process closer to objectivity. With a set of example behaviors, managers can check their subjective opinions against specific, concrete models.
How we researched the landscape of career development frameworks
We didn’t start from a blank page. Many companies, Sourcegraph now among them, have published their career development frameworks online for anyone to see and borrow from (examples of which we link to below).
We didn’t want Sourcegraph to be a place where we unthinkingly copy what someone else did. The career framework was an opportunity for us to innovate and to produce something new for the industry.
And so, we carefully studied available frameworks, identified the differences between the frameworks, and determined which aspects we felt fit Sourcegraph.
Some of the examined companies included:
The research culminated in a nine-page document. Here are some of the major differences we found.
Cycle frequency
A major logistical difference between these frameworks was cycle frequency: how often career development reviews occurred. Generally, companies fall into three options: annual, semi-annual, and quarterly.
This may feel like a small difference but it plays a major role in how engineering managers and ICs feel about reviews. Too often and the reviews become cumbersome, something merely to get through; too infrequent, and the results will lag behind the reality of a given IC’s work.
Axes of evaluation
An axis of evaluation is a category by which an engineering manager evaluates an IC.
Two major differences became clear: different companies have different numbers of categories (typically between three and six, but as many as 16) and some companies add axes as ICs advance to higher levels (e.g. a leadership axis).
Square, for instance, uses three axes for L3 and L4: Ownership, Teamwork and Collaboration, and Technical Execution. But once ICs reach L5, Square adds Team Building and once ICs reach L6, Square replaces Collaboration with Leadership and Collaboration.
Progression
Progression–how engineers advance to higher levels–breaks down into two different types: across-the-board and single-axis.
In across-the-board progression, a Level 3 IC, for example, would have to meet expectations for a Level 3 IC in each axis. In single-axis progression, an IC might, for example, be a Level 6 in a technical axis and Level 3 in a communication axis.
Medium and Rent The Runway both use single-axis progression; Buffer and CircleCI both use across-the-board progression.
Designation
Designation is how a company refers to different levels of ICs. The examples we looked at came in three types:
- Titles like Senior Engineer or Staff Engineer.
- Numbered levels like L2 or L3.
- Descriptive levels based on scope.
Spotify is an example of the last format. According to the titles section of their career framework, an IC’s “step” (Spotify’s term for level) doesn’t map to a title. Internally, Spotify uses titles to describe an IC’s scope of impact and an IC’s given role. Externally, employees have “tremendous flexibility” for representing themselves.
Scope requirements
Multiple companies we researched divided their levels by scope, meaning that as engineers advanced, the scope of their impact was expected to change.
Spotify, for instance, matches levels directly to scope:
- At the Individual Step, ICs are “focused primarily on being a useful contributor.”
- At the Squad/Chapter Step, ICs are “a resource for your chapter or your squad.”
- At the Tribe/Guild Step, ICs “have an impact across squads or chapters.”
- At the Technology/Company Step, ICs focus on providing significant support for “tech-wide or company-wide initiatives.”
Generally, the expectation is that as an IC advances, their impact increases in proportion with their level.
The framework roadshow: What the team had to say about work in progress
The career development framework process was transparent from day one but up until the end of the project, most of the responses we got were variations of “LGTM!”
It wasn’t until the end, when the work was mostly done, that more substantive feedback started to come in.
Even so, it wasn’t as much engagement as we would have liked. We reached out to engineering managers and asked for their feedback directly. We also paid special attention to engineers who had been at Sourcegraph a long time so as to make sure there were no concerns from the people who knew Sourcegraph best.
In some cases, those feedback requests resulted in a cautious thumbs-up. Engineers liked the idea of a framework, and wanted to try what the working group had produced, but also wanted to remain open to changing the framework if it didn’t work. Luckily for them, that was the plan all along (see the calibration to come section below).
We found it surprisingly easy to get buy-in. A potential cause of this is how much time we invested into the effort. We always took the time to hash out different situations and possibilities. Our detailed discussions ensured that most eventualities were built into the framework by the time the rest of the team saw it.
With a thumbs up (some of the thumbs being cautious and some enthusiastic), we formed the career development framework.
The Sourcegraph career development framework
The resulting career development framework, based on our research as well as feedback from the wider organization, prioritized a few aspects:
- Simplicity: where other frameworks were overly complex, with multiple different categories and levels, ours was simple. We wanted ICs to be able to reference it easily and efficiently as they went about their work. There was a tradeoff–other companies’ frameworks may indeed be more comprehensive–but we felt it was worth it. Better, we thought, to focus on the practical, real-world value released by something easier to use.
- Clarity: where other frameworks were long, having 10 or more categories and complex point systems, ours was short and clear. We wanted ICs to be able to grasp the framework intuitively and avoid getting lost in the weeds.
- Impact: where other frameworks emphasized scope, ours emphasized impact. At all times, we wanted engineering managers to be asking their ICs, and ICs to be asking themselves, “What is the impact you’re having?” That impact might be cross-team and that impact might be on one team, but it needs to be significant.
- Example-driven: where other frameworks provided checklists of items to be completed, ours provided illustrative examples and models. We wanted to design a system that couldn’t be hacked or gamed, a system where ICs have to consistently demonstrate a certain level of impact.
- Bias-resistant: where the lack of a framework opens a company up to promoting different people for different (often unfair or unjust) reasons, ours provided objective criteria by which engineering managers could evaluate team members.
Below, we highlight some especially important aspects of the resulting career development framework, all of which you can read about in detail in our handbook.
Expectations
Levels for Sourcegraph ICs range from L1 to L7 and each level contains three axes:
- Proficiency
- Execution
- Teamwork
A fewer number of axes is easier to assess and discuss. More axes, we found in our research, led to confusion because the distinctions between can get fuzzy. You can imagine how a given behavior might be hard to categorize in either a “Mentorship” or “Community” axis, for example.
Our framework notes that “what is listed in the level descriptions are example behaviors, and not checkboxes for promotion.” Our framework also makes clear that “doing everything listed there is neither necessary nor sufficient for a promotion.”
The above three axes cross-influence each other. You can’t, in other words, be great technically while having no teamwork or execution skills.
We also wanted to avoid what’s often known as the “rockstar developer” or “10x programmer” character, the person who ships code but refuses to work with the rest of the team. We didn’t want to reward someone who has a strong focus on one axis and neglects teamwork and execution or vice versa.
Progression happens across the board, which we believe will simplify the mapping of levels to compensation bands and better ensure fairness and consistency.
Promotions
Promotions happen, according to our framework, “when your manager can make the case that you’ve had at least one quarter of high performance at your current level, and one quarter performing at the next level, in all three of the categories.”
Our framework notes too that it takes time to demonstrate the consistent meeting of expectations. This is important because no one wants to promote someone to a level they’d struggle in.
In-band compensation increases can happen at any time but promotions are considered during quarterly talent reviews.
Talent reviews
Talent reviews happen once a quarter: far enough apart for significant growth to happen in between and often enough that the stakes of each review aren’t intimidatingly high.
Before talent reviews, engineering managers evaluate each of their ICs according to their corresponding level descriptions to assess whether each IC is meeting, exceeding, or struggling to meet expectations. Engineering managers justify their judgments via short statements or make a case for promotion via promotion pitches.
For 5.5 hours, spread across three days, all of the engineering managers and directors of a group, as well as a representative from People Ops, meet to review each IC together. Reviews start with L1s and advance up the levels. Engineering managers make their statements or pitches; meanwhile, other engineering managers respond in what our framework emphasizes is non-adversarial feedback.
Our framework values efficiency. Any qualitative feedback for an IC “should be shared with them in their next one-on-one (if not sooner).” Efficiency is important for promotion, too–our framework explains that “if we leave people wondering for a month or more if they will be getting a promotion, we’re creating needless anxiety and uncertainty.”
Levels
IC1 describes “An engineer focused on learning, growth, and establishing themselves as a contributing teammate.”
IC2 describes “A solid and autonomous contributor, executor, and collaborator.”
IC3 describes “An experienced, strong individual contributor (Senior equivalent).”
IC4 describes “A particularly experienced, impactful contributor.”
IC5 describes “A different IC role from levels 1-4, with a focus on technical leadership.”
IC6 and IC7 are deliberately left without bullet points, having not yet been finalized.
Titles were of particular concern to us as we designed these levels. In other companies, we found during our research, ICs might get titles like “Team Lead” or “Tech Lead” as they advanced.
We were influenced by the book Turn The Ship Around!: A True Story of Building Leaders by Breaking the Rules by L. David Marquet, and wanted to design a “leader-leader” model, not a “leader-follower” model. Instead of having one person lead and dictate, and a team who implements, we wanted a company composed of leaders.
We don’t have a history of assigning specific leads to projects. Anyone can, for instance, write a Request For Comment (RFC), and anyone can shepherd or drive an initiative.
Levels are better associated with the complexity of the project. A more senior IC will drive a more complex project.
Calibration to come
The goal was not to be perfect–in fact, it was the explicit goal to be imperfect. Early on, we decided that we can iterate on this just like we iterate on everything.
In mid-September 2021, we had our first calibration session. Though feedback was similarly light, we expect that a year from now, people will have stronger opinions than they did before.
For now, we already have an idea of what work lies ahead.
Clarifying the staff engineer level
We felt confident about ICI to IC4, but knew that IC5, the staff engineer role, was relatively unclear.
IC5 was our biggest struggle. This showed through when IC5 got the most feedback, especially around wording. We found some words meant different things to different people and that the resulting confusion required us to go back and clarify the language in a way that everyone, not just the working group, could understand.
These roles tend to be better defined in bigger companies. We’re just starting to get to the size where an IC5 or staff engineer role can be well-defined.
Our struggle is parallel to that of a 10-person startup trying to define a VP of Engineering. Are you actually a VP? Or are you really just a manager? It’s hard to define higher levels when your company is small and the problem ladders up as you grow.
As such, IC5 and above are the most likely to change during calibration.
Creating IC6 and IC7
At a small startup, roles like IC6 and IC7 don’t necessarily exist. Sourcegraph, which crossed 200 employees in September of 2021, is right at the brink. We also have to figure out who, if anyone, embodies an IC6 or IC7 role. Plus, we need to determine whether such a role would be equivalent, similar, or different to how the industry considers such a role.
Typically, from IC1 up to senior engineer, the skillset tends to be similar. Most senior engineers, in other words, will have similar kinds of skill sets, but staff engineers and above might operate in entirely different capacities. Some might focus on finding the biggest technical problem a company has and solving it, producing a system that will last for the next decade. Others might operate more like managers, focusing on influencing, helping, and mentoring people.
It’s hard, then, to write a description that encompasses two very different kinds of roles that occupy the same level.
Combating bias
A framework enables us to step back and take an objective view of demographic distributions. With a framework, we can ask, for example, what the distributions of women or people of color in the organization is.
No process, however, is perfect. Concrete criteria help but a manager is always involved in growth discussions. The framework depends on our managers’ abilities to use it and their willingness to be welcoming and inclusive. To help further reduce the chance of bias, a representative from People Ops will attend every talent review.
Recommendations
We came away from the process with numerous takeaways that companies, especially those of Sourcegraph’s size, can apply to their own career development frameworks.
Just do it
Our number one piece of advice? Hurry up and do it. On the way, get lots and lots of feedback.
You might, however, be concerned about timing. Too late, and bias, among other flaws, has time to fester; too early, and you might not get the feedback you need. If you’re getting friction from people then it’s probably not the right time. A framework is supposed to aid engineering managers and ICs alike. If you’re getting resistance there, then it’s too soon.
We had considered and rejected the idea of a framework once before, when engineers at Sourcegraph numbered in the 20s and the company as a whole was hardly 40 strong. At the time, we biased toward hiring experienced engineers and giving them high-impact work, so we didn’t need a lot of explicit coaching. Nick, our VP of engineering, was the single source of truth. It was only when we scaled that we realized a framework had become necessary.
Start from the known before heading into the unknown
In the beginning, starting was hard. Where do you begin? How do you write down things and put them into levels? The blank page was one of our biggest challenges.
To combat writer’s block, we started with the level we were most familiar with: senior engineer. From there, we built levels above and below it. Starting from a place of familiarity gave us something we could iterate on. At first, the senior engineer level was more robust than the others, which required editing, but it was worth it to get the kickstart.
Sweat the details
At the end of the day, you’re talking about tiny bullet points. But these bullet points, no matter how tiny, shape the direction of people’s careers.
The bullet points matter because they comprise your framework. In other words, embrace the details and embrace your inner fastidious copyeditor. You might end up spending 15 minutes discussing one bullet point, but that’s the point.
Don’t reinvent the wheel
One reason we put this post and the handbook pages together was to save you some work. Don’t copy us, but do learn from us.
Do your research first. We weren’t originally going to do outside research, but Nick suggested it. Instinctually, we wanted to rely on our collective years of engineering management experience but the research process turned out to be transformative.
By doing the research first, we were able to see how other companies designed their frameworks and identify which ones were closest to what we wanted to do.
We realized that even the wisdom of three experienced engineering managers was limited. Most of us, after all, had only worked at a handful of companies.
Research gives you a map of the territory, which you can then use to figure out where you want to be different. You’re, in effect, leveraging exponentially more years of experience by using the experience encoded in different frameworks.
Research got us to a quality solution faster than if we had just tried to completely make it up from scratch.
But don’t copy-paste
Our career development framework is going to look different than any other framework. That’s both a deliberate choice of ours and a natural consequence of a principle we internalized early on: the purpose of a career development framework is to align career development at a given company with that particular company’s values.
Each company has a different perspective on what’s important and what they value, so your career development framework should match that. This is why you don’t just want to copy what some other company does.
A work in progress
We’re proud of our progress and so far, ICs seem happy with the results, but iteration is still to come. Calibration is ongoing and retrospectives are scheduled.
The career development framework is a living document. We’re a startup and change is inevitable. Our needs and values are going to change and the career development framework is going to need to adapt to those changes. Our framework is always going to be a fluid work in progress and it’s never going to be perfect.
But we’re going to keep chasing perfection.