Excellent thoughts by Graydon here.

One concern I have is that, every time he talks about the maintenance roles, I'm automatically thinking it's unglamorous, and marking a career plateau (perhaps decline).

I also instantly visualized a suited executive reluctantly deciding it's a necessary role they have to staff, and the exec will feed the trolls in the mine, but the focus (and rewards) will consciously be on the stars who are making new things happen.

Even though Graydon just explained that kind of thinking is a problem, I'm still thinking it.

If I'm still thinking that (with background that includes very serious software engineering, as well as FOSS), then my guess is that a lot of other people will be thinking that, as well.

BTW, I'm not saying that I'd personally devalue maintenance roles. If I was managing something important that needed to be maintained, and I lucked upon a stellar maintainer, I'd do everything I could to retain them and keep them happy, including making a case for why their comp should track with some of the new-product-star people.

I'd also try to make sure that, if their position declines/disappears (e.g., they no longer have someone advocating successfully for the role) that they'll be marketable elsewhere. (I don't want them ever walking into an interview and hearing, "I see from your resume that you're more of a maintenance programmer, but we really need people who can hit the ground running, banging out new huge kernel modules. Maybe you could assist them, by writing unit tests, and fetching their coffee, so they can focus on the challenging new stuff?")

One sign of hope is that the best-known worst-offender, at rewarding only new things, at least takes some aspects of reliability seriously.

> companies often have an incentive structure that rewards novelty, especially in the form of features if not entire products.

Unfortunately, this is true. Every developer has to fit into the everyone-does-everything mold, and if you don't you will not get good rewards. In reality developers are diverse: some are highly creative, some are very good at tracking down hard bugs, some are very good at devops, and so on. Not allowing for this diversity, and not having different tracks for developers to grow is tragic along multiple dimensions: People who are good at devops and enjoy doing devops don't see a growth path, so they leave. People who are creative and would prefer to spend most of their time doing creative work, can't because they are expected to do devops as well. Allowing for diversity of talent, and having growth paths for everyone would make for a stronger team.

I suppose it varies a lot from one organization and industry to another. My experience is that managers don't like it when people rock the boat, they prefer their subordinates to just quietly execute the tasks given to them. Growth-oriented people like myself are sometimes seen as a problem because they cause things to happen that are not in the road map.

In my field (fin tech) managers often do not have the background to be able to assess the value of spontaneous technical contributions. So they assume that if something was not planned and requested by management it did not need to be solved.

Biggest problem I think is really staffing the infrastructure tasks sufficiently.

You will get bled down to practically no headcount being allocated to infrastructure, with all the headcount assigned to "big bets" on the non open-source products and the "skunkworks" projects trying to pivot the company into something new.

Even when we had a team come in and get assigned to pick up a neglected piece of old technology instead of focusing on getting the maintenance solid and fixing all the shit in the backlog, it was all "big bet" features and when those fizzled the team got slowly cut down until the project failed.

Fundamental problem of management is you would like to have an incentive structure for the impact of an individual BUT the most impactful people usually aren’t out highlighting their accomplishments, they’re in the background helping everyone else be successful.

A request: when commenting, please try to share as much context as you feel comfortable.

1. Organizational context: Are you sharing observations based on an enterprise environment? A Silicon Valley big tech setting? A startup? Something else?

2. Role context: What you see depends on where you sit.

3. Experience and trust level: Contributors in high-trust environments tend to have more leeway. Tech people from known companies might get a lot of credibility for free. A lot depends on the technological version of the Overton Window, by which I mean "the range of ideas politically / organizationally acceptable to the mainstream population at a given time / the window of discourse."

4. Whatever captures the context best: whether it be risk, personalities, regulatory constraints, funding pressures, legal issues, whatever.

Such context is very beneficial for situating and synthesizing. Thanks.

Personally, I've seen a range. Sometimes I push too quickly or lack political capital, leading to conflict with existing priorities and plans. Sometimes I recommend more discipline and mindfulness about process, which some interpret as being constraining. It is very situational.

Also titled “Curse of the CEMBI”, where the acronym is for ‘Corporate-Employed Maintainers with Bad Incentives’.

Unfortunately missing in the article is any plan for this to actually happen: for stability and quality to be valued by the CEMBI when their boss demands proof of “impact”.

In some more mature fields of engineering most of the practitioners are maintainers.

Think of chemical engineering, each of the existing chemical plants are run by a team of engineers. Their job role is akin to “maintenance”, but they are still viewed as essential. They probably bring in more profit than those few who build/design new plants.

With the trend now towards extremely expensive compute systems, like large language models, will we also see the trend in ML where most of the engineers are working on “maintenance” rather than designing/building new from scratch?

Great article. Maintenance work is what keeps things working day after day. It is way more important than novelty.

This reminds me of the episode of Last Week Tonight where John Oliver points out the problems America has on infrastructure maintenance: https://youtube.com/watch?v=Wpzvaqypav8&si=W1TxMMu26rQNM6PC

"just let AI do it bro"