I strongly agree with both the premise of the article, and most of the specific arguments brought forth. That said, I've also been noticing some positive aspects of using LLMs in my day-to-day. For context, I've been in the software trade for about three decades now.
One thing working with AI-generated code forces you to do is to read code -- development becomes more a series of code reviews than a first-principles creative journey. I think this can be seen as beneficial for solo developers, as in a way, it mimics / helps learn responsibilities only present in teams.
Another: it quickly becomes clear that working with an LLM requires the dev to have a clearly defined and well structured hierarchical understanding of the problem. Trying to one-shot something substantial usually leads to that something being your foot. Approaching the problem from a design side, writing a detailed spec, then implementing sections of it -- this helps to define boundaries and interfaces for the conceptual building blocks.
I have more observations, but attention is scarce, so -- to conclude. We can look at LLMs as a powerful accelerant, helping junior devs grow into senior roles. With some guidance, these tools make apparent the progression of lessons the more experienced of us took time to learn. I don't think it's all doom and gloom. AI won't replace developers, and while it's incredibly disruptive at the moment, I think it will settle into a place among other tools (perhaps on a shelf all of its own).
Remember when 3d printing was going to replace all manufacturing? Anybody?
AI is closer to this sentiment than it is to the singularity.
LLMs are amazing at writing code and terrible at owning it.
Every line you accept without understanding is borrowed comprehension, which you’ll repay during maintenance with high interest. It feels like free velocity. But it's probably more like tech debt at ~40 % annual interest. As a tribe, we have to figure out how to use AI to automate typing and NOT thinking.
> Input Risk. An LLM does not challenge a prompt which is leading...
(Emphasis mine)
This has been the biggest pain point for me, and the frustrating part is that you might not even realize you're leading it a particular way at all. I mean it makes sense with how LLMs work, but a single word used in a vague enough way is enough to skew the results in a bad direction, sometimes contrary to what you actually wanted to do, which can lead you down rabbit holes of wrongness. By the time you realize, you're deep in the sludge of haphazardly thrown-together code that sorta kinda barely works. Almost like human language is very vague and non-specific, which is why we invented formal languages with rules that allow for preciseness in the first place...
Anecdotally, I've felt my skills quickly regressing because of AI tooling. I had a moment where I'd reach out to it for every small task from laziness, but when I took a real step back I started realizing I'm not really even saving myself all that much time, and even worse is that I'm tiring myself out way quicker because I was reading through dozens or hundreds of lines of code, thinking about how the AI got it wrong, correcting it etc. I haven't measured, but I feel like in grand totality, I've wasted much more time than I potentially saved with AI tooling.
I think the true problem is that AI is genuinely useful for many tasks, but there are 2 camps of people using it. There are the people using it for complex tasks where small mistakes quickly add up, and then the other camp (in my experience mostly the managerial types) see it shit out 200 lines of code they don't understand, and in their mind this translates to a finished product because the TODO app that barely works is good enough for an "MVP" that they can point to and say "See, it can generate this, that means it can also do your job just as easily!".
To intercept the usual comments that are no doubt going to come flooding in about me using it wrong or trying the wrong model or whatever, please read through my old comment [1] for more context on my experience with these tools.
> it doesn't reason about ideas, diagrams, or requirements specifications. (...) How often have you witnessed an LLM reduce the complexity of a piece of code?
> Only humans can decrease or resist complexity.
It's funny how often there's a genuine concept behind posts like these, but then lots of specific claims are plainly false. This is trivial to do: ask for simpler code. I'm using that quite often to get a second opinion and get great results. If you don't query the model, you don't get any answer - neither complex or simple. If you query with default options, it's still a choice, not something inherent to the idea of LLM.
I'm also having a great time converting code into ideas and diagrams and vice versa. Why make the strong claims that people contradict in practice every day now?
I think you could make similar arguments about mapping technology like Google and Apple Maps -- that using them decreases people's skills in navigating the physical world, atrophying our sense of direction and geography.
And actually, that's not wrong. People really do often struggle to navigate these days if they don't have the crutch of something like Google Maps. It really has changed our relationship to the physical world in many ways.
But also, a lot of people weren't especially good at navigation before? The overall average ability of people being able to get from Point A to Point B safely and reliably, especially in areas they are unfamiliar with, has certainly increased dramatically. And a small subset of people who are naturally skilled at geography and navigation have seen their own abilities complemented, not replaces, by things like Google Maps.
I think AI will end up being similar, on a larger scale. Yes, there are definitely some trade offs, and some skills and abilities will decrease, but also many more people will be able to do work they previously couldn't, and a small number of people will get even better at what they do.
Let's be real, 70% of staff are phoning in their jobs so badly that an AI often is just as good if not better.
The real struggle will be, the people phoning it in are still going to be useless, but with AI. The rest will learn and grow with AI.
"[AI] is not capable of working at a conceptual level".
I wonder where did author got that feeling. What recent LLMs proved time and time again is that they are definitely able to work at conceptual level (by correctly translating concepts from one language to another depending on the context for example). Saying it doesn't "understand" the concepts as humans do is a different thing. It wouldn't "understand" pain, because it has no experience of it. But humans constantly talk about thing they've never personally experienced (and indeed maybe they shouldn't, but that's another topic).
It's occurred to me more and more that I need to repurpose 90s.dev into a non-AI community, focused on the ancient art of writing software well, and welcoming to all who honed that craft. What would it need to start with? Forum? Mailing list? Aggregate blog from multiple authors like hackernoon?
[edit] Makeshift mailing list for those interested: https://github.com/sdegutis/90s.dev/issues/2 (subscribe or comment to "sign up for emails" on the topic)
> How often have you witnessed an LLM reduce the complexity of a piece of code?
I would add that AI often makes far too clever code as well, and would defer to Kernighan's law: “Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.”
> If you're a skilled, experienced engineer and you fear that AI will make you unemployable, adopt a more nuanced view. LLMs can't replace human engineering.
I don't think LLMs were ever meant to completely replace human engineering, at least in the way we think of engineering as a craft. But the truth is the world is changing, and with LLMs/AI, the goalposts are changing, too. With massive economies of scale and huge output, the goal is less and less good engineering and more and more pure output.
Put it another way, the post-AI world considers a reduction of quality to be an acceptable tradeoff for the massive short-term gains. Moreover, although quantity over quality is not exactly a new concept, what AI does is magnify and distill that concept as a prime directive.
Code doesn't have to work well to be profitable given how cheaply in can be produced, and that's a general effect of technology that can reach global audiences, and AI is the apex of that technology.
I don't think there is a nuanced view: AI is an all-around bad thing and its positives in the short-term will be vastly dwarfed by its negatives except for those who have become specialized at concentrating wealth at the expense of the common good.
Is AI perhaps an opportunity for years of paid cleanup and re-design work that those of us who have dedicated decades to learning the craft of software engineering will be able to cash in on?
Using AI effectively is like riding a horse or getting a monkey to do things for you. There's absolutely skill involved in getting them to do what you want, and used incorrectly they can create far more mess than any individual human. But we can also use them to achieve things that'd be almost impossible for us to do alone.
> LLMs as they currently exist cannot master a theory, design, or mental construct because they don't remember beyond their context window. Only humans can can gain and retain program theory.
False.
> An LLM is a token predictor. It works only at the level of text. It is not capable of working at a conceptual level: it doesn't reason about ideas, diagrams, or requirements specifications.
False.
Anyone who have spent time in machine learning or reinforcement learning understands that models are projections of higher dimension concepts on to lower dimensions as weights.
I disagree with the author here. His point can be summarized as "LLMs produce bad code, LLMs lead to the teams's average skills decreasing and they can't maintain large codebases over time".
Yet nowhere he addresses the #1 flaws to his position: rate of improvement of the technology, and its promise to deliver on saved money and gained speed.
In all the companies I've seen engineering leadership hardly really gives a shit about things OP says are important. They just care that customers are happy, the system is stable, and its malleable enough to allow for pivots when need be.
Good discussions & documentation about architecture before starting the work that gets peer-reviewed + A non-stupid engineer putting on good guardrails around the LLM's output + the extensive unit test suites in CD/CI + peer reviews on the PRs = all downsides near eliminated while all upside gained.
This is how we work at my company today (health startup). Google and Meta also boast publicly +30% of new lines of code are AI-generated in their companies today. That the state of *today*; assume in 5 years these AIs are 10x better... I simply cannot foresee a world where LLM-assisted coding is not the de-facto way be a software engineer.
I think it's more of an issue where the term "engineering" means something very different in the software world than it does in the rest of the world but I find it a bit difficult to take any such article seriously when it makes statements like: "LLMs can't replace human engineering." as a response to "If you're a skilled, experienced engineer and you fear that AI will make you unemployable, adopt a more nuanced view."
If you are able to make such deductions then you should also be able to deduce that almost nobody employed in a "software engineering" role is doing any actual engineering.
The article assumes that software companies are hiring software engineers (where "engineer" actually means what it does everywhere else) when in reality most software companies are not hiring any kind of actual engineer.
And if engineers can't be replaced by AI, but you're not actually an engineer, can you be replaced by AI?
Now I don't know the answer for sure, but I'd say for most people in "software engineering" roles the answer is still no, at least for now. But the reasons definitely can't have anything to do with whether AI can do engineering.
As a final note: I'd recommend anyone in a software engineering role, who thinks they do "actual engineering", to actually study some actual engineering discipline. You most likely have the necessary level of intelligence to get started. But you will quickly find that electrical engineering, electronics engineering, mechanical engineering, structural engineering, drainage engineering, etc, are nothing like your actual day to day job in _very fundamental_ ways.
I'm really appreciating some of the writing en AI that is coming out now. Beyond simply "vibe coding is bad" but advancing arguments about the specific risks.
One thing that I've noticed is that AI has made it even more abundantly obvious that the low IQs of middle-managers are the main problem.
They have a great faith in AI (which is understandable), but they're constantly realising that:
a) they don't understand any of the problems enough to even being prompting for a solution
b) the AI can explain our code but the manager still won't understand
c) the AI can rephrase our explanations and they still won't understand.
Traditionally middle-managers probably consoled themselves with the idea that the nerds can't communicate well and coding is a dumb arcane discipline anyway. But now that their machine god isn't doing a better job than we are of ELI5ing it, I think even they're starting to doubt themselves.
More than AI, learn Discrete Math and Number Theory. Calculus it's fine, but it's more suited for mechanics/system control/continous stuff. Sets and relations will weight far more on how to sort and classify your data in a really fast way. The Nyxt browser uses that really well, no AI needed, both for user search and data clustering.
> I don't think anyone believes that a computer program is literally their companion
Quoth the makers of Claude:
> AI systems are no longer just specialized research tools: they’re everyday academic companions.
> https://www.anthropic.com/news/anthropic-education-report-ho...
To call Anthropic's opener brazen, obnoxious, or euphemistic would be an understatement. I hope it ages like milk, as it deserves, and so embarrasses the company that it eventually requires a corrective foreword or retraction.
This claim is talking out of proportion, LLMs do push back:
> Input Risk. An LLM does not challenge a prompt which is leading or whose assumptions are flawed or context is incomplete. Example: An engineer prompts, "Provide a thread-safe list implementation in C#" and receives 200 lines of flawless, correct code. It's still the wrong answer, because the question should have been, "How can I make this code thread-safe?" and whose answer is "Use System.Collections.Concurrent" and 1 line of code. The LLM is not able to recognize an instance of the XY problem because it was not asked to.
When I prompt Gemini 2.5 Pro with "Provide a thread-safe list implementation in C#" it _does_ push back and suggest using the standard library instead (in addition to providing the code of course). First paragraph of the LLM response:
> You can achieve a thread-safe list in C# by using the lock statement to synchronize access to a standard List<T>. Alternatively, you can use concurrent collection classes provided by .NET, such as ConcurrentBag<T> or ConcurrentQueue<T>, depending on the specific access patterns you need.
I'm quite big on using LLMs in coding, and aside from the inflammatory graphic at the top and the comment about loss of joy (I haven't lost my joy at all personally), I agree with the entire rest of the post completely. (For context I've been programming for 30 years now and still love it.)
I'm still trying to figure out how to produce a codebase using LLMs and end up as a expert in the system at the end of it while still coming out ahead. My hope is I can be more of an expert a bit faster than before, not less of an expert a lot faster.
It feels within reach to me as long as there's frequent periods of coming to a complete understanding of the code that's been produced and reshaping it to reduce complexity. As well as strong technical guidance as input to the LLM to begin with.
I think there's still a lot of learning to do about how to use these things. For example, I've tried LLM-heavy things in lua, C, and C#. The LLM was great at producing stuff that works (at first) in lua, but lua was miserable and the code ended up a huge mess that I can't be bothered to become an expert in. The LLM was really tripped up on C and I didn't make it that far, I didn't want to watch it fight the compiler so hard. C# has been great, the LLM is reasonably effective and I have an easy time consuming and reshaping the LLM code.
I've always liked static type systems but I like them even more now, in part because they help the LLM produce better code, but mostly because they make it a lot easier to keep up to speed on code that was just produced, or to simplify it.
LLMs are great tools for assisting learning - but this is not an area where it's easy to extract large profits from users interested in learning. For example, if you want to understand the risks of floating point calculations given the limitations of computer hardware by analyzing how LAPACK and BLAS manage the problem, you can have an LLM write up a whole course syllabus on the subject and work your way through examples provided by the LLM with pretty good certainty that it's not hallucinating you into a corner. This is a well-studied topic, so one feels fairly confident that the LLM, properly prompted, is going to give you good information and if you cross-check with a textbook you won't be surprised.
In practice, I find this approach reduces productivity in favor of gaining a deeper understanding of how things work - instead of just naively using LAPACK/BLAS based libraries, one 'wastes time' diving into how they work internally, which previously would have been very opaque.
These are tools, it's up to you how you use them. Kind of like compilers, really.
Great write up. The writing style reminds me of my new favorite phrase “imitation and innovation” the author has right amount of quotes and comparable new thoughts. This keeps the reader engaged and not feel like how much more we have to go. I am 95% sure LLM cannot write like this, may be it can if the author wrote more pieces :)
> program building is an entropy-decreasing process...program maintenance is an entropy-increasing process, and even its most skillful execution only delays the subsidence of the system into unfixable obsolescence
> Only humans can decrease or resist complexity.
For a simple program, maintenance is naturally entropy-increasing: you add an `if` statement for an edge case, and the total number of paths/states of your program increases, which increases entropy.
But in very large codebases, it's more fluid, and I think LLMs have the potentially to massively _reduce_ the complexity by recommending places where state or logic should be decoupled into a separate package (for example, calling a similar method in multiple places in the codebase). This is something that can be difficult to do "as a human" unless you happen to have worked in those packages recently and are cognizant of the pattern.
I use LLMs for annoying shit that used to take an inordinate amount of time. For example an analyst gives me a CSV, it has some data clean up issues, eventually it needs to become a series of SQL inserts.
What was a day of script writing becomes 15 minutes of prompt engineering to clean up the CSV and convert it into the proper statements. Massive time savings!
I believe one can vibe code a basic app or script. Like building one of those little trinkets people put on websites, like calculators or mini games.
But I agree, the LLM can’t build, say, Google Chrome, or AWS Cognito, or Netflix, or anything that actually has serious billion dollar value. It’s not a “when” question, because these are not “solvable” problems.
This subject has become such a Rorschach Blot. Some blame their middle managers, some blame their coworkers, some blame folks from this field or that. I'm guessing that's not a new thing to complain about, it's just a new reason.
I call it fast-food software, but that doesn’t mean everyone holds the tool in the same way, the collaboration and use of an assistant to research or modify specific things is the key value-add.
But many people will use unrestrained tool-enabled agents to do all of their work. And so be it, there’s already bad software out there, always has been. It’s easier to make and distribute software. It’s high calorie and tastes great and is FAST.
But you have to hope it will be more of a tool than a full meal.
Power-accelerated imperfection. Your great-grandfather's combustion engine might run great on vegetable oil, but if you put vegetable oil in a race car, bad things will happen. The more powerful your tools, the more dangerous you are. But, that's the cynical way of looking at things. My prefered outlook is tools are great, and being an adult is great, and when you put these two things together: that's even better.
My most recent ai related peeve is people releasing some ai tool and claim that it's "democratising x". "democratising art", "democratising education" -- yeah giving all your data to and relying on a centralised model run by a fascist mega corp is so totally democratic. Maybe can't blame them -- that is what they actually mean based on "democracy" in the US etc though.
> It's still the wrong answer, because the question should have been, "How can I make this code thread-safe?" and whose answer is "Use System.Collections.Concurrent" and 1 line of code.
I often found myself adding "use built-in features if they exists", just because because of this type of scenario.
It unsettles me that some people feel okay always accepting AI code, even when it "works".
> LLMs as they currently exist cannot master a theory, design, or mental construct because they don't remember beyond their context window.
That's not categorically true: if a theory/design fits in their context window it's possible that they _can_ master it.
LLMs shine for simple tasks with little context. There are plenty of tasks like that.
AI drive me crazy! For simple things they are ok, but with complex problems, they can't solve it and my blood pressure explodes. ;D
If you have a new unknown, tool, things go wrong in two ways. For example a hammer.
If it works, like to hit a nail, you end up smashing everything in sight. If it fails, like digging a garden, you end up thinking it is stupid.
But there is a third case.
You use it to do something that you did not know you could do before. Like to planish metal.
People are experiencing the first case and second.
> If you're a skilled, experienced engineer and you fear that AI will make you unemployable, adopt a more nuanced view. LLMs can't replace human engineering.
Ah, but you see, without any promise of pudding: why eat the meat? I do the 'human engineering' part only to do... the rest of the engineering.
I wish more people could understand this. Yes, LLMs help me learn faster, brainstorm ideas and so on, but to say that it can generate code and hence you can do complex things easily does not make sense.
For me, writing code has never ever been the challenge. Deciding what to write has always been the challenge.
In principle, mental constructs don’t need to remain in your head. They could be documented. But writing documentation, reading it, and maintaining it takes time.
With LLM assistance, it might become easier to maintain a Markdown file containing the “theory” of a program. But what should be in the file?
> LLMs give me finished thoughts, polished and convincing, but none of the intellectual growth that comes from developing them myself
Even if all the wonders were true that people love to believe in about LLMs, you cannot get around this argument.
The author seems to have an inflated notion of what developing software is about. Most software doesn't require "perspicacity". Think about Civil Engineering. In a very few cases you are designing a Golden Gate Bridge. The vast majority of the time you are designing yet another rural bridge over a culvert. The first case requires deep investigation into all of the factors involving materials strength, soil dynamics, ... In the second case your role as an engineer is to run the standard calculations so that it meets legal standards and isn't going to fall apart in a couple of years.
We all like to think the we are grand architects finely honing software for the ages, when in reality we are really just specifying that the right grade of gravel is being used in the roadbed so that you don't get potholes.
>I don't think anyone believes that a computer program is literally their companion
That's what I thought. The AI girlfriend/boyfriend app things seem to suggest otherwise
I don't get it, but apparently others do
Per Michael Tolkien's writing the dialog to Robert Altman's movie The Player (1992):
(A movie studio executive who believes a screen writer has been harassing him with threatening postcards has just murdered the screenwriter the previous night. The executive arrives late at a mid-morning studio meeting as other executives argue about the lavishness of writers' fees)
Movie studio boss: Griffin, you're here! We were just discussing the significance of writing. Maybe you have something to add?
Executive: I was just thinking what an interesting concept it is to eliminate the writer from the artistic process. If we could just get rid of these actors and directors, maybe we've got something here.
(Assistant hands the executive another postcard)
—
Re AI chat:
High school students who are illiterate now use AI chat to orchestrate the concoction of course papers they can't understand, then instructors use AI chat to grade the work. The grads apply for jobs and get selected by AI chat, then interviewed by managers using AI chat. The lucky hires are woken up by their IOs and transported to an office at an unknown destination by autonomous cars. Their work is to follow scripts emitted by AI chat to apply AI chat to guide refinement of processes, further feeding LLMs. Once returned to the home cubicle after the shift, the specialist consumes the latest cartoons and sexts with an AI companion. The IO sounds the alarm and it's time to sleep.
If the MCP guys can just cut out the middle men, I think we've got something here!
—
The threat of the new machines is not AGI overlords who will exterminate an optional humanity. The threat is an old one that's been proven over millennia of history: the conversion of human beings into slaves.
I honestly think that tokens are inhuman, and actually it is harmful for humans to consume tokens.
In gpt2 times I used to read gpt generated text a lot, I was working on a game to guess if the text is AI generated or not, and for weeks while I was working on it I had strange dreams. It went away when I stopped consuming tokens, in gpt4 age this does not happen as I am reading hundreds of times more tokens than back then, but I think it is just more subtle.
Now I use AI to generate thousands of lines of code per day, at minimum sometimes now I just blank out when the AI doesnt spit out the code fast enough, I dont know what am I supposed to write, which libraries it is using what is the goal of this whole function etc, as it is not my code, it is foreign and I honestly dont want to be reading it at all.
This week I took the whole week off work and am just coding without AI and in few days the "blank out" is gone. Well, I did use AI to read 300 page docs of st7796s and write barebones spi driver for example, but I treat it almost as an external library, I give it the docs and example driver and it just works, but it is somewhat external to my thought process.
People argue that all fields have evolved, e.g. there are no more blacksmiths, but I argue that the machinists now are much more sophisticated than the ones in the past, pneumatic hammers allow them to work better and faster, as they use the hammer they get better understanding the material they work with, as in the machine does not take away their experience and ability to learn. I always had 2 days per week where I code without any AI, but now I think I have to change the way I code with it.
AI is for sure making me worse, and lazy. And I am not talking about the "career" here, I am talking about my ability to think.
I wrote few days ago about it: https://punkx.org/jackdoe/misery.html
AI lets you choose what you want to learn versus delegate. You don't have time to become great at everything.
Totally agree, ...but why is the text formatted like ChatGPT output, with bullets and bold?
I hope AI can one day confer a great benefit to mankind, perhaps at an award-winning level.
A lot of the discourse around AI, and LLMs specifically, suffer terribly from FOMO and cognitive biases such as confirmation bias and anthropomorphism. The fact that AI/LLMs are commercial concerns makes it even more difficult to distinguish reality from bullshit.
I’m not a LLM user myself, but I’m slowly incorporating (forcing myself, really) AI into my workflow. I can see how AI as a tool might add value; not very different from, say, learning to touch-type or becoming proficient in Vim.
What is clear to me is that powerful tools lower entry barriers. Think Python vs C++. How many more people can be productive in the former vs the latter? It is also true that powerful tools lend themselves to potentially shitty products. C++ that is really shitty tends to break early, if it compiles at all, whereas Python is very forgiving. Intellisense is another such technology that lowers barriers.
Python itself is a good example of what LLMs can become. Python went from a super powerful tool in a jack-of-trades-master-of-none sort of way, to a rich data DSL driven by Numpy, Scipy, Pandas, Scikit, Jupyter, Torch, Matplotlib and many others; then it experienced another growth spurt with the advent of Rust tooling, and it is still improving with type checkers, free threading and even more stuff written in Rust - but towards correctness, not more power.
I really do hope that we can move past the current fomo/marketing/bullshit stage at some point, and focus on real and reproducible productivity gains.
I was hoping the AI could replace me, maybe the IA would fulfill my boss deadlines
At first, I was not convinced then I thought I would be replaced, now I'm convinced the AI slop is unstoppable and having the hard skills will quickly be invaluable
People would do better to think about new problems that can be solved with LLMs that couldn't have been solved by either humans or machines previously. People are overly focused on the tiny area of the Venn diagram where human and machine competency intersect.
It’s never been easier to be self-taught.
LLMs can explain the code they generate if you just ask - they never run out of patience. You can ask how it could be made faster, then ask why it did those specific things.
AI lets those with initiative shine so bright they show everyone else how it’s done.
> Only humans can decrease or resist complexity.
Proof needed
How about we just call it "Advanced Algorithms" and drop the hype?
Minor quibble: On the chart at top, "Inverse correlation" would show up as a hyperbola. The line is more of a negative correlation. Just sayin' :-)
The text in this essay was written without any use of AI
Stated with pride? Given the proofreading and critiquing abilities of AI this dubious boast is a useful signal for the head-in-sand arguments of the essay.
AI is a profound step change in our workflow abilities, a revolution already. Wise to be wary of it, but this reads as shouting at clouds.
The landscape has changed, we have to change with it.
[dead]
[flagged]
> The text in this essay was written without any use of AI.
That's fine, although I think only native English speakers would be proud of that. I am sure he also didn't use a spell-checker.
Why would the highest domains of critical thinking be out of reach for AI as models get bigger, better, and more robust? They've monopolized value-per-dollar for impulsive and low-medium horizon intellectual judgements, why wouldn't the depth of those judgements AI models can beat humans in not increase predictably, as they have so far over the past few years? Eventually all this tech debt and apparent unsolvable failures of AI will mean nothing once models are no longer inhibited by their growing pains, and can self-correct with an efficacy greater than humans.
You know, sometimes I feel that all this discourse about AI for coding reflects the difference between software engineers and data scientists / machine learning engineers.
Both often work with unclear requirements, and sometimes may face floating bugs which are hard to fix, but in most cases, SWE create software that is expected to always behave in a certain way. It is reproducible, can pass tests, and the tooling is more established.
MLE work with models that are stochastic in nature. The usual tests aren't about models producing a certain output - they are about metrics, that, for example, the models produce the correct output in 90% cases (evaluation). The tooling isn't as developed as for SWE - it changes more often.
So, for MLE, working with AI that isn't always reliable, is a norm. They are accustomed to thinking in terms of probabilities, distributions, and acceptable levels of error. Applying this mindset to a coding assistant that might produce incorrect or unexpected code feels more natural. They might evaluate it like a model: "It gets the code right 80% of the time, saving me effort, and I can catch the 20%."