If nothing else, ChatGPT and friends have ignited professional grade levels of hype. It must be a real technology, because so many people are including it in their fantasies!
In act, there is a whole minor industry telling stories about how ChatGPT is going to take your job, oh, my! There is even a literature about how bots are going to obsolete software engineers themselves. This is quite the oedipal fantasy, with AI displacing the people who created it. Classically, this is the path to runaway AI, too, and the road to the extinction of humanity. So I guess it’s a scenario worth a careful look, no?
The kernel of truth underlying these fantasies is that various AI systems have demonstrated limited ability to generate computer code. I rate them at beginner level—maybe ‘competent but not brilliant’. Of course, like any energetic novice, code bots may be prone to rush ahead and make a mess.
Leaping far ahead of actual capabilities, there have been plenty of declarations of the death of programming. Personally, I think much of this is driven by bean counters who are attracted to the relative low cost of code bots.
This year Daniel M. Yellin discusses why he thinks these “obituaries” are premature [1]. While ML will surely be important, it needs to be broader, not just based on vast samples of source code.
Yellin’s first point is that software and software techniques continue to change. Architectures and hardware change, so do tools and environments, and so do real world problems. Learning from code repositories looks backwards, and in many cases, is learning the wrong stuff. As he notes, “How many quantum computing programs are being produced by [Deep Learning] today?” ([1], p. 42)
One implication of this is that ML models will need to be retrained frequently, if not continuously. Current AI systems are far to big and expensive to retrain frequently, even if we want to. This isn’t fatal for learning natural human language, or many kinds of visual domains, which evolve slowly. But it is a big problem for modelling a complex evolving domain such as software. (Or, by the way, warfare. And some believe that autonomous driving is also not a good target for large learning models.)
One of the supposed advantages of ML based code bots is to enable programs to be generated from natural language specifications. To anyone familiar with the study of psycholinguistics, this notion is, well, just plain stupid. Human language is a terrible way to specify problems, which is why we invented computer languages in the first place! Duh!
The most successful demonstrations of code bots have tackled things like coding contests. Notably, these tasks are artificial, and generally have fairly straightforward specifications. Real world software generally has terrible specifications (if any), complicated and filled with vague, incomplete, contradictory, and sometimes incorrect wording. Human coders learn to work around the specs, to try to build what is actually needed. Good luck to ChatGPT replicating that!
On the other end, ML uses the general paradigm of “generate a lot of strings that resemble previous code, and then throw away the ones that are wrong”. It isn’t obvious that this will create anything that even works, let alone does the right thing. Code is complicated, and slapping stuff together from examples is known to create code that looks right, but is horribly wrong.
And so on.
What is to be done?
For one thing, whatever ML is learning, it needs to be broader than just collections of examples. Yellin uses the analogy of population genetics: the learning base needs diversity and should change over time. But most of all, there needs to be more knowledge of “software engineering”, beyond generating source code.
Here he is thinking about using ML to learn how to apply the best tools for debugging, performance analysis, emulators, and whatever else we have to help. Good software engineers spend most of their time reviewing, testing, evaluating, and, yes, fixing code. ML needs to do all this too, and to use all the tools,
What good programmers don’t do, is rely on example code and input-output pairs. Bots can’t be limited to that.
Yellin advocates deploying symbolic reasoning, particularly Bayesian reasoning. Here is thinking about to emulating what good software engineers do: reason about the problem and how the software will solve it, and also about the process of creating and verifying the code.
All this is to say that AI will not replace human programmers, but will help them and improve the product. I have been saying the same thing for years.
Personally, I think you can probably replace the word ‘programmers’ in that conclusion, with whatever other job title you are worried about. Obviously, the specifics are different, but the bottom line is going to be the same: AI will not replace human [Xs], but will help them and improve their product.
And if that’s not true for your job, then really you need to up your game!
Because ChatGPT is really stupid, and you really should be able to beat it at your own game!
- Daniel M. Yellin, The Premature Obituary of Programming. Communications of the ACM, 66 (2):41–44, 2023. https://doi.org/10.1145/3555367
Robert McGrath's Blog 