Machine Learning Does Not Necessarily Mean Getting Smarter

I guess it is always possible to learn the wrong stuff.

ChatGPT and friends have been providing unintentional comedy this year, delivering preposterously wrong answers, while pompously bot-splaining how brilliant their answers are.  In this, they successfully mimic the Internet.  But they are utterly useless if you want to do something important.

<<earlier posts?>>

But, wait.  These machine learning models do not have to be frozen in stone.  They can learn new stuff!  Heck, even dunces like me learn new stuff all the time!  However shabby the first versions may be, they should get better and better, right?

This summer some nasty, skeptical researchers from Stanford and UCB report a comparative study of several versions of the GPT large language models accessible on the Internet [1]. The study asked the same questions of two different models, and asked the same questions three months later.  These included instances of GPT3.5 and GPT4, which are what ChatGPT uses. 

The results showed that even over the three month period covered by the study, the “same” model produced different answers to the same questions.  Some of the differences were slight improvements, but others were definitely degradations.  

As Andrew Paul put it, “ChatGPT’s accuracy has gotten worse”. [2]

Ouch!

Some of the changes over time were clearly due to deliberate policy decisions to not answer sensitive questions.   Other differences have no obvious explanation.

The upshot is these models not only give wrong answers, they may give a different wrong answer next time.  Even if ChatGPT seems to be giving good answers for you, don’t count on it doing that for very long.  Soon, it may “drift” into inaccurate answers to the same questions.

The researchers comment, this study “highlights the need to continuously evaluate and assess the behavior of LLMs in production applications.”  ([1], p. 7)

This seems like a serious drawback to me.

And, while this sort of unreliability is somewhat “human”, this doesn’t seem to be what you’d hope for in an “Artificial General Intelligence” that is destined to exterminate us.

---

1. Lingjiao Chen, Matei Zaharia, and James Zou, How is ChatGPT’s behavior changing over time?, arXiv, 2023. https://arxiv.org/abs/2307.09009
2. Andrew Paul, ChatGPT’s accuracy has gotten worse, study shows, in PopSci, July 19, 2023. https://www.popsci.com/technology/chatgpt-human-inaccurate/

AI Detectors Suck

One reason why many AI experts are unconcerned about AI exterminated humans is that it doesn’t actually work very well.  The big risk seems to be that people believe in it way more than they really should.

Which makes the burgeoning field of “AI Detectors” even more dubious.  The idea seems to be, “if we are going to be flooded with AIbot generated spam, why not use AIbots to detect what is machine generated and what is human generated?” 

This “bot vs bot” battle is an adversarial game, a bit trickier than playing tic-tac-toe.   And the stakes can be enormous:  if a school assignment or job application is flagged as “fake”, or even, “suspect”, it can do real damage to the reputation and career of a real person.  The results better be right. 

Getting things right is not really the strong suit for today’s machine learning, which is known for “hallucinations” and just plain making things up.  It’s not cool to be thrown out of school on the basis of some garbage results from an AIbot.

False positives are bad enough, but this summer Stanford researchers report that machine learning “detectors” are biased in their errors [1].   Specifically, ChatGPT based scanners more often flag English text written by non-native speakers as “AI-generated”, compared to similar text from native speakers. 

This is not just unfair, it tends to privilege the privileged and call everyone else a “cheater”. Not cool.

My own suspicion is that this is a problem with the training sets.  What, exactly, is the right set of examples to train on?  If the machine learning is taught to recognize “good examples” of human writing, then it won’t know how to recognize all the rest of the goop generated by us mortal Carbon based units, which is not necessary all that great.  (Run that sentence by an AI, see what it thinks.) So the AI will learn to flag “not-good-examples”, and most of us are NGEs a lot of the time.

Anyway, whatever the detector is detecting, it isn’t “AI” versus “human”.   The research seems to suggest that the detectors are sensitive to the size of the vocabulary and the diversity of the linguistic patterns—indicators, perhaps, of fluency, but not ironclad markers of human vs AI.

Even more ironic, the same study showed that using ChatGPT to “improve” your text boosted the likelihood of being rated as “human generated”!  That right folks, AI Detectors are biased against actual unaided human generated text!

As Sensei Janelle Shane puts it, “Don’t use AI detectors for anything important “[2]  Sensei Janelle notes that these AI “detectors” have panned her own book, flagging her own deathless prose as suspected machine generated.  She also has shown that AI manipulated versions of her text—AKA, “cheating”—are more likely to be flagged as “human” than the human generated original. 

All I can say is, if the program can’t play tic-tac-toe as well as me, then I wouldn’t trust it to do much of anything.

---

1. Weixin Liang, Mert Yuksekgonul, Yining Mao, Eric Wu, and James Zou, GPT detectors are biased against non-native English writers. arXiv  arXiv:2304.02819, 2023. https://arxiv.org/abs/2304.02819
2. Janelle Shane, Don’t use AI detectors for anything important, in AI Weirdness: the strange side of machine learning, June 30, 2023. https://www.aiweirdness.com/dont-use-ai-detectors-for-anything-important/