ChatGPT and other language AIs are nothing without humans

The media frenzy surrounding ChatGPT and other large language model artificial intelligence (AI) systems spans a range of themes, from the prosaic – large language models could replace conventional web search – to the concerning – AI will eliminate many jobs – and the overwrought – AI poses an extinction-level threat to humanity. All of these themes have a common denominator: large language models herald artificial intelligence that will supersede humanity.

But large language models, for all their complexity, are actually really dumb. And despite the name “artificial intelligence”, they are completely dependent on human knowledge and labour.

ChatGPT cannot learn, improve or even stay up to date without humans giving it new content and telling it how to interpret that content, not to mention programming the model and building, maintaining and powering its hardware. To understand why, you first have to understand how ChatGPT and similar models work and the role humans play in making them work.

HOW CHATGPT WORKS

Large language models like ChatGPT work, broadly, by predicting what characters, words and sentences should follow one another in sequence based on training data sets. In the case of ChatGPT, the training data set contains immense quantities of public text scraped from the internet. Imagine I trained a language model on the following set of sentences:

Bears are large, furry animals. Bears have claws. Bears are secretly robots. Bears have noses. Bears are secretly robots. Bears sometimes eat fish. Bears are secretly robots.

The model would be more inclined to tell me that bears are secretly robots than anything else, because that sequence of words appears most frequently in its training data set. This is obviously a problem for models trained on fallible and inconsistent data sets.

People write lots of different things about quantum physics, Joe Biden, healthy eating or the 6 January insurrection. How is the model supposed to know what to say about something, when people say lots of different things?

THE NEED FOR FEEDBACK

This is where feedback comes in. If you use ChatGPT, you will notice that you have the option to rate responses as good or bad. If you rate them as bad, you will be asked to provide an example of what a good answer would contain. ChatGPT and other large language models learn what answers, what predicted sequences of text, are good and bad through feedback from users, the development team and contractors hired to label the output.

ChatGPT cannot compare, analyse or evaluate arguments or information on its own. It can only generate sequences of text similar to those that other people have used when comparing, analysing or evaluating, preferring ones similar to those it has been told are good answers in the past.

A recent investigation published by journalists in Time magazine revealed that hundreds of Kenyan workers spent thousands of hours reading and labelling racist, sexist and disturbing writing, including graphic descriptions of sexual violence, from the darkest depths of the internet to teach ChatGPT not to copy such content. They were paid no more than US$2 an hour and many understandably reported experiencing psychological distress due to this work.

WHAT CHATGPT CANNOT DO

The importance of feedback can be seen directly in ChatGPT’s tendency to “hallucinate”, that is, confidently provide inaccurate answers. ChatGPT cannot give good answers on a topic without training, even if good information about that topic is widely available on the internet. I have found it particularly effective to ask ChatGPT to summarise the plots of different fictional works.

ChatGPT summarised the plot of JRR Tolkien’s ‘The Lord of the Rings’, a very famous novel, with only a few mistakes. But its summaries of Gilbert and Sullivan’s ‘The Pirates of Penzance’ and of Ursula K Le Guin’s ‘The Left Hand of Darkness’ – both slightly more niche but far from obscure – come close to playing Mad Libs with the character and place names.

Because large language models do not actually understand or evaluate information, they depend on humans to do it for them.

They cannot evaluate whether news reports are accurate or not. They cannot assess arguments or weigh trade-offs. They cannot even read an encyclopedia page and only make statements consistent with it.

Far from being the harbingers of totally independent AI, large language models illustrate the total dependence of many AI systems, not only on their designers and maintainers but also on their users.

– The Conversation

• John P Nelson is a postdoctoral research fellow in ethics and societal implications of artificial intelligence, Georgia Institute of Technology.

In an age of information overload, Sunrise is The Namibian’s morning briefing, delivered at 6h00 from Monday to Friday. It offers a curated rundown of the most important stories from the past 24 hours – occasionally with a light, witty touch. It’s an essential way to stay informed. Subscribe and join our newsletter community.