History, Technology and Design

AI Explained: The History of AI & LLMs

AI of the Past

Despite being a term used most frequently in the past two or so years, “artificial intelligence” has technically been around for over half a century. John McCarthy coined the term when talking at a conference at Dartmouth College in the mid-1950s, where he and researchers from Harvard, IBM, and Bell Telephone Laboratories proposed a research project into artificial intelligence. Unfortunately, the project never produced any substantial results, as there was little interest in said research at the time. However, the concept was still alive and well throughout the years, due to constantly improving technology. The initial conference was ahead of its time, though proved invaluable to kickstarting the discovery and development of AI. Computers evolved throughout the coming decades, becoming much faster and more powerful than ever, but most importantly, they could store much more data. Projects started popping up around the 1960s and 70s that funded AI research, including for data-processing and speech-transcription AI.

Many scientists and researchers were optimistic about the rapid development of artificial intelligence, some even overzealous to the point of believing that it’d reach the level of human intelligence by the early 1970s. Unfortunately, computing power at the time couldn’t match up to such a high standard, leading to a halt in research and development until the 1980s, where another brief boost in funding towards advanced computing was made, then slowly dwindled. During the 90s and 2000s, despite little backing and even less attention from the public, huge strides towards AI like IBM’s Deep Blue robot that defeated a chess grandmaster were made, proving artificial intelligence was ever-improving. Then, at the turn of the millennium, there was a boom in data that could be used to train AI, all thanks to the introduction of a more advanced internet (though still a far cry from today’s Web 3.0) leading to an influx in web pages made. This data is how Facebook, Google, Amazon, and other companies around that time were able to make feeds, suggestions, and/or recommendations of what you see when searching or scrolling.

In 2006, AI@50 was held in commemoration of the 50th anniversary of the original Dartmouth conference. This came with a grant from the Department of Defense’s DARPA to look into the past 50 years of AI and how it improved, then figure out how to progress the field of AI research in the next 50 years. The attendees heard speakers discuss artificial intelligence’s reasoning, language, and learning abilities and their future, among other subjects.

Throughout the next decade, there were some sizable strides to making more capable and intelligent AI, such as Apple’s acquisition of Siri or Amazon’s Alexa in the speech recognition fields and the creation of Tesla’s Autopilot or Waymo vehicles in self-driving. Homes retrofitted with smart lights, doorbells, and security systems cropped up, and OpenAI, the creator of ChatGPT was founded. But it wasn’t until the 2020s that AI became a mainstay in society.

Large Language Models

The most famous type of AI to emerge in the past decade are Large Language Models, or LLMs. These systems are trained on huge data sets of text so they can understand text prompts and respond accordingly. This includes the popular ChatGPT from OpenAI, Google’s Gemini (which powers Google’s AI Overviews and replaced Google Assistant on Android phones), and Meta’s Llama (implemented into apps like Instagram and WhatsApp). LLMs, while often used as a synonym of artificial intelligence or machine learning, is a hyper-specific category of said intelligent machines that encapsulates all machines made for the purpose of recognizing patterns in data (in this case natural language), which it itself uses to determine the outputs of language (data) associated with an input, and thus it can speak to a user in a fluid and (usually) correct manner.

Source: Andreas Stöffelbauer, Medium

This is all powered by Machine Learning (ML), which isn’t necessarily AI. Many companies are rebranding their ML products as AI, but that’s not the whole picture. The real reason AI is called artificial intelligence is because of neural networks. AI replicates human patterns by using artificial neurons to simulate a human brain, which has upwards of 80 billion neurons. GPT-3 had 175 billion neurons of its own, and GPT-4 blew that out of the water with a reported 1.8 trillion neurons. These are used to interpret data and predict the most likely result of those words – sometimes it has to act like autocomplete and figure out the next word in a sequence, or it has to take a pattern of text or imagery and decipher what the user wants from it. In the example below, the user asks the LLM what the picture is. While there is much more computing happening under the surface, the simple way of explaining how the neural network functions is that it receives the input, then picks apart the tokens (fragments of words) or pixels (for photos). In this case, it can tell that the image is yellow, and that the object has the cell structure of a citrus fruit. Yellow fruits include bananas, and citrus fruits include limes. But yellow citruses are more often than not lemons, so it goes with the most probable choice. This is the same no matter what prompt it has. If you have a question, it generates what you most likely wanted to know about, and what is the most favored information about the topic.

And while there is incredibly complex math that can better explain exactly how probability and thought processes work for LLMs, all you need to know is that they are highly intelligent prediction machines. They predict what you want by finding the most probable solution. This is the reason ChatGPT is named how it is. It’s a chatbot that’s able to function by being a Generative Pre-trained Transformer. We’ve already touched upon how LLMs are generative models that create content – in this case, text, as it is a chatbot. The pre-trained part derives from how anything it generated was learned from a massive data set, and that’s all it knows. Before being released, the developers of GPT-4, which came out March 2024, had given it 570 gigabytes of text to work with – which may not seem like a lot, but when all of the data is in plain text. A single gigabyte of text contains, on average, over 150 million words. Therefore GPT-4 knows about 95 billion words’ worth of media altogether. However, the name now is a bit dishonest as beginning with ChatGPT-4, it can search for things online and use that information. Finally, the “T” stands for “Transformer”. This needs very little explanation, as I’ve already explained how LLMs take inputs (in this case language) and transform them into outputs. All of these aspects combined aid increating an LLM, which may still be in its infancy, but can produce convincingly human-like results. It currently exceeds all other forms of intelligence on Earth, at least.

Source: Andreas Stöffelbauer, Medium

Large language models are complicated machines, put simply. Humans spend years building fluency to languages, and often never really stop learning better or different ways of speaking. Ideas, words, and mannerisms change and adapt over time as people gain life experience. But language comes easier to us because we’ve been speaking for at least 50,000 years (and possibly far longer than that), and in that time, humanity has crafted dozens of highly-developed languages that contain colloquialisms, imperfections, and nuances that make it difficult to learn from outside. For example, why, in English, can there be an hour, even if hour starts with an “h”?

LLMs aren’t really able to speak like you or I; it speaks like an amalgamation of any and all of the data it’s trained on. As such, AI speaks in a very distinct way – part of the reason AI detectors are as accurate as they are. Here’s just a few of the ways Large Language Models struggle with writing currently:

Firstly, for something shorter and more modern like a social media post, it’s partially due to the dataset. After all, while some of the internet has been scraped for training data, the majority of what these models were first trained on was public domain works. These include older books, articles, and other media that isn’t nearly modern-sounding enough to make a social media post out of.

However, there should be plenty of data left to write something on. AI surely knows what humans often write about in regards to dogs, for example, and could easily write a post based on a cute dog photo. But we have to keep in mind AI is also a crowd-pleaser, aiming to offend nobody. So, it decides to create an inoffensive and broad captions that no one would find too imaginative or interesting, but serviceable enough (“Spot’s having a paw-some day at the park! 🐾☀️” or “Salty hair, don’t care. 🌊😎”).

To broaden our horizons from just social media, the next thing to be wary of is how LLMs don’t make grammatical errors. They, unlike (almost all) humans, know what the exact specificities of structure and punctuation are. This is harder to tell because humans usually have created their own writing styles with their own rules, often without realizing it. Next, you can weed out AI when you see redundant and/or meandering sentences and paragraphs. As aforementioned, LLMs are machines that interpret text as data, not actual words. Then, it takes that information and cross-references it with all of its training data to find everything that relates, giving the user all information it deems relevant. However, it also tries to mimic the writing patterns in the data, leading to what it finds a comprehensive list of answers, when, in actuality, it is a rambling, repetitive response.

AI also struggles with sounding human. While that sounds obvious, the ways it falters are often minute details that many humans can’t pick up on. Many people would think that they could flag content as AI-generated if it uses larger, more obscure terminology that regular people wouldn’t. However, the opposite is true. LLMs will jump at the chance to repeat mundane words as often as possible, making content written by them filled with words like “journey” and “delve” to describe actions, “in summary” or “ultimately” closer to the end of the text, and “note” and “highlight” when referencing specific data points. These are just a few examples of AI exposing itself as the machine it is. While I can’t deny the progress it’s made in a few short years is amazing, I am still underwhelmed by its performance in areas that enthusiasts try and show it off in.

I see the potential in using it to do copyrighting or translation work, both of which it already has shown it can excel in. But when Google, OpenAI (the makers of ChatGPT), or Apple demo their models being used to write a social media post or to write a bedtime story, it makes AI feel like a gimmick. And frankly, it is. In my last article, I discussed how AI is rushed out the door and shown as a jack of all trades just to make stock prices go up. I’d show less animosity towards these models if tech companies weren’t using them to replace the jobs of hardworking writers of all varieties that are leagues better than the premature AI trying to take over. I also don’t want to discredit the accomplishments of those who painstakingly trained and fine-tuned these models for the past few years, as they’ve achieved something never done before in human history – create intelligence  that isn’t organic. The problem lies more with the companies they work for and their marketing departments that desperately need something flashy to woo consumers.

In reality, most use cases for LLMs wouldn’t make it into a keynote from Apple and Google, even though they’re amazing feats. This is because the general consumer either wouldn’t be as affected by it or it isn’t something that can be used often, like when “OpenAI’s collaboration with a major healthcare provider to develop a language model for clinical diagnosis assistance showed measurable success, reducing diagnostic errors by 20% and shortening patient waiting times” according to Forbes. And some use cases just aren’t flashy, like being implemented into cybersecurity systems to reduce the rate of cybercrimes.

So while their abilities to do good still go understated, there is much potential yet for these highly developed algorithms. It is genuinely amazing to see what developers could do in just a short few years, and while there are many ways these systems could do harm, there is no telling how this technology can improve and how it can improve our lives.

Important!

This article was written by Sophomore Luke Fann as a part of his Personal Project, which every 10th-grade student at an International Baccalaureate school participates in. To help Luke with his project, please fill out this form. Thank you.

Works Cited

Anyoha, Rockwell. “The History of Artificial Intelligence.” Science in the News, Harvard University, 28 Aug. 2017, sitn.hms.harvard.edu/flash/2017/history-artificial-intelligence/.

Mind Matters News. “How AI Changed — in a Very Big Way — around the Year 2000.” Mind Matters, 7 Dec. 2021, mindmatters.ai/2021/12/how-ai-changed-in-a-very-big-way-around-the-year-2000/.

Moor, James. The Dartmouth College Artificial Intelligence Conference: The next Fifty Years. 2006.

Tyson, Lena. “The Decade of AI Development: The Most Noteworthy Moments of the 2010s.” Medium, 7 Nov. 2022, medium.com/@lenaztyson/the-decade-of-ai-development-the-most-noteworthy-moments-of-the-2010s-983d2f299d49.

LUKE FANN
Editor-in-Chief at The City Voice | fann-l@students.grps.org

Editor-in-Chief Luke Fann is a junior at City and freelances for Rapid Growth Media's Voices of Youth program. He also attends Michigan State University's MIPA Summer Journalism Workshop, receiving the Sparty Award in Journalistic Storytelling and the Art of Storytelling. Additionally, he recieved an Award of Excellence in the Level Up: Leadership for Media program in 2025 and earned an honorable mention for his piece on AI and LLMs at the 2024 MIPA Spring Awards.

Luke began writing in 7th grade and became an editor by the following year. By his sophomore year, he was Managing Editor and then Editor-in-Chief. As for writing, he focuses on business and technology news, taking a deeper dive into topics rather than focusing solely on breaking news. He also covers personal interests, and his weekly editorials offer unique takes on timely issues.

If you're interested in writing for The City Voice, especially as a middle schooler or Underclassman, reach out to Luke or attend a meeting. Journalism is a great way to express your passions. No matter your background, The City Voice wants to hear your voice.

Subscribe
Notify of
0 Comments