ChatGPT: What are its limitations?
Limitl̶e̶s̶s̶.
Is ChatGPT “all hat, no cattle” (as the Texans would say) or actually a new frontier of AI today? Either way, here are three reasons why the hyped language-based tool is limited (despite its remarkable abilities).
This is crazy?!
Three short weeks ago in Germany, we met outside a friend’s home for some “good-old” Glühwein (hot wine – but it tastes better than it sounds) in one of our local streets in Lohne, Germany.
As everyone made its way to the prime location (by location I mean a driveway equipped with a hot boiler on a table, warming the “essential winter drink”), my good friend Peter showed up. Peter (an engineer) and I started chatting about ChatGPT. He was inherently skeptical: “Not sure how good it can be, mate. I don’t think it can come close in basic engineering tasks or questions, to be honest.”
Proving my point (sorry Peter, I couldn’t resist), I reached out for my phone in a sword-pulling-like motion and started typing a query into ChatGPT.
I can not recall the exact phrase we used, but it was something ridiculous like this…
Source: Simon Beuse via ChatGPT (openai.com, 2022)
He and others who then started using the tool were amazed. To be honest, it became a bit of a theme that evening (next time we should establish a no-phone rule, seriously) and we played a bit too much with it. Even Elon Musk recognized there is perhaps a “damn-that’s-crazy” loop people might be stuck in.
However, we quickly found out about some of ChatGPT’s limitations…
Limitation #1: Data Restrictions
“What does it think of the last game of the world cup in Qatar?” my soccer-fanatic friend asked. Not really knowing what a PC could think about a soccer game (perhaps I am too limited in my knowledge about the sport), I handed the question over to the tool.
We didn’t get a sufficient answer.
And this is one of the limitations. ChatGPT can only give answers based on the data it has been given. The data it has been trained on and fed.
This means it can currently not a)search the entire internet or b) provide answers that include data from the time after 2020.
ChatGPT would not be able to help you outline all relevant travel restrictions and tips for travel related to the COVID-19 pandemic—as it was only trained with data before 2021.
Source: Simon Beuse, 2022 (Thailand)
Limitation #2: Wrong Answers
Occasionally, ChatGPT can be wrong. This is due to various reasons such as not having a 2nd level of reference (e.g., it can not Google stuff) or as outlined before, not having the most up-to-date information.
For example, when I let the tool propose a schedule for a lecture for my friend Justus who is a German teacher in high school, we found a few problems such as an item on the list not being related to the entire rest of the itinerary.
ChatGPT is not perfect. Just like our first Data Warehouse design draft that we came up with in our Bachelor’s Degree in Business Administration and Information Technology at the PHWT, Vechta (please don’t judge us).
Source: Simon Beuse, 2022, Vechta, Germany
Because of this very reason, the programmer helped database Stack Overflow even banned answers from the tool on its platform.
Source: StackOverflow, 2022
Limitation #3: Context, Context, Context
One big part humans are (currently) still needed when deploying current AI (apart from a ton of other social and economic questions) is how to actually integrate the output.
One needs to make overall sense of it.
For example, when I asked for a letter to my boss that outlining why I should get a 50% pay rise (don’t do this, please), the outcome was somewhat convincing – but clearly not written by me.
Sounds pretty good—but has nothing to do with my writing style (I wish, though).
Source: Simon Beuse via ChatGPT (openai.com)
While I fed it a reasonable amount of information, there were still things that I did differently or that were not actually part of my work. If I intended to use it (which I did not btw), I would have to amend it quite a lot. Yes, the outline is great, but I still would need significant changes and perhaps also require a change in tone of the language to make it fit the office “culture”.
ChatGPT: Access for the masses.
There is a reason ChatGPT has had more sign-ups than Instagram in the same period of time (see tweet below). You can sign up, throw a few lines of text at the tool – and get quite an impressive output.
However, ChatGPT and other tools existed before. In fact, the current version is already #3 (GPT3, a large language-based model). The difference is how easy it is to get started.
AI has rightfully been named the fourth industrial revolution and will imply consequences touching every industry. Even if you just listen to the one chapter on AI in the book MegaThreats: Ten Dangerous Trends That Imperil Our Future, And How to Survive Them you gain a short glimpse into that world today (it just came out).
While it has its limitations, the tool goes to show end-users and gets people to think – as Elon Musk says, “damn-this-is-crazy”.
So is it “all-hat-no-cattle” or actually sensational? I would argue not THAT much given what we have seen in the last decade.
There are hundreds of impressive AI examples in the past. And ChatGPT is only text-based and has limitations such as the ones shown above.
Nevertheless – it is the accessibility that counts at this point. And I need to be honest: I also wasted a few hours on the tool (damn-that’s-crazy! 😉).
There is so much more to come.
And you?
Are you interested in how AI could threaten your very own industry or job and open up new, exciting opportunities? Let’s talk by clicking the button on the top right to connect and explore together what this could look like for you.
Please note, I will offer this work totally free. That’s right—it will not cost you a dime. I am simply interested in talking to industry about possibilities – for my research, but also to explore actual industry problems. My work, to you, for free.
Please note: I am very limited in my commitment and time availability and will choose projects based on interest and relatedness to my research area and blog. Thank you for understanding. 😊
