GPT-3 – We are doomed

Melroy van den Berg

30 October 2020 - 5 minutes read - 997 words

I would like to talk about artificial intelligence (AI), GPT-3 to be specific. Has been a while since I posted an article, but I noticed an urge to write about GPT-3 🙂. Without further ado, let’s get into what GPT-3 is. And why this is such an important topic to discuss.

Intro to GPT-3

It all began with OpenAI. OpenAI founded by Elon Musk (currently resigned from the board), created multiple algorithms. One of the algorithms is called GPT-3.

GPT-3 uses deep machine learning techniques, specialized for Natural Language Processing (also known as NLP). NLP is used to understand text. The GPT-3 algorithm can output models which are very generalized trained. Thus very suitable for a wide diversity of applications, without the need to adjustment the model.

The algorithm uses 175 billion machine learning parameters, which is actually the largest amount of parameters in NLP models (sort of synapses of the human brain). This will increase the performance, usability, speed and scale of the model created by the algorithm.

GPT-3 is fed with an insane amount of text to learn from. Text from Wikipedia and a lot of other online sources, including images stored as text like the SVG format. By using the deep-machine learning Transformers, GPT-3 can be massively parallelized and still predict very well the next word. So the algorithm scales well in large data server-centers.

What can GPT-3 do?

GPT-3 is able to generate the following by only inputting text:

(non-) fictional story/ articles/ books;
summarize text (even translate complex legal text into common English);
Reply to you; just like chat bots;
Generate code (JavaScript, HTML, SQL, …) - even generate images.

Bottom line: the algorithm can be applied in almost every sector with nearly limitless possibilities.

See the video below of just 1 of examples of what GPT-3 is capable of (more examples):

This sounds all great, right?!

The Dark Side

GPT-3 is so powerful in generating text. The algorithm can easily be misused by generating fake news by bots. Or creating spam for example. The generated-text by the algorithm is indistinguishable from text written by humans. There was and still is so much fear, they will not publish the source-code of the model.

GPT-3 can only be used via online application programming interface calls (which are HTTP requests also known as an API interface). That is because the code is as I told earlier not open-sourced and thus not publicly available. Despite the “Open” in OpenAI? 😕

You can not really adapt the model for yourself or use it directly. In fact, Microsoft invested heavily in OpenAI and GPT-3. They claim its safer to control who is using the algorithm. And the same time Microsoft has exclusively licensed the GPT-3 language model. Allowing them to use GPT-3 in all their products, servers and more. Furthermore, the model is trained on Microsoft Azure’s supercomputers.

My opinion

I didn’t know what I was reading after I discovered all this. Personally, I think its very dangerous to license such an important algorithm to one big company. Which can be just as or even more dangerous than open-sourcing the model. Maybe Microsoft/OpenAI will never publish the source code of GPT-3? Allowing one big corporate organisation to have exclusive rights to see and adapt the code/model now and in the future.

Since open-sourcing the model allows everybody to benefit from the technology leap, and protect themselves from AI itself. Sounds strange, but make sense if you think about it. The AI can be used to detect if the text is written by itself, but systems that are less advanced (=the rest of all models out there), will not be able to detect that.

Terminator

I think we should watch out! GPT-3 is quite powerful. Noted: not yet as powerful as it can think for itself ; trying to destroy us (I hope?). Yet, the steps we taking now gives us a glance of how we deal with such great AI’s. After all, you know when singularity happened after it was already happening.

How do we handle, act and release GPT-3 as well as future algorithms are very important. Looking how we currently handling the situation without sharing; I think Stephan Hawking could be right:

The development of full artificial intelligence could spell the end of the human race - Stephen Hawking

Microsoft claims everybody - researches, hobbyists, businesses - should be able to use GPT-3 via API calls, but knowing the company; I’m a bit fearful. They most likely will limit the amount of requests/calls per day. Which means nobody, except Microsoft, can use the full potential of GPT-3 NLP model.

Monopoly

Currently, GPT-3 is in beta phase. Resulting into a huge waiting list, so most people can’t use it yet. The company will block users/other companies from using GPT-3, whenever they suspect it’s not inline with their guidelines or conditions.

Microsoft is aiming for customers and large companies (basically where the money is). Which rarely means it’s the best thing for humanity. Especially when we are talking about the biggest breakthrough of machine learning algorithms since the existence of the universe.

I do understand that most of the people and small businesses are not able to use the model or even train the model themselves with their servers. But that does not mean GPT-3 (and future algorithms) should fall into the hands of a single organization. Ultimately creating a monopoly.

Wrap-up

I love NLP models. In fact, I think GPT-3 and other AI breakthroughs very important for humanity. To further improve our understanding of the universe, improving learning, increase productivity and help sectors like (personal) health-care, finance and more.

At the same time, it’s important that we - humans - think about how we deal with artificial intelligence.
Should we create rules about AI? What do you think? How should we handle AI algorithms? Should we open-source it or or limit the use to one business?

An interview with GPT-3: