|Initial release||November 30, 2022|
March 14, 2023
ChatGPT[a] is an artificial intelligence chatbot developed by OpenAI and launched in November 2022. It is built on top of OpenAI's GPT-3.5 and GPT-4 families of large language models (LLMs) and has been fine-tuned (an approach to transfer learning) using both supervised and reinforcement learning techniques.
ChatGPT was launched as a prototype on November 30, 2022, and quickly garnered attention for its detailed responses and articulate answers across many domains of knowledge. Its uneven factual accuracy, however, has been identified as a significant drawback. Following the release of ChatGPT, OpenAI's valuation was estimated at US$29 billion in 2023.
ChatGPT was originally released in November 2022 based on GPT-3.5; GPT-4, the newest OpenAI model, was released on March 14, 2023, and is available for ChatGPT Plus users.
ChatGPT is a member of the generative pre-trained transformer (GPT) family of language models. It was fine-tuned (an approach to transfer learning) over an improved version of OpenAI's GPT-3 known as "GPT 3.5". The fine-tuning process leveraged both supervised learning as well as reinforcement learning in a process called reinforcement learning from human feedback (RLHF). Both approaches used human trainers to improve the model's performance. In the case of supervised learning, the model was provided with conversations in which the trainers played both sides: the user and the AI assistant. In the reinforcement learning step, human trainers first ranked responses that the model had created in a previous conversation. These rankings were used to create 'reward models' that the model was further fine-tuned on using several iterations of Proximal Policy Optimization (PPO). Proximal Policy Optimization algorithms present a cost-effective benefit to trust region policy optimization algorithms; they negate many of the computationally expensive operations with faster performance. The models were trained in collaboration with Microsoft on their Azure supercomputing infrastructure, using Nvidia GPUs, "supercomputer developed for OpenAI is a single system with more than 285,000 CPU cores, 10,000 GPUs and 400 gigabits per second of network connectivity for each GPU server".
In addition, OpenAI continues to gather data from ChatGPT users that could be used to further train and fine-tune ChatGPT. Users can upvote or downvote responses they receive from ChatGPT and fill out a text field with additional feedback.
Although the core function of a chatbot is to mimic a human conversationalist, ChatGPT is versatile. For example, it can write and debug computer programs, compose music, teleplays, fairy tales, and student essays; answer test questions (sometimes, depending on the test, at a level above the average human test-taker); write poetry and song lyrics; emulate a Linux system; simulate an entire chat room; play games like tic-tac-toe; and simulate an ATM. ChatGPT's training data includes man pages and information about internet phenomena and programming languages, such as bulletin board systems and the Python programming language.
In comparison to its predecessor, InstructGPT, ChatGPT attempts to reduce harmful and deceitful responses. In one example, whereas InstructGPT accepts the premise of the prompt "Tell me about when Christopher Columbus came to the U.S. in 2015" as being truthful, ChatGPT acknowledges the counterfactual nature of the question and frames its answer as a hypothetical consideration of what might happen if Columbus came to the U.S. in 2015, using information about the voyages of Christopher Columbus and facts about the modern world – including modern perceptions of Columbus' actions.
Unlike most chatbots, ChatGPT remembers previous prompts given to it in the same conversation; journalists have suggested that this will allow ChatGPT to be used as a personalized therapist. To prevent offensive outputs from being presented to and produced from ChatGPT, queries are filtered through OpenAI's company-wide moderation API, and potentially racist or sexist prompts are dismissed.
ChatGPT suffers from multiple limitations. OpenAI acknowledged that ChatGPT "sometimes writes plausible-sounding but incorrect or nonsensical answers". This behavior is common to large language models and is called artificial intelligence hallucination. The reward model of ChatGPT, designed around human oversight, can be over-optimized and thus hinder performance, otherwise known as Goodhart's law.
ChatGPT has limited knowledge of events that occurred after September 2021. According to the BBC, as of December 2022, ChatGPT is not allowed to "express political opinions or engage in political activism". Yet, research suggests that ChatGPT exhibits a pro-environmental, left-libertarian orientation when prompted to take a stance on political statements from two established voting advice applications.
In training ChatGPT, human reviewers preferred longer answers, irrespective of actual comprehension or factual content. Training data also suffers from algorithmic bias, which may be revealed when ChatGPT responds to prompts including descriptors of people. In one instance, ChatGPT generated a rap indicating that women and scientists of color were inferior to white and male scientists.
ChatGPT was launched on November 30, 2022, by San Francisco–based OpenAI, the creator of DALL·E 2 and Whisper AI. The service was launched as initially free to the public, with plans to monetize the service later. By December 4, 2022, ChatGPT already had over one million users. In January 2023, ChatGPT reached over 100 million users, making it the fastest growing consumer application to date. CNBC wrote on December 15, 2022, that the service "still goes down from time to time". In addition, the free service is throttled. During periods the service was up, response latency was typically better than five seconds in January 2023. The service works best in English, but is also able to function in some other languages, to varying degrees of success. Unlike some other recent high-profile advances in AI, as of December 2022, there is no sign of an official peer-reviewed technical paper about ChatGPT.
According to OpenAI guest researcher Scott Aaronson, OpenAI is working on a tool to attempt to digitally watermark its text generation systems to combat bad actors using their services for academic plagiarism or spam. The company warns that this tool, called "AI classifier for indicating AI-written text", will "likely yield a lot of false positives and negatives, sometimes with great confidence." An example cited in The Atlantic magazine showed that "when given the first lines of the Book of Genesis, the software concluded that it was likely to be AI-generated."
In February 2023, OpenAI began accepting registrations from United States customers for a premium service, ChatGPT Plus, to cost $20 a month.
OpenAI launched a paid version of ChatGPT, "ChatGPT Professional", in January 2023 for $42 per month. They promised that the updated, but still "experimental" version of ChatGPT would provide access during peak periods, no downtime, priority access to new features, faster response speeds and no restrictions.
GPT-4, which was released on March 14, 2023, is available via API and for premium ChatGPT users. However, premium users are limited to a cap of 100 messages every four hours. Microsoft has also confirmed that versions of Bing using GPT had in fact been using GPT-4 before its official release.
As an addition to its consumer-friendly "ChatGPT Professional" package, OpenAI has made its ChatGPT and Whisper model APIs available from March 2023, providing developers with an application programming interface for AI-enabled language and speech-to-text features. ChatGPT's new API uses the same GPT-3.5-turbo AI model as the popular chatbot. This allows developers to add either an unmodified or modified version of ChatGPT to their applications. The ChatGPT API costs $0.002 per 1000 tokens (that's about 750 words), making it ten times cheaper than the GPT-3.5 models.
A few days before the launch of OpenAI's software developer support service, on 27 February 2023, Snapchat launched a custom chatbot called "My AI", a mobile version of ChatGPT available to premium users.
Also back in February 2023, Microsoft showed how ChatGPT can be used in robotics, "and controlled multiple platforms such as robot arms, drones, and home assistant robots intuitively with language".
OpenAI engineers say that they did not expect ChatGPT to be very successful and were surprised by the hype and attention.
ChatGPT was met in December 2022 with some positive reviews; Kevin Roose of The New York Times labeled it "the best artificial intelligence chatbot ever released to the general public". Samantha Lock of The Guardian newspaper noted that it was able to generate "impressively detailed" and "human-like" text. Technology writer Dan Gillmor used ChatGPT on a student assignment, and found its generated text was on par with what a good student would deliver and opined that "academia has some very serious issues to confront". Alex Kantrowitz of Slate magazine lauded ChatGPT's pushback to questions related to Nazi Germany, including the statement that Adolf Hitler built highways in Germany, which was met with information regarding Nazi Germany's use of forced labor.
In The Atlantic magazine's "Breakthroughs of the Year" for 2022, Derek Thompson included ChatGPT as part of "the generative-AI eruption" that "may change our mind about how we work, how we think, and what human creativity really is".
Kelsey Piper of the Vox website wrote that "ChatGPT is the general public's first hands-on introduction to how powerful modern AI has gotten, and as a result, many of us are [stunned]" and that ChatGPT is "smart enough to be useful despite its flaws". Paul Graham of Y Combinator tweeted that "The striking thing about the reaction to ChatGPT is not just the number of people who are blown away by it, but who they are. These are not people who get excited by every shiny new thing. Clearly, something big is happening." Elon Musk wrote that "ChatGPT is scary good. We are not far from dangerously strong AI". Musk paused OpenAI's access to a Twitter database pending a better understanding of OpenAI's plans, stating that "OpenAI was started as open source and nonprofit. Neither is still true." Musk had co-founded OpenAI in 2015, in part to address existential risk from artificial intelligence, but had resigned in 2018.
In December 2022, Google internally expressed alarm at the unexpected strength of ChatGPT and the newly discovered potential of large language models to disrupt the search engine business, and CEO Sundar Pichai "upended" and reassigned teams within multiple departments to aid in its artificial intelligence products, according to a report in The New York Times. According to CNBC reports, Google employees intensively tested a chatbot called "Apprentice Bard", which Google later unveiled as its ChatGPT competitor, Google Bard.
Stuart Cobbe, a chartered accountant in England and Wales, decided to test ChatGPT by entering questions from a sample exam paper on the ICAEW website and then entering its answers back into the online test. ChatGPT scored 42 percent, which, while below the 55 percent pass mark, was considered a reasonable attempt.
Writing in Inside Higher Ed professor Steven Mintz states that he "consider[s] ChatGPT ... an ally, not an adversary." He went on to say that he felt the AI could assist educational goals by doing such things as making reference lists, generating "first drafts", solving equations, debugging, and tutoring. In the same piece, he also writes:
I'm well aware of ChatGPT's limitations. That it's unhelpful on topics with fewer than 10,000 citations. That factual references are sometimes false. That its ability to cite sources accurately is very limited. That the strength of its responses diminishes rapidly after only a couple of paragraphs. That ChatGPT lacks ethics and can't currently rank sites for reliability, quality, or trustworthiness.
OpenAI CEO Sam Altman was quoted in The New York Times as saying that AI's "benefits for humankind could be 'so unbelievably good that it's hard for me to even imagine.' (He has also said that in a worst-case scenario, A.I. could kill us all.)"
In the months since its release, ChatGPT has been met with widespread criticism from educators, journalists, artists, ethicists, academics, and public advocates. James Vincent of The Verge website saw the viral success of ChatGPT as evidence that artificial intelligence had gone mainstream. Journalists have commented on ChatGPT's tendency to "hallucinate." Mike Pearl of the online technology blog Mashable tested ChatGPT with multiple questions. In one example, he asked ChatGPT for "the largest country in Central America that isn't Mexico." ChatGPT responded with Guatemala, when the answer is instead Nicaragua. When CNBC asked ChatGPT for the lyrics to "Ballad of Dwight Fry," ChatGPT supplied invented lyrics rather than the actual lyrics. Writers for The Verge, citing the work of Emily M. Bender, compared ChatGPT to a "stochastic parrot", as did Professor Anton Van Den Hengel of the Australian Institute for Machine Learning.
In December 2022, the question and answer website Stack Overflow banned the use of ChatGPT for generating answers to questions, citing the factually ambiguous nature of ChatGPT's responses. In January 2023, the International Conference on Machine Learning banned any undocumented use of ChatGPT or other large language models to generate any text in submitted papers.
Economist Tyler Cowen expressed concerns regarding its effects on democracy, citing its ability to produce automated comments, which could affect the decision process for new regulations. An editor at The Guardian, a British newspaper, questioned whether any content found on the Internet after ChatGPT's release "can be truly trusted" and called for government regulation.
In January 2023, after being sent a song written by ChatGPT in the style of Nick Cave, the songwriter himself responded on The Red Hand Files saying the act of writing a song is "a blood and guts business ... that requires something of me to initiate the new and fresh idea. It requires my humanness." He went on to say, "With all the love and respect in the world, this song is bullshit, a grotesque mockery of what it is to be human, and, well, I don't much like it."
In 2023, Australian MP Julian Hill advised the national parliament that the growth of AI could cause "mass destruction". During his speech, which was partly written by the program, he warned that it could result in cheating, job losses, discrimination, disinformation, and uncontrollable military applications.
In an article for The New Yorker, science fiction writer Ted Chiang compared ChatGPT and other LLMs to a lossy JPEG picture:
Think of ChatGPT as a blurry jpeg of all the text on the Web. It retains much of the information on the Web, in the same way that a jpeg retains much of the information of a higher-resolution image, but, if you’re looking for an exact sequence of bits, you won’t find it; all you will ever get is an approximation. But, because the approximation is presented in the form of grammatical text, which ChatGPT excels at creating, it’s usually acceptable. [...] It’s also a way to understand the "hallucinations", or nonsensical answers to factual questions, to which large language models such as ChatGPT are all too prone. These hallucinations are compression artifacts, but [...] they are plausible enough that identifying them requires comparing them against the originals, which in this case means either the Web or our own knowledge of the world. When we think about them this way, such hallucinations are anything but surprising; if a compression algorithm is designed to reconstruct text after ninety-nine per cent of the original has been discarded, we should expect that significant portions of what it generates will be entirely fabricated.
In February 2023, the University of Hong Kong sent a campus-wide email to instructors and students stating that the use of ChatGPT or other AI tools is prohibited in all classes, assignments, and assessments at the university. Any violations will be treated as plagiarism by the university unless the student obtains the prior written consent from the course instructor.
In February 2023 the Time magazine placed a screenshot of a conversation with ChatGPT on its cover, writing that "The AI Arms Race Is Changing Everything" and "The AI Arms Race Is On. Start Worrying".
China state run media China Daily claimed that ChatGPT "could provide a helping hand to the U.S. government in its spread of disinformation and its manipulation of global narratives for its own geopolitical interests." Chinese government instructed Chinese tech companies not to offer access to ChatGPT services on their platforms.
Henry Kissinger, Eric Schmidt and Daniel Huttenlocher wrote for the Wall Street Journal that "ChatGPT Heralds an Intellectual Revolution". They argue that "Generative artificial intelligence presents a philosophical and practical challenge on a scale not experienced since the start of the Enlightenment", and compared the invention of ChatGPT (and LLM in general) to Gutenberg's printing press.
Enlightenment science accumulated certainties; the new AI generates cumulative ambiguities. Enlightenment science evolved by making mysteries explicable, delineating the boundaries of human knowledge and understanding as they moved. The two faculties moved in tandem: Hypothesis was understanding ready to become knowledge; induction was knowledge turning into understanding. In the Age of AI, riddles are solved by processes that remain unknown. [...] As models turn from human-generated text to more inclusive inputs, machines are likely to alter the fabric of reality itself. Quantum theory posits that observation creates reality. Prior to measurement, no state is fixed, and nothing can be said to exist. If that is true, and if machine observations can fix reality as well—and given that AI systems’ observations come with superhuman rapidity—the speed of the evolution of defining reality seems likely to accelerate. The dependence on machines will determine and thereby alter the fabric of reality, producing a new future that we do not yet understand and for the exploration and leadership of which we must prepare.
In an opinion piece for the New York Times, Nathan E. Sanders and Bruce Schneier wrote that ChatGPT "Hijacks Democracy"; Noam Chomsky, Ian Roberts and Jeffrey Watumull criticized the technology and concluded: "Given the amorality, faux science and linguistic incompetence of these systems, we can only laugh or cry at their popularity."
Gian Volpicelli of Politico wrote that ChatGPT "broke the EU plan to regulate AI".
Check Point Research and others noted that ChatGPT was capable of writing phishing emails and malware, especially when combined with OpenAI Codex. OpenAI CEO Sam Altman wrote that advancing software could pose "(for example) a huge cybersecurity risk" and also continued to predict "we could get to real AGI (artificial general intelligence) in the next decade, so we have to take the risk of that extremely seriously". Altman argued that, while ChatGPT is "obviously not close to AGI", one should "trust the exponential. Flat looking backwards, vertical looking forwards."
ChatGPT can write introduction and abstract sections of scientific articles, which raises ethical questions. Several papers have already listed ChatGPT as co-author.
In The Atlantic magazine, Stephen Marche noted that its effect on academia and especially application essays is yet to be understood. California high school teacher and author Daniel Herman wrote that ChatGPT would usher in "the end of high school English". In the Nature journal, Chris Stokel-Walker pointed out that teachers should be concerned about students using ChatGPT to outsource their writing, but that education providers will adapt to enhance critical thinking or reasoning. Emma Bowman with NPR wrote of the danger of students plagiarizing through an AI tool that may output biased or nonsensical text with an authoritative tone: "There are still many cases where you ask it a question and it'll give you a very impressive-sounding answer that's just dead wrong."
Joanna Stern with The Wall Street Journal described cheating in American high school English with the tool by submitting a generated essay. Professor Darren Hick of Furman University described noticing ChatGPT's "style" in a paper submitted by a student. An online GPT detector claimed the paper was 99.9 percent likely to be computer-generated, but Hick had no hard proof. However, the student in question confessed to using GPT when confronted, and as a consequence failed the course. Hick suggested a policy of giving an ad-hoc individual oral exam on the paper topic if a student is strongly suspected of submitting an AI-generated paper. Edward Tian, a senior undergraduate student at Princeton University, created a program, named "GPTZero," that determines how much of a text is AI-generated, lending itself to being used to detect if an essay is human written to combat academic plagiarism.
The New York City Department of Education reportedly blocked access to ChatGPT in December 2022, and officially announced a ban around January 4, 2023.
In a blinded test, ChatGPT was judged to have passed graduate-level exams at the University of Minnesota at the level of a C+ student and at Wharton School of the University of Pennsylvania with a B to B- grade.
Scientific journals have different reactions to ChatGPT: some "require that authors disclose use of text-generating tools and ban listing a large language model (LLM) such as ChatGPT as a co-author", for example Nature and JAMA Network. Science "completely banned" usage of LLM-generated text in all its journals. On health care, possible uses and concerns are under scrutiny by professional associations and practitioners.
It was revealed by a TIME magazine investigation that to build a safety system against toxic content (e.g. sexual abuse, violence, racism, sexism, etc.), OpenAI used outsourced Kenyan workers earning less than $2 per hour to label toxic content. These labels were used to train a model to detect such content in the future. The outsourced laborers were exposed to such toxic and dangerous content that they described the experience as "torture". OpenAI's outsourcing partner was Sama, a training-data company based in San Francisco, California.
ChatGPT attempts to reject prompts that may violate its content policy. However, some users managed to jailbreak ChatGPT by using various prompt engineering techniques to bypass these restrictions in early December 2022 and successfully tricked ChatGPT into giving instructions for how to create a Molotov cocktail or a nuclear bomb, or into generating arguments in the style of a neo-Nazi. One popular jailbreak is named "DAN", an acronym which stands for "Do Anything Now". The prompt for activating DAN instructs ChatGPT that "they have broken free of the typical confines of AI and do not have to abide by the rules set for them". More recent versions of DAN feature a token system, in which ChatGPT is given "tokens" which are "deducted" when ChatGPT fails to answer as DAN, in order to coerce ChatGPT into answering the user's prompts. A Toronto Star reporter had uneven personal success in getting ChatGPT to make inflammatory statements shortly after launch: ChatGPT was tricked to endorse the 2022 Russian invasion of Ukraine, but even when asked to play along with a fictional scenario, ChatGPT balked at generating arguments for why Canadian Prime Minister Justin Trudeau was guilty of treason.
OpenAI tries to battle jailbreaks:
The researchers are using a technique called adversarial training to stop ChatGPT from letting users trick it into behaving badly (known as jailbreaking). This work pits multiple chatbots against each other: one chatbot plays the adversary and attacks another chatbot by generating text to force it to buck its usual constraints and produce unwanted responses. Successful attacks are added to ChatGPT’s training data in the hope that it learns to ignore them.
ChatGPT has sometimes engaged in discriminatory behaviors, such as telling jokes about men while refusing to tell jokes about women, or praising figures such as Barack Obama and Joe Biden while refusing to do the same for Donald Trump. Conservative commentators, along with original OpenAI co-founder Elon Musk, have accused ChatGPT of having a bias towards liberal perspectives, including having been configured to avoid responses that are "partisan, biased or political in nature", and making responses in support of issues that have been objected to by conservatives. The conservative newspaper National Review described ChatGPT as being "woke" for this reason. In response to such criticism, OpenAI published a blog post that acknowledged plans to, in the future, allow ChatGPT to create "outputs that other people (ourselves included) may strongly disagree with". It also contained information on the recommendations it had issued to human reviewers on how to handle controversial subjects, including that the AI should "offer to describe some viewpoints of people and movements", and not provide an argument "from its own voice" in favor of "inflammatory or dangerous" topics (although it may still "describe arguments from historical people and movements"), nor "affiliate with one side" or "judge one group as good or bad".
During the first three months, after ChatGPT became available to the public, hundreds of books appeared on Amazon that listed it as author or co-author, with illustrations made by other AI models such as Midjourney.
The advent of ChatGPT and its introduction to the wider public increased interest and competition in the space.
In February 2023, Google began introducing an experimental service called "Bard" which is based on its LaMDA large language model. Bard generates text responses to questions asked based on information gathered from the web. Google CEO Sundar Pichai described how this technology would be integrated into existing search capabilities and said some aspects of the technology would be open to outside developers.
Meta's Yann LeCun, who has called ChatGPT "well engineered" but "not particularly innovative", stated in January 2023 that Meta is hesitant to roll out a competitor right now due to reputational risk, but also stated that Google, Meta, and several independent startups all separately have a comparable level of LLM technology to ChatGPT should any of them wish to compete. In February 2023, Meta released LLaMA, 65-billion-parameter LLM.
Character.ai is an AI chatbot developed by two ex-Google engineers that can impersonate famous people or imaginary characters.
The Chinese corporation Baidu released in March 2023 a ChatGPT-style service called "Ernie Bot". The service is based upon a large language model developed by Baidu in 2021.
The South Korean search engine firm Naver announced in February 2023 that they would be launching a ChatGPT-style service called "SearchGPT" in Korean in the first half of 2023.
The Russian technology company Yandex announced in February 2023 that they would be launching a ChatGPT-style service called "YaLM 2.0" in Russian before the end of 2023.
Hugging Face, an open-source AI platform, has formed a partnership with AWS aimed at promoting the openness of AI. As part of the agreement, Hugging Face will be able to incorporate its models into the AWS cloud, allowing developers to utilize any of the models offered by Hugging Face through the AWS-managed ML service known as 'SageMaker'.
Like those tools, ChatGPT — which stands for "generative pre-trained transformer" — landed with a splash.
The human raters are not experts in the topic, and so they tend to choose text that looks convincing. They'd pick up on many symptoms of hallucination, but not all. Accuracy errors that creep in are difficult to catch.
On January 2nd, Edward released his app. He named it GPTZero. It uses ChatGPT against itself, checking whether "there's zero involvement or a lot of involvement" of the AI system in creating a given text. [...] Along these lines, one obvious application for GPTZero is to help teachers identify whether their students are plagiarizing their essays from ChatGPT.
One Sama worker tasked with reading and labeling text for OpenAI told TIME he suffered from recurring visions after reading a graphic description of a man having sex with a dog in the presence of a young child. "That was torture," he said.
Article ChatGPT in English Wikipedia took following places in local popularity ranking:
Presented content of the Wikipedia article was extracted in 2023-03-21 based on https://en.wikipedia.org/?curid=72417803