GPT-3, the latest natural-language system, generates tweets, pens poetry, summarizes email, answers trivia questions, learns language by analyzing online texts, and even writes its own computer programs.
Before it was unveiled this summer by OpenAI, an artificial intelligence lab in San Francisco, GPT-3 had spent months learning the ins and outs of natural language by analyzing thousands of digital books and nearly a trillion words posted to blogs, social media, and the rest of the internet. It uses everything it learns from that vast sea of digital text to generate new language on its own. It even learned to predict the next word in a sequence. (If you type a few words into GPT-3, it will keep going, completing your thought with entire paragraphs of text.)
But in acquiring this specific skill, the system learned much more. During its months of training, GPT-3 identified more than 175 billion parameters—mathematical representations of patterns—from the online texts it analyzed. These patterns amount to a map of human language: a mathematical description of the way we piece characters together, whether we are writing blogs or coding software programs. Using this map, GPT-3 can perform all sorts of tasks it was not built to do.
For many artificial intelligence researchers, it is an unexpected step toward machines that can understand the vagaries of human language—and perhaps even tackle other human skills.
GPT-3 is the culmination of several years of work inside the world’s leading artificial intelligence labs, including labs at Google and Facebook. At Google, a similar system helps answer queries on the company’s search engine.
These systems—known as universal language models—can help power a wide range of tools, like services that automatically summarize news articles and “chatbots” designed for online conversation. So far, their impact on real-world technology has been small. But GPT-3—which learned from a far larger collection of online text than previous systems—opens the door to a wide range of new possibilities, such as software that can speed the development of new smartphone apps, or chatbots that can converse in far more human ways than past technologies.
GPT-3 is far from flawless, however. It often spews biased and toxic language. Jerome Pesenti, who leads the Facebook AI lab, called GPT-3 “unsafe,” pointing to sexist, racist, and otherwise toxic language the system generated when asked to discuss women, Black people, and the Holocaust.
With systems like GPT-3, Pesenti says this problem is endemic. “Everyday language is inherently biased and often hateful, particularly on the internet. Because GPT-3 learns from such language, it, too, can show bias and hate.”
It’s unclear how effective systems like GPT-3 will ultimately be. If GPT-3 generates the right text only half of the time, can it satisfy professionals? And it’s also unclear whether this technique of learning language by analyzing online texts is a path to truly conversational machines, let alone truly intelligent systems.
“It is very articulate. It is very good at producing reasonable-sounding text,” said Mark Riedl, a professor and researcher at the Georgia Institute of Technology. “What it does not do, however, is think in advance. It does not plan out what it is going to say. It does not really have a goal.”
Read Full Article from The New York Times (NY) (11/24/20)
Author: Metz, Cade
News summaries © copyright 2020 SmithBucklin
