ChatGPT Plugins Are Transforming the Internet Browsing Experience
OpenAI released Plugins for ChatGPT, allowing users to access third-party services and run computations from the chat interface, changing software development and potentially transforming the internet
by Mona Hamdy, CSO & Shira Eisenberg, intern
Overview
OpenAI recently released Plugins for ChatGPT. “Plugins are tools designed specifically for language models with safety as a core principle,” reads the blog announcement. They “help ChatGPT access up-to-date information, run computations, or use third-party services.”
Among the initial batch of plugins released to a small set of invited users and users off a waitlist are Expedia, FiscalNote, Instacart, KAYAK, Klarna Shopping, Milo Family AI, OpenTable, Shop, Speak, Wolfram, and Zapier. These extensions will enable users to do things like shop for groceries, make dining reservations, do computations, book flights, shop for products, learn new languages, and access data all from the chat interface.
In addition to these plugins, there is a code interpreter plugin that enables code to compile in a sandbox from the chat interface. OpenAI is also making code for a knowledge base retrieval plugin available to the public, making it possible for developers to add their data directly to ChatGPT. There is a waiting list for developer access to building new plugins.
The release of ChatGPT plugins is a big first step toward transforming conversational AI into a more capable, intelligent agent, rather than a knowledgeable advisor. This new intelligent agent can accomplish tasks for users rather than just giving them advice on how to accomplish those tasks themselves.
Plugins enable ChatGPT to become an “everything app,” extending its capabilities to the realm of the internet. ChatGPT already has 100 million users. Now, it has an app store. It is becoming, in its own way, an operating system and might reorient the entire internet and browsing experience, taking ChatGPT from a service that was relatively siloed, with outputs and ambitions limited by its training data, to an all capable agent limited solely by the services it is plugged into.
With plugins, ChatGPT goes from another tab in your browser to an app with control over your browser, with the capabilities of an all-purpose assistant. This introduces a fundamentally new way of interacting with our computers and phones, potentially replacing the home screen and Google search and information feeds with a customizable chat-based interface, transforming ChatGPT from a demo to a platform of its own.
How to build a plugin & how plugins are changing software development
In addition to remodeling the internet browsing experience, plugins are changing the way software is developed and coding is performed. Typically, to connect two programs, a developer needs to understand how a system interacts with different systems, via an API, and write “glue code” so two services can communicate with each other. Plugins offer a different approach.
To create a ChatGPT plugin, a developer simply needs to instruct the model properly. The developer provides ChatGPT with a manifest of the service’s API, in plain English. ChatGPT reads it and generates the glue code for them. The developer still needs to write code for the initial API.
Software development has a long history of adding new layers of abstraction to hide the complexity of the binary logic behind human-friendly interfaces, going from assembly language to higher-level coding environments to Graphical User Interfaces (GUIs). OpenAI’s newest developments begin to fulfill a promise of a natural language programming tool that lets people use English to create code and instruct computers.
Today’s large language models can take instructions from programmers using human language. While the results are far from perfect, passing in error messages enables the language model to self-correct and return new, better code. In short, the results are far better than anyone could expect a few months to a year ago. Additionally, ChatGPT can explain code and answer questions about how it works.
This doesn’t necessarily mean programmers are out of a job just yet. Generative AI is still a work in progress and still hallucinates information to fill gaps in knowledge. It works best as an assistant for programmers rather than a replacement for them. While human programmers will still be needed to invent and engineer new kinds of systems and technologies and to remedy problems that AI cannot yet solve, there may be less demand for the rote labor of wiring up software systems to work together, which makes up a lot of current development work.
Thanks to the code interpreter plugin for ChatGPT, the model can now use Python and manage uploads and downloads in a sandbox environment. The code interpreter helps solve mathematical problems, analyze data, display tables, and convert file formats, among other tasks.
Concluding remarks
For the regular user, plugins act as a model’s eyes and ears, giving them more access to recent, particular, or personal data. For a programmer, plugins are a new distribution channel for apps and services. With authorization credentials added to the manifest, plugins make it possible to carry out restricted and secure actions on behalf of users.
While availability is still limited to a select group of developers and ChatGPT Plus users, OpenAI plans to expand access gradually, over time.