WOW! ChatGPT plugin

After less than ten days, OpenAI released another big news after GPT-4.

Just today, OpenAI announced that it has implemented initial support for plug-ins in ChatGPT, helping ChatGPT access the latest information, run calculations or use third-party services. At the same time, OpenAI personally provided two plug-ins: a web browser and a code interpreter. Additionally, OpenAI has open-sourced the code for the knowledge base retrieval plugin, which can be self-hosted by any developer providing the information they wish to enhance ChatGPT.

To put it simply, ChatGPT is finally "connected to the Internet", and through third-party plug-ins, it can better serve users in terms of food, clothing, housing, work, and study.

It should be noted that it is still in the "Alpha stage", and OpenAI will gradually open the plugin from the waiting list, initially giving priority to a small number of developers and ChatGPT Plus users.

However, it will only be a matter of time before OpenAI "opens up on a larger scale".

If ChatGPT is the "iPhone moment" of AI, then today, it built the "App Store" by itself.

Finally "online", NewBing embarrassed?

Previously, the training database of ChatGPT was only updated to 2021, and the training data of GPT-4 was only tracked to September 2021.

And today, OpenAI personally launched the "web browser" plug-in, and this "shortcoming" will no longer exist.

OpenAI stated, "Inspired by the work of WebGPT, GopherCite, BlenderBot2, LaMDA2, etc., allowing language models to read information from the Internet will strictly expand the amount of content it can discuss, from the training corpus to the current Fresh information."

Take the recent hot spot, the 95th Academy Awards Ceremony as an example. Previously, ChatGPT would politely point out that its training data did not contain enough information for it to answer.

Now with the support of the "web browser", ChatGPT can retrieve the latest information about the latest Oscars, and then perform a familiar ChatGPT poetry performance, browsing has become a way of adding experience.

Source: OpenAI

It should be noted that the above plug-ins only support GET requests, not POST. In other words, the scope of users using this plug-in is limited to retrieving information, and does not include "transactional operations", such as form submission.

Meanwhile, OpenAI's Browse uses the Bing Search API to retrieve content from the web. Thanks to this, ChatGPT also inherits a lot of Microsoft's work on the reliability and authenticity of the source of information and the "safety model" that prevents retrieval of problematic content. Also, the plugin runs in a separate server, so ChatGPT browsing activity will be separated from the rest of the infrastructure.

Although OpenAI has a close relationship with Microsoft, Microsoft also integrated ChatGPT into the Bing search engine and launched NewBing. According to data as of March 20, since Microsoft's NewBing, page visits on Bing have increased by 15.8%, while visits to Google's search engine have dropped by nearly 1 percentage point.

However, with the release of the "Internet Browser" plug-in, the frequency of use of NewBing will naturally be affected. Don't worry too much, Microsoft is already prepared.

Microsoft announced as early as a week ago that it would implant the GPT-4 model through Microsoft 365 Copilot. Credit Suisse estimates that if 10% of Microsoft's 370 million commercial Office 365 users upgrade to the AI-enhanced version, it could generate $14.9 billion in revenue over the next five years.

Solving math problems, data visualization, format conversion, more uses to be tapped

In addition to the "web browser" plug-in, another plug-in launched by OpenAI is the "code interpreter".

Specifically, OpenAI provides the model with a working Python interpreter in a sandboxed, firewalled execution environment, along with some temporary disk space.

Code run by an interpreter plugin is evaluated in a persistent session, and subsequent calls can build on each other. Currently, the feature supports uploading files to the current session workspace and downloading work results.

Source: OpenAI

Currently, from initial user research, OpenAI has identified examples where using a code interpreter is particularly useful: solving quantitative and qualitative mathematical problems, performing data analysis and visualization, and converting files between formats.

In the relevant demonstration video, OpenAI mainly demonstrates three parts: generating and modifying charts through functions; parsing the content of CSV files, obtaining and analyzing table data, and generating visual charts; degree, hue, and generate a download link.

However, OpenAI also stated that this code interpreter mainly provides some of the most basic functions of the computer, and provides a smoother interface for some junior programmers and enthusiasts.

In the future, with the increase of users, more functions of this code interpreter will be gradually discovered.

And in terms of "safety", the first precaution for connecting a ChatGPT model to a programming language interpreter is to properly sandbox the execution so that the AI-generated code does not have unintended side effects in the real world. OpenAI executes code in a secure environment and uses strict network controls to prevent external internet access from executing code. Additionally, OpenAI imposes resource limits on a per-session basis.

In addition to setting up a sandbox to ensure isolation, OpenAI has also conducted multiple red team drills with internal and external personnel to ensure that plug-in developers will not perform prompt injection, send fraud and spam, and bypass the plug-in defense system to install Behaviors such as backdoor programs.

All-round food, clothing, housing and transportation, creating an "App Store" AI ecosystem

When GPT-4 was released, in addition to performance improvements, OpenAI stated that many companies have already installed GPT-4 in their products.

Includes: Duolingo, a language learning tool; Be My Eyes, a software for visually impaired users; Stripe, an international financial services company; Morgan Stanley and Khan Academy. Academy).

In addition, OpenAI also announced the service scenario of the cooperation between GPT-4 and the Icelandic government.

If the above-mentioned "landing" is external empowerment, it is equivalent to making wedding dresses for others.

Then, OpenAI's launch of the plug-in function is equivalent to an endogenous upgrade, and it will eventually form a huge ecology with an attitude of inclusiveness.

Currently, OpenAI's first batch of third-party plug-ins open to use includes Expedia (travel), FiscalNote, Instacart (shopping), KAYAK (flight accommodation travel), Klarna, Milo, OpenTable (catering and takeaway), Shopify, Slack, Speak, Wolfram, Zapier.

Source: OpenAI

It's important to note that Zapier is the leader in no-code automation—integrating with more than 5,000 apps from partners like Google, Salesforce, and Microsoft.

As an automation tool, Zapier can help users create automated workflows and help you connect your applications, software & various services. The overall concept of Zapier is to create automated commands by triggering conditions & executing conditions, "If this condition is triggered, then execute this action".

And this also means that the ChatGPT physical examination will be more convenient and smoother.

The outside world generally believes that the launch of the ChatGPT plug-in system is just like Apple's opening of the App Store. And OpenAI is also building its own AI ecological empire through this move.

For developers, if the ChatGPT plug-in can add a "payment" mechanism like the App Store, it will also be a "win-win" situation.