Category: Computing

Large language models (LLMs) such as ChatGPT have embedded within them and can make use of the huge amount of information they were fed during training. A user is able to access that embedded knowledge by giving the LLM instructions during the course of a conversation with it. At present, however, the LLM has a limited capacity to remember the details of a conversation; that capacity is determined by the size of its context window.

The context window is the part of the input text that the LLM processes when producing the next word in its response to an instruction. Although it varies across different LLMs, the context window is typically a few thousand words. Once the conversation exceeds the size of the context window, the LLM is unable to make use of everything the user has input over the course of the conversation; it ‘forgets’ things from earlier parts of the conversation. The context window can be increased in size, but that increases the amount of processing that has to be done to produce a response and that further increase soon becomes impractical. Researchers at UC Berkeley are exploring one approach to get around this limitation and have explained it in their paper MemGPT: Towards LLMs as Operating Systems.

In MemGPT, they have given the LLM a memory system similar in principle to that of a personal computer (PC). They call the context window the LLM’s main context and view this as its short-term memory, analogous to a PC’s Random Access Memory (RAM). In addition, MemGPT has been given an external context analogous to a PC’s disk drive or solid state drive. The external context comprises:
recall storage which stores the entire history of events processed by the LLM processor, and
archival storage which serves as a general read-write datastore that can serve as overflow for the main context.

During a conversation, archival storage allows MemGPT to store facts, experiences, preferences, etc. about the user, while recall storage allows MemGPT to find past interactions related to a particular query or within a specific time period. For document analysis, archival storage can be used to search over (and add to) an expansive document database.

To achieve the above, MemGPT’s main context is divided into three components:
system instructions set out the logic for how MemGPT functions control the interaction with external context;
conversational context holds a first-in-first-out (FIFO) queue of recent event history (e.g., messages between the LLM and the user); and
working context serves as a working memory scratchpad.

System instructions are read-only and pinned to main context (they do not change during the lifetime of the MemGPT agent).
Conversational context is read-only with a special eviction policy (if the queue reaches a certain size, a portion of the front is truncated or compressed via recursive summarization).
Working context is writeable by the LLM processor via function calls.

Combined, the three parts of main context cannot exceed the underlying LLM processor’s maximum context size.

The capabilities of ChatGPT are increasing at pace. The latest upgrade turns it into a multimodal AI. Instead of being restricted to text-only input and output, ChatGPT can now accept prompts with images or voice as well as text and can output its responses in one of five AI-generated voices. A user can switch seamlessly between text, image and voice prompts within the same conversation.

Browse with Bing enables ChatGPT to search the internet to help answer questions that benefit from recent information.

Advanced Data Analysis (formerly called Code Interpreter) enables ChatGPT to upload and download files, analyze data, do maths, and create and interpret Python code. These are powerful capabilities but there are restrictions which include: no internet access; a limited set of preinstalled packages; maximum upload and runtime limits; state is cleared (along with any generated files or links) when the environment dies.

Open Interpreter is an open source project which seeks to overcome the restrictions of Advanced Data Analysis. Open Interpreter runs in your local computer and interacts with ChatGPT. It has full access to the internet, is not restricted by time or file size, and can utilize any code package or library. Thus Open Interpreter combines the power of GPT-4’s Advanced Data Analysis with the flexibility of your local development environment.

Plugins enable ChatGPT to interact with functionality provided by other systems. Examples are:
Wolfram Plugin for ChatGPT gives it access to powerful computation, accurate maths, curated knowledge, real-time data and visualization through Wolfram|Alpha and Wolfram Language.
Show Me ChatGPT Plugin allows users to create and edit diagrams directly within a conversation in ChatGPT. 
There is a growing number of plugins; some are shown here.

Being a Large Language Model neural net, ChatGPT cannot by itself do non-trivial computations nor be relied upon to produce correct data. Recent months, however, have seen ChatGPT being linked with Wolfram|Alpha and the Wolfram Language to give it a powerful computational capability. In his blog post ChatGPT Gets Its “Wolfram Superpowers”!, Stephen Wolfram uses some examples to explain the current scope of this combined capability and to hint at the revolutionary power of its future potential.

Steve Blank’s blog post Playing With Fire – ChatGPT looks at that combined capability from another perspective. He highlights that not only is ChatGPT good at what is was designed to do but that it is demonstrating emergent behaviours (things it was not designed to do) which were not seen in its smaller-scale predecessors. He points out, also, that ChatGPT is beginning to interact with a variety of other applications through an application programming interface. These applications can be used by ChatGPT to enhace its own capabilities. Conversely, the applications can harness ChatGPT’s capabilities for their separate, third-party purposes. These increasingly complex systems will display emergent properties, ie properties that the individual parts of the system do not have on their own but which emerge when the parts interact as a whole. Some of the emergent properties will occur by design, but it is inevitable that there will be some which cannot be predicted.

We are still some way from artificial general intelligence, but that is the direction of travel and we should be concerned that the continued development of this technology is driven by for-profit companies, venture capitalists and autocratic governments without any means of control.

ChatGPT can respond to a question presented to it in natural language and is proving to be good at producing a human-like answer. But the answer is not always correct, and this is especially the case when the question involves quantitative data. In this respect ChatGPT is similar to most humans: we find it easy to write an essay but struggle to include correct facts and figures about the subject where these require us to do complicated calculations. Give us a pocket calculator, however, and we can do very much better. Is there a pocket calculator that ChatGPT could use?

Stephen Wolfram believes there is. In Wolfram|Alpha as the Way to Bring Computational Knowledge Superpowers to ChatGPT, he explains that Wolfram|Alpha is able to accept questions in natural language which it then converts into “precise, symbolic computational language [the Wolfram Language] on which it can apply its computational knowledge power” and then produce an answer in natural language. In other words, because ChatGPT communicates using natural language it is in principle able to use Wolfram|Alpha as its pocket calculator.

A possible next step, which Stephen Wolfram says has already started, is for ChatGPT to learn how to use Wolfram Language directly in the same way that humans do. This could enable ChatGPT to produce computational essays which bring together three elements: text to describe context and motivation; computer input in Wolfram Language for a precise specification of what is being talked about; and computer output for facts and results, often in graphical form. A key point here is that the Wolfram Language enables each piece of computer input to be short, not more than a line or two, and to be understandable both by the computer and by a human reading the essay.