Code Interpreter

How to build an AI capable of generating and running code

It was also inspired by this tutorial made by E2B.

In this recipe, we will build a simple version of ChatGPT Code Interpreter using OpenAI GPT-4 and E2B.

Here's the final result:

What's the plan?

Building a code interpreter is pretty straightforward; we'll proceed as follows:

  1. We'll use the AgentLabs SDK to listen for user's input in the Chat.

  2. We'll send the request to GPT-4 and ask it to generate some code snippets if asked by the user.

  3. GPT is returning some code to run; then, we'll use E2B to run the code in a sandboxed cloud environment.

  4. We'll use the AgentLabs SDK again to display the result to the user

Using E2B is not mandatory, but it's a great solution to run code safely in a sandboxed environment, so we highly recommend you check this out.

Let's code

Prepare the environment variables

In this tutorial, we'll need many environment variables. If you look at our example, you'll see we verify every variable is present before running the app, but you're free to manage your environment variables as you want.

We'll need the following variables:

  • AgentLabs Project ID, Agent ID, URL and Secret (that you can find on your console)

  • Open AI Api Key

  • E2B Api Key

Handle user requests

Once our variables are ready, we'll import and init the AgentLabs SDK.

Here, we instantiate the project and the agent.

We start listening for user messages using the .onChatMessage() method, we'll complete it later in this recipe.

We don't forget to use the .connect() method to open the socket connection.

import { Project } from "@agentlabs/node-sdk";

const project = new Project({
    projectId: 'your-project-id',
    secret: 'your-secret',
    url: 'agentlabs-url'
});

const agent = project.agent(agentId);

project.onChatMessage((userMessage) => {
  // We'll fill it out later
});

project.connect();

Talk to GPT-4

Whenever a user sends a message, we'll forward it to GPT-4 using the ChatCompletionAPI. We expect GPT-4 to send us a response with potentially the code to execute.

We'll provide GPT with two kinds of information:

  • The context: a list of fake history messages so GPT understands what we expect, plus in the end the message sent by the user.

  • The function calling config: we leverage the function call API so ChatGPT will hopefully output some code we can directly execute in the sandbox.

import OpenAI from 'openai';

// ...other parts

// We define the options for the function call API.
const functions = [
    {
        name: 'exec_code',
        description: 'Executes the passed JavaScript code using Nodejs and returns the stdout and stderr',
        parameters: {
            type: 'object',
            properties: {
                code: {
                    type: 'string',
                    description: 'The JavaScript code to execute.',
                },
            },
            required: ['code'],
        },
    },
]

project.onChatMessage(async (userMessage) => {
    const conversationId = userMessage.conversationId;

    // We call the ChatCompletionAPI with some context
    // and our function call API config
    const chatCompletion = await openai.chat.completions.create({
        model: 'gpt-4',
        messages: [
            {
                role: 'system',
                content: 'You are a senior developer that can code in JavaScript. Always produce valid JSON.',
            },
            {
                role: 'user',
                content: 'Write hello world',
            },
            {
                role: 'assistant',
                content: '{"code": "print("hello world")"}',
                name: 'exec_code',
            },
            {
                role: 'user',
                // Here, we send the message of the user
                content: userMessage.text,
            }
        ],
        functions,
    });
});

project.connect();

Don't forget the OpenAI SDK needs an environment variable named OPENAI_API_KEY to be set.

Parsing GPT's response

If you look at the chatCompletion result, you will see it contains something like this: 

{
  ...
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "function_call": {
          "name": "exec_code",
          "arguments": "{\n  \"code\": \"\n console.log('something') \n  \"\n}"
        }
      },
      "finish_reason": "function_call"
    }
  ],
  ...
}

From this result we want to:

  • Know if the choices[0].function_call.name equals exec_code. If yes, it means we have to execute the code.

  • Parse the arguments of choices[0].function_call.arguments in order to extract the value of the code to execute.

// Inside the onChatMessage handler

const message = chatCompletion.choices[0].message;

const func = message["function_call"];
    
// If function_call is undefined, we don't handle this.
if (!func) { return; }

const funcName = func.name;

// Get rid of newlines and leading/trailing spaces in the raw function arguments JSON string.
// This sometimes help to avoid JSON parsing errors.
let args = func["arguments"];
args = args.trim().replace(/\n|\r/g, "");
// Parse the cleaned up JSON string.
const funcArgs = JSON.parse(args);

// We don't handle that case in this example
if (funcName !== "exec_code") {
  return; 
}

// Here we have our code ready to be executed
const code = funcArgs["code"];

Executing the code

Now we have extracted the code to run; we can use E2B to execute it safely in a sandboxed environment.

We will use their process API to start a new process that will run our code.

Here's the plan:

  1. Initiate an E2B session using Session.create() method.

  2. Write the code we want to execute in a file named index.js in the sandbox.

  3. Create a new process, running the node index.js command to run our code remotely.

  4. We'll keep the onStdout and onStderr callbacks empty for now but we'll use them later.

  5. We'll wait for the process execution to be completed.

import { Session } from '@e2b/sdk'


// The following part will still run inside the onChatMessage function.
const session = await Session.create({
    id: 'Nodejs',
    apiKey: e2bApiKey,
});

await session.filesystem.write('/index.js', code);

const proc = await session.process.start({
    cmd: 'node /index.js',
    onStdout: (data) => {
        // we'll use it later
    },
    onStderr: (data) => {
        // we'll use it later
    }
});

await proc.finished;

The code above will run our code until the execution is completed. Every output in Stdout or Stderr will trigger a call of one of the given callbacks.

Sending results and feedback to the user in real time

So far we're able to listen for messages, ask completions to OpenAI, and eventually execute the code.

We still need to implement some real-time feedback to the users so they know what's going on, and they will view the output generated by the code interpreter.

To do so, we'll leverage the agent.createSteam() method provided by AgentLabs.

We can add some feedback whenever we want.

Let's create a stream if the ChatCompletion contains a function_call. 

const func = message["function_call"];
    
// If function_call is undefined, we don't handle this.
if (!func) { return; }

// We open a stream
const stream = agent.createStream({
    conversationId,
}, {
    format: 'Markdown',
});

Here, we open a stream channel, and we indicate we'll send some Markdown, so the Chat UI knows how to display the code output.

Now we can use stream.write to write on the chat every time we need to.

For example, we can indicate we received some code to execute.

// This is the code we wrote before
let args = func["arguments"];
args = args.trim().replace(/\n|\r/g, "");
const funcArgs = JSON.parse(args);

stream.write(`Here is the code I have to execute:\n`);
stream.write(`\`\`\`js\n${funcArgs["code"]}\n\`\`\`\n\n`);

Note we stream in markdown format, so we use ``` in front of the code so it's shown in a code block.

Then, we want to notify the user we'll start executing the code, and crucially we want to output the result of the code interpreter in real time. To do so, we'll write to our stream before executing the code, and every time the onStdout or onStderr callbacks are called.

And once we're done, we'll end the stream.

await session.filesystem.write('/index.js', code);
stream.write(`Executing the code...\n\n`);

// This is to open a code block
stream.write(`\`\`\`\n`);

const proc = await session.process.start({
    cmd: 'node /index.js',
    onStdout: (data) => {
       stream.write(data.line + '\n');
    },
    onStderr: (data) => {
        stream.write(data.line + '\n');
    }
});

await proc.finished;
// We close the code block
stream.write(`\n\`\`\`\n`);
stream.end();

Congrats, you just rebuilt your own code interpreter!

Feel free to check out the full example here.

PreviousChatGPT with LangChainNextMutli-Agent with AutoGen

Last updated