Clean the model output

Problem

How to force a return scheme on openAI model response ?

Control Points

• We want to be 100% sure that the result will have to same

Answer

• It is possible ! Even though it is not the primary usage for functions

Strategy/Solution

Use OpenAI's functions to force a return scheme We can give functions to openAI with the functions parameter. When the model decide to use a function it add a field in the response with the name of the function to call and its arguments. It knows what arguments to use based on its description (see example). So we can give the model a fake function with some well described arguments and the model will put the right data in the right argument. For example we can have a function with a description send_code_to_user with a field output_code with a description "Code to send back to the user". And the model will put the code in this field and not add any text or description.

Furthermore, we can 100% force the model to use the function by doing

function_call={"name": "function_name"} #By default it's set to "auto"

Limitations

None

Example

An output from ChatCompletionApi with functions looks like this :

{
  //...
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "null", // <= classic text response
        // extra properties added with the use of functions
        "function_call": {
          "name": "generate_test",
          "arguments": "{\n  \"name_of_the_test\": \"HistoryOperationCard\",\n  \"render\": \"renderWithProviders(<HistoryOperationCard Icon={<Icon />} title=\\\"Test Title\\\" date={new Date()} amount={100} />)\",\n  \"integration_tests\": \"expect(screen.getByText('Test Title')).toBeTruthy();\\nexpect(screen.getByText('+\u20ac100.00')).toBeTruthy();\"\n}"
        } // This is JSON !
      },
      "finish_reason": "function_call"
    }
  ],
  "usage": {
    // ...
  }
}

Here is the function used :

 function = {
    "name": "generate_test", # fake function
    "description": "Generate a Jest test",
    "parameters": {
        "type": "object",
        "properties": { # scheme of the response we want to force
            "name_of_the_test": {
                "type": "string",
                "description": "Name of the test",
            },
            "mocks": {
                "type": "string",
                "description": "All the mocks needed for the test",
            },
            "render": {
                "type": "string",
                "description": "The render method",
            },
            "integration_tests": {
                "type": "string",
                "description": "The integration tests with actions and expect",
            },
        },
    },
}

# We can call the api by doing
openai.ChatCompletion.create(..., functions=[function])