Fine-tuning and tool calling
Models that use the chat completions API support tool calling. Functions defined in your chat completion calls don’t always perform as expected. Fine-tuning your model with tool calling examples can improve model output:
| Benefit | Description |
|---|
| Reduce prompt tokens | Get similarly formatted responses even when the full function definition isn’t present. |
| Improve accuracy | Get more accurate and consistent outputs. |
function_call and functions have been deprecated in favor of tools.
Use the tools parameter instead.
Constructing a training file
When constructing a training file of tool calling examples, take a function definition like this:
{
"messages": [
{ "role": "user", "content": "What is the weather in San Francisco?" },
{
"role": "assistant",
"tool_calls": [
{
"id": "call_id",
"type": "function",
"function": {
"name": "get_current_weather",
"arguments": "{\"location\": \"San Francisco, USA\", \"format\": \"celsius\"}"
}
}
]
}
],
"tools": [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and country/region, eg. San Francisco, USA"
},
"format": { "type": "string", "enum": ["celsius", "fahrenheit"] }
},
"required": ["location", "format"]
}
}
}
]
}
.jsonl training file as below:
{"messages":[{"role":"user","content":"What is the weather in San Francisco?"},{"role":"assistant","tool_calls":[{"id":"call_id","type":"function","function":{"name":"get_current_weather","arguments":"{\"location\": \"San Francisco, USA\", \"format\": \"celsius\"}"}}]}],"tools":[{"type":"function","function":{"name":"get_current_weather","description":"Get the current weather","parameters":{"type":"object","properties":{"location":{"type":"string","description":"The city and country/region, eg. San Francisco, USA"},"format":{"type":"string","enum":["celsius","fahrenheit"]}},"required":["location","format"]}}}]}
Optimize for cost
To use fewer prompt tokens after fine-tuning on full function definitions, experiment with the following strategies:
| Strategy | Description |
|---|
| Omit descriptions | Remove the description field from function and parameters. |
| Omit parameters | Remove the entire properties field from the parameters object. |
| Omit function entirely | Remove the entire function object from the functions array. |
Optimize for quality
To improve the quality of tool calling output, keep the function definitions in your fine-tuning training dataset and subsequent chat completion calls identical.
Customize model responses to function outputs
You can also fine-tune on tool calling examples to improve the model’s response to function outputs. Include examples consisting of function response messages and assistant response messages where the function response is interpreted and put into context by the assistant.
{
"messages": [
{"role": "user", "content": "What is the weather in San Francisco?"},
{"role": "assistant", "tool_calls": [{"id": "call_id", "type": "function", "function": {"name": "get_current_weather", "arguments": "{\"location\": \"San Francisco, USA\", \"format\": \"celsius\"}"}}]}
{"role": "tool", "tool_call_id": "call_id", "content": "21.0"},
{"role": "assistant", "content": "It is 21 degrees celsius in San Francisco, CA"}
],
"tools": [] // same as before
}
.jsonl training file would be a single line:
{"messages":[{"role":"user","content":"What is the weather in San Francisco?"},{"role":"assistant","tool_calls":[{"id":"call_id","type":"function","function":{"name":"get_current_weather","arguments":"{\"location\": \"San Francisco, USA\", \"format\": \"celsius\"}"}}]},{"role":"tool","tool_call_id":"call_id","content":"21.0"},{"role":"assistant","content":"It is 21 degrees celsius in San Francisco, CA"}],"tools":[]}
Constructing a training file
When constructing a training file of function calling examples, take a function definition like this:
{
"messages": [
{"role": "user", "content": "What is the weather in San Francisco?"},
{"role": "assistant", "function_call": {"name": "get_current_weather", "arguments": "{\"location\": \"San Francisco, USA\", \"format\": \"celsius\"}"}
],
"functions": [{
"name": "get_current_weather",
"description": "Get the current weather",
"parameters": {
"type": "object",
"properties": {
"location": {"type": "string", "description": "The city and country, eg. San Francisco, USA"},
"format": {"type": "string", "enum": ["celsius", "fahrenheit"]}
},
"required": ["location", "format"]
}
}]
}
.jsonl training file as below:
{"messages": [{"role": "user", "content": "What is the weather in San Francisco?"}, {"role": "assistant", "function_call": {"name": "get_current_weather", "arguments": "{\"location\": \"San Francisco, USA\", \"format\": \"celsius\"}"}}], "functions": [{"name": "get_current_weather", "description": "Get the current weather", "parameters": {"type": "object", "properties": {"location": {"type": "string", "description": "The city and country/region, eg. San Francisco, USA"}, "format": {"type": "string", "enum": ["celsius", "fahrenheit"]}}, "required": ["location", "format"]}}]}
Optimize for cost
To use fewer prompt tokens after fine-tuning on full function definitions, experiment with the following strategies:
| Strategy | Description |
|---|
| Omit descriptions | Remove the description field from function and parameters. |
| Omit parameters | Remove the entire properties field from the parameters object. |
| Omit function entirely | Remove the entire function object from the functions array. |
Optimize for quality
To improve the quality of function calling output, keep the function definitions in your fine-tuning training dataset and subsequent chat completion calls identical.
Customize model responses to function outputs
You can also fine-tune on function calling examples to improve the model’s response to function outputs. Include examples consisting of function response messages and assistant response messages where the function response is interpreted and put into context by the assistant.
{
"messages": [
{"role": "user", "content": "What is the weather in San Francisco?"},
{"role": "assistant", "function_call": {"name": "get_current_weather", "arguments": "{\"location\": \"San Francisco, USA\", \"format\": \"celsius\"}"}}
{"role": "function", "name": "get_current_weather", "content": "21.0"},
{"role": "assistant", "content": "It is 21 degrees celsius in San Francisco, CA"}
],
"functions": [...] // same as before
}
.jsonl training file would be a single line:
{"messages": [{"role": "user", "content": "What is the weather in San Francisco?"}, {"role": "assistant", "function_call": {"name": "get_current_weather", "arguments": "{\"location\": \"San Francisco, USA\", \"format\": \"celsius\"}"}}, {"role": "function", "name": "get_current_weather", "content": "21.0"}, {"role": "assistant", "content": "It is 21 degrees celsius in San Francisco, CA"}], "functions": []}
Next steps