We expect your custom LLM to be an OpenAI compatible server.

​
Adding your Custom LLM using dashboard

  1. Click on LLM select dropdown as shown in the image

Choose LLM

  1. From the dropdown click on Add your own LLM.

Click Add your own LLM

  1. A dialog box will be displayed. Fill in the following details:
  • LLM URL: the endpoint of your custom LLM
  • LLM Name: a name for your custom LLM
click on Add Custom LLM to connect this LLM to Bolna

Add your custom LLM

  1. Refresh the page
  2. In the LLM settings tab, choose Custom in the first dropdown to select LLM Providers

Choose custom LLM Provider

  1. In the LLM settings tab, you’l now see your custom LLM model name appearing. Select this and save the agent.

Choose custom LLM Model

Using the above steps will make sure the agent uses your Custom LLM URL.

​
Demo video

Here’s a working video highlighting the flow:

Add custom LLM demo video