List LLMs
Retrieves a list of LLM configurations accessible to the caller, with optional filtering.
LABEL FILTERS: Supply dynamic query parameters in the form label.<key>=<value> (for example, label.environment=production).
PERMISSION-BASED FILTERING: With LIST_LLM_OWN permission, you can only see your own LLMs (ownerId filter is ignored if set to another user). With LIST_LLM_ANY permission, you can see all LLMs or filter by any ownerId. This is a read-only operation with no side effects.
In: header
Query Parameters
Filter LLMs by owner ID. With LIST_LLM_ANY permission, omitting this shows all accessible LLMs; providing it filters by that owner. With LIST_LLM_OWN permission, only your own LLMs are shown regardless of this parameter.
Filter LLMs by provider type. Allowed values match the LLMProviderType schema.
"OPENAI" | "LITELLM_PROXY" | "OPEN_ROUTER" | "VLLM" | "OLLAMA" | "LLAMA_CPP" | "CUSTOM_OPENAI_COMPATIBLE"Response Body
curl -X GET "http://localhost:8080/v1/llms?ownerId=550e8400-e29b-41d4-a716-446655440000&providerType=OPENAI"{
"llms": [
{
"llmId": "550e8400-e29b-41d4-a716-446655440000",
"displayName": "GPT-4 Turbo",
"description": "OpenAI's GPT-4 Turbo model for chat completions",
"providerType": "OPENAI",
"endpointUrl": "https://api.openai.com/v1",
"apiPath": "/chat/completions",
"modelIdentifier": "gpt-4-turbo-preview",
"credentials": {
"apiKey": {
"inlineSecret": "sk-1234567890abcdef",
"secretRef": {
"uri": "vault://path/to/secret"
},
"headerName": "Authorization",
"prefix": "Bearer "
},
"gcpAdc": {
"scopes": [
"https://www.googleapis.com/auth/cloud-platform"
],
"quotaProjectId": "my-quota-project"
}
},
"version": "1.0.0",
"monitoringEndpoint": "https://monitoring.example.com/llms/status",
"capabilities": {
"supportsChat": true,
"supportsCompletion": true,
"supportsFunctionCalling": true,
"supportsSystemMessages": true,
"supportsStreaming": true,
"supportsSamplingParameters": true
},
"defaultSamplingParams": {
"maxTokens": 2048,
"temperature": 0.7,
"topP": 0.9,
"topK": 50,
"frequencyPenalty": 0,
"presencePenalty": 0,
"stopSequences": [
"\n\n",
"END"
]
},
"maxContextLength": 32768,
"ownerId": "550e8400-e29b-41d4-a716-446655440000",
"createdAt": 1617293472000,
"updatedAt": 1617293472000,
"createdById": "550e8400-e29b-41d4-a716-446655440000",
"updatedById": "550e8400-e29b-41d4-a716-446655440000"
}
]
}