LLMs

LLM management.

Methods on this page are called through client.llms.

client.llms.create

client.llms.create(request: LlmsCreateRequest, requestOptions?: ProviderApiKeyOptions): Promise<CreateLLMResponseShape>

Creates a new LLM configuration for text generation services. LLMs represent connections to different language model API services (like OpenAI, vLLM, etc.) and include all the necessary configuration to use them for text generation. DUPLICATE DETECTION: Returns HTTP 409 Conflict (ALREADY_EXISTS) if another LLM exists with identical {ownerId, providerType, endpointUrl, apiPath, modelIdentifier, credentialsFingerprint} after URL canonicalization. Uniqueness is enforced per-owner. Credentials are hashed (SHA-256) for uniqueness while remaining encrypted. The apiPath field defaults to '/chat/completions' if omitted. Requires CREATE_LLM_OWN permission (or CREATE_LLM_ANY for admin users).

HTTP: POST /v1/llms

Parameters

Returns: Promise<CreateLLMResponseShape>

Example

const llmResponse = await client.llms.create(
  {
    displayName: "Doc LLM",
    modelIdentifier: "gpt-4o-mini",
    labels: { env: "docs" },
  },
  { apiKey: "sk-..." },
);
const llm = llmResponse.llm;

client.llms.delete

client.llms.delete(id: string, requestOptions?: RequestOptions): Promise<void>

Permanently deletes an LLM configuration. This operation cannot be undone and removes the LLM record and securely deletes stored credentials. IMPORTANT: This does NOT invalidate or delete any previously generated content using this LLM - existing generations remain accessible. Requires DELETE_LLM_OWN permission for LLMs you own (or DELETE_LLM_ANY for admin users).

HTTP: DELETE /v1/llms/{id}

Parameters

Returns: Promise<void>

Example

await client.llms.delete("your-llm-id");

client.llms.get

client.llms.get(id: string, requestOptions?: RequestOptions): Promise<LLMResponseShape>

Retrieves the details of a specific LLM configuration by its unique identifier. Requires READ_LLM_OWN permission for LLMs you own (or READ_LLM_ANY for admin users to view any user's LLMs). This is a read-only operation with no side effects.

HTTP: GET /v1/llms/{id}

Parameters

Returns: Promise<LLMResponseShape>

Example

const fetchedLlm = await client.llms.get("your-llm-id");
console.log(fetchedLlm.displayName);

client.llms.list

client.llms.list(options?: LlmsListOptions, requestOptions?: RequestOptions): Promise<Array<LLMResponseShape>>

Retrieves a list of LLM configurations accessible to the caller, with optional filtering. LABEL FILTERS: Label filters accept either label.<key>=<value> or label[key]=value (for example, label.environment=production or label[environment]=production). PERMISSION-BASED FILTERING: With LIST_LLM_OWN permission, you can only see your own LLMs (ownerId filter is ignored if set to another user). With LIST_LLM_ANY permission, you can see all LLMs or filter by any ownerId. This is a read-only operation with no side effects.

HTTP: GET /v1/llms

Parameters

Returns: Promise<Array<LLMResponseShape>>

Example

for (const item of await client.llms.list()) {
  console.log(item.llmId, item.displayName);
}

client.llms.update

client.llms.update(id: string, request: LLMUpdateRequest, requestOptions?: RequestOptions): Promise<LLMResponseShape>

Updates an existing LLM configuration including display information, endpoint configuration, model parameters, credentials, and labels. All fields are optional - only specified fields will be updated. SUPPORTED_MODALITIES UPDATE: If the array contains >=1 elements, it replaces the stored set; if empty or omitted, no change occurs and it does not count as an update by itself. IMPORTANT: providerType is IMMUTABLE after creation and cannot be changed. Requires UPDATE_LLM_OWN permission for LLMs you own (or UPDATE_LLM_ANY for admin users).

HTTP: PUT /v1/llms/{id}

Parameters

Returns: Promise<LLMResponseShape>

Example

const updatedLlm = await client.llms.update("your-llm-id", {
  displayName: "Doc LLM (updated)",
  mergeLabels: { version: "2" },
});
console.log(updatedLlm.llmId);

Data Models

Enum Values

LLMProviderType

OPENAI, LITELLM_PROXY, OPEN_ROUTER, VLLM, OLLAMA, LLAMA_CPP, CUSTOM_OPENAI_COMPATIBLE

Interfaces

CreateLLMResponse

Response containing the created LLM and any informational status messages

LLMCapabilities

Capabilities and features supported by an LLM service

LLMCreationRequest

Request body for creating a new LLM. An LLM represents a configuration for text generation services.

LLMResponse

LLM configuration information

LLMSamplingParams

Sampling and generation parameters for controlling LLM text output

LLMUpdateRequest

Request body for updating an existing LLM. All fields are optional - only specified fields will be updated. supportedModalities replaces the stored set only when the array contains at least one value; empty or omitted leaves it unchanged and does not count as an update by itself.

ListLLMsResponse

Response containing a list of LLM configurations

Response Shapes

Response shape types model values returned by the SDK after forward-compatible unknown enum strings are coerced to null.

CreateLLMResponseShape

LLMCapabilitiesResponseShape

LLMResponseShape

LLMSamplingParamsResponseShape

ListLLMsResponseShape