LLMs

LLM management.

Methods on this page are called through client.llms.

client.llms.create

client.llms.create(request: LlmsCreateRequest, requestOptions?: ProviderApiKeyOptions): Promise<CreateLLMResponseShape>

Creates a new LLM configuration for text generation services. LLMs represent connections to different language model API services (like OpenAI, vLLM, etc.) and include all the necessary configuration to use them for text generation. DUPLICATE DETECTION: Returns HTTP 409 Conflict (ALREADY_EXISTS) if another LLM exists with the same effective provider connection and model configuration for this owner after endpoint canonicalization and provider-default resolution. Equivalent credentials participate in the comparison. The apiPath field defaults to '/chat/completions' if omitted. OWNER DEFAULTS: Owner defaults to the authenticated principal unless ownerId is provided; CREATE_LLM is evaluated against the proposed LLM and owner.

HTTP: POST /v1/llms

Parameters

Returns: Promise<CreateLLMResponseShape>

Example

const llmResponse = await client.llms.create(
  {
    displayName: "Doc LLM",
    modelIdentifier: "gpt-4o-mini",
    labels: { env: "docs" },
  },
  { apiKey: "sk-..." },
);
const llm = llmResponse.llm;

client.llms.delete

client.llms.delete(id: string, requestOptions?: RequestOptions): Promise<void>

Permanently deletes an LLM configuration. This operation cannot be undone and removes the LLM record and securely deletes stored credentials. IMPORTANT: This does NOT invalidate or delete any previously generated content using this LLM - existing generations remain accessible. Requires DELETE_LLM on the requested LLM.

HTTP: DELETE /v1/llms/{id}

Parameters

Returns: Promise<void>

Example

await client.llms.delete("your-llm-id");

client.llms.get

client.llms.get(id: string, options?: LlmsGetOptions, requestOptions?: RequestOptions): Promise<LLMResponseShape>

Retrieves the details of a specific LLM configuration by its unique identifier. Requires READ_LLM on the requested LLM. The service distinguishes a missing LLM from an existing LLM the caller cannot read. This is a read-only operation with no side effects.

HTTP: GET /v1/llms/{id}

Parameters

Returns: Promise<LLMResponseShape>

Example

const fetchedLlm = await client.llms.get("your-llm-id");
console.log(fetchedLlm.displayName);

client.llms.list

client.llms.list(options?: LlmsListOptions, requestOptions?: RequestOptions): Promise<Array<LLMResponseShape>>

Retrieves a list of LLM configurations accessible to the caller, with optional filtering. LABEL FILTERS: Label filters accept either label.<key>=<value> or label[key]=value (for example, label.environment=production or label[environment]=production). AUTHORIZATION: Requires LIST_LLM on the GoodMem instance. Each returned LLM must also be visible through READ_LLM; unauthorized LLMs are filtered in PostgreSQL. The ownerId parameter filters that already-authorized result set and does not grant additional visibility. This is a read-only operation with no side effects.

HTTP: GET /v1/llms

Parameters

Returns: Promise<Array<LLMResponseShape>>

Example

for (const item of await client.llms.list()) {
  console.log(item.llmId, item.displayName);
}

client.llms.update

client.llms.update(id: string, request: LLMUpdateRequest, requestOptions?: RequestOptions): Promise<LLMResponseShape>

Updates an existing LLM configuration including display information, endpoint configuration, model parameters, credentials, and labels. All fields are optional - only specified fields will be updated. SUPPORTED_MODALITIES UPDATE: If the array contains >=1 elements, it replaces the stored set; if empty or omitted, no change occurs and it does not count as an update by itself. IMPORTANT: providerType is IMMUTABLE after creation and cannot be changed. Requires UPDATE_LLM on the requested LLM.

HTTP: PUT /v1/llms/{id}

Parameters

Returns: Promise<LLMResponseShape>

Example

const updatedLlm = await client.llms.update("your-llm-id", {
  displayName: "Doc LLM (updated)",
  mergeLabels: { version: "2" },
});
console.log(updatedLlm.llmId);

Data Models

Enum Values

LLMProviderType

OPENAI, LITELLM_PROXY, OPEN_ROUTER, DASHSCOPE, VLLM, OLLAMA, LLAMA_CPP, CUSTOM_OPENAI_COMPATIBLE

Interfaces

CreateLLMResponse

Response containing the created LLM and any informational status messages

LLMCapabilities

Capabilities and features supported by an LLM service

LLMCreationRequest

Request body for creating a new LLM. An LLM represents a configuration for text generation services.

LLMResponse

LLM configuration information

LLMSamplingParams

Sampling and generation parameters for controlling LLM text output

LLMUpdateRequest

Request body for updating an existing LLM. All fields are optional - only specified fields will be updated. supportedModalities replaces the stored set only when the array contains at least one value; empty or omitted leaves it unchanged and does not count as an update by itself.

ListLLMsResponse

Response containing a list of LLM configurations

Response Shapes

Response shape types model values returned by the SDK after forward-compatible unknown enum strings are coerced to null.

CreateLLMResponseShape

LLMCapabilitiesResponseShape

LLMResponseShape

LLMSamplingParamsResponseShape

ListLLMsResponseShape