Llm

Services

LLMService Service

Service for managing LLM configurations in the GoodMem system.

Authentication: gRPC metadata x-api-key: <api-key>

Global errors: All RPCs may return DEADLINE_EXCEEDED, CANCELLED, UNAVAILABLE, RESOURCE_EXHAUSTED, INTERNAL.

Authorization model:

• Every RPC evaluates a typed authorization operation against the GoodMem instance or a particular LLM.
• Ownership, code-defined role assignments, and direct grants can supply authority; an API-key ceiling may further restrict the authenticated principal's effective authority.
• Listing requires LIST_LLM on the instance and returns only LLMs on which the caller has READ_LLM.

CreateLLM

Creates a new LLM configuration.

Auth: gRPC metadata x-api-key: <api-key>

Authorization Required: CREATE_LLM on the proposed LLM, evaluated using its proposed owner.

Summary:

• Owner defaults to the authenticated principal unless owner_id is provided; authorization is evaluated against that proposed owner
• ALREADY_EXISTS: another LLM for the same owner has the same effective provider connection and model configuration after endpoint canonicalization and provider-default resolution; equivalent credentials participate in this comparison
• The response omits stored credentials; use GetLLM with include_credentials=true for separately authorized disclosure

Side Effects:

• Persists the LLM and its credentials; sets audit fields

Error Codes:

• UNAUTHENTICATED: missing/invalid auth
• PERMISSION_DENIED: lacks CREATE_LLM on the proposed LLM
• INVALID_ARGUMENT: bad URL(s); empty/invalid fields; UNSPECIFIED enums; labels exceed limits/charset; unsupported modality; invalid sampling parameters
• ALREADY_EXISTS: matching LLM as defined above
• INTERNAL: unexpected server error

Idempotency: Non-idempotent; clients SHOULD NOT blindly retry on unknown failures.

Examples:

grpcurl -plaintext \
-H 'x-api-key: gm_xxx' \
-d '{
"display_name": "GPT-4 Turbo",
"provider_type": "LLM_PROVIDER_TYPE_OPENAI",
"endpoint_url": "https://api.openai.com",
"model_identifier": "gpt-4-turbo-preview",
"capabilities": {"supports_chat": true, "supports_streaming": true},
"credentials": "sk-***",
"labels": {"env":"prod","team":"ai"}
}' \
localhost:8080 goodmem.v1.LLMService/CreateLLM

Note: bytes fields in JSON must be base64.

GetLLM

Retrieves details of a specific LLM.

Auth: gRPC metadata x-api-key: <api-key>

Authorization Required: READ_LLM on the requested LLM. When include_credentials is true, the caller must also hold READ_LLM_CREDENTIALS; only the current owner or an instance administrator can supply that authority.

Side Effects: None

Error Codes:

• UNAUTHENTICATED: missing/invalid auth
• PERMISSION_DENIED: lacks READ_LLM, or requests credentials without READ_LLM_CREDENTIALS
• INVALID_ARGUMENT: invalid LLM ID format
• NOT_FOUND: LLM does not exist
• INTERNAL: unexpected server error

Idempotency: Read-only; safe to retry; results may change over time.

Examples:

grpcurl -plaintext \
-H 'x-api-key: gm_xxx' \
-d '{ "llm_id": "BASE64_UUID_BYTES_HERE" }' \
localhost:8080 goodmem.v1.LLMService/GetLLM

Note: bytes fields in JSON must be base64.

ListLLMs

Lists LLMs accessible to the authenticated principal.

Auth: gRPC metadata x-api-key: <api-key>

Authorization Required: LIST_LLM on the GoodMem instance and READ_LLM on each returned LLM. The standard user role may enumerate the catalog and read owned LLMs; direct grants can expose additional LLMs.

Request Parameters:

• owner_id (optional, bytes UUID): filter the already-authorized result set by owner
• provider_type (optional): UNSPECIFIED ignored
• label_selectors (optional): AND of exact key=value matches (case-sensitive)
• Stored credentials are always omitted from list responses

Note: bytes fields in JSON must be base64.

Side Effects: None

Error Codes:

• UNAUTHENTICATED: missing/invalid auth
• PERMISSION_DENIED: lacks instance-level LIST_LLM
• INVALID_ARGUMENT: invalid filters or parameters
• INTERNAL: unexpected server error

Idempotency: Read-only; safe to retry; results may change over time.

Examples:

grpcurl -plaintext \
-H 'x-api-key: gm_xxx' \
-d '{ "provider_type":"LLM_PROVIDER_TYPE_OPENAI", "label_selectors":{"env":"prod"} }' \
localhost:8080 goodmem.v1.LLMService/ListLLMs

UpdateLLM

Updates mutable properties of an LLM.

Auth: gRPC metadata x-api-key: <api-key>

Authorization Required: UPDATE_LLM on the requested LLM.

Summary:

• Mutable fields include display_name, description, endpoint_url, api_path, model_identifier, supported_modalities, credentials, version, monitoring_endpoint, capabilities, default_sampling_params, max_context_length, client_config, and labels
• For supported_modalities, a non-empty list replaces the stored set; omission or an empty list leaves it unchanged and does not count as an update by itself
• provider_type and owner_id are immutable after creation
• Omitting credentials preserves the stored value; a present empty payload is invalid
• The response always omits stored credentials

Side Effects:

• Persists changes; updates updated_at and updated_by_id

Error Codes:

• UNAUTHENTICATED: missing/invalid auth
• PERMISSION_DENIED: lacks UPDATE_LLM on the requested LLM
• INVALID_ARGUMENT: invalid fields or formats
• NOT_FOUND: LLM does not exist
• INTERNAL: unexpected server error

Idempotency: Idempotent with identical input; safe to retry.

Examples:

grpcurl -plaintext \
-H 'x-api-key: gm_xxx' \
-d '{
"llm_id": "BASE64_UUID_BYTES_HERE",
"replace_labels": { "items": {} }
}' \
localhost:8080 goodmem.v1.LLMService/UpdateLLM

Note: bytes fields in JSON must be base64.

DeleteLLM

Permanently deletes an LLM configuration.

Auth: gRPC metadata x-api-key: <api-key>

Authorization Required: DELETE_LLM on the requested LLM.

Side Effects:

• Removes the LLM record; securely deletes stored credentials
• Does not cancel in-flight generations; previously generated content is unaffected

Error Codes:

• UNAUTHENTICATED: missing/invalid auth
• PERMISSION_DENIED: lacks DELETE_LLM on the requested LLM
• INVALID_ARGUMENT: invalid LLM ID format
• NOT_FOUND: LLM does not exist
• INTERNAL: unexpected server error

Idempotency: Safe to retry; may return NOT_FOUND if already deleted or never existed.

Examples:

grpcurl -plaintext \
-H 'x-api-key: gm_xxx' \
-d '{ "llm_id": "BASE64_UUID_BYTES_HERE" }' \
localhost:8080 goodmem.v1.LLMService/DeleteLLM

Note: bytes fields in JSON must be base64.

Messages

LLMCapabilities

Capabilities and features supported by an LLM service.

Defines the interface features and generation modes available from the model endpoint. These capabilities determine which API methods and request formats can be used.

Multi-modal capabilities:

• Vision support is derived from IMAGE in supported_modalities field
• Audio support is derived from AUDIO in supported_modalities field

Capability interactions:

• supports_chat and supports_completion may be independent; some providers expose only one mode
• supports_function_calling requires supports_chat in most implementations
• supports_system_messages enhances both chat and completion modes when available
• supports_streaming applies to both chat and completion generation modes
• supports_sampling_parameters controls whether stochastic generation knobs (e.g., temperature, top_p) are available; deterministic-only models will disable this

See also: goodmem.v1.Modality

LLMSamplingParams

Sampling and generation parameters for controlling LLM text output.

These parameters fine-tune the generation behavior, creativity, and output constraints of the language model. Different providers may support different subsets of parameters.

Parameter interactions:

• temperature and top_p work together; lower values increase determinism
• top_k is primarily used by local/open-source models (HuggingFace, Ollama)
• frequency_penalty and presence_penalty help reduce repetitive output
• stop_sequences provide precise generation termination control

Provider compatibility:

• OpenAI-style APIs commonly support temperature/top_p/frequency/presence controls, with OpenAI using temperature 0-2 and penalties -2..2; top_k support varies.
• Anthropic-compatible APIs cap or ignore some OpenAI sampling fields.
• Cohere, Vertex/Gemini, local models, and custom providers use provider-specific ranges and supported subsets.

LLM

Represents a connection to a Large Language Model service for text generation.

LLMs provide generative AI capabilities for chat completion, text completion, and function calling by interfacing with both hosted services (OpenAI, API gateways) and self-hosted models (vLLM, Ollama). Each configuration includes connection details, model parameters, generation capabilities, and access credentials.

Security:

• credentials are returned only by GetLLM when explicitly requested and separately authorized. Create, update, and list responses omit them.

Immutability:

• provider_type is IMMUTABLE after creation.
• owner_id is set at creation and cannot be modified.

Notes:

• All timestamps are UTC (google.protobuf.Timestamp).
• Complex configuration objects (capabilities, default_sampling_params, client_config) are stored as JSONB.

See also: LLMProviderType, LLMCapabilities, LLMSamplingParams, goodmem.v1.Modality

LLM.LabelsEntry

CreateLLMRequest

CreateLLMRequest.LabelsEntry

CreateLLMResponse

Response message for the CreateLLM RPC.

Contains the newly created LLM configuration and any informational statuses generated during the creation process, such as results from capability inference.

GetLLMRequest

ListLLMsRequest

ListLLMsRequest.LabelSelectorsEntry

ListLLMsResponse

Response message for the ListLLMs RPC.

Contains a list of LLM configurations that match the request filters and are accessible by the authenticated principal.

UpdateLLMRequest

DeleteLLMRequest

Enums

LLMProviderType

LLM provider type for text generation services.

Organized by deployment model with distinct value ranges:

• 0: Invalid type
• 1-99: Reserved for future general providers
• 10-19: Cloud/SaaS providers and API gateways
• 100-199: Self-hosted and local deployment solutions
• 999: Generic compatibility fallback

GoodMem's DashScope integration