Llm

Services

LLMService Service

Service for managing LLM configurations in the GoodMem system.

Authentication: gRPC metadata authorization: Bearer <api-key>

Global errors: All RPCs may return DEADLINE_EXCEEDED, CANCELLED, UNAVAILABLE, RESOURCE_EXHAUSTED, INTERNAL.

Permissions model:

• *_OWN: operate on caller-owned LLMs
• *_ANY: operate on any user's LLMs (requires elevated role)

CreateLLM

Creates a new LLM configuration.

Auth: gRPC metadata authorization: Bearer <api-key>

Permissions Required: CREATE_LLM_OWN or CREATE_LLM_ANY

Summary:

• Owner defaults to the authenticated user unless owner_id is provided (requires *_ANY if differs)
• ALREADY_EXISTS: another LLM exists with identical {owner_id, provider_type, endpoint_url, api_path, model_identifier, credentials_fingerprint} after URL canonicalization: trailing slash stripped; host comparison case-insensitive; default ports removed; path and model_identifier comparison case-sensitive; credentials_fingerprint is SHA-256 hash of normalized credentials

Side Effects:

• Persists LLM; encrypts credentials; sets audit fields

Error Codes:

• UNAUTHENTICATED: missing/invalid auth
• PERMISSION_DENIED: lacks CREATE_LLM_*
• INVALID_ARGUMENT: bad URL(s); empty/invalid fields; UNSPECIFIED enums; labels exceed limits/charset; unsupported modality; invalid sampling parameters
• ALREADY_EXISTS: matching LLM as defined above
• INTERNAL: unexpected server error

Idempotency: Non-idempotent; clients SHOULD NOT blindly retry on unknown failures.

Examples:

grpcurl -plaintext \
-H 'authorization: Bearer gm_xxx' \
-d '{
"display_name": "GPT-4 Turbo",
"provider_type": "LLM_PROVIDER_TYPE_OPENAI",
"endpoint_url": "https://api.openai.com",
"model_identifier": "gpt-4-turbo-preview",
"capabilities": {"supports_chat": true, "supports_streaming": true},
"credentials": "sk-***",
"labels": {"env":"prod","team":"ai"}
}' \
localhost:8080 goodmem.v1.LLMService/CreateLLM

Note: bytes fields in JSON must be base64.

GetLLM

Retrieves details of a specific LLM.

Auth: gRPC metadata authorization: Bearer <api-key>

Permissions Required: READ_LLM_OWN or READ_LLM_ANY

Side Effects: None

Error Codes:

• UNAUTHENTICATED: missing/invalid auth
• PERMISSION_DENIED: lacks READ_LLM_*
• INVALID_ARGUMENT: invalid LLM ID format
• NOT_FOUND: LLM does not exist
• INTERNAL: unexpected server error

Idempotency: Read-only; safe to retry; results may change over time.

Examples:

grpcurl -plaintext \
-H 'authorization: Bearer gm_xxx' \
-d '{ "llm_id": "BASE64_UUID_BYTES_HERE" }' \
localhost:8080 goodmem.v1.LLMService/GetLLM

Note: bytes fields in JSON must be base64.

ListLLMs

Lists LLMs accessible to the authenticated user.

Auth: gRPC metadata authorization: Bearer <api-key>

Permissions Required: LIST_LLM_OWN or LIST_LLM_ANY

Request Parameters:

• owner_id (optional, bytes UUID): filter by owner. If caller has LIST_LLM_ANY and owner_id is omitted, all LLMs visible to the role are returned; otherwise only caller-owned LLMs are returned
• provider_type (optional): UNSPECIFIED ignored
• label_selectors (optional): AND of exact key=value matches (case-sensitive)

Note: bytes fields in JSON must be base64.

Side Effects: None

Error Codes:

• UNAUTHENTICATED: missing/invalid auth
• PERMISSION_DENIED: lacks LIST_LLM_*
• INVALID_ARGUMENT: invalid filters or parameters
• INTERNAL: unexpected server error

Idempotency: Read-only; safe to retry; results may change over time.

Examples:

grpcurl -plaintext \
-H 'authorization: Bearer gm_xxx' \
-d '{ "provider_type":"LLM_PROVIDER_TYPE_OPENAI", "label_selectors":{"env":"prod"} }' \
localhost:8080 goodmem.v1.LLMService/ListLLMs

UpdateLLM

Updates mutable properties of an LLM.

Auth: gRPC metadata authorization: Bearer <api-key>

Permissions Required: UPDATE_LLM_OWN or UPDATE_LLM_ANY

Side Effects:

• Persists changes; updates updated_at and updated_by_id

Error Codes:

• UNAUTHENTICATED: missing/invalid auth
• PERMISSION_DENIED: lacks UPDATE_LLM_*
• INVALID_ARGUMENT: invalid fields or formats
• NOT_FOUND: LLM does not exist
• INTERNAL: unexpected server error

Idempotency: Idempotent with identical input; safe to retry.

Examples:

grpcurl -plaintext \
-H 'authorization: Bearer gm_xxx' \
-d '{
"llm_id": "BASE64_UUID_BYTES_HERE",
"replace_labels": { "items": {} }
}' \
localhost:8080 goodmem.v1.LLMService/UpdateLLM

Note: bytes fields in JSON must be base64.

DeleteLLM

Permanently deletes an LLM configuration.

Auth: gRPC metadata authorization: Bearer <api-key>

Permissions Required: DELETE_LLM_OWN or DELETE_LLM_ANY

Side Effects:

• Removes the LLM record; securely deletes stored credentials
• Does not cancel in-flight generations; previously generated content is unaffected

Error Codes:

• UNAUTHENTICATED: missing/invalid auth
• PERMISSION_DENIED: lacks DELETE_LLM_*
• INVALID_ARGUMENT: invalid LLM ID format
• NOT_FOUND: LLM does not exist
• INTERNAL: unexpected server error

Idempotency: Safe to retry; may return NOT_FOUND if already deleted or never existed.

Examples:

grpcurl -plaintext \
-H 'authorization: Bearer gm_xxx' \
-d '{ "llm_id": "BASE64_UUID_BYTES_HERE" }' \
localhost:8080 goodmem.v1.LLMService/DeleteLLM

Note: bytes fields in JSON must be base64.

Messages

LLMCapabilities

Capabilities and features supported by an LLM service.

Defines the interface features and generation modes available from the model endpoint. These capabilities determine which API methods and request formats can be used.

Multi-modal capabilities:

• Vision support is derived from IMAGE in supported_modalities field
• Audio support is derived from AUDIO in supported_modalities field

Capability interactions:

• supports_chat and supports_completion may be independent; some providers expose only one mode
• supports_function_calling requires supports_chat in most implementations
• supports_system_messages enhances both chat and completion modes when available
• supports_streaming applies to both chat and completion generation modes
• supports_sampling_parameters controls whether stochastic generation knobs (e.g., temperature, top_p) are available; deterministic-only models will disable this

See also: goodmem.v1.Modality

LLMSamplingParams

Sampling and generation parameters for controlling LLM text output.

These parameters fine-tune the generation behavior, creativity, and output constraints of the language model. Different providers may support different subsets of parameters.

Parameter interactions:

• temperature and top_p work together; lower values increase determinism
• top_k is primarily used by local/open-source models (HuggingFace, Ollama)
• frequency_penalty and presence_penalty help reduce repetitive output
• stop_sequences provide precise generation termination control

Provider compatibility:

• OpenAI: supports all except top_k
• Local models (vLLM, Ollama): typically support all parameters
• Custom providers: parameter support varies by implementation

LLM

Represents a connection to a Large Language Model service for text generation.

LLMs provide generative AI capabilities for chat completion, text completion, and function calling by interfacing with both hosted services (OpenAI, API gateways) and self-hosted models (vLLM, Ollama). Each configuration includes connection details, model parameters, generation capabilities, and access credentials.

Security:

• credentials is INPUT_ONLY and is omitted from all responses.

Immutability:

• provider_type is IMMUTABLE after creation.
• owner_id is set at creation and cannot be modified.

Notes:

• All timestamps are UTC (google.protobuf.Timestamp).
• Complex configuration objects (capabilities, default_sampling_params, client_config) are stored as JSONB.

See also: LLMProviderType, LLMCapabilities, LLMSamplingParams, goodmem.v1.Modality

LLM.LabelsEntry

CreateLLMRequest

CreateLLMRequest.LabelsEntry

CreateLLMResponse

Response message for the CreateLLM RPC.

Contains the newly created LLM configuration and any informational statuses generated during the creation process, such as results from capability inference.

GetLLMRequest

ListLLMsRequest

ListLLMsRequest.LabelSelectorsEntry

ListLLMsResponse

Response message for the ListLLMs RPC.

Contains a list of LLM configurations that match the request filters and are accessible by the authenticated user.

UpdateLLMRequest

DeleteLLMRequest

Enums

LLMProviderType

LLM provider type for text generation services.

Organized by deployment model with distinct value ranges:

• 0: Invalid type
• 1-99: Reserved for future general providers
• 10-19: Cloud/SaaS providers and API gateways
• 100-199: Self-hosted and local deployment solutions
• 999: Generic compatibility fallback