Microsoft NLWeb

Overview

GoodMem is a memory layer for AI agents that handles embedding, vector search, and reranking server-side. This integration serves NLWeb retrieval from a GoodMem space: GoodMemRetrievalProvider implements nlweb_core.retriever.RetrievalProvider, so search() returns real RetrievedItem objects, and GoodMemObjectLookupProvider implements ObjectLookupProvider for full-object lookup by URL. Both are registered purely by configuration — NLWeb imports the class by path from YAML, so nothing is added to NLWeb's own source tree.

The integration targets nlweb-core, the pip-installable package with the config-driven provider architecture. The nlweb-ai/NLWeb reference implementation has a different interface and a hardcoded provider table, so a third-party package cannot register with it.

Installation

pip install goodmem-nlweb

The pip package is goodmem-nlweb; the import name is goodmem_nlweb. It depends on nlweb-core and the goodmem Python SDK, and a running GoodMem server is required.

Configuration

NLWeb's ask handler asks for the retrieval provider named default, so register the provider under that name:

retrieval:
  default:
    import_path: goodmem_nlweb
    class_name: GoodMemRetrievalProvider
    base_url: https://your-goodmem-server.example.com
    api_key: gm_xxxxxxxxxxxxxxxxxxxxxxxx
    space_name: nlweb

Every key beside import_path and class_name is passed to the constructor as a keyword argument. NLWeb reads a key ending in _env from the environment instead, so api_key_env: GOODMEM_API_KEY keeps the key out of the file.

space_id, embedder_id, and reranker_id must be UUIDs; any other value raises ValueError when the provider is built, before a request is made. In code you can also pass client= to inject your own AsyncGoodmem client, which the provider borrows but never closes (GoodMemConnection handles this ownership).

Searching by site

NLWeb filters every call by site; GoodMem has no such concept, so a site is stored as memory metadata and filtered server-side with GoodMem's filter grammar. Values are escaped, never interpolated.

await provider.search("noodle soup", "recipes.example.com")   # one site
await provider.search("noodle soup", ["a.com", "b.com"])      # OR across sites
await provider.search("noodle soup", "all")                   # every site
await provider.search_all_sites("noodle soup")                # same thing
await provider.get_sites()                                    # ['a.com', 'b.com']

Results keep the server's ordering — a GoodMem vector score is a negative inner product and a reranker score is a different scale, so neither is exposed as an NLWeb relevance number. When the server reports a problem, whatever it did return is still returned and the statuses are logged; a degraded retrieval that returned nothing yields an empty list plus a UserWarning, never an exception, so an ask request can still answer from another endpoint.

Metadata filters

search() also takes metadata_filter, a mapping of metadata field to value. Each pair becomes a server-side equality, AND-ed with the others and with the site filter. GoodMem compares metadata under an explicit cast, so each value is cast by its Python type:

None and any other type (a list, a dict, …) raise ValueError before a request is made.

await provider.search("noodle soup", "recipes.example.com",
                      metadata_filter={"vegetarian": True, "servings": 4})

metadata_filter is an argument to search(), not a provider option: a metadata_filter: key in the provider's YAML entry is ignored.

Ingesting Schema.org documents

RetrievalProvider only reads, so ingestion is a helper rather than part of the interface:

from goodmem_nlweb import GoodMemRetrievalProvider, upload_documents

provider = GoodMemRetrievalProvider(
    space_name="nlweb",
    base_url="https://your-goodmem-server.example.com",
    api_key="gm_xxxxxxxxxxxxxxxxxxxxxxxx",
)
await upload_documents(provider, [
    {"@type": "Recipe", "url": "https://ex.com/r/laksa", "name": "Singapore Laksa",
     "description": "A coconut curry noodle soup."},
], site="recipes.example.com")

Each document maps onto a GoodMem memory:

The embedded text is not the raw JSON: it is extracted from name, headline, description, articleBody, and similar fields, while the untouched object is kept alongside for NLWeb to return verbatim. The mapping functions schema_to_text and to_memory_fields are exported if you need them directly.

upload_documents waits for indexing by default (wait=True, timeout=180.0) and returns the created memory ids. It inspects every item of the batch response — the server returns HTTP 200 even when an item failed — and a partial failure raises GoodMemUploadError, which carries the ids that did land in created_memory_ids.

Looking objects up by URL

NLWeb enriches a truncated search result with the complete Schema.org object when an object_storage provider named default is configured:

object_storage:
  default:
    import_path: goodmem_nlweb
    class_name: GoodMemObjectLookupProvider
    base_url: https://your-goodmem-server.example.com
    api_key: gm_xxxxxxxxxxxxxxxxxxxxxxxx
    space_name: nlweb

from goodmem_nlweb import GoodMemObjectLookupProvider

lookup = GoodMemObjectLookupProvider(
    space_name="nlweb",
    base_url="https://your-goodmem-server.example.com",
    api_key="gm_xxxxxxxxxxxxxxxxxxxxxxxx",
)
await lookup.get_by_id("https://ex.com/r/laksa")   # the full Schema.org object

The id NLWeb passes here is the item's URL, not a GoodMem id, so it is not required to be a UUID.

Learn more

The full README, examples, and API reference are available on the PyPI package page.