Knowledge and Search

The Semantic Query Engine makes your documents searchable by meaning, not only by keywords. Agents use it to answer questions based on your manuals, procedures or tickets, and cite the sources they used.

This section is for developers who load documents into the service. All requests need a token for the Semantic Query Engine and the headers described in Request basics. Paths are relative to ${SQE_BASE_URL}.

The complete, interactive API reference of the Semantic Query Engine is available at ${SQE_BASE_URL}/docs.

Concepts

Concept	What it is
Datasource	A named collection of documents, for example all product manuals. Access is controlled per datasource.
Document	A text you load, together with its metadata.
Node	A searchable piece of a document. Long documents are split into several nodes.
Metadata	Key-value attributes of a document, such as device model or document type. Used for filtering. Some keys, such as title and url, are used to show sources. See Source metadata.

flowchart LR
    S[Your source system] -->|documents| DS[Datasource]
    DS -->|split and index| N[Nodes]
    A[Agent] -->|search| N
    App[Your application] -->|search| N

Create a datasource

PUT /api/v1/data/datasources/ProductManuals
Content-Type: application/json

{
  "name": "ProductManuals",
  "description": "Product manuals and troubleshooting documentation",
  "do_not_split": false,
  "access_control": { "execute_roles": ["support-team"] }
}

Field	Description
name	Name of the datasource. Letters, digits and dashes, up to 28 characters.
description	What the datasource contains
do_not_split	true stores each document as one node without splitting. See Ingestion best practices.
access_control.execute_roles	Roles that may search the datasource
custom_transforms	Optional transformations applied before indexing

Transformations

Name	What it does
json_to_yaml	Converts JSON documents to YAML before indexing
json_to_markdown	Converts JSON documents to Markdown before indexing

{ "custom_transforms": [{ "name": "json_to_markdown" }] }

Both remove empty values by default. Set "params": {"clean_empty": false} to keep them. Documents that are not valid JSON are indexed unchanged.

Manage datasources

Operation	Request
List datasources	GET /api/v1/data/datasources
Get one datasource	GET /api/v1/data/datasources/{name}
Create or update	PUT /api/v1/data/datasources/{name}
Delete a datasource with all its documents	DELETE /api/v1/data/datasources/{name}

In this section

• Ingesting documents: load, update and delete documents
• Ingestion best practices: IDs, metadata and splitting
• Search and filters: query datasources directly