> ## Documentation Index
> Fetch the complete documentation index at: https://developer.watson-orchestrate.ibm.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Connecting a knowledge base to an external content source (Public Preview)

> [!WARNING]
>
> **Important:**
>
> This feature is currently in public preview. Functionality and behavior may change in future updates.

Use a connector-backed knowledge base to ingest documents from a third-party content source such as Box, SharePoint, or OneDrive. You create a connection for the source credentials, identify source paths in the knowledge base specification, and import the specification with the ADK CLI.

watsonx Orchestrate supports only `key_value` connections. Use `--kind key_value` or `--kind kv` for each source. Other OAuth connection kinds are not validated end-to-end for knowledge base content sources.

A connector-backed knowledge base does not upload document files through the ADK. The knowledge base specification identifies the external system in `content_source` and identifies the source paths in `documents`. During a sync, watsonx Orchestrate retrieves the files, converts and chunks their content, and indexes the chunks in the built-in vector store.

The source documents must meet these requirements:

* Give each file a unique name.
* Include no more than 100 files in one knowledge base specification.
* Limit each `.xlsx` file to 1 MB.
* Limit each `.docx`, `.pdf`, and `.pptx` file to 25 MB.
* Limit each `.csv`, `.html`, and `.txt` file to 5 MB.

## Before you begin

* Register an application or service account with your content provider.
* Collect the required credentials from the provider.
* If you use Developer Edition, start the server with connector support:

```bash BASH theme={null}
orchestrate server start -e <path-to-env-file> --with-ingestion-from-external-source
```

## Required credentials

Each third-party source requires a different set of credentials from its own developer console or app registration. Pass each credential as a separate `--entries "<key>=<value>"` option on `orchestrate connections set-credentials`.

| **content\_source.type** | **Required key/value fields** |
| - | - |
| `box` | `client_id`, `client_secret`, `enterprise_id`, `public_key`, `private_key`, `private_key_password` |
| `sharepoint-file` | `host`, `site`, `tenant_id`, `client_id`, `client_secret`, `refresh_token` |
| `onedrive` | `tenant_id`, `client_id`, `client_secret`, `username`, `password` |

## Connecting to a third-party content source

**1. Create a connection**

Create a connection with an application ID that identifies the external source:

```bash BASH theme={null}
orchestrate connections add -a <app-id>
```

Configure a team connection in the draft environment:

```bash BASH theme={null}
orchestrate connections configure \
  -a <app-id> \
  --kind key_value \
  --type team \
  --env draft
```

Set the credentials. Use the fields for your source from the [required credentials](#required-credentials) table:

```bash BASH theme={null}
orchestrate connections set-credentials \
  -a <app-id> \
  --entries "<field1>=<value1>" \
  --entries "<field2>=<value2>" \
  --env draft
```

To use the knowledge base in the live environment, configure the connection and set its credentials again with `--env live`.

> [!NOTE]
>
> **Note:**
>
> Keep private keys and passwords out of source control. The connection stores these values separately from the knowledge base specification.

**2. Define the knowledge base**

Create a YAML file with `content_source` and `documents`:

```yaml YAML theme={null}
spec_version: v1
kind: knowledge_base
name: <knowledge-base-name>
description: <description>
content_source:
  type: <source-type>
documents:
  - <path/to/file.ext>
  - <path/to/folder/>
```

**content\_source.type** `string` *required*

The connector to use. Accepted values are `box`, `sharepoint-file`, and `onedrive`.

**documents** `array` *required*

List of file and folder paths in the source system. Add a trailing `/` to a path to identify a folder — the sync ingests files directly in that folder but not files in its subfolders.

> [!WARNING]
>
> Do not set `content_source.connection_id` in the specification. The import command resolves the connection ID automatically from the connection that matches `--app-id`.

**3. Import and perform the initial sync**

Import the specification and provide the application ID from the connection:

```bash BASH theme={null}
orchestrate knowledge-bases import \
  -f <path-to-knowledge-base.yaml> \
  -a <app-id>
```

On the first import, watsonx Orchestrate starts a sync and the CLI waits for it to finish.

When you import an existing knowledge base, the command updates the specification without starting a sync. Add `--sync` to update the specification and sync its content:

```bash BASH theme={null}
orchestrate knowledge-bases import \
  -f <path-to-knowledge-base.yaml> \
  -a <app-id> \
  --sync
```

If the sync fails, the knowledge base remains available. Correct the credentials or connector configuration, and then start another sync.

**4. Check status and sync content**

Check the detailed status:

```bash BASH theme={null}
orchestrate knowledge-bases status \
  --name <knowledge-base-name> \
  --verbose
```

The response includes `sync_state`, `sync_state_msg`, and `last_sync_stats` for a connector-backed knowledge base.

| `sync_state` | Meaning |
| - | - |
| `started` | The platform accepted the sync request and started retrieving files. |
| `in_progress` | The platform processes document changes and builds embeddings. |
| `ready_for_promotion` | The platform prepared the new index for promotion. |
| `stable` | No sync is active and the production index is healthy. |
| `failed` | The latest sync failed. Check `sync_state_msg`, correct the problem, and sync again. |

A detailed response can contain these fields:

```json JSON [expandable] theme={null}
{
  "id": "<knowledge-base-id>",
  "name": "<knowledge-base-name>",
  "ready": true,
  "built_in_index_status": "ready",
  "sync_state": "stable",
  "sync_state_msg": "Last sync completed. 1 added (/path/to/file.ext).",
  "last_sync_stats": {
    "added": ["/path/to/file.ext"],
    "deleted": [],
    "modified": [],
    "not_found": []
  },
  "documents": [
    {
      "id": "<document-id>",
      "metadata": {
        "source_uri": "<source-type>:///path/to/file.ext",
        "source_type": "<source-type>"
      }
    }
  ]
}
```

After a sync, use the verbose status output to confirm which files were indexed. If a path does not exist in the source, the import still succeeds but watsonx Orchestrate does not index those files. Correct the paths in the specification and re-import with `--sync`.

Start an on-demand sync after you change files in the source or update the document paths:

```bash BASH theme={null}
orchestrate knowledge-bases sync --name <knowledge-base-name>
```

The command waits for the sync to finish. A knowledge base supports one active sync at a time. During a sync, the knowledge base continues to answer questions with the previous index. A successful sync promotes the new index. A failed sync keeps the previous index available.

## Remove the knowledge base and connection

Remove the knowledge base:

```bash BASH theme={null}
orchestrate knowledge-bases remove --name <knowledge-base-name>
```

This command removes the indexed content from watsonx Orchestrate. It does not remove files from the external source.

Remove the connection when you no longer need it:

```bash BASH theme={null}
orchestrate connections remove -a <app-id>
```

## Examples

**Box**

Box uses a JWT service account. Get the credential values from your Box Developer Console application and its generated key pair.

```bash BASH theme={null}
orchestrate connections add -a box

orchestrate connections configure -a box --kind key_value --type team --env draft

orchestrate connections set-credentials -a box \
  --entries "client_id=<box_client_id>" \
  --entries "client_secret=<box_client_secret>" \
  --entries "enterprise_id=<box_enterprise_id>" \
  --entries "public_key=<box_public_key_id>" \
  --entries "private_key=<box_private_key_pem>" \
  --entries "private_key_password=<box_private_key_passphrase>" \
  --env draft
```

Create the knowledge base specification:

```yaml YAML [expandable] theme={null}
spec_version: v1
kind: knowledge_base
name: <kb-name>
description: <description>
content_source:
  type: box
documents:
  - <path/to/file.ext>
  - <path/to/folder/>
```

Import the specification:

```bash BASH theme={null}
orchestrate knowledge-bases import -f <path/to/knowledge_base.yaml> -a box
```

**SharePoint**

SharePoint uses a Microsoft Entra ID application registration and a delegated OAuth refresh token that you store as a key/value credential.

```bash BASH theme={null}
orchestrate connections add -a sharepoint

orchestrate connections configure -a sharepoint --kind key_value --type team --env draft

orchestrate connections set-credentials -a sharepoint \
  --entries "host=<yourtenant.sharepoint.com>" \
  --entries "site=<site-name>" \
  --entries "tenant_id=<azure_tenant_id>" \
  --entries "client_id=<azure_app_client_id>" \
  --entries "client_secret=<azure_app_client_secret>" \
  --entries "refresh_token=<oauth_refresh_token>" \
  --env draft
```

Create the knowledge base specification:

```yaml YAML [expandable] theme={null}
spec_version: v1
kind: knowledge_base
name: <kb-name>
description: <description>
content_source:
  type: sharepoint-file
documents:
  - <path/to/file.ext>
  - <path/to/folder/>
```

Import the specification:

```bash BASH theme={null}
orchestrate knowledge-bases import -f <path/to/knowledge_base.yaml> -a sharepoint
```

**OneDrive**

OneDrive uses a Microsoft Entra ID application registration with a username and password.

```bash BASH theme={null}
orchestrate connections add -a onedrive

orchestrate connections configure -a onedrive --kind key_value --type team --env draft

orchestrate connections set-credentials -a onedrive \
  --entries "tenant_id=<azure_tenant_id>" \
  --entries "client_id=<azure_app_client_id>" \
  --entries "client_secret=<azure_app_client_secret>" \
  --entries "username=<user@yourdomain.onmicrosoft.com>" \
  --entries "password=<user_password>" \
  --env draft
```

Create the knowledge base specification:

```yaml YAML [expandable] theme={null}
spec_version: v1
kind: knowledge_base
name: <kb-name>
description: <description>
content_source:
  type: onedrive
documents:
  - <path/to/file.ext>
  - <path/to/folder/>
```

Import the specification:

```bash BASH theme={null}
orchestrate knowledge-bases import -f <path/to/knowledge_base.yaml> -a onedrive
```

## Troubleshooting

| Problem | Resolution |
| - | - |
| The connection application ID already exists. | Use the existing connection or choose another application ID. |
| The CLI reports `No connection exists with the app-id`. | Create the connection or correct the value passed to `--app-id`. |
| The CLI reports that sync is unavailable in Developer Edition. | Restart Developer Edition with `--with-ingestion-from-external-source`. |
| `last_sync_stats.not_found` contains paths. | Correct the source paths in `documents`, import the specification again, and add `--sync`. |
| `sync_state` is `failed`. | Review `sync_state_msg`, correct the credentials or connector configuration, and run `orchestrate knowledge-bases sync`. |
| The CLI reports that the knowledge base is not connector-backed. | Use `knowledge-bases sync` only with a knowledge base that defines `content_source`. |