83 lines
4.2 KiB
Markdown
83 lines
4.2 KiB
Markdown
# Overview
|
|
|
|
## Installing the dependencies
|
|
## Ubuntu 24.04
|
|
1. Install the .NET SDK: `sudo apt update && sudo apt install dotnet-sdk-8.0 -y`
|
|
2. Install the python SDK: `sudo apt install python3 python3.12 python3.12-dev`
|
|
## Windows
|
|
Download the [.NET SDK](https://dotnet.microsoft.com/en-us/download) or follow these steps to use WSL:
|
|
1. Install Ubuntu in WSL (`wsl --install` and `wsl --install -d Ubuntu`)
|
|
2. Enter your WSL environment `wsl.exe` and configure it
|
|
3. Update via `sudo apt update && sudo apt upgrade -y && sudo snap refresh`
|
|
4. Continue here: [Ubuntu 24.04](#Ubuntu-24.04)
|
|
# Configuration
|
|
The configuration is located in `src/Indexer` and conforms to the [ASP.NET configuration design pattern](https://learn.microsoft.com/en-us/aspnet/core/fundamentals/configuration/?view=aspnetcore-9.0), i.e. `src/Indexer/appsettings.json` is the base configuration, and `/src/Indexer/appsettings.Development.json` overrides it.
|
|
|
|
If you plan to use multiple environments, create any `appsettings.{YourEnvironment}.json` (e.g. `Development`, `Staging`, `Prod`) and set the environment variable `DOTNET_ENVIRONMENT` accordingly on the target machine.
|
|
## Setup
|
|
If you just installed the server and want to configure it:
|
|
1. Open `src/server/appsettings.Development.json`
|
|
2. If your search server is not on the same machine as the indexer, update "BaseUri" to reflect the URL to the server.
|
|
3. If your search server requires API keys, (i.e. it's operating outside of the "Development" environment) set `"ApiKey": "<your key here>"` beneath `"BaseUri"` in the `"Embeddingsearch"` section.
|
|
4. Create your own indexing script(s) in `src/Indexer/Scripts/` and configure their use as
|
|
## Structure
|
|
```json
|
|
"EmbeddingsearchIndexer": {
|
|
"Worker":
|
|
[ // This is a list; you can have as many "workers" as you want
|
|
{
|
|
"Name": "example",
|
|
"Searchdomains": [
|
|
"example"
|
|
],
|
|
"Script": "Scripts/example.py",
|
|
"Calls": [ // This is also a list. You can have as many different calls as you need.
|
|
{
|
|
"Type": "interval", // See: Call types
|
|
"Interval": 60000
|
|
}
|
|
]
|
|
}
|
|
]
|
|
}
|
|
```
|
|
## Call types
|
|
- `interval`
|
|
- What does it do: The script gets called periodically based on the specified `Interval` parameter.
|
|
- Parameters:
|
|
- Interval (in milliseconds)
|
|
- `schedule` (WIP)
|
|
- What does it do: The script gets called based on the provided schedule
|
|
- Parameters: (WIP)
|
|
- `fileupdate` (WIP)
|
|
- What does it do: The script gets called whenever a file is updated in the specified subdirectory
|
|
- Parameters: (WIP)
|
|
# Scripting
|
|
## Python
|
|
To ease scripting, tools.py contains all definitions of the .NET objects passed to the script. This includes attributes and methods.
|
|
|
|
These are not yet defined in a way that makes them 100% interactible with the Dotnet CLR, meaning some methods that require anything more than strings or other simple data types to be passed are not yet supported. (WIP)
|
|
### Required elements
|
|
Here is an overview of required elements by example:
|
|
```python
|
|
from tools import * # Import all tools that are provided for ease of scripting
|
|
|
|
def init(toolset: Toolset): # defining an init() function with 1 parameter is required.
|
|
pass # Your code would go here.
|
|
# DO NOT put a main loop here! Why?
|
|
# This function prevents the application from initializing and maintains exclusive control over the GIL
|
|
|
|
def update(toolset: Toolset): # defining an update() function with 1 parameter is required.
|
|
pass # Your code would go here.
|
|
```
|
|
### Using the toolset passed by the .NET CLR
|
|
The use of the toolset is laid out in good example by `src/Indexer/Scripts/example.py`.
|
|
|
|
Currently, `Toolset`, as provided by the IndexerService to the Python script, contains 3 elements:
|
|
1. (only for `update`, not `init`) `callbackInfos` - an object that provides all information regarding the callback. (e.g. what file was updated)
|
|
2. `client` - a .NET object that has the functions as described in `src/Indexer/Scripts/tools.py`. It's the client that - according to the configuration - communicates with the search server and executes the API calls.
|
|
3. `filePath` - the path to the script, as specified in the configuration
|
|
## Golang
|
|
TODO
|
|
## Javascript
|
|
TODO |