🤖 Self-hosted, community-driven, local OpenAI-compatible API with Keycloak Auth Flak app as frontend. 🏠

History

mudler 3a90ea44a5 Update readme and examples		2 years ago
..
data	example(add): document query example	2 years ago
models	example(add): document query example	2 years ago
.gitignore	example(add): document query example	2 years ago
README.md	example(add): document query example	2 years ago
docker-compose.yml	example(add): document query example	2 years ago
query.py	Update readme and examples	2 years ago
store.py	Update readme and examples	2 years ago

README.md

Data query example

This example makes use of Llama-Index to enable question answering on a set of documents.

It loosely follows the quickstart.

Requirements

For this in order to work, you will need a model compatible with the llama.cpp backend. This is will not work with gpt4all.

The example uses WizardLM. Edit the config files in models/ accordingly to specify the model you use (change HERE).

You will also need a training data set. Copy that over data.

Setup

Start the API:

# Clone LocalAI
git clone https://github.com/go-skynet/LocalAI

cd LocalAI/examples/query_data

# Copy your models, edit config files accordingly

# start with docker-compose
docker-compose up -d --build

Create a storage:

export OPENAI_API_BASE=http://localhost:8080/v1
export OPENAI_API_KEY=sk-

python store.py

After it finishes, a directory "storage" will be created with the vector index database.

Query

export OPENAI_API_BASE=http://localhost:8080/v1
export OPENAI_API_KEY=sk-

python query.py