You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
gregandev
77800c1636
|
1 year ago | |
---|---|---|
.github | 1 year ago | |
.vscode | 2 years ago | |
api | 1 year ago | |
cmd/grpc | 1 year ago | |
examples | 1 year ago | |
extra | 1 year ago | |
internal | 1 year ago | |
models | 2 years ago | |
pkg | 1 year ago | |
prompt-templates | 2 years ago | |
tests | 1 year ago | |
.dockerignore | 1 year ago | |
.env | 1 year ago | |
.gitignore | 1 year ago | |
Dockerfile | 1 year ago | |
Earthfile | 2 years ago | |
LICENSE | 2 years ago | |
Makefile | 1 year ago | |
README.md | 1 year ago | |
assets.go | 2 years ago | |
docker-compose.yaml | 2 years ago | |
entrypoint.sh | 1 year ago | |
go.mod | 1 year ago | |
go.sum | 1 year ago | |
main.go | 1 year ago | |
renovate.json | 2 years ago |
README.md
LOCAL AI
USAGE
- Installation et démarrage:
# Clone LocalAI
git clone https://github.com/go-skynet/LocalAI
cd LocalAI
# (optional) Checkout a specific LocalAI tag
# git checkout -b build <TAG>
# Download gpt4all-j to models/
wget https://gpt4all.io/models/ggml-gpt4all-j.bin -O models/ggml-gpt4all-j
# Use a template from the examples
cp -rf prompt-templates/ggml-gpt4all-j.tmpl models/
# (optional) Edit the .env file to set things like context size and threads
# vim .env
# start with docker-compose
# docker-compose up -d --pull always
# or you can build the images with:
docker-compose up -d --build
# Now API is accessible at localhost:8080
curl http://localhost:8080/v1/models
# {"object":"list","data":[{"id":"ggml-gpt4all-j","object":"model"}]}
curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
"model": "ggml-gpt4all-j",
"messages": [{"role": "user", "content": "How are you?"}],
"temperature": 0.9
}'
# {"model":"ggml-gpt4all-j","choices":[{"message":{"role":"assistant","content":"I'm doing well, thanks. How about you?"}}]}
- Python implementation:
import openai
openai.api_base = "http://localhost:8080/v1"
# create a chat completion
chat_completion = openai.ChatCompletion.create(model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Hello world"}])
# print the completion
print(completion.choices[0].message.content)
TO DO
- Flask app frontend
- Keycloak auth
- speech to text avec openVINO