Deploying haystack model/workflow

Question

I'm trying to deploy a haystack model for Question Answering for my application as a REST API /API. I want to query and get my answers directly and I need to do it soon so I'm finding a way to do it on Algorithmia. Any suggestions, tutorials, examples or any help is appreciated. Thanks!!

For reference, this could be an example model.

Answer 1

Not sure about Alorithmia, but here's a simple option to deploy a Haystack service incl. a REST API on any standard machine (e.g. AWS EC2 instance):

# Clone haystack repo
git clone https://github.com/deepset-ai/haystack.git
cd haystack
# Start (demo) containers
docker-compose pull
docker-compose up
# Run a query
curl -X 'POST' \
  'http://127.0.0.1:8000/query' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "query": "Who is the father of Arya Stark?",
  "params": {}
  }'

This basically spins up:

1. Haystack REST API using this docker image
2. Elasticsearch with some demo data (see comment in the docker-compose.yaml for how to replace this with an empty instance for your own data)
3. A simple streamlit-based UI (you can easily remove this from the docker-compose if you don't need it)

---

If you want to customize the pipeline being deployed in the API (e.g. change a model):

• Edit the pipelines.yaml in your cloned repo (in haystack/rest_api/pipeline/)
• Mount this folder as a volume into the container by uncommenting this part in the docker-compose.yaml

If you want to deploy on a GPU machine, just execute instead:

docker-compose -f docker-compose-gpu.yml pull
docker-compose -f docker-compose-gpu.yml up

For more details, see the official documentation of the REST API here.