Data loss when using Fiware Orion Broker, QuantumLeap and CrateDB

163 Views Asked by At

I'm using Fiware Orion Broker, QuantumLeap and CrateDB, with the aim of recording all temporal data in cratedb.

My docker-compose configuration is this:

orion:
    image: fiware/orion:${ORION_VERSION}
    hostname: orion
    container_name: fiware-orion
    depends_on:
        - mongo-db
    networks:
        - fiware
    expose:
        - "${ORION_PORT}"
    ports:
        - "${ORION_PORT}:${ORION_PORT}"
    command: -dbhost mongo-db
    healthcheck:
        test: curl --fail -s http://orion:${ORION_PORT}/version || exit 1
        interval: 5s

mongo-db:
    image: mongo:latest
    hostname: mongo-db
    container_name: db-mongo
    expose:
        - "${MONGO_DB_PORT}"
    ports:
        - "${MONGO_DB_PORT}:${MONGO_DB_PORT}"
    networks:
        - fiware
    volumes:
        -  ./volumes/mongo-db:/data
    healthcheck:
        test: |
            host=`hostname --ip-address || echo '127.0.0.1'`; 
            mongo --quiet $host/test --eval 'quit(db.runCommand({ ping: 1 }).ok ? 0 : 2)' && echo 0 || echo 1
        interval: 5s

quantumleap:
    image: orchestracities/quantumleap:latest
    hostname: quantumleap
    container_name: fiware-quantumleap
    ports:
        - "${QUANTUMLEAP_PORT}:${QUANTUMLEAP_PORT}"
    depends_on:
        - crate-db
        - redis-db
    environment:
        - CRATE_HOST=crate-db
        - LOGLEVEL=WARNING
    healthcheck:
        test: curl --fail -s http://quantumleap:${QUANTUMLEAP_PORT}/version || exit 1
    networks:
        - fiware

crate-db:
    image: crate:latest
    hostname: crate-db
    container_name: db-crate
    ports:
        - "4200:4200"
        - "4300:4300"
    command: crate -Cauth.host_based.enabled=false  -Ccluster.name=democluster -Chttp.cors.enabled=true -Chttp.cors.allow-origin="*" -Cdiscovery.type=single-node
    environment:
        - CRATE_HEAP_SIZE=2g 
    volumes:
        - ./volumes/crate-db:/data
    networks:
        - fiware

I'm running performance tests through Apache JMeter, and consecutive requests are sent for 1 minutes to evaluate their performance, but I'm having a problem where some data is not being registered in the CrateDB, that is, in the last test I did, it was done about 18000 requests, and in CrateDB only about 10000 are registered.

I also tried using the TimescaleDB database in QuantumLeap, but the same problem happens, so I assume that the problem is not with the database.

Does anyone know what the problem could be?

0

There are 0 best solutions below