How to apply Hasura JSON metadata, which was generated from Hasura console, to Hasura docker container?

Question

I plan to deploy Hasura Docker container into Kubernetes cluster.
So I got a Hasura metadata JSON file (hasura_metadata_2023_02_23_16_59_45_407.json), which was generated form one Hasura via its setting in web console.

My question how can I use this metadata JSON file to automatically pre-configure a Hasura container (which will be created inside a Pod in Kubernetes) each time the container is created ?

The configuration would be something like : connect to multiple databases, set role permissions, set event-trigger ... etc. I want all this configurations to be all finished when the Hasura container is created.

Answer 1

You can use the hasura/graphql-engine:<version>.cli-migrations-v3 docker image which will automatically apply metadata and migrations on build.

From the Hasura docs

The migrations and metadata directories created by the Hasura CLI in a Hasura Project can be mounted at the /hasura-migrations and /hasura-metadata paths of this Docker container and the container's entrypoint script will automatically apply the Migrations and Metadata before starting the server. If no directory is mounted at the designated paths, the server will start and ignore the Migrations and/or Metadata.

Answer 2

My folder structure has all these file generated from Hasura-cli hasura init ... :
metadata, migrations, config.yaml, seeds

And here's command I use to create Hasura container:

docker run -d -p 8080:8080 \ -v metadata:/hasura-metadata \ -v migrations:/hasura-migrations \ -e HASURA_GRAPHQL_ENABLE_CONSOLE=true \ -e HASURA_GRAPHQL_DATABASE_URL=postgres://postgres:[email protected]:5432/postgres \ -e HASURA_GRAPHQL_ADMIN_SECRET=wQuhf5g9T3 \ hasura/graphql-engine:v2.18.0.cli-migrations-v3

Here's the error I saw from the Hasura container on Docker Desktop:

2023-02-24 14:16:09 {"timestamp":"2023-02-24T07:16:09.000+0000","level":"info","type":"startup","detail":{"kind":"migrations-startup","info":"migrations server port env var is not set, defaulting to 9691"}}
2023-02-24 14:16:09 {"timestamp":"2023-02-24T07:16:09.000+0000","level":"info","type":"startup","detail":{"kind":"migrations-startup","info":"server timeout is not set, defaulting to 30 seconds"}}
2023-02-24 14:16:09 {"timestamp":"2023-02-24T07:16:09.000+0000","level":"info","type":"startup","detail":{"kind":"migrations-startup","info":"starting graphql engine temporarily on port 9691"}}
2023-02-24 14:16:09 {"timestamp":"2023-02-24T07:16:09.000+0000","level":"info","type":"startup","detail":{"kind":"migrations-startup","info":"waiting 30 for 9691 to be ready"}}
2023-02-24 14:16:09 {"detail":{"info":{"admin_secret_set":true,"auth_hook":null,"auth_hook_mode":null,"console_assets_dir":null,"console_sentry_dsn":null,"cors_config":{"allowed_origins":"*","disabled":false,"ws_read_cookie":null},"enable_allowlist":false,"enable_console":true,"enable_maintenance_mode":false,"enable_metadata_query_logging":false,"enable_telemetry":true,"enabled_apis":["metadata"],"enabled_log_types":["startup","webhook-log","http-log","jwk-refresh-log","websocket-log"],"events_fetch_batch_size":100,"experimental_features":[],"graceful_shutdown_timeout":60,"infer_function_permissions":true,"jwt_secret":[],"live_query_options":{"batch_size":100,"refetch_delay":1},"log_level":"info","port":9691,"remote_schema_permissions":false,"server_host":"HostAny","stringify_numeric_types":false,"transaction_isolation":"ISOLATION LEVEL READ COMMITTED","unauth_role":null,"use_prepared_statements":true,"v1-boolean-null-collapse":false,"websocket_compression_options":"NoCompression","websocket_connection_init_timeout":"Refined (Seconds {seconds = 3s})","websocket_keep_alive":"KeepAliveDelay {unKeepAliveDelay = Refined (Seconds {seconds = 5s})}"},"kind":"server_configuration"},"level":"info","timestamp":"2023-02-24T07:16:09.811+0000","type":"startup"}
2023-02-24 14:16:10 {"detail":{"info":{"database_url":"postgres://postgres:[email protected]:5432/postgres","retries":1},"kind":"postgres_connection"},"level":"info","timestamp":"2023-02-24T07:16:09.811+0000","type":"startup"}
2023-02-24 14:16:11 {"detail":{"info":{"message":"source \"motor_db\" has not been initialized yet.","source":"motor_db"},"kind":"source_catalog_migrate"},"level":"info","timestamp":"2023-02-24T07:16:11.511+0000","type":"startup"}
2023-02-24 14:16:12 {"detail":{"info":{"message":"source \"health_db\" has not been initialized yet.","source":"health_db"},"kind":"source_catalog_migrate"},"level":"info","timestamp":"2023-02-24T07:16:11.511+0000","type":"startup"}
2023-02-24 14:16:15 {"detail":{"info":{"message":"source \"user_db\" has not been initialized yet.","source":"user_db"},"kind":"source_catalog_migrate"},"level":"info","timestamp":"2023-02-24T07:16:15.691+0000","type":"startup"}
2023-02-24 14:16:16 {"detail":"Thread sourcePingPoller (re)started","level":"info","timestamp":"2023-02-24T07:16:15.691+0000","type":"unstructured"}
2023-02-24 14:16:16 {"detail":{"info":"Already at the latest catalog version (47); nothing to do.","kind":"catalog_migrate"},"level":"info","timestamp":"2023-02-24T07:16:15.691+0000","type":"startup"}
2023-02-24 14:16:16 {"detail":{"info":"Schema sync enabled. Polling at Refined (Milliseconds {milliseconds = 1s})","kind":"schema-sync"},"level":"info","timestamp":"2023-02-24T07:16:15.691+0000","type":"startup"}
2023-02-24 14:16:16 {"detail":{"info":{"instance_id":"622c5f64-7425-443a-96a9-ce79d9aacd56","message":"listener thread started","thread_id":"ThreadId 50"},"kind":"schema-sync"},"level":"info","timestamp":"2023-02-24T07:16:15.691+0000","type":"startup"}
2023-02-24 14:16:16 {"detail":{"info":{"instance_id":"622c5f64-7425-443a-96a9-ce79d9aacd56","message":"processor thread started","thread_id":"ThreadId 56"},"kind":"schema-sync"},"level":"info","timestamp":"2023-02-24T07:16:15.691+0000","type":"startup"}
2023-02-24 14:16:16 {"detail":{"info":"starting workers","kind":"event_triggers"},"level":"info","timestamp":"2023-02-24T07:16:15.691+0000","type":"startup"}
2023-02-24 14:16:16 {"detail":{"info":"preparing data","kind":"scheduled_triggers"},"level":"info","timestamp":"2023-02-24T07:16:15.691+0000","type":"startup"}
2023-02-24 14:16:16 {"detail":{"info":"Help us improve Hasura! The graphql-engine server collects anonymized usage stats which allows us to keep improving Hasura at warp speed. To read more or opt-out, visit https://hasura.io/docs/latest/graphql/core/guides/telemetry.html","kind":"telemetry"},"level":"info","timestamp":"2023-02-24T07:16:15.691+0000","type":"startup"}
2023-02-24 14:16:16 {"detail":"Thread SchemeUpdate.listener (re)started","level":"info","timestamp":"2023-02-24T07:16:15.691+0000","type":"unstructured"}
2023-02-24 14:16:16 {"detail":"Thread update JWK (re)started","level":"info","timestamp":"2023-02-24T07:16:15.691+0000","type":"unstructured"}
2023-02-24 14:16:16 {"detail":"Thread ourIdleGC (re)started","level":"info","timestamp":"2023-02-24T07:16:15.691+0000","type":"unstructured"}
2023-02-24 14:16:16 {"detail":"Thread SchemeUpdate.processor (re)started","level":"info","timestamp":"2023-02-24T07:16:15.691+0000","type":"unstructured"}
2023-02-24 14:16:16 {"detail":"Thread processEventQueue (re)started","level":"info","timestamp":"2023-02-24T07:16:15.691+0000","type":"unstructured"}
2023-02-24 14:16:16 {"detail":"Thread asyncActionsProcessor (re)started","level":"info","timestamp":"2023-02-24T07:16:15.691+0000","type":"unstructured"}
2023-02-24 14:16:16 {"detail":"Thread asyncActionSubscriptionsProcessor (re)started","level":"info","timestamp":"2023-02-24T07:16:15.691+0000","type":"unstructured"}
2023-02-24 14:16:16 {"detail":"Thread runCronEventsGenerator (re)started","level":"info","timestamp":"2023-02-24T07:16:15.691+0000","type":"unstructured"}
2023-02-24 14:16:16 {"detail":{"info":"Received metadata resource version MetadataResourceVersion {getMetadataResourceVersion = 57} as an initial version. Not updating the schema cache.","thread_type":"processor"},"level":"info","timestamp":"2023-02-24T07:16:15.691+0000","type":"schema-sync"}
2023-02-24 14:16:16 {"detail":"Thread processScheduledTriggers (re)started","level":"info","timestamp":"2023-02-24T07:16:15.691+0000","type":"unstructured"}
2023-02-24 14:16:16 {"detail":"Thread checkForUpdates (re)started","level":"info","timestamp":"2023-02-24T07:16:15.691+0000","type":"unstructured"}
2023-02-24 14:16:17 {"timestamp":"2023-02-24T07:16:17.000+0000","level":"info","type":"startup","detail":{"kind":"migrations-startup","info":"port 9691 is ready"}}
2023-02-24 14:16:17 {"timestamp":"2023-02-24T07:16:17.000+0000","level":"info","type":"startup","detail":{"kind":"migrations-startup","info":"env var HASURA_GRAPHQL_MIGRATIONS_DIR is not set, defaulting to /hasura-migrations"}}
2023-02-24 14:16:17 {"timestamp":"2023-02-24T07:16:17.000+0000","level":"info","type":"startup","detail":{"kind":"migrations-startup","info":"env var HASURA_GRAPHQL_METADATA_DIR is not set, defaulting to /hasura-metadata"}}
2023-02-24 14:16:17 {"timestamp":"2023-02-24T07:16:17.000+0000","level":"info","type":"startup","detail":{"kind":"migrations-apply","info":"applying metadata from /hasura-metadata"}}
2023-02-24 14:16:17 {"detail":{"info":{"message":"starting API server","time_taken":7.123395446},"kind":"server"},"level":"info","timestamp":"2023-02-24T07:16:16.934+0000","type":"startup"}
2023-02-24 14:16:17 {"detail":"Thread runTelemetry (re)started","level":"info","timestamp":"2023-02-24T07:16:16.934+0000","type":"unstructured"}
2023-02-24 14:16:18 {"detail":{"http_info":{"content_encoding":null,"http_version":"HTTP/1.1","ip":"127.0.0.1","method":"GET","status":200,"url":"/v1/version"},"operation":{"request_id":"329e86da-6353-421c-b465-d7305ce1bc7b","request_mode":"non-graphql","response_size":16,"uncompressed_response_size":16},"request_id":"329e86da-6353-421c-b465-d7305ce1bc7b"},"level":"info","timestamp":"2023-02-24T07:16:17.936+0000","type":"http-log"}
2023-02-24 14:16:18 {"detail":{"http_info":{"content_encoding":"gzip","http_version":"HTTP/1.1","ip":"127.0.0.1","method":"POST","status":200,"url":"/v1/query"},"operation":{"query_execution_time":0.245242231,"request_id":"0688a267-1f53-45d4-a4e3-9dd625699188","request_mode":"non-graphql","request_read_time":6.78e-7,"response_size":3067,"uncompressed_response_size":27436,"user_vars":{"x-hasura-role":"admin"}},"request_id":"0688a267-1f53-45d4-a4e3-9dd625699188"},"level":"info","timestamp":"2023-02-24T07:16:17.936+0000","type":"http-log"}
2023-02-24 14:16:18 {"detail":{"http_info":{"content_encoding":null,"http_version":"HTTP/1.1","ip":"127.0.0.1","method":"GET","status":200,"url":"/v1/version"},"operation":{"request_id":"d76729a0-6e8c-41ae-b031-a3b3fa1021a6","request_mode":"non-graphql","response_size":16,"uncompressed_response_size":16},"request_id":"d76729a0-6e8c-41ae-b031-a3b3fa1021a6"},"level":"info","timestamp":"2023-02-24T07:16:17.936+0000","type":"http-log"}
2023-02-24 14:16:18 {"detail":{"http_info":{"content_encoding":null,"http_version":"HTTP/1.1","ip":"127.0.0.1","method":"GET","status":404,"url":"/v1alpha1/config"},"operation":{"error":{"code":"not-found","error":"resource does not exist","path":"$"},"request_id":"9b6971c7-1bb6-499e-bd4e-715bf9957848","request_mode":"error","response_size":65,"uncompressed_response_size":65},"request_id":"9b6971c7-1bb6-499e-bd4e-715bf9957848"},"level":"error","timestamp":"2023-02-24T07:16:17.936+0000","type":"http-log"}
2023-02-24 14:16:19 {"detail":{"http_info":{"content_encoding":null,"http_version":"HTTP/1.1","ip":"127.0.0.1","method":"POST","status":200,"url":"/v1/metadata"},"operation":{"query_execution_time":0.35049015,"request_id":"c74c8780-61b2-44a3-a42c-cc4926e312b5","request_mode":"non-graphql","request_read_time":1.603e-6,"response_size":438,"uncompressed_response_size":438,"user_vars":{"x-hasura-role":"admin"}},"request_id":"c74c8780-61b2-44a3-a42c-cc4926e312b5"},"level":"info","timestamp":"2023-02-24T07:16:18.965+0000","type":"http-log"}
2023-02-24 14:16:19 {"detail":{"http_info":{"content_encoding":null,"http_version":"HTTP/1.1","ip":"127.0.0.1","method":"POST","status":400,"url":"/v1/metadata"},"operation":{"error":{"code":"parse-failed","error":"key \"tables\" not found","path":"$.args.metadata"},"request_id":"bfd71f4c-38bf-4a47-ba12-60b8bf272f1a","request_mode":"error","response_size":83,"uncompressed_response_size":83,"user_vars":{"x-hasura-role":"admin"}},"request_id":"bfd71f4c-38bf-4a47-ba12-60b8bf272f1a"},"level":"error","timestamp":"2023-02-24T07:16:18.965+0000","type":"http-log"}
2023-02-24 14:16:18 {"level":"info","msg":"Help us improve Hasura! The cli collects anonymized usage stats which\nallow us to keep improving Hasura at warp speed. To opt-out or read more,\nvisit https://hasura.io/docs/latest/graphql/core/guides/telemetry.html\n","time":"2023-02-24T07:16:18Z"}
2023-02-24 14:16:18 {"level":"info","msg":"Applying metadata...","time":"2023-02-24T07:16:18Z"}
2023-02-24 14:16:20 time="2023-02-24T07:16:20Z" level=fatal msg="error applying metadata \n{\n  \"code\": \"parse-failed\",\n  \"error\": \"key \\\"tables\\\" not found\",\n  \"path\": \"$.args.metadata\"\n}"

Answer 3

Apparently it's not supported yet, here is a ticket about it https://github.com/hasura/graphql-engine/issues/8423

As an alternative, what you can currently do is to use the yaml metadata files generated by hasura init . to automatically pre-configure a Hasura container.

• doc on how to hasura init
• doc on Hasura metadata
• doc on applying migrations and metadata with docker