This is our rough setup(application, config-server, vault, git), this is what we want to achieve (proxy vault/git through config-server) and this is our problem (we can only get requests one of the config-server backends and not both at the same time)
We used token (X-Config-Token) based Authentication to fetch config from Git and Vault when ever requesting the configurations through config server.
We have tried with following configuration setup in Config server.
application.yml
spring:
profiles:
active: vault , git
cloud:
config:
server:
accept-empty: false
git:
order: 2
uri: https://gitlab/project/config-sample-nonprod/{profile}}
deleteUntrackedBranches: true
tryMasterBranch: false
vault:
order: 1
host: vault-dev-project.com
backend: kv/project/sample-platform-sandbox/
default-key: sample-collector/mmg-sample
port: 443
scheme: https
authentication: TOKEN
kv-version: 2
token: xxxxxewdsdjbcxxxxxxxx
failOnCompositeError: false
We are testing our API requests using CURL to see the configuration of client application coming through config server from different backend as below:
Git as backend
Applications should get its configuration from the config server using the following URL pattern:
https://<config-service>/<name>/<repo>/<branch or tag>
Example as below:
curl -X GET https://sample-config-service-dev/sample-collector/mmg-sample/develop -H X-Config-Token:0cd3253
We are getting the configuration as response from our Gitlab
Vault as backend
curl -X GET https://config-service-dev.com/sample-collector/mmg-sample,dev -H X-Config-Token:0cd3253
Response:
{"name":"sample-collector","profiles":["mmg-sample,dev"],"label":null,"version":null,"state":null,"propertySources":[{"name":"vault:sample-collector/mmg-sample,dev","source":{"env":"development","project":"sample","user":"test1"}}]}
When we send the request to Vault as config-service-dev.com/sample-collector/mmg-sample,dev to get the secrets related to development as above, we are getting the desired results.
But when we are trying to fetch the results for below request from Vault,
curl -X GET https://config-service-dev.com/sample-collector/mmg-sample/develop -H X-Config-Token:0cd3253
It throws me an error as below
{"timestamp":"2024-02-04T15:37:24","status":404,"error":"Not Found","path":"/sample-collector/mmg-sample/develop"}
Logs from Config server:
{"@timestamp":"2024-02-04T15:37:23.947Z","log.level":"DEBUG","message":"Matching [/actuator/**] against [/sample-collector/mmg-sample/develop]...",,"process.thread.name":"http-nio-8080-exec-4","log.logger":"com.mm.mspconfigservice.SecurityConfiguration","transaction.id":"80fa2d228749ba6f","trace.id":"d63fcb6f26f57454b72a58ec3e81fabc"}
{"@timestamp":"2024-02-04T15:37:24.007Z","log.level": "INFO","message":"URI 'https://sample.com/g/m/config-mcp-nonprod/mmg-sample': following HTTP redirect #0 https://sample.com/g/m/config-mcp-nonprod/mmg-sample/ -> https://sample.com/g/m/config-mcp-nonprod/mmg-sample.git/",,"process.thread.name":"http-nio-8080-exec-4","log.logger":"org.eclipse.jgit.transport.TransportHttp","transaction.id":"80fa2d228749ba6f","trace.id":"d63fcb6f26f57454b72a58ec3e81fabc"}
{"@timestamp":"2024-02-04T15:37:24.145Z","log.level": "WARN","message":"**Error occured cloning to base directory**.",,"process.thread.name":"http-nio-8080-exec-4","log.logger":"org.springframework.cloud.config.server.environment.JGitEnvironmentRepository","transaction.id":"80fa2d228749ba6f","trace.id":"d63fcb6f26f57454b72a58ec3e81fabc","error.type":"org.eclipse.jgit.api.errors.InvalidRemoteException","error.message":"Invalid remote: origin","error.stack_trace":[
"org.eclipse.jgit.api.errors.InvalidRemoteException: Invalid remote: origin",
"Caused by: org.eclipse.jgit.errors.NoRemoteRepositoryException: https://sample.com/g/m/config-mcp-nonprod/mmg-sample: https://sample.com/g/m/config-mcp-nonprod/mmg-sample.git/info/refs?service=git-upload-pack not found: Not Found",
{"@timestamp":"2024-02-04T15:37:24.147Z","log.level": "INFO","message":"**Error adding environment** for org.springframework.cloud.config.server.environment.MultipleJGitEnvironmentRepository@29a4dec2",,"process.thread.name":"http-nio-8080-exec-4","log.logger":"org.springframework.cloud.config.server.environment.SearchPathCompositeEnvironmentRepository","transaction.id":"80fa2d228749ba6f","trace.id":"d63fcb6f26f57454b72a58ec3e81fabc"}
{"@timestamp":"2024-02-04T15:37:24.147Z","log.level": "WARN","message":"Error getting the Environment with name=sample-collector profiles=mmg-sample label=develop includeOrigin=false",,"process.thread.name":"http-nio-8080-exec-4","log.logger":"org.springframework.cloud.config.server.environment.EnvironmentController","transaction.id":"80fa2d228749ba6f","trace.id":"d63fcb6f26f57454b72a58ec3e81fabc","error.type":"org.springframework.cloud.config.server.environment.EnvironmentNotFoundException",**"error.message":"Profile Not found",**"error.stack_trace":[
"org.springframework.cloud.config.server.environment.EnvironmentNotFoundException: Profile Not found",
Even though, we gave in the API request to fetch the details from Vault secrets, it is first redirecting to Git and throwing an error without checking in Vault.