I am making ECS Service withe Terraform. Now I have to route my ECS Service log to S3 my bucket.
I looked it up and they told me to use AWS Firelens. But It dosen't work. Here is my setting.
resource "aws_ecs_task_definition" "xxx-terraform-task-definition" {
family = "xxx-terraform-task"
execution_role_arn = aws_iam_role.ecs_task_execution_role.arn
network_mode = "awsvpc"
requires_compatibilities = ["FARGATE"]
cpu = var.fargate_cpu
memory = var.fargate_memory
container_definitions = jsonencode([
{
name : var.app_name,
image : var.app_image,
cpu : var.fargate_cpu,
memory : var.fargate_memory
networkMode : "awsvpc",
logConfiguration : {
"logDriver" : "awslogs",
"options" : {
"awslogs-group" : var.xxx_logs,
"awslogs-region" : var.aws_region,
"awslogs-stream-prefix" : "ecs"
}
},
portMappings : [
{
"containerPort" : 5001,
"hostPort" : 5001
}
]
},
{
essential : true,
image : "public.ecr.aws/aws-observability/aws-for-fluent-bit:latest",
name : "log_router",
firelensConfiguration : {
"type" : "fluentbit",
"config-file-type" : "file",
"config-file-value" : "./firelens.config"
},
logConfiguration : {
"logDriver" : "awslogs",
"options" : {
"awslogs-group" : "firelens-container",
"awslogs-region" : var.aws_region,
"awslogs-create-group" : "true",
"awslogs-stream-prefix" : "firelens"
}
},
memoryReservation : 50
},
{
essential : true,
image : "httpd",
name : "app",
logConfiguration : {
"logDriver" : "awsfirelens",
"options" : {
"Name" : "s3",
"region" : var.aws_region,
"bucket" : "xxx-logs-bucket",
"total_file_size" : "1M",
"upload_timeout" : "1m",
"use_put_object" : "On",
"retry_limit" : "2"
}
},
memoryReservation : 100
}
])
}
This is my firelens.config
[OUTPUT]
Name s3
Match *
bucket xxx-logs-bucket
region ap-northeast-2
total_file_size 50M
use_put_object on
compression gzip
s3_key_format /$TAG/%Y/%m/%d/%H_%M_%S/$UUID.gz
s3_key_format_tag_delimiters .-
After completing the terraform apply, when I check the ECS Service log, some logs are repeatedly generated over and over. It is never stop.
[error] [output:s3:s3.1] Could not send chunk with tag app-firelens-dd6592001de147d09588e566aa5e60ba
[error] [output:s3:s3.1] PutObject request failed
[error] [aws_client] could not sign request
[error] [signv4] Provider returned no credentials, service=s3
[ warn] [aws_credentials] No cached credentials are available and a credential refresh is already in progress. The current co-routine will retry.
[ warn] [imds] unable to evaluate IMDS version
[error] [aws_client] connection initialization error
[error] [aws_credentials] Failed to retrieve credentials for AWS Profile default
[error] [aws_credentials] Shared credentials file /root/.aws/credentials does not exist
[error] [aws_client] could not sign request
[error] [signv4] Provider returned no credentials, service=s3
[ warn] [aws_credentials] No cached credentials are available and a credential refresh is already in progress. The current co-routine will retry.
[ warn] [imds] unable to evaluate IMDS version
[error] [aws_client] connection initialization error
[error] [aws_credentials] Failed to retrieve credentials for AWS Profile default
[error] [aws_credentials] Shared credentials file /root/.aws/credentials does not exist
I would appreciate it if you could tell me which one I should handle.