Is there a way to create a snowflake external stage, which reads data from s3 access point?

738 Views Asked by At

I want to create an external stage in snowflake, which will use S3 access point to access files stored in S3 bucket. I've started by creating a IAM Role, with attached policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "s3:GetBucketLocation",
            "Resource": "<s3_bucket_arn>"
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:ListBucket",
                "s3:GetObjectVersion"
            ],
            "Resource": [
                "<s3_bucket_arn>/*",
                "<s3_bucket_arn>"
            ],
            "Condition": {
                "StringEquals": {
                    "s3:DataAccessPointArn": "<s3_access_point_arn>"
                }
            }
        }
    ]
}

The access point has a policy

{
    "Version":"2012-10-17",
    "Statement": [
    {
        "Effect": "Allow",
        "Principal": {
            "AWS": "<role_arn>"
        },
        "Action": ["s3:GetObject", "s3:GetObjectVersion"],
        "Resource": "<access_point_arn>/object/*"
    },
    {
        "Effect": "Allow",
        "Principal": {
            "AWS": "<role_arn>"
        },
        "Action": "s3:ListBucket",
        "Resource": "<access_point_arn>"
    }]
}

In snowflake, I have created the storage integration with sql:

create storage integration s3ap
    type = external_stage
    storage_provider = s3
    storage_aws_role_arn = '<role_arn>'
    enabled = true
    storage_allowed_locations = ('s3://<access_point_alias>/');

Then I've updated Iam role trust policy and created a stage

create stage test_stage
storage_integration = s3ap
url = 's3://<access_point_alias>';

I can list the stage with command ls @test_stage and it shows objects in s3 correctly, but when I try to read data with command select $1 from @test_stage I get an error Failed to access remote file: access denied. Please check your credentials.

I've tried also adding kms privileges, as it was mentioned on this site, to the iam role policy, but the error is still raised.

Am I missing something, or S3 access point cannot be used to create an external stage in snowflake?

1

There are 1 best solutions below

0
On

S3 access points is not supported yet for external Stages. You need to use S3 Bucket URL to work with External Tables.