We are in the process of evaluating the combination of SnowflakeIO and S3 bucket. As per the Snowflake document below:
https://docs.snowflake.com/en/user-guide/data-load-s3-config
There are three options to configure the access to S3 bucket: 1) Storage integration; 2) IAM role; 3) IAM user. The second option is deprecated, so we ruled it out.
As per the SnowflakeIO document below:
https://beam.apache.org/documentation/io/built-in/snowflake/#using-snowflakeio-with-aws-s3
Both access key and secret key are mentioned in the code snippet, it means the option of IAM user is used, so we can rule the first option out as the purpose of the storage integration is to avoid the need to supply the security credentials. Actually, I also did the test with the storage integration and failed, the error said both keys need to be provided.
Can you please confirm that only the option of IAM user is supported by SnowflakeIO connector so far? If yes, is there any plan to support it? Thanks.