Can't specify digitalocean endpoint in sitemap_generator s3 aws adaptor

Question

I am using sitemap_generator in our Rails application & trying to create and upload the sitemaps to DigitalOcean spaces (which is s3 compatible as far as I know) but I can't override the endpoint attribute in the sitemap adaptor, so it always assumes my endpoint is amazon and not digitalocean

The error I get is:

Aws::Errors::NoSuchEndpointError: Encountered a `SocketError` while attempting to connect to:

  https://BUCKET_NAME.s3.XXXX.amazonaws.com/sitemaps/en/english.xml.gz

This is typically the result of an invalid `:region` option or a
poorly formatted `:endpoint` option.

My code in the sitemap.rb configuration for this part is:

SitemapGenerator::Sitemap.adapter = SitemapGenerator::AwsSdkAdapter.new(
  ENV["BUCKET_NAME"],
  aws_access_key_id: ENV["S3_KEY"],
  aws_secret_access_key: ENV["S3_SECRET"],
  aws_endpoint: "https://XXXX.digitaloceanspaces.com",
  aws_region: 'XXXX'
)

I tried using the other adaptor but I get an even more ambiguous error (apparently the same, failing to connect)

SitemapGenerator::Sitemap.adapter = SitemapGenerator::S3Adapter.new(
  aws_access_key_id: ENV["S3_KEY"],
  aws_secret_access_key: ENV["S3_SECRET"],
  fog_provider: 'AWS',
  fog_directory: ENV["BUCKET_NAME"],
  endpoint: "https://XXXX.digitaloceanspaces.com",
  fog_region: 'XXXX'
)

Answer 1

I believe the issue here is that the AWS sdk adapter for this gem does not currently support the "endpoint" option. Here are the currently supported options:

def initialize(bucket, options = {})
  @bucket = bucket
  @aws_access_key_id = options[:aws_access_key_id]
  @aws_secret_access_key = options[:aws_secret_access_key]
  @aws_region = options[:aws_region]
end

The endpoint option is what tells the AWS sdk to override the default service endpoint to use DigitalOcean instead of AWS: https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Endpoint.html

To get the sitemap_generator gem's AWS sdk adapter working with DigitalOcean spaces, a new pull request will need to be merged that adds the endpoint option.