AWS RDS to S3 Error. "The XML WAS NOT WELL FORMED"

Question

I am trying to migrate specific data from RDS(MYSQL) to S3 using AMAZON DATA PIPELINE. but i get the error

"The XML you provided was not well-formed or did not validate against our published schema (Service: Amazon S3; Status Code: 400; Error Code: Malformed XML"

enter image description here

enter image description here

Answer 1

Instead of building a custom architecture, i used the RDS - S3 full copy template. that seems to eliminate the error. although not yet getting the data in the s3 bucket, but i can access the empty csv file.

Answer 2

It might be caused by using invalid-XML-characters in some part of the S3 objects. See https://github.com/aws/aws-sdk-java/issues/333 .

To dig deeper, please provide more information:

• Is this caused by an API call? If so what SDK do you use and what exactly is the call?
• Can you share the pipeline configuration?

Answer 3

You need to set LifecycleTransition properties Days and Storage Class.

LifecycleConfiguration newConfiguration = new LifecycleConfiguration
        {
            Rules = new List<LifecycleRule>
            {
                new LifecycleRule
                        {
                             Id = "some id here",
                             Filter = new LifecycleFilter()
                             {
                                 LifecycleFilterPredicate = new LifecyclePrefixPredicate()
                                 {
                                    
                                 }
                             },
                             Status = LifecycleRuleStatus.Enabled,
                             Transitions = new List<LifecycleTransition>
                             {
                                  new LifecycleTransition
                                  {
                                       Days = 0,
                                       StorageClass = S3StorageClass.Glacier
                                  }
                              },
                             Expiration = new LifecycleRuleExpiration()
                             {
                                   Days = 1
                             }
                        }
            }
        };