I am building an autmomated data streaming pipeline system from Kafka Cluster to S3 storage. It should be flexible to use all object storages that using S3 protocol to transfer data (Ceph, MinIO, Hitachi Storage, ...), not only AWS S3.
Then, I need to find an open-source Kafka Sink Connector to stream data from Kafka brokers to the S3 storage.
I have searched for a few days and all I found are many sink connectors to specific kinds of storage, not all S3-protocol services.
So I really want ask if anyone had already found or developed any Sink Connectors that meets my requirements: is open-source (under Apache 2.0 License) & streams to all S3-protocol object storages?
Thank you so much!