Can we use kafka consumer with Ballerina as a AWS Lambda function?

Question

I am trying out on ballerina and want to know if we can have a ballerina program which is a kafka consumer can be exposed as a AWS Lambda function.

Answer 1

You cannot create a Kafka Listener as an AWS lambda, but you can create a non-listener type Kafka consumer as an AWS lambda function.

You can write ballerina function to handle Kafka SimpleConsumer object's functions (Current Ballerina-By-Guide for Kafka is treating Kafka consumer as a service listener), and use that function to create AWS lambdas.

WSO2-Kafka connector offers various functionalities to handle Kafka consumer manually. Read the documentation for more info, it is available on WSO2-Kafka release.

You can generate AWS lambdas by adding AWS lambda annotation on top of the function.

@awslambda:Function
function kafkaConsumerService(awslambda:Context ctx, json inputs) returns json|error {

  kafka:ConsumerConfig consumerConfig = {
    // Consumer configs
  };

  kafka:SimpleConsumer kafkaConsumer = new(consumerConfig);

  // You can implement Kafka consume functionalities here.
  // For an example, you can poll consumer using kafkaConsumer->poll(<duration>);
  var results = kafkaConsumer->poll(1000);

  if (results is error) {
    // Handle error
  } else {
    // Handle records received
  }
}

Then you can generate AWS Lambdas by following the Ballerina documentation on AWS Lambdas.

Answer 2

AWS Lambda functions needs to be triggered from somewhere (Not working as listeners). In that case you wont be able expose Ballerina Kafka consumer with Amazon AWS.