SnapStart function priming - Doesn't seem to be working

Question

I’m somewhat new to Kotlin/Java, but I have been using AWS Lambda for several years now (all Python and Node). I’ve been trying to “successfully” enable SnapStart on a SpringBoot Lambda using Kotlin running on java11 corretto (the only runtime supported currently), but it doesn’t seem to be working as I would have expected.

I have hooked into the CRaC lifecycle methods beforeCheckpoint and afterRestore. In beforeCheckpoint I’ve initialized the SpringBoot application and I can see it in the deployment logs (AWS creates log streams for the deployment phase with SnapStart lambdas).

However, the concerning thing is I’m also seeing the SpringBoot app get initialized in the function invocation logs too. I would have expected that to only happen during the deployment/initialization phase when the snapshot is being created. As a result I’m not really seeing a tremendous improvement on latency or overall.

Any ideas why this is happening?

Answer 1

I ran into essentially the same issue (with Java instead of Kotlin) and the solution was to switch the runtime->handler from

org.springframework.cloud.function.adapter.aws.SpringBootStreamHandler

to

org.springframework.cloud.function.adapter.aws.FunctionInvoker::handleRequest

Answer 2

It would probably be worth mentioning that as of 2023-02-20 SnapStart isn't engaged for $LATEST version of an AWS Lambda function, i.e. make sure you are invoking a particular published version. Otherwise, Best practices for working with Lambda SnapStart article says that the main performance killers are dynamically loaded classes, and network connections that need to be re-established from time to time.

From Snapstart Integration issue raised for Spring Cloud Function on GitHub I tend to think that switching to org.springframework.cloud.function.adapter.aws.FunctionInvoker probably somewhat helps, but doesn't address the performance challenges mentioned above. I'm not sure if I'm interpreting olegz's advice correctly, but what worked best so far for my AWS lambda function built with Spring Boot/Spring Cloud Function is a "warm-up" config. It hooks into the CRaC lifecycle via beforeCheckpoint() and issues dummy requests to S3 and DynamoDB before the VM snapshot is made. This way most dynamically-loaded classes are pre-loaded, and network connections are pre-established, before any subsequent function invocation takes place.

package eu.mycompany.mysamplesystem.attachmentstore.configuration;

import com.amazonaws.services.lambda.runtime.events.S3Event;
import eu.mycompany.mysamplesystem.attachmentstore.handlers.MainEventHandler;
import lombok.extern.slf4j.Slf4j;
import org.crac.Core;
import org.crac.Resource;
import org.springframework.context.annotation.Configuration;
import software.amazon.awssdk.services.s3.model.NoSuchKeyException;

import java.util.ArrayList;
import java.util.List;

@Configuration
@Slf4j
public class WarmUpConfig implements Resource {

    private final MainEventHandler mainEventHandler;

    public WarmUpConfig(final MainEventHandler mainEventHandler) {
        Core.getGlobalContext().register(this);
        this.mainEventHandler = mainEventHandler;
    }

    @Override
    public void beforeCheckpoint(org.crac.Context<? extends Resource> context) {
        log.debug("Warm-up MainEventHandler by issuing dummy requests");
        dummyS3Invocation();
        dummyDynamoDbInvocation();
    }

    @Override
    public void afterRestore(org.crac.Context<? extends Resource> context) {

    }

    public void dummyS3Invocation() {
        S3Event s3Event = generateWarmUpEvent("ObjectCreated:Put");

        try {
            mainEventHandler.handleRequest(s3Event, null);
            throw new IllegalStateException("Warm-up event processing should have reached S3 and failed with S3Exception");
        } catch (NoSuchKeyException e) {
            log.debug("S3Exception is expected, since it is a warm-up");
        }
    }

    public void dummyDynamoDbInvocation() {
        S3Event s3Event = generateWarmUpEvent("ObjectRemoved:Delete");
        mainEventHandler.handleRequest(s3Event, null);
    }

    private S3Event generateWarmUpEvent(String eventName) {
        S3Event.S3BucketEntity s3BucketEntity = new S3Event.S3BucketEntity("hopefully_non_existing_bucket", null, null);
        S3Event.S3ObjectEntity s3ObjectEntity = new S3Event.S3ObjectEntity("hopefully/non/existing.key", 0L, null, null, null);
        S3Event.S3Entity s3Entity = new S3Event.S3Entity(null, s3BucketEntity, s3ObjectEntity, null);
        List<S3Event.S3EventNotificationRecord> records = new ArrayList<>();
        records.add(new S3Event.S3EventNotificationRecord(null, eventName, null, null, null, null, null, s3Entity, null));
        return new S3Event(records);
    }
}

P.S.: The MainEventHandler is basically the entry point to all the business logic exposed by the Function.

@SpringBootApplication
@RequiredArgsConstructor
public class Lambda {

    private final MainEventHandler mainEventHandler;

    public static void main(String... args) {
        SpringApplication.run(Lambda.class, args);
    }

    @Bean
    public Function<Message<S3Event>, String> defaultFunctionLambda() {
        return message -> {
            Context context = message.getHeaders().get("aws-context", Context.class);
            return mainEventHandler.handleRequest(message.getPayload(), context);
        };
    }
}