AWS CDK - using CodePipeline my pipeline bounces back and forth between Build stage and UpdatePipeline stage

Question

I am using CodePipeline in aws cdk my pipeline bounces back and forth between the Build stage and the UpdatePipeline stage. I solved it once by toggling selfMutation to false. Now I tried to add a lambda function and I receive an error

-> Resource handler returned message: "Error occurred while GetObject. S3 Error Code: NoSuchKey. S3 Error Message: The specified key does not exist. (Service: Lambda, Status Code: 400

I figured it is due to my pipeline not updating and creating that asset inside s3 for the lambda function.

When I toggle selfMutation to true the pipeline just runs and runs and runs for hours and never finishes. It just bounces back and forth between the Build stage and the UpdatePipeline stage.

How do I go about fixing this? Has anyone else had this experience?

I have tried to run cdk deploy and it deployed but still with error -> Error occurred while GetObject. S3 Error Code: NoSuchKey. S3 Error Message: The specified key does not exist. (Service: Lambda, Status Code: 400

I tried toggling selfMutation to true -> pipeline just endlessly runs bouncing back and forth between the Build stage and the UpdatePipeline stage

PipelineStack Code Below

export class FloraPipelineStack extends cdk.Stack {
  constructor(
    scope: Construct,
    id: string,
    myEnvVars: any,
    props?: cdk.StackProps
  ) {
    super(scope, id, props);
    dotenv.config();

    const MyEcsRepo = Repository.fromRepositoryName(
      this,
      "My-Repo-Name",
      "My-Repo-Name"
    );

    const pipeline = new CodePipeline(this, "Pipeline", {
      pipelineName: "AppPipeline",
      synth: new CodeBuildStep("SynthStep", {
        input: CodePipelineSource.codeCommit(MyEcsRepo, "develop"),
        installCommands: ["npm install -g aws-cdk"],
        commands: ["npm ci", "npm run build", "npx cdk synth"],
        env: { ...(myEnvVars as { [key: string]: string }) },
      }),
      selfMutation: false,
      selfMutationCodeBuildDefaults: {
        cache: Cache.local(LocalCacheMode.SOURCE),
      },
      synthCodeBuildDefaults: {
        cache: Cache.local(LocalCacheMode.SOURCE),
      },
    });

    const deployAurora = new MyPipelineAuroraAppStage(
      this,
      "MyAuroraDeploy"
    );
    pipeline.addStage(deployAurora);
    
    const deployECS = new MyPipelineEcsAppStage(this, "MyEcsDeploy");
    pipeline.addStage(deployECS);
  }
}

Answer 1

Without seeing your code, my guess is that some part of your pipeline definition changes everytime you do build. Perhaps you have used a random value or a current date based value somewhere in pipeline definition.

My suggestion would be to:

1. Build/Synthesize your CDK locally
2. Copy pipeline stack into an external file
3. Build/Synthesize again
4. Do a byte by byte comparison between copied template and new template and see what changed.
5. Find root cause of that change and eliminate this from your code.