NoSuchUpload 404 Error when uploading files via signedurl (MultiPartUpload)

35 Views Asked by At

So I am trying to do multi part upload viacjs sdk on my nextjs app. But I am getting Request failed with status code 404 and No Such Upload error when making a put request to the presignurl for the first part itself.

Earlier I dont how but it used to work but when I used to perform the CompleteMultiPartUpload I used to get NoSuchUpload due to having diffrent key for creating and completing multi part upload, but when I fixed that I started getting this error. I am pretty sure I have messed up somwhere which I am not able to find.

Below are the frontend and backend part (and please ignore the mess I have created).

This is the api part which created Multi part upload and generates signed url

I know mutating variables is bad, but I am desparate to get it work then I will look at making this better

  let UploadId = uploadId;
  let KEY: string | undefined = objectKey;
  try {
    console.log('[STARTING MULTI PART]::', chunkIndex);
    if (chunkIndex === 1) {
      console.log({ KEY });
      KEY = v4();

      const createMultipartUploadCmd = new CreateMultipartUploadCommand({
        Bucket: env.AWS_S3_BUCKET_NAME,
        Key: KEY,
        ContentType: type,
      });

      UploadId = (await s3Client.send(createMultipartUploadCmd)).UploadId;
    }

    console.log({ UploadId, k: `${KEY}/chunk_${chunkIndex}` });

    const uploadPartCmd = new UploadPartCommand({
      Bucket: env.AWS_S3_BUCKET_NAME,
      Key: `${KEY}/chunk_${chunkIndex}`,
      PartNumber: chunkIndex,
      UploadId,
    });
    console.log('[GENERATING SIGNED URL]');

    const signedUrl = await getSignedUrl(s3Client, uploadPartCmd, {
      expiresIn: 60 * 5,
    });
    console.log('[DONE SIGNED URL]', { signedUrl });

    return NextResponse.json(
      {
        message: 'success',
        data: { url: signedUrl, uploadId: UploadId, objectKey: KEY },
      },
      { status: 200 }
    );

This the frontend useMutationHook

  const useVideoUpload = useMutation({
    mutationKey: ['upload-video-s3'],
    mutationFn: async () => {
      // requesting signed url
      if (!videoFile) {
        throw new Error('File not found');
      }
      let upUrl: string;
      let upId: string;
      console.log('[STRATING VIDEO UPLOAD]');

      const PART_SIZE = 1024 * 1024 * 5; // 5 MB

      const totalParts = Math.ceil(videoFile.size / PART_SIZE);
      const partsETags: { ETag: string; PartNumber: number }[] = [];

      // Request a signed URL for the first part to get the uploadId
      const firstPart = videoFile.slice(0, PART_SIZE);
      const checksumForPart1 = await computeSHA256(firstPart);
      console.log({ size: videoFile.size, totalParts });

      let uploadId: string;
      let objectKey: string;
      console.log('STARTIN FIRST PART');

      try {
        const res = await axios.post('/api/channel/get-signed-url/video', {
          fileName: videoFile.name,
          type: videoFile.type,
          size: firstPart.size,
          checksum: checksumForPart1,
          chunkIndex: 1,
        });

        if (res.status !== 200) {
          console.log('EHHHT');

          throw new Error(res.data);
        }
        const { data: urlData } = (await res.data) as {
          data: GetSignedUrlResponse;
        };
        uploadId = urlData.uploadId;
        objectKey = urlData.objectKey;
        upUrl = urlData.url;
        const putResponse = await axios.put(urlData.url, firstPart, {
          headers: {
            'Content-Type': videoFile.type,
          },
        });

        partsETags.push({
          ETag: putResponse.headers.etag.trim().replaceAll('"', ''),
          PartNumber: 1,
        });
        console.log('DONE FIRST PART', { putResponse });
      } catch (error) {
        console.error('[ERROR REQUESTING SIGNED URL]', error);
        throw error;
      }

      upId = uploadId;

      console.log('STARTIN DATA', { uploadId, objectKey });

      for (let i = 1; i < totalParts; i++) {
        console.log('IN THE Loop', i, { uploadId });

        const start = i * PART_SIZE;
        const end = Math.min(start + PART_SIZE, videoFile.size);
        const chunk = videoFile.slice(start, end);
        const checksum = await computeSHA256(videoFile);

        try {
          const res = await axios.post('/api/channel/get-signed-url/video', {
            fileName: videoFile?.name,
            type: videoFile?.type,
            size: chunk?.size,
            checksum,
            objectKey: objectKey,
            uploadId,
            chunkIndex: i + 1,
          });

          if (res.status !== 200) {
            throw new Error(res.data);
          }
          const { data: signedUrlData } = (await res.data) as {
            message: string;
            data: { url: string; uploadId: string };
          };
          console.log('IT UP IS SAME', upId === uploadId);
          console.log('Signed URL IS SAME', upUrl === signedUrlData.url);

          console.log(`[STARTING VIDEO PART ${i + 1}]`);
          const response = await axios.put(signedUrlData.url, chunk, {
            headers: {
              'Content-Type': videoFile.type,
            },
          });
          console.log(`[DONE VIDEO PART ${i + 1}]`);
          console.log('video res', { response });
          partsETags.push({
            ETag: response.headers.etag.trim().replaceAll('"', ''),
            PartNumber: i + 1,
          });
          console.log('pushed etag');
        } catch (error) {
          console.error(`[ERROR UPLOADING VIDEO PART ${i + 1}]`, error);
          throw error;
        }
      }
      console.log(`[COMPLETING VIDEO PART ]`);
      console.log('[STARTING COMPLETE]');

      const completeUploadRes = await axios.post(
        '/api/channel/get-signed-url/video/complete',
        {
          partsETags,
          objectKey: objectKey,
          uploadId,
        }
      );
      if (completeUploadRes.status === 201) {
        throw new Error(completeUploadRes.data);
      }
      console.log('[UPLOAD COMPLETE]');
      return await completeUploadRes.data;
    },
  });

This is the api part for completing the Uplaod

const completeMultipartUploadCmd = new CompleteMultipartUploadCommand({
    Bucket: env.AWS_S3_BUCKET_NAME,
    Key: objectKey,
    UploadId: uploadId,
    MultipartUpload: {
      Parts: partsETags.sort((a, b) => a.PartNumber - b.PartNumber),
    },
  });

    await s3Client.send(completeMultipartUploadCmd);

The thing is I am able to succesfullt genrate presigned url but the when making the put request with genrated url throws the 404 NoSuchUpload error

0

There are 0 best solutions below