Input buffer contains unsupported image format

Question

I have raised this on the lovell/sharp issues board too, but I was hoping there is someone out there who can give any tips on this.

I have a lambda that is being triggered by SQS with a reference to a source image. Then I somehow source that image and I try to sharpify :D

An extract of this is:

import axios from "axios";
import sharp from "sharp";
//first get the source image
    try {
        const source = (
            await axios({
                url: rawImage.sourceUrl,
                responseType: "arraybuffer"
            })
        ).data;
        sourceImageBuffer = Buffer.from(source);
        console.log("sourceImageBuffer", sourceImageBuffer);
    } catch (error) {
        problemWithImage = "can't access source image";
        console.log("getObject can't access source image", error);
    }

    //convert image to a sharp object
    try {
        if (!!sourceImageBuffer) {
            imageBuffer = await sharp(sourceImageBuffer).toBuffer();
            imageMetadata = await sharp(imageBuffer).metadata();
        }
    } catch (error) {
        console.log(
            "can't process source image. is it an image? error ",
            error
        );
        problemWithImage = "can't process source image. is it an image?";
        throw new Error(error);
    }

Every time following error is being thrown: 2020-09-29T09:05:07.436Z 2d64036b-3a05-474f-a22a-93e3446204d6 INFO can't process source image. is it an image? error [Error: Input buffer contains unsupported image format]

I've tried sourcing the image with s3.getObject or using some sort of request library, I'm getting the same result. If I console.log the sourceImageBuffer I correctly get a <Buffer ff d8 ff e0 00 10 4a 46 49 46 00 01 01 00 00 01 00 01 00 00 ff db 00 43 00 03 02 02 02 02 02 03 02 02 02 03 03 03 03 04 06 04 04 04 04 04 08 06 06 05 ... 25119 more bytes>.

When I run the code locally (on a mac, not docker) it runs perfectly.

After 5 days, 500 grey hairs and reading all bug tickets the google might offer me... I'm out of ideas.

Instead of buffer should I try using a stream? Or am I missing a library in the lambda? No idea.

sample image: https://s3.eu-west-1.amazonaws.com/rp-p1-stage-core-images-cold-storage/724/724100/11121/1%3A1/kfmrixsc.jpeg

    versions: {
      cairo: '1.16.0',
      exif: '0.6.22',
      expat: '2.2.9',
      ffi: '3.3',
      fontconfig: '2.13.92',
      freetype: '2.10.2',
      fribidi: '1.0.10',
      gdkpixbuf: '2.40.0',
      gettext: '0.21',
      gif: '5.1.4',
      glib: '2.65.2',
      gsf: '1.14.47',
      harfbuzz: '2.7.1',
      jpeg: '2.0.5',
      lcms: '2.11',
      orc: '0.4.31',
      pango: '1.46.1',
      pixman: '0.40.0',
      png: '1.6.37',
      svg: '2.49.4',
      spng: '0.6.0',
      tiff: '4.1.0',
      vips: '8.10.0',
      webp: '1.1.0',
      xml: '2.9.10',
      zlib: '1.2.11'
    }

format: {
  jpeg: {
    id: 'jpeg',
    input: { file: true, buffer: true, stream: true },
    output: { file: true, buffer: true, stream: true }
  },
  png: {
    id: 'png',
    input: { file: true, buffer: true, stream: true },
    output: { file: true, buffer: true, stream: true }
  },
  webp: {
    id: 'webp',
    input: { file: true, buffer: true, stream: true },
    output: { file: true, buffer: true, stream: true }
  },
  tiff: {
    id: 'tiff',
    input: { file: true, buffer: true, stream: true },
    output: { file: true, buffer: true, stream: true }
  },
  magick: {
    id: 'magick',
    input: { file: false, buffer: false, stream: false },
    output: { file: false, buffer: false, stream: false }
  },
  openslide: {
    id: 'openslide',
    input: { file: false, buffer: false, stream: false },
    output: { file: false, buffer: false, stream: false }
  },
  dz: {
    id: 'dz',
    input: { file: false, buffer: false, stream: false },
    output: { file: true, buffer: true, stream: true }
  },
  ppm: {
    id: 'ppm',
    input: { file: false, buffer: false, stream: false },
    output: { file: false, buffer: false, stream: false }
  },
  fits: {
    id: 'fits',
    input: { file: false, buffer: false, stream: false },
    output: { file: false, buffer: false, stream: false }
  },
  gif: {
    id: 'gif',
    input: { file: true, buffer: true, stream: true },
    output: { file: false, buffer: false, stream: false }
  },
  svg: {
    id: 'svg',
    input: { file: true, buffer: true, stream: true },
    output: { file: false, buffer: false, stream: false }
  },
  heif: {
    id: 'heif',
    input: { file: false, buffer: false, stream: false },
    output: { file: false, buffer: false, stream: false }
  },
  pdf: {
    id: 'pdf',
    input: { file: false, buffer: false, stream: false },
    output: { file: false, buffer: false, stream: false }
  },
  vips: {
    id: 'vips',
    input: { file: true, buffer: false, stream: false },
    output: { file: true, buffer: false, stream: false }
  },
  raw: {
    id: 'raw',
    input: { file: false, buffer: true, stream: true },
    output: { file: false, buffer: true, stream: true }
  }
}

sharp version in package.json "sharp": "^0.26.1"

Answer 1

hm. so here it goes. In another part of the project, I use sharp-phash to generate a unique hash for the source image, so I can check wether I need to resize or the old source and new source are matching.

I'm suspecting that the sharp dependency in sharp-phash and my project were conflicting and the whole thing just blew up.

As soon as I removed the sharp-phash import the problem went away.