Get foreground from AdobeLabsUXMagicSelectionView (Adobe Creative SDK)

Question

I'm trying to use AdobeLabsUXMagicSelectionView and I'm facing with 2 problems. I wanna "cut" the selected area (foreground) using these 2 methods:

Method 1) getForeground:andMatte: It doesn't give me the correct foreground. When I select an area and call getForeground:andMatte I gives me foreground and background (mixed).

Selecting dog's face

Cutting dog's face

Documentation says:

Alternatively, if you don’t need to process the underlying bitmap directly, and intend to use the results as inputs to CoreGraphics or CoreImage, you can call:

Method 2) After this I'm trying to "cut" as the documentation does

extension AdobeLabsUXMagicSelectionView {

    func foregroundCGImage() -> CGImage {
        let w = size_t(self.image.size.width)
        let h = size_t(self.image.size.height)
        let data = UnsafeMutablePointer<UInt8>(malloc(4 * w * h * sizeof(UInt8)))
        self.readForegroundAndMatteIntoBuffer(data)

        for var i = 0; i < 4 * w * h; i += 4 {
            let alpha: UInt8 = UInt8(data[i + 3]) / 255
            data[i] *= alpha
            data[i + 1] *= alpha
            data[i + 2] *= alpha
        }

        let bitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.NoneSkipLast.rawValue)
        let ctx = CGBitmapContextCreate(data, w, h, 8, 4 * w, CGColorSpaceCreateDeviceRGB(), bitmapInfo.rawValue)
        let imageRef = CGBitmapContextCreateImage(ctx)!

        return imageRef
    }

}

But it only paints (black) the non-selected portion (background) of the image.

Anyone can help? What I want is get a final image of selected area.

UPDATE:

As @DonWoodward said I've create this Cattegory:

@implementation AdobeLabsUXMagicSelectionView (Foreground)

- (UIImage *)getForeground {
    // show the results
    // first create a UIImage of just the foreground bits per the documentation in AdobeLabsUXMagicSelectionView.h
    size_t w = self.image.size.width;
    size_t h = self.image.size.height;

    uint8_t *data = (uint8_t *)malloc(4*w*h*sizeof(uint8_t));
    [self readForegroundAndMatteIntoBuffer:data];

    // Paint the non-selected portion of the image black
    for (int i = 0; i < 4*w*h; i += 4) {
        float alpha = (float)data[i + 3] / 255;
        data[i    ] *= alpha;
        data[i + 1] *= alpha;
        data[i + 2] *= alpha;
    }
    CGContextRef ctx = CGBitmapContextCreate(data, w, h, 8, 4*w, CGColorSpaceCreateDeviceRGB(), (CGBitmapInfo)kCGImageAlphaNoneSkipLast);
    CGImageRef imageRef = CGBitmapContextCreateImage(ctx);
    UIImage * foregroundBits = [UIImage imageWithCGImage:imageRef];
    CGImageRelease(imageRef);

    return foregroundBits;
}

@end

But the result has a lot of black pixel around the "foreground".

What I need? Get a "clean" foreground (selected area without black pixels) to put over an UIImageView

Answer 1

In this case, you're premultiplying the RGB channels by your alpha channel, which implies that you would want to use kCGImageAlphaPremultipliedLast when creating your context, as in:

// Paint the non-selected portion of the image black
for (int i = 0; i < 4*w*h; i += 4)
{
    float alpha = (float)data[i + 3] / 255;
    data[i    ] *= alpha;
    data[i + 1] *= alpha;
    data[i + 2] *= alpha;
}
CGContextRef ctx = CGBitmapContextCreate(data, w, h, 8, 4*w, CGColorSpaceCreateDeviceRGB(), (CGBitmapInfo)kCGImageAlphaPremultipliedLast);
CGImageRef imageRef = CGBitmapContextCreateImage(ctx);
UIImage * foregroundBits = [UIImage imageWithCGImage:imageRef];
CGImageRelease(imageRef);

More information can be found in the CGImage docs

Also, you could also try using CGImageCreateWithMask with the output of

- (void)getForeground:(UIImage **)foregroundImage andMatte:(UIImage **)matteImage

as described in the Quartz 2D Programming Guide

UIImage *fg;
UIImage *matte;

[_magicSelectionView getForeground:&fg andMatte:&matte];

CGImageRef mattedForeground = CGImageCreateWithMask(fg.CGImage, matte.CGImage);
UIImage *foregroundBits = [UIImage imageWithCGImage:mattedForeground];

// show the results
_resultsView = [[UIImageView alloc] initWithFrame: CGRectMake(0, VIEW_Y_OFFSET, self.view.bounds.size.width, self.view.bounds.size.height-VIEW_Y_OFFSET)];
_resultsView.contentMode = UIViewContentModeScaleAspectFit;
[_resultsView setImage: foregroundBits];
[self.view addSubview: _resultsView];

If neither of these works, it's possible there might be some issues with the configuration of the UIImageViews being used to render the results, but without that code it's hard to say.

Answer 2

I think the problem is that your scale factor is of type uint8_t and so you're clipping the scale factor to zero. Make it a float and it should work. Here's the code from MagicPuppy (in objective c) that does it:

    // show the results
    // first create a UIImage of just the foreground bits per the documentation in AdobeLabsUXMagicSelectionView.h
    size_t w = _magicSelectionView.image.size.width;
    size_t h = _magicSelectionView.image.size.height;

    uint8_t *data = (uint8_t *)malloc(4*w*h*sizeof(uint8_t));
    [_magicSelectionView readForegroundAndMatteIntoBuffer:data];

    // Paint the non-selected portion of the image black
    for (int i = 0; i < 4*w*h; i += 4)
    {
        float alpha = (float)data[i + 3] / 255;
        data[i    ] *= alpha;
        data[i + 1] *= alpha;
        data[i + 2] *= alpha;
    }
    CGContextRef ctx = CGBitmapContextCreate(data, w, h, 8, 4*w, CGColorSpaceCreateDeviceRGB(), (CGBitmapInfo)kCGImageAlphaNoneSkipLast);
    CGImageRef imageRef = CGBitmapContextCreateImage(ctx);
    UIImage * foregroundBits = [UIImage imageWithCGImage:imageRef];
    CGImageRelease(imageRef);

    // show the results
    _resultsView = [[UIImageView alloc] initWithFrame: CGRectMake(0, VIEW_Y_OFFSET, self.view.bounds.size.width, self.view.bounds.size.height-VIEW_Y_OFFSET)];
    _resultsView.contentMode = UIViewContentModeScaleAspectFit;
    [_resultsView setImage: foregroundBits];
    [self.view addSubview: _resultsView];