How can I apply scipy.interpolate.RBFInterpolator on an image / ndarray?

Question

For example how do I apply RBFInterpolator on image loaded by opencv? I need to apply interpolation using vector-operations of numpy (which is fast)

I need to do non-affine transformation on image by defining interpolation between image points.

How do I do it?

Answer 1

import cv2
import numpy as np
from scipy.interpolate import RBFInterpolator

# Load the image
image = cv2.imread('image.png')
width,height = image.shape[:2]
w=width
h=height

# Your source points and corresponding destination points
src_points = np.array([
    [0, 0],
    [w,h],
    [w,0],
    [0,h],
])

dst_points = np.array([
    [0, 0],
    [w,h],
    [w,0],
    [0,h],
])

# Create the RBF interpolator instance
rbfx = RBFInterpolator(src_points,dst_points[:,0],kernel="thin_plate_spline")
rbfy = RBFInterpolator(src_points,dst_points[:,1],kernel="thin_plate_spline")

# Create a meshgrid to interpolate over the entire image
img_grid = np.mgrid[0:width, 0:height]

# flatten grid so it could be feed into interpolation
flatten=img_grid.reshape(2, -1).T

# Interpolate the displacement using the RBF interpolators
map_x = rbfx(flatten).reshape(width,height).astype(np.float32)
map_y = rbfy(flatten).reshape(width,height).astype(np.float32)
# Apply the remapping to the image using OpenCV
warped_image = cv2.remap(image, map_y, map_x, cv2.INTER_LINEAR)

# Save or display the result
cv2.imwrite('remap.png', warped_image)

The code above should produce same image as input image.

Change source and destination points to do image transformation.

I believe there is a faster way to do this interpolation, but this is what I come up with.

If you do a lot of transformations on same image multiple times, i suggest to cache flatten and reuse it instead of creating it each time.