Color Transfer Between Two Image with One Image as Plain Color

Question

I came across this particular color-transfer tutorial using OpenCV:

https://www.pyimagesearch.com/2014/06/30/super-fast-color-transfer-images/

and implemented it like this:

def color_transfer(source, target):
    # compute color statistics for the source and target images
    source = cv2.cvtColor(source, cv2.COLOR_BGR2LAB).astype("float32")
    target = cv2.cvtColor(target, cv2.COLOR_BGR2LAB).astype("float32")
    
    # compute color stats for both images
    (lMeanSrc, lStdSrc, aMeanSrc, aStdSrc, bMeanSrc, bStdSrc) = self.image_stats(source)
    (lMeanTar, lStdTar, aMeanTar, aStdTar, bMeanTar, bStdTar) = self.image_stats(target)
    
    # split the color space
    (l, a, b) = cv2.split(target)
    
    # substract the means from target image
    l -= lMeanTar
    a -= aMeanTar
    b -= bMeanTar
    
    # check values
    print(lStdSrc, aStdSrc, bStdSrc)
    print(lStdTar, aStdTar, bStdTar)
    print(lMeanSrc, aStdSrc, bMeanSrc)
    
    # process lab computation
    l = (lStdSrc / lStdTar) * l
    a = (aStdSrc / aStdTar) * a
    b = (bStdSrc / bStdTar) * b
    
    # add the source mean
    l += lMeanSrc
    a += aMeanSrc
    b += bMeanSrc
    
    # clipping the pixels between 0 and 255
    l = np.clip(l, 0, 255)
    a = np.clip(a, 0, 255)
    b = np.clip(b, 0, 255)
    
    # merge the channels
    transfer = cv2.merge([l, a, b])
    
    # converting back to BGR
    transfer = cv2.cvtColor(transfer.astype("uint8"), cv2.COLOR_LAB2BGR)
    
    return transfer

In this particular code:

    # process lab computation
    l = (lStdSrc / lStdTar) * l
    a = (aStdSrc / aStdTar) * a
    b = (bStdSrc / bStdTar) * b

it gets the standard deviation of the source, so when we combine the source and the target image, it will become a plain image as well since the lab will all be 0.

How can I fix this? It works when the source image is not a plain image with color.