Shadow Removal in Traffic Lane Lines

Question

I am working on lane lines detection. My current working strategy is:

1. defining a region of interest where lane lines could be

2. Warping the image to get a bird eye view

3. Converting the image to YUV color space

4. Normalizing the Y channel

5. Fitting the second order polynomial and sliding window approach

every thing works fine but where there are shadows the algorithm do not work. I have tried adaptive thresholding, otssu thresholding but not succeeded.

Source Image without Shadow:

Processed Source Image without Shadow:

Source Image with Shadow:

Processed Source Image with Shadow:

In the second Image it can be seen that the shadowed area is not detected. Actually shadows drops the image values down so i tried to threshold the image with new values lower than the previous one new image can be found here:

This technique does not work as it comes with a lot of noise

Currently I am trying background subtraction and shadow removal techniques but its not working. I am struck in this problem from last 2 3 weeks. Any help will really be appreciated...

import cv2
import matplotlib.pyplot as plt
import numpy as np 
from helper_functions import undistort, threshholding, unwarp,sliding_window_polyfit
from helper_functions import polyfit_using_prev_fit,calc_curv_rad_and_center_dist
from Lane_Lines_Finding import RoI
img = cv2.imread('./test_images/new_test.jpg')
new =undistort(img)
new = cv2.cvtColor(new, cv2.COLOR_RGB2BGR)
#new = threshholding(new)
h,w = new.shape[:2]
# define source and destination points for transform
imshape = img.shape
vertices = np.array([[
                      (257,670),
                      (590, 446),
                      (722, 440),
                      (1150,650)
                      ]], 
                      dtype=np.int32)  
p1 = (170,670)
p2 = (472, 475)
p3 = (745, 466)
p4 = (1050,650)

vertices = np.array([[p1,
                      p2,
                      p3,
                      p4
                      ]], 
                      dtype=np.int32)  
masked_edges = RoI(new, vertices)
#masked_edges = cv2.cvtColor(masked_edges, cv2.COLOR_RGB2BGR)


src = np.float32([(575,464),
                  (707,464), 
                  (258,682), 
                  (1049,682)])
dst = np.float32([(450,0),
                  (w-450,0),
                  (450,h),
                  (w-450,h)])
warp_img, M, Minv = unwarp(masked_edges, src, dst)
warp_img = increase_brightness_img(warp_img)
warp_img = contrast_img(warp_img)
YUV = cv2.cvtColor(warp_img, cv2.COLOR_RGB2YUV)
Y,U,V = cv2.split(YUV)
Y_equalized= cv2.equalizeHist(Y)
YUV = cv2.merge((Y,U,V))
thresh_min = 253
thresh_max = 255
binary = np.zeros_like(Y)
binary[(Y_equalized>= thresh_min) & (Y_equalized <= thresh_max)] = 1

kernel_opening= np.ones((3,3),np.uint8)
opening = cv2.morphologyEx(binary, cv2.MORPH_OPEN, kernel_opening)
kernel= np.ones((7,7),np.uint8)
dilation = cv2.dilate(opening,kernel,iterations = 3)