Open CV: Identify cow based on its body pattern

Question

I want to identify a cow based on its body pattern. I am attempting to use template matching as I can gather data and the camera is fairly constant as well as the cows position. However the results are not perfect as i the matching includes the background as well. An example of the Cow is shown below:

The background is more or less static:

I have tried simple subtraction but the results are terrible, i think its because the background has similar pixel values. An example output i get for the subtraction is:

I would like to continue using template matching for the recognition for simplicity. I get okayish results using the following code, which has no preprocessing or filtering. I am sure theres ways to improve my results. I have tried different types of thresholding but doesnt improve the results that much, again i think its the sun thats the issue.

import cv2
import os
import numpy as np
cap = cv2.VideoCapture('Cow data/cow5/cow5.mp4')

while (cap.isOpened()):
      ret, frame = cap.read()
      img_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

template = cv2.imread('Cow data/cow5/Cow5.jpg', 0)
w, h = template.shape[::-1]

threshold = 0.7
loc = np.where(res >= threshold) #or res1 >= threshold)# or res2>= threshold)## or res3>= threshold)
for pt in zip(*loc[::-1]):
    cv2.rectangle(frame, pt, (pt[0] + w, pt[1] + h), (0, 255, 255), 25)

cv2.imshow('Detected', frame)
cv2.imshow('Other', img_gray)

if cv2.waitKey(1) & 0xFF == ord('c'):
    cv2.imwrite('Saved cows/Cow1/Cow1_extra'+str(save)+'.jpg', frame)
    save+=1

if cv2.waitKey(1) & 0xFF == ord('q'):
    break

cv2.waitKey(0)
cap.release()
cv2.destroyAllWindows()

I want to know what's the best way to isolate the cow. I have tried grabcut, but with poor results and really slow. Whats the best way to improve the results and what kind of preprocessing should i do?

I am really new to image processing and i would like to avoid more complicated image processing like feature extractions with surf and so on.

Any advice will be greatly appreciated!