Is there any method to Average out close lines detected in houghlines?

Question

I have to detect appropriate rectangles (which satisfy the area and aspect ratio condition of doors) from an image to find Doors in it. So I used canny edge detection to find the edges and then I used hough Transform to extract lines for the edges. And I got the result something like in the image below: enter image description here

Now I somehow extracted lines that forms the rectangle which satisfy the 'Door Conditions'. but as you can see in the image the lines are so close that it forms many rectangles which satisfies my door condition. So I thought I would reduce the lines by replacing the close lines with their average line but I am unable to do it as it is becoming much more complex. I tried something like this.

def segregateLines(tobesegLines):
    while True:
        noLinesLeft = True
        for i in range(0 , len(tobesegLines)):
            closeLineFound = False
            for j in range(0, len(tobesegLines)):
                if abs(tobesegLines[i][0][0] - tobesegLines[j][0][0]) < 50:
                    closeLineFound = True
                    noLinesLeft = False
                    newLines.append([[(abs(tobesegLines[i][0][0] +tobesegLines[j][0][0])) /2, tobesegLines[i][0][1]]])

            if not closeLineFound:
                newLines.append(tobesegLines[i])

        if noLinesLeft:
            break
        segregateLines(newLines)
    return newLines

But i am getting 'memory error'.Does anyone have an idea about averaging the close lines in python. Or any other ideas about extracting correct rectangle from the lines to find my door?

P.S : I have Distance of every line from th origin i.e rho. and I have lines segregated as almost horizontal and almost vertical lines.