Method leaks memory until a crash

50 Views Asked by At

I have many goroutines running for a very long time working with RabbitMQ. They receive data, accumulate it into sync.Map, and then every minute flush them to the database.

Before, when data was received, I was writing it straightaway to the database, but after some time it increased database' load dramatically, so I resorted to the solution of doing the flush every so and often.

Since then the memory usage of the app has been going up until it exhausts it and crashes. I didn't run a pprof yet, considering the new approach to be the cause, because when I revert it back the issue disappears.

I have many (a couple of thousand of goroutines) writing to sync.Map under integer key a map[string]interface{}. The question is why the memory usage increases and not gets GCed? This is the code for the flush method:

func flushOrdersToDatabase() error {
    ordersRep := order.NewRepository(structs.Gorm)
    log.Printf("Writing %d updated orders to database", lenSyncMap(&structs.Orders))
    structs.Orders.Range(func(existingOrderID, columnsToUpdate interface{}) bool {
        data := columnsToUpdate.(map[string]interface{})
        errUpdateOrder := ordersRep.UpdateOrder(nil, existingOrderID.(uint), data)
        if errUpdateOrder != nil {
            log.Printf("Error updating data for order with ID %d: %s",
                existingOrderID, errUpdateOrder.Error())
        }
        structs.Orders.Delete(existingOrderID)
        return true
    })

    log.Printf("Flush of orders to database completed")
    return nil
}

I have found that it's recommended to store a pointer to data instead of actual map, but haven't tried it yet.

This is how I populate the map:

queuedColumns, loaded := structs.Orders.LoadOrStore(existingOrder.ID, columnsToUpdate)
if loaded {
    mergedResult := mergeMaps(queuedColumns.(map[string]interface{}), columnsToUpdate)
    structs.Orders.Store(existingOrder.ID, mergedResult)
}

and this is the mergeMaps function just in case:

func mergeMaps(left map[string]interface{}, right map[string]interface{}) map[string]interface{} {
    result := make(map[string]interface{})
    for k, v := range left {
        if _, ok := left[k]; ok {
            result[k] = v
        }
    }

    for k, v := range right {
        if _, ok := right[k]; ok {
            result[k] = v
        }
    }

    return result
}
0

There are 0 best solutions below