Getting WARNING: DATA RACE in golang

Question

I am using this golang size library https://pkg.go.dev/github.com/streamonkey/size which calculates variable's memory consumption at runtime. Here is the link to source code of the library https://github.com/streamonkey/size/blob/v0.0.1/size.go.

I am passing an interface to the Of function below which then calls the sizeOf function in the library:

func Of(v interface{}) int {
    // Cache with every visited pointer so we don't count two pointers
    // to the same memory twice.
    cache := make(map[uintptr]bool)
    return sizeOf(reflect.Indirect(reflect.ValueOf(v)), cache)
}

sizeOf goes through each field in the interface and check the type of the data it is. Based on the type it calculates the size of variable on runtime.

The issue i am having when the type is map I am getting an Data Race warning. Below is the piece of the code which is throwing an error and you can also find it in the source code link I posted above.

    case reflect.Map:
        // return 0 if this node has been visited already (infinite recursion)
        if cache[v.Pointer()] {
            return 0
        }
        cache[v.Pointer()] = true
        sum := 0
        keys := v.MapKeys()
        for i := range keys {
            val := v.MapIndex(keys[i])
            // calculate size of key and value separately
            sv := sizeOf(val, cache)
            if sv < 0 {
                return -1
            }
            sum += sv
            sk := sizeOf(keys[i], cache)
            if sk < 0 {
                return -1
            }
            sum += sk
        }
        // Include overhead due to unused map buckets.  10.79 comes
        // from https://golang.org/src/runtime/map.go.
        return sum + int(v.Type().Size()) + int(float64(len(keys))*10.79)

Below is the error I am receiving:

Read At

WARNING: DATA RACE
00:03:27  Read at 0x00c000996330 by goroutine 98:
00:03:27    reflect.maplen()
00:03:27        /home/git/.gimme/versions/go1.21.1.linux.amd64/src/runtime/map.go:1411 +0x0
00:03:27    reflect.Value.MapKeys()
00:03:27        /home/git/.gimme/versions/go1.21.1.linux.amd64/src/reflect/value.go:1788 +0x1a4

WRITE at

 Previous write at 0x00c000996330 by goroutine 99:
00:03:27    runtime.mapdelete_fast64()
00:03:27        /home/git/.gimme/versions/go1.21.1.linux.amd64/src/runtime/map_fast64.go:273 +0x0
00:03:27    net/http.(*Transport).setReqCanceler()
00:03:27        /home/git/.gimme/versions/go1.21.1.linux.amd64/src/net/http/transport.go:1156 +0x204
00:03:27    net/http.(*persistConn).roundTrip.func1()
00:03:27        /home/git/.gimme/versions/go1.21.1.linux.amd64/src/net/http/transport.go:2622 +0x9a
00:03:27    runtime.deferreturn()
00:03:27        /home/git/.gimme/versions/go1.21.1.linux.amd64/src/runtime/panic.go:477 +0x30
00:03:27    net/http.(*Transport).roundTrip()

Also i tried to add the mutex lock around keys := v.MapKeys() but no luck.

Would need some suggestion what exactly is happening or any suggestions are welcomed. Thanks

Answer 1

short answer is: this Of() function is not thread safe and mustn't be used on values that may be modified in other goroutines.

---

You didn't paste your own code (the one where you call Of(...) on one of your variables), but from the looks of the data race, it looks like you are trying to get the size of an http.Client or Request or Response, which is also actively used elsewhere (e.g: in an actual, live http request).

Depending on your usage context: my guess would be that this library wasn't intended to be used in production code, or at least only on structures where you are in complete control of their lifetime.
If you are in a test function, just check size.Of() after all these requests have completed.