How do I use RWMutex when concurrently modifying a map while iterating over it

536 Views Asked by At

I want to use a map's keys to request something from an API and then update the corresponding values based on the API's response.

My guess would be the following code.

Or maybe scratch this approach, collect the map-keys in an array before iterating and then use the array entries to make a request and modify the map

wg := &sync.WaitGroup{}
wg.Add(len(someMap))

sem := semaphore.NewWeighted(maxWorkers)
ctx := context.TODO()
mutex := &sync.RWMutex{}

mutex.RLock()
for k, v := range someMap {
    mutex.RUnlock()
    go func(k, v) {
        defer wg.Done()

        sem.Acquire(ctx, 1)
        res, err := API.REQUEST(k)
        sem.Release(1)

        if err != nil {
            return
        }

        v.SomeElement = res.SomeElement
        mutex.Lock()
        someMap[k] = v
        mutex.Unlock()
    }(k, v)
    mutex.RLock()
}
mutex.RUnlock()

wg.Wait()
1

There are 1 best solutions below

0
On

What you're doing should work. However, note that between runlock and rlock all you do is create a goroutine, so you'll have a lot of goroutines waiting for the lock. First iterating the map and then modifying is easier to read and debug. Or, you can change the map and have map[keytype]struct{value *valuetype}, and then you don't need to lock the map, and run your goroutines without a need for mutexes. Each goroutine will modify map[k].value instead of map[k].