How do I access an element in a CuArray of Julia and change its value?

Question

I want to change only one element, as shown in the code below.

using Flux, CuArrays

a = rand(3,3) |> gpu
CuArrays.allowscalar(false)
a[1, 1] = 1.0f0

Because allowscalar is set to false, it is natural that it will appear as below.

ERROR: scalar setindex! is disallowed

But if allowscalar is removed, it will appear as below.

Performing scalar operations on GPU arrays: This is very slow, consider disallowing these operations with allowscalar(false)

I turned "allowscalar" on and off before and after the part that accesses the element. Then, it was about 20 times slower than when "allowscalar" was set to true.

Next, I tried to create another matrix once on the CPU and then add up the matrices on the GPU, as shown below.

b = zeros(Float32, 3, 3)
b[1, 1] = 1.0f0
b = b |> gpu
a .+= b

However, it is about 4 times faster if I assume that I can do it on the GPU alone such as below.

a .*= 1.0f0 # Dummy calculations that do some processing on the GPU
a .+= a # Dummy calculations that do some processing on the GPU

How do I access an element in a CuArray and change its value? I look forward to hearing from you soon.

Answer 1

I turned "allowscalar" on and off before and after the part that accesses the element. Then, it was about 20 times slower than when "allowscalar" was set to true.

Toggling allowscalar should not affect performance. In fact, CuArrays itself does so when it needs to inspect individual elements with certain APIs. A macro version of the function makes it easy to do so:

julia> a = CuArrays.rand(3,3);

julia> CuArrays.allowscalar(false)

julia> a[1, 1] = 1.0f0
ERROR: scalar setindex! is disallowed

julia> CuArrays.@allowscalar a[1, 1] = 1.0f0
1.0f0

julia> a
3×3 CuArray{Float32,2,Nothing}:
 1.0       0.277899   0.333898
 0.126213  0.0881365  0.794662
 0.94518   0.586488   0.656359

julia> a[1, 1] = 1.0f0