Julia MethodError: no method matching (::Dense{typeof(logistic),CuArray{Float32,2,Nothing},CuArray{Float32,1,Nothing}})(::Float32)

Question

I have the following training data in CuArrays.

X: 300×8544 CuArray{Float32,2,Nothing}
y: 5×8544 Flux.OneHotMatrix{CuArray{Flux.OneHotVector,1,Nothing}}

and I have the following model I want to train:

# define activation
logistic(x) = 1. / (1 .+ exp.(-x))

# first define a 2-layer MLP model
model = Chain(Dense(300, 64, logistic),
          Dense(64, c),
          softmax) |> gpu

# define the loss
loss(x, y) = Flux.crossentropy(model(x), y)

# define the optimiser
optimiser = ADAM()

but if I do

Flux.train!(loss, params(model), zip(X, y), optimiser)

I get the following error:

MethodError: no method matching (::Dense{typeof(logistic),CuArray{Float32,2,Nothing},CuArray{Float32,1,Nothing}})(::Float32)

How should I resolve this?

Answer 1

@D.Danier Please provide minimal working examples (MWE), that means complete code that people can copy and paste and run. Below is an example

#Pkg.activate("c:/scratch/flux-test")

using CuArrays, Flux
CuArrays.allowscalar(false)

# define activation
# you don't the broadcast dots
logistic(x) = 1 / (1 + exp(-x))

# ensure your code works on GPU
CuArrays.@cufunc logistic(x) = 1 / (1 + exp(-x))

X = cu(rand(300, 8544))
y = cu(rand(5, 8544))
c = 5

# first define a 2-layer MLP model
model = Chain(Dense(300, 64, logistic),
          Dense(64, c),
          softmax) |> gpu

# define the loss
loss(x, y) = Flux.crossentropy(model(x), y) |> gpu

model(X)

# define the optimiser
optimiser = ADAM()

loss(X, y)

Flux.train!(loss, params(model), zip(eachcol(X), eachcol(y)), optimiser)

When you Flux.train!, you must tell Flux that you want to pair up the columns of X and y to compute the loss. BTW, this is probably less than ideal as it's computing too many iterations. You may want to group them up into mini-batches. Or if your problem is genuinely this small, then you can want to compute the whole thing in one go e.g.

Flux.train!(loss, params(model), (X, y), optimiser)

which basically say compute the loss based on the whole of X and y.