How to make a custom kernel (or bias) weights initializer which makes -1 or 1 values randomly for Keras Dense?

Question

In pure Python I can make it easily:

from random import choice


def init_by_plus_or_menus() -> int:
    return choice([-1, 1])

Keras documentation uses in the example Keras.backend (tf) function:

def my_init(shape, dtype=None):
    return tf.random.normal(shape, dtype=dtype)

layer = Dense(64, kernel_initializer=my_init)

, but there is no function is similar to standard Python random.choice or its element-wise analogue.

I tried this:

def init_plus_or_minus(shape, dtype=None):
    return backend.constant(value=choice([-1, 1]), shape=shape, dtype=dtype)

# Testing
if __name__ == '__main__':
   print(init_plus_or_minus(shape=(10,), dtype='int8'))

Terminal had turned me this:

tf.Tensor([1 1 1 1 1 1 1 1 1 1], shape=(10,), dtype=int8)

As you see the resulting tensor is filled by random value generated ONCE, it's not what I want.

Answer 1

I think I solved the problem using array generated by numpy.choice in tensorflow.constant's "value" argument:

from keras.backend import constant
from numpy.random import choice


def init_plus_or_minus(shape, dtype=None, name=None):
    return constant(value=choice([-1, 1], size=shape), dtype=dtype, name=name)


# Testing
if __name__ == '__main__':
   print(init_plus_or_minus(shape=(10, 5,), dtype='int8'))

Terminal turns this back:

tf.Tensor(
[[ 1 -1  1 -1  1]
 [ 1  1 -1 -1  1]
 [ 1  1 -1  1  1]
 [ 1 -1  1  1  1]
 [-1  1 -1 -1  1]
 [ 1 -1 -1  1 -1]
 [ 1 -1 -1  1 -1]
 [-1 -1 -1 -1 -1]
 [-1  1  1 -1  1]
 [-1  1  1 -1  1]], shape=(10, 5), dtype=int8)