How to add different priors for each of ARD lengthscales hyperparameters in GPflow

Question

I've seen how to add a prior to the lengthscales hyperpameter in the MCMC notebook:

model.kernel.lengthscales.prior = tfd.Gamma(f64(1.0), f64(1.0))

From above, it seems we can only add a prior to a single length-scale of the RBF (radial basis function) kernel. I would like to add a prior to each of the lengthscales of a ARD (Automatic Relevance Determination) kernel. Any suggestions on how to do this is much appreciated.

Answer 1

Your code snippet,

model.kernel.lengthscales.prior = tfd.Gamma(f64(1.0), f64(1.0))

does add a prior to all the lengthscales of your ARD RBF kernel, but it's the same prior for each dimension.

To assign different priors to different dimensions of the ARD lengthscale, you can simply use the batch feature of tfp Distributions, e.g.

model.kernel.lengthscales.prior = tfd.Gamma(
    concentration=np.array([1.0, 2.0]),
    rate=np.array([3.0, 4.0]),
)

You can probably make it work for different distributions, too (there's tfd.JointDistribution); for that you'll have to consult the tensorflow_probability docs.