R dotplot bind color to group

Question

I'm trying to create a dotplot where I want the dots to be colored by group. But I can't bind a specific color to a specific group. My grouping variable is always c(1,2,3) where 1=black, 2=red and 3=green. But in cases where there are no group=2, dotplot handles the grouping variable as c(1,2) not c(1,3).

Consider the following example:

test <- data.frame(
    year = c(rep(2021, 3), rep(2022, 3), rep(2023, 3)),
    category = rep(c("A", "B", "C"), 3),
    value = c(1.23, 2.34, 1.54, 6.23, 8.67, 6.67, 4.78, 3.56, 6.90),
    grp = c(2, 2, 2, 3, 3, 2, 1, 1, 1)
)

lattice::dotplot(factor(year) ~ value | category,
    data = test,
    group = grp,
    col = c("#000000", "#ff0000", "#00ff00"),
    pch = 16,
    cex = 2,
    layout = c(1, 3)
)

The reference is always the current year, so that dot is black for all categories (grp=1). All dots lower than the reference value is red and green when higher.

In the example below the grouping variable only contains c(1,3) and all dot values are higher than the reference values, and I want the dots to be green.

test <- data.frame(
    year = c(rep(2021, 3), rep(2022, 3), rep(2023, 3)),
    category = rep(c("A", "B", "C"), 3),
    value = c(3.23, 6.34, 4.54, 4.23, 4.67, 6.67, 2.78, 3.01, 4.00),
    grp = c(3, 3, 3, 3, 3, 3, 1, 1, 1)
)

lattice::dotplot(factor(year) ~ value | category,
    data = test,
    group = grp,
    col = c("#000000", "#ff0000", "#00ff00"),
    pch = 16,
    cex = 2,
    layout = c(1, 3)
)

But dotplot seems to grab the colors in order of appearance. Is there a way to bind group values to specific colors?