position = position_dodge does not work with date axis

Question

I have a dummy data set that looks like this,

structure(list(yr_mth = structure(c(19631, 19631, 19662, 19662
), class = "Date"), key = c("login", "active", "login", "active"
), value = c(1000, 100, 2000, 300)), class = "data.frame", row.names = c(NA, 
-4L))

and I tried to draw a dodge bar plot to show the trend, however, if I keep the yr_mth column as date, then the position_dodge in geom_text seems not working:

read_excel("sample_date.xlsx", 
    col_types = c("date", "text", "numeric")) %>% 
    ggplot(aes(x = yr_mth, y = value, group = key, fill = key)) + 
  geom_col(position="dodge")  +
  geom_text(aes(x = yr_mth , y = value, group = key,label = value), 
            position = position_dodge(width = .9))

if I keep the yr_mth column as a string, then it works perfectly

read_excel("sample_date.xlsx", 
    col_types = c("text", "text", "numeric")) %>% 
    ggplot(aes(x = yr_mth, y = value, group = key, fill = key)) + 
  geom_col(position="dodge")  +
  geom_text(aes(x = yr_mth , y = value, group = key,label = value), 
            position = position_dodge(width = .9))

of course, I could force it to be a string but it will need a lot of extra work for data format, is there a way to fix this?