Holoviews Heatmap not displaying when using explicit color mapping

Question

I'm attempting to create a categorical heatmap in holoviews. My pandas dataframe looks like this:

                                  defline msa alnpos loc  pos  clustal   
236522  GCF_012184425_1_000000000010_4705   T   1151   O  420  #ffffff   
236523  GCF_012184425_1_000000000010_4705   R   1152   O  421  #ffffff   
236524  GCF_012184425_1_000000000010_4705   -   1153   -    -  #ffffff   
236525  GCF_012184425_1_000000000010_4705   -   1154   -    -  #ffffff   
236526  GCF_012184425_1_000000000010_4705   -   1155   -    -  #ffffff   
236527  GCF_012184425_1_000000000010_4705   -   1156   -    -  #ffffff   
236528  GCF_012184425_1_000000000010_4705   -   1157   -    -  #ffffff   
236529  GCF_012184425_1_000000000010_4705   -   1158   -    -  #ffffff   
236530  GCF_012184425_1_000000000010_4705   -   1159   -    -  #ffffff   
236531  GCF_012184425_1_000000000010_4705   D   1160   B  422  #ffffff   
236532  GCF_012184425_1_000000000010_4705   T   1161   B  423  #ffffff   
236533  GCF_012184425_1_000000000010_4705   A   1162   B  424  #80a0f0   
236534  GCF_012184425_1_000000000010_4705   A   1163   B  425  #ffffff   
236535  GCF_012184425_1_000000000010_4705   L   1164   B  426  #80a0f0   
236536  GCF_012184425_1_000000000010_4705   L   1165   B  427  #ffffff   
236537  GCF_012184425_1_000000000010_4705   M   1166   B  428  #80a0f0   
236538  GCF_012184425_1_000000000010_4705   Q   1167   B  429  #ffffff   
236539  GCF_012184425_1_000000000010_4705   V   1168   B  430  #80a0f0   
236540  GCF_012184425_1_000000000010_4705   D   1169   B  431  #ffffff   
236541  GCF_012184425_1_000000000010_4705   F   1170   P  432  #80a0f0   

I'd like defline on the y-axis, alnpos on the x-axis, with clustal being the color of the heatmap cell at that position.

Library setup is:

import os
import sys
import argparse
import numpy as np
from itertools import islice
import pandas as pd
import bokeh.io
import bokeh.plotting
bokeh.io.output_notebook()
from bokeh.models import CategoricalColorMapper
from bokeh.models import HoverTool
import holoviews as hv
hv.extension('bokeh')

Attempting hv.HeatMap(data=alignment_df, kdims=['alnpos','defline'], vdims=['clustal']) results in a error in color mapping:

failed to validate LinearColorMapper(id='2912aef4-31cd-46e6-b9e2-635da2a8556b', ...).low: expected either None or a value of type Float, got '#14a4a4'

So I created a new column mapping color hex to a number using:

color_to_number = {'#ffffff':1,
'#14a4a4':2,
'#16c015':3,
'#80a0f0':4,
'#c048bf':5,
'#c0c000':6,
'#f01405':7,
'#f08080':8,
'#f09147':9}

hv.HeatMap(data=alignment_df, kdims=['alnpos','defline'], vdims=['color_by_numbers']) generates a plot just fine, but attempting to then use explicit color mapping to map color using the reverse of the dictionary above (hv.HeatMap(data=alignment_df, kdims=['alnpos','defline'], vdims=['color_by_numbers']).opts(cmap=number_to_color)) prints nothing. Anyone know what's going wrong here? Is Heatmap just incapable of accepting a dictionary in cmap? I'm hoping to avoid a color cycle to avoid problems when one color is not present in the data.