cuDF not working on 'apply' function for string return type

Question

I'm getting below error while calling the apply function on cuDF. AttributeError: 'CPUDispatcher' object has no attribute 'closure'

File /opt/conda/lib/python3.10/site-packages/cudf/utils/cudautils.py:78, in make_cache_key(udf, sig)
     76 names = udf.__code__.co_names
---> 78 if udf.__closure__ is not None:
     79     cvars = tuple(x.cell_contents for x in udf.__closure__)

AttributeError: 'CPUDispatcher' object has no attribute '__closure__'

Which results below error as well: ValueError: user defined function compilation failed.

File /opt/conda/lib/python3.10/site-packages/cudf/core/indexed_frame.py:2288, in IndexedFrame._apply(self, func, kernel_getter, *args, **kwargs)
   2284     kernel, retty = _compile_or_get(
   2285         self, func, args, kernel_getter=kernel_getter
   2286     )
   2287 except Exception as e:
-> 2288     raise ValueError(
   2289         "user defined function compilation failed."
   2290     ) from e
   2292 # Mask and data column preallocated
   2293 ans_col = _return_arr_from_dtype(retty, len(self))

ValueError: user defined function compilation failed.

Sample code to reproduce:

import numba, cudf
print(numba.__version__)    # 0.58.1
print(cudf.__version__)     # 23.08.00

@numba.njit
def f(row):
    st = row['str_col']
    scale = row['scale']
    if len(st) == 0:
            return 'a' + str(scale) + 'x'
    elif st.startswith('a'):
            return 'b'
    elif 'example' in st:
            return 'c'
    else:
            return 'd'

df1 = cudf.DataFrame({
    'str_col': ['', 'abc', 'some_example'],
    'scale': [1, 2, 3]
})
df1['abc'] = df1.apply(f, axis=1)  
df1.head(100)

Answer 1

As a workaround, I think you can do:

df1['abc'] = df1.apply(f.py_func, axis=1) 

In the long term this either needs a fix in cuDF (to special-case Numba dispatcher objects) or Numba (to implement dunder attributes like __closure__ for its dispatcher objects) - I'm not sure which is the correct way to go for the long term without giving it a bit more thought.