Exception: Unable to add DataPipe function name sharding_filter as it is already taken

Question

torchdata.datapipes is not working in Google Colab.

Even after installing the torchdata library, it raises an exception when datapipes function are imported.

I installed the dependencies

!pip install torchdata or !pip install --pre torchdata -f https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html But when I write

import torchdata.datapipes as dp or

from torchdata.datapipes.iter import StreamReader,FileOpener,FileLister
 
from torchdata.dataloader2 import DataLoader2, MultiProcessingReadingService

it throws an error :

Exception                                 Traceback (most recent call last)
<ipython-input-15-01a976fbf894> in <cell line: 1>()
----> 1 from torchdata.datapipes.iter import StreamReader,FileOpener,FileLister
      2 
      3 from torchdata.dataloader2 import DataLoader2, MultiProcessingReadingService

6 frames
/usr/local/lib/python3.9/dist-packages/torch/utils/data/datapipes/datapipe.py in register_datapipe_as_function(cls, function_name, cls_to_register, enable_df_api_tracing)
    134     def register_datapipe_as_function(cls, function_name, cls_to_register, enable_df_api_tracing=False):
    135         if function_name in cls.functions:
--> 136             raise Exception("Unable to add DataPipe function name {} as it is already taken".format(function_name))
    137 
    138         def class_function(cls, enable_df_api_tracing, source_dp, *args, **kwargs):

Exception: Unable to add DataPipe function name sharding_filter as it is already taken

What can be the possible causes and solutions?

Answer 1

The issue is automatically solved now. Library is working fine. Since datapipes are in beta development phase, my guess is that they might have updated some code.