Error on AWS Lambda when using deoldify/ColorizeArtistic_gen

24 Views Asked by At

I'm encountering a difficulty when using deoldify/ColorizeArtistic_gen in AWS Lambda.

[ERROR] OSError: [Errno 30] Read-only file system: '/home/sbx_user1051'
Traceback (most recent call last):
  File "/var/lang/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/var/task/app.py", line 20, in <module>
    colorizer = visualize.get_image_colorizer(artistic=True)
  File "/var/task/deoldify/visualize.py", line 442, in get_image_colorizer
    return get_artistic_image_colorizer(root_folder=root_folder, render_factor=render_factor)
  File "/var/task/deoldify/visualize.py", line 465, in get_artistic_image_colorizer
    learn = gen_inference_deep(root_folder=root_folder, weights_name=weights_name)
  File "/var/task/deoldify/generators.py", line 88, in gen_inference_deep
    learn = gen_learner_deep(
  File "/var/task/deoldify/generators.py", line 100, in gen_learner_deep
    return unet_learner_deep(
  File "/var/task/deoldify/generators.py", line 131, in unet_learner_deep
    body = create_body(arch, pretrained)
  File "/var/lang/lib/python3.9/site-packages/fastai/vision/learner.py", line 56, in create_body
    model = arch(pretrained)
  File "/var/lang/lib/python3.9/site-packages/torchvision/models/resnet.py", line 320, in resnet34
    return _resnet("resnet34", BasicBlock, [3, 4, 6, 3], pretrained, progress, **kwargs)
  File "/var/lang/lib/python3.9/site-packages/torchvision/models/resnet.py", line 296, in _resnet
    state_dict = load_state_dict_from_url(model_urls[arch], progress=progress)
  File "/var/lang/lib/python3.9/site-packages/torch/hub.py", line 571, in load_state_dict_from_url
    os.makedirs(model_dir)
  File "/var/lang/lib/python3.9/os.py", line 215, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/var/lang/lib/python3.9/os.py", line 215, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/var/lang/lib/python3.9/os.py", line 215, in makedirs
    makedirs(head, exist_ok=exist_ok)
  [Previous line repeated 1 more time]
  File "/var/lang/lib/python3.9/os.py", line 225, in makedirs
    mkdir(name, mode)

this is Dockerfile. in stageA i'm downloading the model in '/tmp' dir as lambda allows only '/tmp' directory as writeable. So i first download the model in tmp dir then copy it to working dir (/var/task). I do not know whether this approach is good or not. your suggestions are highly appreciable..

# Stage A: Download Model
FROM public.ecr.aws/lambda/python:3.9 AS stageA

ENV LANG=en_US.UTF-8
ENV TZ=:/etc/localtime
ENV PATH=/var/lang/bin:/usr/local/bin:/usr/bin/:/bin:/opt/bin
ENV LD_LIBRARY_PATH=/var/lang/lib:/lib64:/usr/lib64:/var/runtime:/var/runtime/lib:/var/task:/var/task/lib:/opt/lib
ENV MPLCONFIGDIR=/tmp/matplotlib

ENV NUMBA_CACHE_DIR=/tmp

# Create directory to store the model
RUN mkdir -p $NUMBA_CACHE_DIR/models

# Download the model to /tmp
RUN curl -o $NUMBA_CACHE_DIR/models/ColorizeArtistic_gen.pth https://data.deepai.org/deoldify/ColorizeArtistic_gen.pth

# Stage B: Final Image
FROM public.ecr.aws/lambda/python:3.9

ENV LANG=en_US.UTF-8
ENV TZ=:/etc/localtime
ENV PATH=/var/lang/bin:/usr/local/bin:/usr/bin/:/bin:/opt/bin
ENV LD_LIBRARY_PATH=/var/lang/lib:/lib64:/usr/lib64:/var/runtime:/var/runtime/lib:/var/task:/var/task/lib:/opt/lib
ENV MPLCONFIGDIR=/tmp/matplotlib

ENV LAMBDA_TASK_ROOT=/var/task

WORKDIR $LAMBDA_TASK_ROOT

# Create directory to store the model in the final image
RUN mkdir -p $LAMBDA_TASK_ROOT/models

# Copy the downloaded model from Stage A to Stage B
COPY --from=stageA /tmp/models/ColorizeArtistic_gen.pth $LAMBDA_TASK_ROOT/models/

# Copy other necessary files
COPY requirements.txt .
COPY app.py .
COPY deoldify ./deoldify
COPY result_images ./result_images

# Install dependencies
RUN yum -y update && \
    yum install -y python3-pip ffmpeg libSM libXext libXrender mesa-libGL && \
    yum clean all && \
    pip3 install -r requirements.txt

# Clean up unnecessary files
RUN rm -rf /var/cache/yum && \
    rm -rf /root/.cache && \
    rm -rf /tmp/*

ENTRYPOINT ["/lambda-entrypoint.sh"]
CMD ["app.handler"]
0

There are 0 best solutions below