LoadLibrary error while using libtorch with ROS2

Question

I am trying to build and run a ROS2 package which uses libtorch.

In CMakeLists.txt I wrote

set(Torch_DIR ~/libtorch/share/cmake/Torch/)
find_package(Torch)

ament_auto_add_library(my_lib SHARED  
 src/node.cpp
)

target_link_libraries(my_lib 
 ${TORCH_LIBRARIES}
)

Package can be built without errors, but there is an error while running it:

terminate called after throwing an instance of 'class_loader::LibraryLoadException'
  what():  Could not load library LoadLibrary error: libtorch_cuda.so: cannot open shared object file: No such file or directory, at /tmp/binarydeb/ros-foxy-rcutils-1.1.5/src/shared_library.c:84

If I add path to libtorch to LD_LIBRARY_PATH: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/libtorch/lib/ there is another error while running package:

terminate called after throwing an instance of 'class_loader::LibraryLoadException'
  what():  Could not load library LoadLibrary error: /home/[email protected]/libtorch/lib/libgomp-75eea7e8.so.1: version `GOMP_4.5' not found (required by /lib/x86_64-linux-gnu/libpcl_common.so.1.10), at /tmp/binarydeb/ros-foxy-rcutils-1.1.5/src/shared_library.c:84

By the way, if I build simple executable (without ROS2), I can run it without adding path to libtorch to LD_LIBRARY_PATH. So it seems that path to libtorch is someway baked to an executable by linker (in the build without ROS), but ROS loads .so files in a different way...