runtime error

Exit code: 1. Reason: n _load_unlocked File "<frozen importlib._bootstrap_external>", line 883, in exec_module File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed File "/usr/local/lib/python3.10/dist-packages/diffusers/pipelines/pipeline_utils.py", line 47, in <module> from ..models import AutoencoderKL File "<frozen importlib._bootstrap>", line 1075, in _handle_fromlist File "/usr/local/lib/python3.10/dist-packages/diffusers/utils/import_utils.py", line 810, in __getattr__ module = self._get_module(self._class_to_module[name]) File "/usr/local/lib/python3.10/dist-packages/diffusers/utils/import_utils.py", line 822, in _get_module raise RuntimeError( RuntimeError: Failed to import diffusers.models.autoencoders.autoencoder_kl because of the following error (look up to see its traceback): /usr/local/lib/python3.10/dist-packages/flash_attn_2_cuda.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN3c104cuda9SetDeviceEi The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/app/app.py", line 3, in <module> from diffusers import DiffusionPipeline File "<frozen importlib._bootstrap>", line 1075, in _handle_fromlist File "/usr/local/lib/python3.10/dist-packages/diffusers/utils/import_utils.py", line 811, in __getattr__ value = getattr(module, name) File "/usr/local/lib/python3.10/dist-packages/diffusers/utils/import_utils.py", line 810, in __getattr__ module = self._get_module(self._class_to_module[name]) File "/usr/local/lib/python3.10/dist-packages/diffusers/utils/import_utils.py", line 822, in _get_module raise RuntimeError( RuntimeError: Failed to import diffusers.pipelines.pipeline_utils because of the following error (look up to see its traceback): Failed to import diffusers.models.autoencoders.autoencoder_kl because of the following error (look up to see its traceback): /usr/local/lib/python3.10/dist-packages/flash_attn_2_cuda.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN3c104cuda9SetDeviceEi

Container logs: