root@SERV-SAE:/mnt/f/deep-pavlov/dream# sudo docker-compose -f docker-compose.yml -f assistant_dists/dream_persona_rullama_prompted/docker-compose.override.yml -f assistant_dists/dream_persona_rullama_prompted/dev.yml up --build transformers-lm-llama7bru
[+] Building 2.7s (16/16) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [internal] load .dockerignore 0.1s
=> => transferring context: 35B 0.1s
=> [internal] load metadata for docker.io/pytorch/pytorch:2.0.0-cuda11.7-cudnn8-devel 1.3s
=> [ 1/11] FROM docker.io/pytorch/pytorch:2.0.0-cuda11.7-cudnn8-devel@sha256:96ccb2997a131f2455d70fb78dbb284bafe 0.0s
=> [internal] load build context 1.1s
=> => transferring context: 11.57kB 1.1s
=> CACHED [ 2/11] WORKDIR /src 0.0s
=> CACHED [ 3/11] COPY ./services/transformers_peft_lm/requirements.txt /src/requirements.txt 0.0s
=> CACHED [ 4/11] RUN pip install --upgrade pip && pip install -r /src/requirements.txt 0.0s
=> CACHED [ 5/11] RUN apt-get update && apt-get install -y --allow-unauthenticated git wget 0.0s
=> CACHED [ 6/11] RUN pip install git+https://github.com/huggingface/transformers.git@15641892985b1d77acc74c9065 0.0s
=> CACHED [ 7/11] RUN pip install git+https://github.com/huggingface/peft.git@382b178911edff38c1ff619bbac2ba556b 0.0s
=> CACHED [ 8/11] RUN LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu/ 0.0s
=> CACHED [ 9/11] RUN pip install bitsandbytes 0.0s
=> CACHED [10/11] COPY services/transformers_peft_lm /src 0.0s
=> CACHED [11/11] COPY common /src/common 0.0s
=> exporting to image 0.1s
=> => exporting layers 0.0s
=> => writing image sha256:026b653e510ce1e7a880b0206f6c0559b977812c39ee90ca89fff48644a7ea77 0.0s
=> => naming to Docker 0.0s
[+] Running 1/1
Container dream-transformers-lm-llama7bru-1 Recreated 0.1s
Attaching to dream-transformers-lm-llama7bru-1
dream-transformers-lm-llama7bru-1 |
dream-transformers-lm-llama7bru-1 | ==========
dream-transformers-lm-llama7bru-1 | == CUDA ==
dream-transformers-lm-llama7bru-1 | ==========
dream-transformers-lm-llama7bru-1 |
dream-transformers-lm-llama7bru-1 | CUDA Version 11.7.0
dream-transformers-lm-llama7bru-1 |
dream-transformers-lm-llama7bru-1 | Container image Copyright (c) 2016-2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
dream-transformers-lm-llama7bru-1 |
dream-transformers-lm-llama7bru-1 | This container image and its contents are governed by the NVIDIA Deep Learning Container License.
dream-transformers-lm-llama7bru-1 | By pulling and using the container, you accept the terms and conditions of this license:
dream-transformers-lm-llama7bru-1 | https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license
dream-transformers-lm-llama7bru-1 |
dream-transformers-lm-llama7bru-1 | A copy of this license is made available in this container at /NGC-DL-CONTAINER-LICENSE for your convenience.
dream-transformers-lm-llama7bru-1 |
dream-transformers-lm-llama7bru-1 | WARNING: The NVIDIA Driver was not detected. GPU functionality will not be available.
dream-transformers-lm-llama7bru-1 | Use the NVIDIA Container Toolkit to start this container with GPU support; see
dream-transformers-lm-llama7bru-1 | Welcome — NVIDIA Cloud Native Technologies documentation .
dream-transformers-lm-llama7bru-1 |
dream-transformers-lm-llama7bru-1 | *************************
dream-transformers-lm-llama7bru-1 | ** DEPRECATION NOTICE! **
dream-transformers-lm-llama7bru-1 | *************************
dream-transformers-lm-llama7bru-1 | THIS IMAGE IS DEPRECATED and is scheduled for DELETION.
dream-transformers-lm-llama7bru-1 | doc/support-policy.md · master · nvidia / container-images / cuda · GitLab
dream-transformers-lm-llama7bru-1 |
dream-transformers-lm-llama7bru-1 | * Serving Flask app “server”
dream-transformers-lm-llama7bru-1 | * Environment: production
dream-transformers-lm-llama7bru-1 | WARNING: This is a development server. Do not use it in a production deployment.dream-transformers-lm-llama7bru-1 | Use a production WSGI server instead.
dream-transformers-lm-llama7bru-1 | * Debug mode: off
dream-transformers-lm-llama7bru-1 | /opt/conda/lib/python3.10/site-packages/bitsandbytes/cextension.py:33: UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers, 8-bit multiplication, and GPU quantization are unavailable.
dream-transformers-lm-llama7bru-1 | warn(“The installed version of bitsandbytes was compiled without GPU support. "
dream-transformers-lm-llama7bru-1 | 2023-04-23 06:37:52,882 - server - INFO - PRETRAINED_MODEL_NAME_OR_PATH = IlyaGusev/llama_7b_ru_turbo_alpaca_lora
dream-transformers-lm-llama7bru-1 |
dream-transformers-lm-llama7bru-1 | ===================================BUG REPORT===================================
dream-transformers-lm-llama7bru-1 | Welcome to bitsandbytes. For bug reports, please run
dream-transformers-lm-llama7bru-1 |
dream-transformers-lm-llama7bru-1 | python -m bitsandbytes
dream-transformers-lm-llama7bru-1 |
dream-transformers-lm-llama7bru-1 | and submit this information together with your error trace to: Issues · TimDettmers/bitsandbytes · GitHub
dream-transformers-lm-llama7bru-1 | ================================================================================
dream-transformers-lm-llama7bru-1 | bin /opt/conda/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cpu.so
dream-transformers-lm-llama7bru-1 | CUDA SETUP: Loading binary /opt/conda/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cpu.so…
Loading checkpoint shards: 100%|██████████| 33/33 [00:06<00:00, 4.93it/s]
dream-transformers-lm-llama7bru-1 | 2023-04-23 06:39:50,287 - server - INFO - context inside generate_responses seen as: You are a SpaceX Assistant.
dream-transformers-lm-llama7bru-1 | Человек: What is the goal of SpaceX?
dream-transformers-lm-llama7bru-1 | 2023-04-23 06:39:50,309 - server - ERROR - “addmm_impl_cpu_” not implemented for ‘Half’
dream-transformers-lm-llama7bru-1 | Traceback (most recent call last):
dream-transformers-lm-llama7bru-1 | File “/src/server.py”, line 85, in
dream-transformers-lm-llama7bru-1 | example_response = generate_responses(
dream-transformers-lm-llama7bru-1 | File “/src/server.py”, line 49, in generate_responses
dream-transformers-lm-llama7bru-1 | chat_history_ids = model.generate(
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/peft/peft_model.py”, line 631, in generate
dream-transformers-lm-llama7bru-1 | outputs = self.base_model.generate(**kwargs)
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/torch/autograd/grad_mode.py”, line 27, in decorate_context
dream-transformers-lm-llama7bru-1 | return func(*args, **kwargs)
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/transformers/generation/utils.py”, line 1562, in generate
dream-transformers-lm-llama7bru-1 | return self.beam_sample(
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/transformers/generation/utils.py”, line 3136, in beam_sample
dream-transformers-lm-llama7bru-1 | outputs = self(
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py”, line 1194, in _call_impl
dream-transformers-lm-llama7bru-1 | return forward_call(*input, **kwargs)
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py”, line 687, in forward
dream-transformers-lm-llama7bru-1 | outputs = self.model(
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py”, line 1194, in _call_impl
dream-transformers-lm-llama7bru-1 | return forward_call(*input, **kwargs)
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py”, line 577, in forward
dream-transformers-lm-llama7bru-1 | layer_outputs = decoder_layer(
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py”, line 1194, in _call_impl
dream-transformers-lm-llama7bru-1 | return forward_call(*input, **kwargs)
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py”, line 292, in forward
dream-transformers-lm-llama7bru-1 | hidden_states, self_attn_weights, present_key_value = self.self_attn(
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py”, line 1194, in _call_impl
dream-transformers-lm-llama7bru-1 | return forward_call(*input, **kwargs)
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py”, line 196, in forward
dream-transformers-lm-llama7bru-1 | query_states = self.q_proj(hidden_states).view(bsz, q_len, self.num_heads, self.head_dim).transpose(1, 2)
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py”, line 1194, in call_impl
dream-transformers-lm-llama7bru-1 | return forward_call(*input, **kwargs)
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/peft/tuners/lora.py”, line 426, in forward
dream-transformers-lm-llama7bru-1 | return F.linear(x, transpose(self.weight, self.fan_in_fan_out), bias=self.bias)dream-transformers-lm-llama7bru-1 | RuntimeError: "addmm_impl_cpu” not implemented for ‘Half’
dream-transformers-lm-llama7bru-1 | Traceback (most recent call last):
dream-transformers-lm-llama7bru-1 | File “/opt/conda/bin/flask”, line 8, in
dream-transformers-lm-llama7bru-1 | sys.exit(main())
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/flask/cli.py”, line 966, in main
dream-transformers-lm-llama7bru-1 | cli.main(prog_name=“python -m flask” if as_module else None)
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/flask/cli.py”, line 586, in main
dream-transformers-lm-llama7bru-1 | return super(FlaskGroup, self).main(*args, **kwargs)
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/click/core.py”, line 1055, in main
dream-transformers-lm-llama7bru-1 | rv = self.invoke(ctx)
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/click/core.py”, line 1657, in invoke
dream-transformers-lm-llama7bru-1 | return _process_result(sub_ctx.command.invoke(sub_ctx))
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/click/core.py”, line 1404, in invoke
dream-transformers-lm-llama7bru-1 | return ctx.invoke(self.callback, **ctx.params)
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/click/core.py”, line 760, in invokedream-transformers-lm-llama7bru-1 | return __callback(*args, **kwargs)
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/click/decorators.py”, line 84, in new_func
dream-transformers-lm-llama7bru-1 | return ctx.invoke(f, obj, *args, **kwargs)
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/click/core.py”, line 760, in invokedream-transformers-lm-llama7bru-1 | return __callback(*args, **kwargs)
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/flask/cli.py”, line 848, in run_command
dream-transformers-lm-llama7bru-1 | app = DispatchingApp(info.load_app, use_eager_loading=eager_loading)
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/flask/cli.py”, line 305, in init
dream-transformers-lm-llama7bru-1 | self._load_unlocked()
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/flask/cli.py”, line 330, in _load_unlocked
dream-transformers-lm-llama7bru-1 | self._app = rv = self.loader()
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/flask/cli.py”, line 388, in load_app
dream-transformers-lm-llama7bru-1 | app = locate_app(self, import_name, name)
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/flask/cli.py”, line 240, in locate_app
dream-transformers-lm-llama7bru-1 | import(module_name)
dream-transformers-lm-llama7bru-1 | File “/src/server.py”, line 93, in
dream-transformers-lm-llama7bru-1 | raise e
dream-transformers-lm-llama7bru-1 | File “/src/server.py”, line 85, in
dream-transformers-lm-llama7bru-1 | example_response = generate_responses(
dream-transformers-lm-llama7bru-1 | File “/src/server.py”, line 49, in generate_responses
dream-transformers-lm-llama7bru-1 | chat_history_ids = model.generate(
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/peft/peft_model.py”, line 631, in generate
dream-transformers-lm-llama7bru-1 | outputs = self.base_model.generate(**kwargs)
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/torch/autograd/grad_mode.py”, line 27, in decorate_context
dream-transformers-lm-llama7bru-1 | return func(*args, **kwargs)
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/transformers/generation/utils.py”, line 1562, in generate
dream-transformers-lm-llama7bru-1 | return self.beam_sample(
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/transformers/generation/utils.py”, line 3136, in beam_sample
dream-transformers-lm-llama7bru-1 | outputs = self(
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py”, line 1194, in _call_impl
dream-transformers-lm-llama7bru-1 | return forward_call(*input, **kwargs)
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py”, line 687, in forward
dream-transformers-lm-llama7bru-1 | outputs = self.model(
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py”, line 1194, in _call_impl
dream-transformers-lm-llama7bru-1 | return forward_call(*input, **kwargs)
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py”, line 577, in forward
dream-transformers-lm-llama7bru-1 | layer_outputs = decoder_layer(
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py”, line 1194, in _call_impl
dream-transformers-lm-llama7bru-1 | return forward_call(*input, **kwargs)
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py”, line 292, in forward
dream-transformers-lm-llama7bru-1 | hidden_states, self_attn_weights, present_key_value = self.self_attn(
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py”, line 1194, in _call_impl
dream-transformers-lm-llama7bru-1 | return forward_call(*input, **kwargs)
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py”, line 196, in forward
dream-transformers-lm-llama7bru-1 | query_states = self.q_proj(hidden_states).view(bsz, q_len, self.num_heads, self.head_dim).transpose(1, 2)
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py”, line 1194, in call_impl
dream-transformers-lm-llama7bru-1 | return forward_call(*input, **kwargs)
dream-transformers-lm-llama7bru-1 | File “/opt/conda/lib/python3.10/site-packages/peft/tuners/lora.py”, line 426, in forward
dream-transformers-lm-llama7bru-1 | return F.linear(x, transpose(self.weight, self.fan_in_fan_out), bias=self.bias)dream-transformers-lm-llama7bru-1 | RuntimeError: "addmm_impl_cpu" not implemented for ‘Half’
dream-transformers-lm-llama7bru-1 exited with code 1
1 Like