Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ERR Server error error="failed to load model with internal loader: grpc service not ready" ip=172.17.0.1 latency=40.041976229s method=POST status=500 url=/v1/audio/transcriptions #4857

Open
skodavalla opened this issue Feb 18, 2025 · 2 comments
Open

ERR Server error error="failed to load model with internal loader: grpc service not ready" ip=172.17.0.1 latency=40.041976229s method=POST status=500 url=/v1/audio/transcriptions #4857

skodavalla opened this issue Feb 18, 2025 · 2 comments
Labels
bug Something isn't working unconfirmed

Comments

@skodavalla
Copy link

skodavalla commented Feb 18, 2025 •
edited
Loading

LocalAI version:

localai/localai:latest-gpu-nvidia-cuda-12

Environment, CPU architecture, OS, and Version:

Docker Desktop 4.38.0 (181591) on Windows 10 Dell laptop with Nvidia GPU, 32GB RAM with nvidia-smi and nvcc installed

Describe the bug

ERR Server error error="failed to load model with internal loader: grpc service not ready" when attempting to use Whisper-1 Model by using CURL as in the example. Note no error when using CPU based container

Command output is as below:

curl http://:8081/v1/audio/transcriptions -H "Content-Type: multipart/form-data" -F file="@/d/2025_01_03_2025-01-03T15_55_49Z.mp3" -F model="whisper-1"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 2412k 100 110 100 2411k 2 61259 0:00:55 0:00:40 0:00:15 29{"error":{"code":500,"message":"failed to load model with internal loader: grpc service not ready","type":""}}

To Reproduce

Use Docker Desktop on Windows with NVidia GPU and then use localai/localai:latest-gpu-nvidia-cuda-12. Install whisper-1 model and then attempt to transcribe an mp3 file using curl command as in the notes. Error message is displayed.

Expected behavior

MP3 file needs to be transcribed and output received when using the curl command.
Logs

$ docker logs local-ai
@@@@@
Skipping rebuild
@@@@@
If you are experiencing issues with the pre-compiled builds, try setting REBUILD=true
If you are still experiencing issues with the build, try setting CMAKE_ARGS and disable the instructions set as needed:
CMAKE_ARGS="-DGGML_F16C=OFF -DGGML_AVX512=OFF -DGGML_AVX2=OFF -DGGML_FMA=OFF"
see the documentation at: https://localai.io/basics/build/index.html
Note: See also #288
@@@@@
CPU info:
model name : Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pdcm pcid sse4_1 ss
e4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti ssbd ibrs ibpb stibp fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt xsaveopt xsavec xgetbv1 xsaves md_clear flush_l1d arch_capabiliti
es
CPU: AVX found OK
CPU: AVX2 found OK
CPU: no AVX512 found
@@@@@
9:46AM INF env file found, loading environment variables from file envFile=.env
9:46AM INF Setting logging to info
9:46AM INF Starting LocalAI using 4 threads, with models path: /build/models
9:46AM INF LocalAI version: v2.26.0 (09941c0)
WARNING: failed to determine nodes: open /sys/devices/system/node: no such file or directory
WARNING: failed to read int from file: open /sys/class/drm/card0/device/numa_node: no such file or directory
WARNING: failed to determine nodes: open /sys/devices/system/node: no such file or directory
WARNING: error parsing the pci address "vgem"
9:46AM INF Preloading models from /build/models
9:46AM INF core/startup process completed!
9:46AM INF LocalAI API is listening! Please connect to the endpoint for API documentation. endpoint=http://0.0.0.0:8080
9:46AM INF Success ip=172.17.0.1 latency=2.271895ms method=GET status=200 url=/
9:46AM INF Success ip=172.17.0.1 latency="29.204µs" method=GET status=200 url=/static/assets/highlightjs.css
9:46AM INF Success ip=172.17.0.1 latency=2.876873ms method=GET status=200 url=/static/assets/highlightjs.js
9:46AM INF Success ip=172.17.0.1 latency="32.704µs" method=GET status=200 url=/static/assets/font2.css
9:46AM INF Success ip=172.17.0.1 latency="56.807µs" method=GET status=200 url=/static/general.css
9:46AM INF Success ip=172.17.0.1 latency=3.01239ms method=GET status=200 url=/static/assets/font1.css
9:46AM INF Success ip=172.17.0.1 latency=2.430415ms method=GET status=200 url=/static/assets/tw-elements.css
9:46AM INF Success ip=172.17.0.1 latency=2.018062ms method=GET status=200 url=/static/assets/fontawesome/css/fontawesome.css
9:46AM INF Success ip=172.17.0.1 latency="29.304µs" method=GET status=200 url=/static/assets/fontawesome/css/brands.css
9:46AM INF Success ip=172.17.0.1 latency="246.432µs" method=GET status=200 url=/static/assets/fontawesome/css/solid.css
9:46AM INF Success ip=172.17.0.1 latency="23.703µs" method=GET status=200 url=/static/assets/tailwindcss.js
9:46AM INF Success ip=172.17.0.1 latency="25.504µs" method=GET status=200 url=/static/assets/flowbite.min.js
9:46AM INF Success ip=172.17.0.1 latency="25.303µs" method=GET status=200 url=/static/assets/htmx.js
9:46AM INF Success ip=172.17.0.1 latency="32.204µs" method=GET status=200 url=/static/assets/tw-elements.js
9:46AM INF Success ip=172.17.0.1 latency="45.106µs" method=GET status=200 url=/static/assets/alpine.js
9:46AM INF Success ip=172.17.0.1 latency="24.703µs" method=GET status=200 url=/static/assets/marked.js
9:46AM INF Success ip=172.17.0.1 latency="23.503µs" method=GET status=200 url=/static/assets/purify.js
9:46AM INF Success ip=172.17.0.1 latency="22.203µs" method=GET status=200 url=/static/assets/fontawesome/webfonts/fa-solid-900.woff2
9:46AM INF Success ip=172.17.0.1 latency=22.907167ms method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuFuYMZg.ttf
9:46AM INF Success ip=172.17.0.1 latency=11.751323ms method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuLyfMZg.ttf
9:46AM INF Success ip=172.17.0.1 latency="27.104µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuGKYMZg.ttf
9:46AM INF Success ip=172.17.0.1 latency="36.204µs" method=GET status=200 url=/favicon.ico
9:47AM INF Success ip=172.17.0.1 latency=641.251336ms method=GET status=200 url=/browse/
9:47AM INF Success ip=172.17.0.1 latency="27.603µs" method=GET status=200 url=/static/assets/highlightjs.css
9:47AM INF Success ip=172.17.0.1 latency="25.504µs" method=GET status=200 url=/static/assets/fontawesome/css/fontawesome.css
9:47AM INF Success ip=172.17.0.1 latency="17.302µs" method=GET status=200 url=/static/assets/font2.css
9:47AM INF Success ip=172.17.0.1 latency="53.207µs" method=GET status=200 url=/static/assets/font1.css
9:47AM INF Success ip=172.17.0.1 latency="51.707µs" method=GET status=200 url=/static/assets/tw-elements.css
9:47AM INF Success ip=172.17.0.1 latency="34.604µs" method=GET status=200 url=/static/general.css
9:47AM INF Success ip=172.17.0.1 latency="26.403µs" method=GET status=200 url=/static/assets/fontawesome/css/brands.css
9:47AM INF Success ip=172.17.0.1 latency="97.012µs" method=GET status=200 url=/static/assets/fontawesome/css/solid.css
9:47AM INF Success ip=172.17.0.1 latency="18.903µs" method=GET status=200 url=/static/assets/tailwindcss.js
9:47AM INF Success ip=172.17.0.1 latency="35.905µs" method=GET status=200 url=/static/assets/flowbite.min.js
9:47AM INF Success ip=172.17.0.1 latency="17.202µs" method=GET status=200 url=/static/assets/htmx.js
9:47AM INF Success ip=172.17.0.1 latency="24.403µs" method=GET status=200 url=/static/assets/highlightjs.js
9:47AM INF Success ip=172.17.0.1 latency="32.904µs" method=GET status=200 url=/static/assets/alpine.js
9:47AM INF Success ip=172.17.0.1 latency="35.605µs" method=GET status=200 url=/static/assets/marked.js
9:47AM INF Success ip=172.17.0.1 latency="24.103µs" method=GET status=200 url=/static/assets/purify.js
9:47AM INF Success ip=172.17.0.1 latency="24.304µs" method=GET status=200 url=/static/assets/fontawesome/webfonts/fa-brands-400.woff2
9:47AM INF Success ip=172.17.0.1 latency="27.704µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuFuYMZg.ttf
9:47AM INF Success ip=172.17.0.1 latency="32.404µs" method=GET status=200 url=/static/assets/fontawesome/webfonts/fa-solid-900.woff2
9:47AM INF Success ip=172.17.0.1 latency="54.707µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuGKYMZg.ttf
9:47AM INF Success ip=172.17.0.1 latency="22.303µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuLyfMZg.ttf
9:47AM INF Success ip=172.17.0.1 latency="27.404µs" method=GET status=200 url=/static/assets/tw-elements.js
9:47AM INF Success ip=172.17.0.1 latency=72.744914ms method=POST status=200 url=/browse/search/models
9:47AM INF Success ip=127.0.0.1 latency="13.101µs" method=GET status=200 url=/readyz
9:47AM INF Success ip=172.17.0.1 latency="267.435µs" method=POST status=200 url=/browse/install/model/localai@whisper-1
9:47AM INF Downloading "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-base.bin"
9:47AM INF Success ip=172.17.0.1 latency="982.325µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:47AM INF Success ip=172.17.0.1 latency="38.505µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:47AM INF Success ip=172.17.0.1 latency="37.005µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:47AM INF Success ip=172.17.0.1 latency="54.107µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:47AM INF Success ip=172.17.0.1 latency="32.404µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:47AM INF Success ip=172.17.0.1 latency="48.106µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:47AM INF Success ip=172.17.0.1 latency="39.004µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:47AM INF Downloading /build/models/ggml-whisper-base.bin.partial: 19.8 MiB/141.1 MiB (14.03%) ETA: 30.953888622s
9:47AM INF Success ip=172.17.0.1 latency="43.105µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:47AM INF Success ip=172.17.0.1 latency="32.104µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:47AM INF Success ip=172.17.0.1 latency="31.404µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:47AM INF Success ip=172.17.0.1 latency="32.203µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:47AM INF Success ip=172.17.0.1 latency="61.208µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:47AM INF Success ip=172.17.0.1 latency="35.204µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:47AM INF Success ip=172.17.0.1 latency="32.504µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:47AM INF Success ip=172.17.0.1 latency="33.904µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:47AM INF Downloading /build/models/ggml-whisper-base.bin.partial: 44.4 MiB/141.1 MiB (31.48%) ETA: 21.91296937s
9:47AM INF Success ip=172.17.0.1 latency="32.104µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:47AM INF Success ip=172.17.0.1 latency="82.01µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:47AM INF Success ip=172.17.0.1 latency="69.109µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:47AM INF Success ip=172.17.0.1 latency="37.203µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:47AM INF Success ip=172.17.0.1 latency="45.603µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:47AM INF Success ip=172.17.0.1 latency="69.606µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:47AM INF Success ip=172.17.0.1 latency="31.903µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:47AM INF Success ip=172.17.0.1 latency="43.003µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:47AM INF Success ip=172.17.0.1 latency="39.803µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:47AM INF Downloading /build/models/ggml-whisper-base.bin.partial: 68.1 MiB/141.1 MiB (48.29%) ETA: 16.136450353s
9:47AM INF Success ip=172.17.0.1 latency="31.402µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:48AM INF Success ip=172.17.0.1 latency="43.603µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:48AM INF Success ip=172.17.0.1 latency="30.702µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:48AM INF Success ip=172.17.0.1 latency="44.504µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:48AM INF Success ip=172.17.0.1 latency="49.504µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:48AM INF Success ip=172.17.0.1 latency="40.503µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:48AM INF Success ip=172.17.0.1 latency="30.802µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:48AM INF Success ip=172.17.0.1 latency=1.630905ms method=GET status=200 url=/
9:48AM INF Success ip=172.17.0.1 latency="35.2µs" method=GET status=200 url=/static/assets/highlightjs.js
9:48AM INF Success ip=172.17.0.1 latency="63.301µs" method=GET status=200 url=/static/assets/highlightjs.css
9:48AM INF Success ip=172.17.0.1 latency="78.1µs" method=GET status=200 url=/static/general.css
9:48AM INF Success ip=172.17.0.1 latency="160.201µs" method=GET status=200 url=/static/assets/font1.css
9:48AM INF Success ip=172.17.0.1 latency="60.8µs" method=GET status=200 url=/static/assets/font2.css
9:48AM INF Success ip=172.17.0.1 latency="62µs" method=GET status=200 url=/static/assets/tw-elements.css
9:48AM INF Success ip=172.17.0.1 latency="40.3µs" method=GET status=200 url=/static/assets/fontawesome/css/brands.css
9:48AM INF Success ip=172.17.0.1 latency="49.6µs" method=GET status=200 url=/static/assets/fontawesome/css/fontawesome.css
9:48AM INF Success ip=172.17.0.1 latency="53.001µs" method=GET status=200 url=/static/assets/fontawesome/css/solid.css
9:48AM INF Success ip=172.17.0.1 latency="32.7µs" method=GET status=200 url=/static/assets/tailwindcss.js
9:48AM INF Success ip=172.17.0.1 latency="26.3µs" method=GET status=200 url=/static/assets/flowbite.min.js
9:48AM INF Success ip=172.17.0.1 latency="96.5µs" method=GET status=200 url=/static/assets/htmx.js
9:48AM INF Success ip=172.17.0.1 latency="25.4µs" method=GET status=200 url=/static/assets/tw-elements.js
9:48AM INF Downloading /build/models/ggml-whisper-base.bin.partial: 91.5 MiB/141.1 MiB (64.88%) ETA: 10.865532992s
9:48AM INF Success ip=172.17.0.1 latency="59.2µs" method=GET status=200 url=/static/assets/alpine.js
9:48AM INF Success ip=172.17.0.1 latency="25.8µs" method=GET status=200 url=/static/assets/marked.js
9:48AM INF Success ip=172.17.0.1 latency="38µs" method=GET status=200 url=/static/assets/purify.js
9:48AM INF Success ip=172.17.0.1 latency="35.4µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuFuYMZg.ttf
9:48AM INF Success ip=172.17.0.1 latency="32.2µs" method=GET status=200 url=/static/assets/fontawesome/webfonts/fa-solid-900.woff2
9:48AM INF Success ip=172.17.0.1 latency="55.3µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuLyfMZg.ttf
9:48AM INF Success ip=172.17.0.1 latency="28µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuGKYMZg.ttf
9:48AM INF Success ip=172.17.0.1 latency="45.1µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:48AM INF Success ip=172.17.0.1 latency="48.901µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:48AM INF Success ip=172.17.0.1 latency="37.1µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:48AM INF Success ip=172.17.0.1 latency="29.1µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:48AM INF Success ip=172.17.0.1 latency="34.2µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:48AM INF Success ip=172.17.0.1 latency="32.3µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:48AM INF Success ip=172.17.0.1 latency="32.2µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:48AM INF Downloading /build/models/ggml-whisper-base.bin.partial: 114.7 MiB/141.1 MiB (81.26%) ETA: 5.825357782s
9:48AM INF Success ip=172.17.0.1 latency="51.807µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:48AM INF Success ip=172.17.0.1 latency="33.205µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:48AM INF Success ip=172.17.0.1 latency="80.91µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:48AM INF Success ip=172.17.0.1 latency="33.404µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:48AM INF Success ip=172.17.0.1 latency="67.409µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:48AM INF Success ip=172.17.0.1 latency="32.204µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:48AM INF Success ip=172.17.0.1 latency="31.704µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:48AM INF Success ip=172.17.0.1 latency="42.406µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:48AM INF Success ip=172.17.0.1 latency="35.605µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:48AM INF Downloading /build/models/ggml-whisper-base.bin.partial: 139.4 MiB/141.1 MiB (98.79%) ETA: 369.246932ms
9:48AM INF File "/build/models/ggml-whisper-base.bin" downloaded and verified
9:48AM INF Preloading models from /build/models

Model name: whisper-1

9:48AM INF Success ip=172.17.0.1 latency="29.605µs" method=GET status=200 url=/browse/job/progress/3c4f3af6-ed14-11ef-829a-0242ac110003
9:48AM INF Success ip=172.17.0.1 latency="64.108µs" method=GET status=200 url=/browse/job/3c4f3af6-ed14-11ef-829a-0242ac110003
9:48AM INF Success ip=172.17.0.1 latency="385.453µs" method=GET status=200 url=/
9:48AM INF Success ip=172.17.0.1 latency="26.804µs" method=GET status=200 url=/static/assets/highlightjs.css
9:48AM INF Success ip=172.17.0.1 latency="23.204µs" method=GET status=200 url=/static/assets/highlightjs.js
9:48AM INF Success ip=172.17.0.1 latency="23.103µs" method=GET status=200 url=/static/assets/font2.css
9:48AM INF Success ip=172.17.0.1 latency="19.702µs" method=GET status=200 url=/static/assets/font1.css
9:48AM INF Success ip=172.17.0.1 latency="15.702µs" method=GET status=200 url=/static/general.css
9:48AM INF Success ip=172.17.0.1 latency="26.404µs" method=GET status=200 url=/static/assets/tw-elements.css
9:48AM INF Success ip=172.17.0.1 latency="32.705µs" method=GET status=200 url=/static/assets/fontawesome/css/fontawesome.css
9:48AM INF Success ip=172.17.0.1 latency="64.709µs" method=GET status=200 url=/static/assets/fontawesome/css/solid.css
9:48AM INF Success ip=172.17.0.1 latency="210.629µs" method=GET status=200 url=/static/assets/tailwindcss.js
9:48AM INF Success ip=172.17.0.1 latency="28.004µs" method=GET status=200 url=/static/assets/fontawesome/css/brands.css
9:48AM INF Success ip=172.17.0.1 latency="37.005µs" method=GET status=200 url=/static/assets/flowbite.min.js
9:48AM INF Success ip=172.17.0.1 latency="39.805µs" method=GET status=200 url=/static/assets/htmx.js
9:48AM INF Success ip=172.17.0.1 latency="25.403µs" method=GET status=200 url=/static/assets/tw-elements.js
9:48AM INF Success ip=172.17.0.1 latency="24.404µs" method=GET status=200 url=/static/assets/alpine.js
9:48AM INF Success ip=172.17.0.1 latency="24.603µs" method=GET status=200 url=/static/assets/marked.js
9:48AM INF Success ip=172.17.0.1 latency="23.103µs" method=GET status=200 url=/static/assets/purify.js
9:48AM INF Success ip=172.17.0.1 latency="37.806µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuFuYMZg.ttf
9:48AM INF Success ip=172.17.0.1 latency="34.805µs" method=GET status=200 url=/static/assets/fontawesome/webfonts/fa-solid-900.woff2
9:48AM INF Success ip=172.17.0.1 latency="39.205µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuLyfMZg.ttf
9:48AM INF Success ip=172.17.0.1 latency="30.204µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuGKYMZg.ttf
9:48AM INF Success ip=172.17.0.1 latency="333.941µs" method=GET status=200 url=/
9:48AM INF Success ip=172.17.0.1 latency="26.803µs" method=GET status=200 url=/static/assets/highlightjs.css
9:48AM INF Success ip=172.17.0.1 latency="37.105µs" method=GET status=200 url=/static/assets/highlightjs.js
9:48AM INF Success ip=172.17.0.1 latency="14.602µs" method=GET status=200 url=/static/general.css
9:48AM INF Success ip=172.17.0.1 latency="32.604µs" method=GET status=200 url=/static/assets/font1.css
9:48AM INF Success ip=172.17.0.1 latency="44.306µs" method=GET status=200 url=/static/assets/font2.css
9:48AM INF Success ip=172.17.0.1 latency="28.103µs" method=GET status=200 url=/static/assets/fontawesome/css/fontawesome.css
9:48AM INF Success ip=172.17.0.1 latency="36.504µs" method=GET status=200 url=/static/assets/tw-elements.css
9:48AM INF Success ip=172.17.0.1 latency="26.503µs" method=GET status=200 url=/static/assets/fontawesome/css/brands.css
9:48AM INF Success ip=172.17.0.1 latency="21.703µs" method=GET status=200 url=/static/assets/fontawesome/css/solid.css
9:48AM INF Success ip=172.17.0.1 latency="20.803µs" method=GET status=200 url=/static/assets/tailwindcss.js
9:48AM INF Success ip=172.17.0.1 latency="25.503µs" method=GET status=200 url=/static/assets/flowbite.min.js
9:48AM INF Success ip=172.17.0.1 latency="29.904µs" method=GET status=200 url=/static/assets/tw-elements.js
9:48AM INF Success ip=172.17.0.1 latency="168.421µs" method=GET status=200 url=/static/assets/htmx.js
9:48AM INF Success ip=172.17.0.1 latency="24.503µs" method=GET status=200 url=/static/assets/alpine.js
9:48AM INF Success ip=172.17.0.1 latency="25.203µs" method=GET status=200 url=/static/assets/marked.js
9:48AM INF Success ip=172.17.0.1 latency="49.207µs" method=GET status=200 url=/static/assets/purify.js
9:48AM INF Success ip=172.17.0.1 latency="26.303µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuFuYMZg.ttf
9:48AM INF Success ip=172.17.0.1 latency="25.803µs" method=GET status=200 url=/static/assets/fontawesome/webfonts/fa-solid-900.woff2
9:48AM INF Success ip=172.17.0.1 latency="43.306µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuLyfMZg.ttf
9:48AM INF Success ip=172.17.0.1 latency="50.506µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuGKYMZg.ttf
9:48AM INF Success ip=127.0.0.1 latency="13.101µs" method=GET status=200 url=/readyz
9:49AM INF Success ip=127.0.0.1 latency="737.874µs" method=GET status=200 url=/readyz
9:50AM INF Success ip=127.0.0.1 latency="46.104µs" method=GET status=200 url=/readyz
9:51AM INF Success ip=127.0.0.1 latency="93.709µs" method=GET status=200 url=/readyz
9:52AM INF Success ip=127.0.0.1 latency="31.003µs" method=GET status=200 url=/readyz
9:53AM ERR Server error error="there is no uploaded file associated with the given key" ip=172.17.0.1 latency=3.088443ms method=POST status=500 url=/v1/audio/transcriptions
9:53AM INF Success ip=127.0.0.1 latency="43.604µs" method=GET status=200 url=/readyz
9:54AM INF Success ip=172.17.0.1 latency="365.535µs" method=GET status=200 url=/
9:54AM INF Success ip=172.17.0.1 latency="29.302µs" method=GET status=200 url=/static/assets/highlightjs.css
9:54AM INF Success ip=172.17.0.1 latency="94.709µs" method=GET status=200 url=/static/assets/highlightjs.js
9:54AM INF Success ip=172.17.0.1 latency="64.706µs" method=GET status=200 url=/static/general.css
9:54AM INF Success ip=172.17.0.1 latency="37.803µs" method=GET status=200 url=/static/assets/font1.css
9:54AM INF Success ip=172.17.0.1 latency="34.803µs" method=GET status=200 url=/static/assets/font2.css
9:54AM INF Success ip=172.17.0.1 latency="22.702µs" method=GET status=200 url=/static/assets/tw-elements.css
9:54AM INF Success ip=172.17.0.1 latency="100.109µs" method=GET status=200 url=/static/assets/fontawesome/css/fontawesome.css
9:54AM INF Success ip=172.17.0.1 latency="25.802µs" method=GET status=200 url=/static/assets/fontawesome/css/solid.css
9:54AM INF Success ip=172.17.0.1 latency="29.702µs" method=GET status=200 url=/static/assets/fontawesome/css/brands.css
9:54AM INF Success ip=172.17.0.1 latency="25.502µs" method=GET status=200 url=/static/assets/tailwindcss.js
9:54AM INF Success ip=172.17.0.1 latency="27.403µs" method=GET status=200 url=/static/assets/flowbite.min.js
9:54AM INF Success ip=172.17.0.1 latency="40.403µs" method=GET status=200 url=/static/assets/htmx.js
9:54AM INF Success ip=172.17.0.1 latency="24.102µs" method=GET status=200 url=/static/assets/tw-elements.js
9:54AM INF Success ip=172.17.0.1 latency="25.102µs" method=GET status=200 url=/static/assets/alpine.js
9:54AM INF Success ip=172.17.0.1 latency="42.104µs" method=GET status=200 url=/static/assets/marked.js
9:54AM INF Success ip=172.17.0.1 latency="166.216µs" method=GET status=200 url=/static/assets/purify.js
9:54AM INF Success ip=172.17.0.1 latency="29.103µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuFuYMZg.ttf
9:54AM INF Success ip=172.17.0.1 latency="27.603µs" method=GET status=200 url=/static/assets/fontawesome/webfonts/fa-solid-900.woff2
9:54AM INF Success ip=172.17.0.1 latency="30.802µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuLyfMZg.ttf
9:54AM INF Success ip=172.17.0.1 latency="32.803µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuGKYMZg.ttf
9:54AM INF Success ip=172.17.0.1 latency="255.817µs" method=GET status=200 url=/talk/
9:54AM INF Success ip=172.17.0.1 latency="27.402µs" method=GET status=200 url=/static/assets/highlightjs.css
9:54AM INF Success ip=172.17.0.1 latency="31.102µs" method=GET status=200 url=/static/assets/highlightjs.js
9:54AM INF Success ip=172.17.0.1 latency="23.302µs" method=GET status=200 url=/static/general.css
9:54AM INF Success ip=172.17.0.1 latency="22.101µs" method=GET status=200 url=/static/assets/font1.css
9:54AM INF Success ip=172.17.0.1 latency="21.601µs" method=GET status=200 url=/static/assets/font2.css
9:54AM INF Success ip=172.17.0.1 latency="26.002µs" method=GET status=200 url=/static/assets/tw-elements.css
9:54AM INF Success ip=172.17.0.1 latency="26.301µs" method=GET status=200 url=/static/assets/fontawesome/css/fontawesome.css
9:54AM INF Success ip=172.17.0.1 latency="36.402µs" method=GET status=200 url=/static/assets/tailwindcss.js
9:54AM INF Success ip=172.17.0.1 latency="18.101µs" method=GET status=200 url=/static/assets/fontawesome/css/solid.css
9:54AM INF Success ip=172.17.0.1 latency="110.908µs" method=GET status=200 url=/static/assets/fontawesome/css/brands.css
9:54AM INF Success ip=172.17.0.1 latency="36.902µs" method=GET status=200 url=/static/assets/flowbite.min.js
9:54AM INF Success ip=172.17.0.1 latency="18.901µs" method=GET status=200 url=/static/assets/htmx.js
9:54AM INF Success ip=172.17.0.1 latency="29.302µs" method=GET status=200 url=/static/assets/alpine.js
9:54AM INF Success ip=172.17.0.1 latency="25.202µs" method=GET status=200 url=/static/talk.js
9:54AM INF Success ip=172.17.0.1 latency="26.202µs" method=GET status=200 url=/static/assets/marked.js
9:54AM INF Success ip=172.17.0.1 latency="21.801µs" method=GET status=200 url=/static/assets/purify.js
9:54AM INF Success ip=172.17.0.1 latency="29.002µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuFuYMZg.ttf
9:54AM INF Success ip=172.17.0.1 latency="22.301µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuLyfMZg.ttf
9:54AM INF Success ip=172.17.0.1 latency="106.807µs" method=GET status=200 url=/static/assets/fontawesome/webfonts/fa-solid-900.woff2
9:54AM INF Success ip=127.0.0.1 latency="14.801µs" method=GET status=200 url=/readyz
9:54AM INF Success ip=172.17.0.1 latency="27.303µs" method=GET status=301 url=/swagger/
9:54AM INF Success ip=172.17.0.1 latency=1.616348ms method=GET status=200 url=/swagger/index.html
9:54AM INF Success ip=172.17.0.1 latency="44.104µs" method=GET status=200 url=/swagger/swagger-ui-bundle.js
9:54AM INF Success ip=172.17.0.1 latency="27.302µs" method=GET status=200 url=/swagger/swagger-ui.css
9:54AM INF Success ip=172.17.0.1 latency="173.916µs" method=GET status=200 url=/swagger/swagger-ui-standalone-preset.js
9:54AM INF Success ip=172.17.0.1 latency=2.359916ms method=GET status=200 url=/swagger/doc.json
9:54AM INF Success ip=172.17.0.1 latency="29.303µs" method=GET status=200 url=/swagger/favicon-32x32.png
9:55AM INF Success ip=127.0.0.1 latency="52.302µs" method=GET status=200 url=/readyz
9:56AM ERR Server error error="there is no uploaded file associated with the given key" ip=172.17.0.1 latency="277.944µs" method=POST status=500 url=/v1/audio/transcriptions
9:56AM INF Success ip=127.0.0.1 latency="30.302µs" method=GET status=200 url=/readyz
9:57AM INF Success ip=127.0.0.1 latency="34.702µs" method=GET status=200 url=/readyz
9:58AM INF Success ip=127.0.0.1 latency="53.404µs" method=GET status=200 url=/readyz
9:58AM INF Success ip=172.17.0.1 latency="61.405µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuGKYMZg.ttf
9:59AM INF Success ip=172.17.0.1 latency="162.012µs" method=GET status=200 url=/swagger/index.html
9:59AM INF Success ip=172.17.0.1 latency="57.104µs" method=GET status=200 url=/swagger/swagger-ui.css
9:59AM INF Success ip=172.17.0.1 latency="37.402µs" method=GET status=200 url=/swagger/swagger-ui-bundle.js
9:59AM INF Success ip=172.17.0.1 latency="126.109µs" method=GET status=200 url=/swagger/swagger-ui-standalone-preset.js
9:59AM INF Success ip=172.17.0.1 latency="202.115µs" method=GET status=200 url=/swagger/doc.json
9:59AM ERR Server error error="there is no uploaded file associated with the given key" ip=172.17.0.1 latency="226.517µs" method=POST status=500 url=/v1/audio/transcriptions
9:59AM INF Success ip=172.17.0.1 latency="258.919µs" method=GET status=200 url=/
9:59AM INF Success ip=172.17.0.1 latency="27.602µs" method=GET status=200 url=/static/assets/highlightjs.css
9:59AM INF Success ip=172.17.0.1 latency="20.002µs" method=GET status=200 url=/static/assets/font2.css
9:59AM INF Success ip=172.17.0.1 latency="39.603µs" method=GET status=200 url=/static/assets/tw-elements.css
9:59AM INF Success ip=172.17.0.1 latency="61.405µs" method=GET status=200 url=/static/assets/fontawesome/css/solid.css
9:59AM INF Success ip=172.17.0.1 latency="46.704µs" method=GET status=200 url=/static/assets/fontawesome/css/fontawesome.css
9:59AM INF Success ip=172.17.0.1 latency="32.203µs" method=GET status=200 url=/static/assets/fontawesome/css/brands.css
9:59AM INF Success ip=172.17.0.1 latency="42.803µs" method=GET status=200 url=/static/general.css
9:59AM INF Success ip=172.17.0.1 latency="43.003µs" method=GET status=200 url=/static/assets/font1.css
9:59AM INF Success ip=172.17.0.1 latency="29.102µs" method=GET status=200 url=/static/assets/tailwindcss.js
9:59AM INF Success ip=172.17.0.1 latency="27.002µs" method=GET status=200 url=/static/assets/flowbite.min.js
9:59AM INF Success ip=172.17.0.1 latency="26.402µs" method=GET status=200 url=/static/assets/htmx.js
9:59AM INF Success ip=172.17.0.1 latency="64.305µs" method=GET status=200 url=/static/assets/highlightjs.js
9:59AM INF Success ip=172.17.0.1 latency="32.903µs" method=GET status=200 url=/static/assets/tw-elements.js
9:59AM INF Success ip=172.17.0.1 latency="39.803µs" method=GET status=200 url=/static/assets/alpine.js
9:59AM INF Success ip=172.17.0.1 latency="18.901µs" method=GET status=200 url=/static/assets/marked.js
9:59AM INF Success ip=172.17.0.1 latency="31.802µs" method=GET status=200 url=/static/assets/purify.js
9:59AM INF Success ip=172.17.0.1 latency="24.602µs" method=GET status=200 url=/static/assets/fontawesome/webfonts/fa-solid-900.woff2
9:59AM INF Success ip=172.17.0.1 latency="24.402µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuFuYMZg.ttf
9:59AM INF Success ip=172.17.0.1 latency="25.302µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuLyfMZg.ttf
9:59AM INF Success ip=172.17.0.1 latency="29.402µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuGKYMZg.ttf
9:59AM INF Success ip=172.17.0.1 latency=584.36306ms method=GET status=200 url=/browse/
9:59AM INF Success ip=172.17.0.1 latency="41.697µs" method=GET status=200 url=/static/assets/highlightjs.css
9:59AM INF Success ip=172.17.0.1 latency="58.096µs" method=GET status=200 url=/static/general.css
9:59AM INF Success ip=172.17.0.1 latency="62.995µs" method=GET status=200 url=/static/assets/tw-elements.css
9:59AM INF Success ip=172.17.0.1 latency="72.396µs" method=GET status=200 url=/static/assets/font1.css
9:59AM INF Success ip=172.17.0.1 latency="16.799µs" method=GET status=200 url=/static/assets/fontawesome/css/fontawesome.css
9:59AM INF Success ip=172.17.0.1 latency="33.398µs" method=GET status=200 url=/static/assets/font2.css
9:59AM INF Success ip=172.17.0.1 latency="39.898µs" method=GET status=200 url=/static/assets/fontawesome/css/brands.css
9:59AM INF Success ip=172.17.0.1 latency="24.998µs" method=GET status=200 url=/static/assets/tailwindcss.js
9:59AM INF Success ip=172.17.0.1 latency="16.499µs" method=GET status=200 url=/static/assets/fontawesome/css/solid.css
9:59AM INF Success ip=172.17.0.1 latency="66.496µs" method=GET status=200 url=/static/assets/flowbite.min.js
9:59AM INF Success ip=172.17.0.1 latency="25.598µs" method=GET status=200 url=/static/assets/htmx.js
9:59AM INF Success ip=172.17.0.1 latency="41.298µs" method=GET status=200 url=/static/assets/highlightjs.js
9:59AM INF Success ip=172.17.0.1 latency="25.598µs" method=GET status=200 url=/static/assets/alpine.js
9:59AM INF Success ip=172.17.0.1 latency="25.898µs" method=GET status=200 url=/static/assets/tw-elements.js
9:59AM INF Success ip=172.17.0.1 latency="25.299µs" method=GET status=200 url=/static/assets/marked.js
9:59AM INF Success ip=172.17.0.1 latency="23.898µs" method=GET status=200 url=/static/assets/purify.js
9:59AM INF Success ip=172.17.0.1 latency="29.298µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuFuYMZg.ttf
9:59AM INF Success ip=172.17.0.1 latency="28.298µs" method=GET status=200 url=/static/assets/fontawesome/webfonts/fa-solid-900.woff2
9:59AM INF Success ip=172.17.0.1 latency="37.297µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuLyfMZg.ttf
9:59AM INF Success ip=172.17.0.1 latency="28.298µs" method=GET status=200 url=/static/assets/UcCO3FwrK3iLTeHuS_fvQtMwCp50KnMw2boKoduKmMEVuGKYMZg.ttf
9:59AM INF Success ip=172.17.0.1 latency="24.798µs" method=GET status=200 url=/static/assets/fontawesome/webfonts/fa-brands-400.woff2
9:59AM INF Success ip=127.0.0.1 latency="70.706µs" method=GET status=200 url=/readyz
9:59AM INF Success ip=172.17.0.1 latency=76.705154ms method=POST status=200 url=/browse/search/models
9:59AM INF Success ip=172.17.0.1 latency=64.648509ms method=POST status=200 url=/browse/search/models
10:00AM INF Success ip=127.0.0.1 latency="49.104µs" method=GET status=200 url=/readyz
10:01AM INF Success ip=127.0.0.1 latency="57.404µs" method=GET status=200 url=/readyz
10:02AM INF Success ip=127.0.0.1 latency="34.702µs" method=GET status=200 url=/readyz
10:02AM INF BackendLoader starting backend=whisper modelID=whisper-1 o.model=ggml-whisper-base.bin

> ****10:03AM ERR failed starting/connecting to the gRPC service error="rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing: dial tcp 127.0.0.1:39853: connect: connection refused\""
> 10:03AM ERR Server error error="failed to load model with internal loader: grpc service not ready" ip=172.17.0.1 latency=40.041976229s****

method=POST status=500 url=/v1/audio/transcriptions
10:03AM INF Success ip=127.0.0.1 latency="12µs" method=GET status=200 url=/readyz
10:04AM INF Success ip=127.0.0.1 latency="12.3µs" method=GET status=200 url=/readyz
10:05AM INF Success ip=127.0.0.1 latency="12.6µs" method=GET status=200 url=/readyz
10:06AM INF Success ip=127.0.0.1 latency="16.201µs" method=GET status=200 url=/readyz
10:07AM INF Success ip=127.0.0.1 latency="19.501µs" method=GET status=200 url=/readyz
10:08AM INF Success ip=127.0.0.1 latency="11.801µs" method=GET status=200 url=/readyz
10:09AM INF Success ip=127.0.0.1 latency="14.001µs" method=GET status=200 url=/readyz

Additional context

Same steps works for CPU based container without error
@skodavalla skodavalla added bug Something isn't working unconfirmed labels Feb 18, 2025
@skodavalla
Copy link
Author

Please let me know if any additional info is needed. Also the container size is more than 40GB can it be reduced?

@Hello-World-Traveler
Copy link

Connection refused

Ensure that the firewall is not blocking incoming connections to the server and check configuration files. You can also change the port if you want too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working unconfirmed
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@skodavalla @Hello-World-Traveler