Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Training notebook - File not found erroe #721

Open
is33 opened this issue Feb 1, 2025 · 5 comments
Open

Training notebook - File not found erroe #721

is33 opened this issue Feb 1, 2025 · 5 comments

Comments

@is33
Copy link

is33 commented Feb 1, 2025

Running the training notebook, extracting files goes fine. I see them in the folders. However when I run the preprocessor, I see its not picking up the files. Checking my CSV file:

wavs/1.wav|Transcript 1.
wavs/2.wav|Transscript 2.
wavs/3.wav|Transscriopt 3.

Error:

/content/piper/src/python
WARNING:preprocess:Missing wavs/1.wav
INFO:preprocess:Single speaker dataset
INFO:preprocess:Wrote dataset config
INFO:preprocess:Processing 2 utterance(s) with 2 worker(s)
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/content/piper/src/python/piper_train/preprocess.py", line 502, in
main()
File "/content/piper/src/python/piper_train/preprocess.py", line 225, in main
for utt_batch in batched(
File "/content/piper/src/python/piper_train/preprocess.py", line 491, in batched
raise ValueError("n must be at least one")
ValueError: n must be at least one

It looks like its still running:

Executing (5m 38s)
<cell line: 0>
navigate_next
system()
navigate_next
_system_compat()
navigate_next
_run_command()
navigate_next
_monitor_process()
navigate_next
_poll_process()

Not sure if its really doing anything because I do not see the GPU being used.

Any help would be appreciated

@is33
Copy link
Author

is33 commented Feb 1, 2025

update, looks like I was saving the file with BOM. Removed that but still getting:

Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/content/piper/src/python/piper_train/preprocess.py", line 502, in
main()
File "/content/piper/src/python/piper_train/preprocess.py", line 225, in main
for utt_batch in batched(
File "/content/piper/src/python/piper_train/preprocess.py", line 491, in batched
raise ValueError("n must be at least one")
ValueError: n must be at least one

and doesnt APPEAR to have any progress being made.

@is33
Copy link
Author

is33 commented Feb 1, 2025

OKAY - so not enough samples it seems.

However when I try to start training I am getting the below:

ImportError: cannot import name '_compare_version' from 'torchmetrics.utilities.imports' (/usr/local/lib/python3.11/dist-packages/torchmetrics/utilities/imports.py)

@is33
Copy link
Author

is33 commented Feb 1, 2025

From what I can tell, this stems from the install section. There are several packages that are no longer available:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
albucore 0.0.19 requires numpy>=1.24.4, but you have numpy 1.24.0 which is incompatible.
chex 0.1.88 requires numpy>=1.24.1, but you have numpy 1.24.0 which is incompatible.
imbalanced-learn 0.13.0 requires numpy<3,>=1.24.3, but you have numpy 1.24.0 which is incompatible.
torchvision 0.20.1+cu124 requires torch==2.5.1, but you have torch 1.13.0+cu117 which is incompatible.
torchaudio 2.5.1+cu124 requires torch==2.5.1, but you have torch 1.13.0+cu117 which is incompatible.
tensorflow 2.18.0 requires numpy<2.1.0,>=1.26.0, but you have numpy 1.24.0 which is incompatible.
seaborn 0.13.2 requires numpy!=1.24.0,>=1.20, but you have numpy 1.24.0 which is incompatible.
pymc 5.19.1 requires numpy>=1.25.0, but you have numpy 1.24.0 which is incompatible.
blosc2 3.0.0 requires numpy>=1.25.0, but you have numpy 1.24.0 which is incompatible.
albumentations 1.4.20 requires numpy>=1.24.4, but you have numpy 1.24.0 which is incompatible.
ERROR: Could not find a version that satisfies the requirement torchtext==0.14.0 (from versions: 0.1.1, 0.2.0, 0.2.1, 0.2.3, 0.3.1, 0.4.0, 0.5.0, 0.6.0, 0.15.1, 0.15.2, 0.16.0, 0.16.1, 0.16.2, 0.17.0, 0.17.1, 0.17.2, 0.18.0)
ERROR: No matching distribution found for torchtext==0.14.0
ERROR: Ignored the following yanked versions: 2.0.0
ERROR: Could not find a version that satisfies the requirement torchaudio==0.13.0 (from versions: 2.0.1, 2.0.2, 2.1.0, 2.1.1, 2.1.2, 2.2.0, 2.2.1, 2.2.2, 2.3.0, 2.3.1, 2.4.0, 2.4.1, 2.5.0, 2.5.1, 2.6.0)
ERROR: No matching distribution found for torchaudio==0.13.0

Updating Numpy version fixes some of these. As well as torch. But doing so breaks....something in the training step:
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/content/piper/src/python/piper_train/main.py", line 7, in
from pytorch_lightning import Trainer
File "/usr/local/lib/python3.11/dist-packages/pytorch_lightning/init.py", line 34, in
from pytorch_lightning.callbacks import Callback # noqa: E402
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/pytorch_lightning/callbacks/init.py", line 14, in
from pytorch_lightning.callbacks.callback import Callback
File "/usr/local/lib/python3.11/dist-packages/pytorch_lightning/callbacks/callback.py", line 25, in
from pytorch_lightning.utilities.types import STEP_OUTPUT
File "/usr/local/lib/python3.11/dist-packages/pytorch_lightning/utilities/init.py", line 18, in
from pytorch_lightning.utilities.apply_func import move_data_to_device # noqa: F401
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/pytorch_lightning/utilities/apply_func.py", line 29, in
from pytorch_lightning.utilities.imports import _compare_version, _TORCHTEXT_LEGACY
File "/usr/local/lib/python3.11/dist-packages/pytorch_lightning/utilities/imports.py", line 153, in
_TORCHTEXT_LEGACY: bool = _TORCHTEXT_AVAILABLE and _compare_version("torchtext", operator.lt, "0.11.0")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/pytorch_lightning/utilities/imports.py", line 71, in _compare_version
pkg = importlib.import_module(package)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/torchtext/init.py", line 18, in
from torchtext import _extension # noqa: F401
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/torchtext/_extension.py", line 64, in
_init_extension()
File "/usr/local/lib/python3.11/dist-packages/torchtext/_extension.py", line 58, in _init_extension
_load_lib("libtorchtext")
File "/usr/local/lib/python3.11/dist-packages/torchtext/_extension.py", line 50, in _load_lib
torch.ops.load_library(path)
File "/usr/local/lib/python3.11/dist-packages/torch/_ops.py", line 1357, in load_library
ctypes.CDLL(path)
File "/usr/lib/python3.11/ctypes/init.py", line 376, in init
self._handle = _dlopen(self._name, mode)
^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: /usr/local/lib/python3.11/dist-packages/torchtext/lib/libtorchtext.so: undefined symbol: _ZN5torch3jit17parseSchemaOrNameERKSs

Updated Libraries code:
`#@markdown # Install software. 📦
#@markdown ---
#@markdown ####In this cell the synthesizer and its necessary dependencies to execute the training will be installed. (this may take a while)

clone:

!git clone -q https://github.com/rmcpantoja/piper
%cd /content/piper/src/python
!wget -q "https://raw.githubusercontent.com/coqui-ai/TTS/dev/TTS/bin/resample.py"
!pip install pip==24.0
#!pip install -q -r requirements.txt
!pip install -q cython>=0.29.0 piper-phonemize==1.1.0 librosa>=0.9.2 numpy==1.24.4 onnxruntime>=1.11.0 pytorch-lightning==1.7.7 torch==2.5.1 --extra-index-url https://download.pytorch.org/whl/cu117
!pip install -q torchtext==0.18.0 torchvision==0.21.0

fixing recent compativility isswes:

!pip install -q torchaudio==2.6.0 torchmetrics==0.11.4 faster_whisper
!pip install --upgrade gdown transformers
!bash build_monotonic_align.sh

Useful vars:

use_whisper = True
print("Done!")`

Output from updated step 2:
/content/piper/src/python
Requirement already satisfied: pip==24.0 in /usr/local/lib/python3.11/dist-packages (24.0)
DEPRECATION: pytorch-lightning 1.7.7 has a non-standard dependency specifier torch>=1.9.. pip 24.1 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of pytorch-lightning or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at pypa/pip#12063
DEPRECATION: pytorch-lightning 1.7.7 has a non-standard dependency specifier torch>=1.9.
. pip 24.1 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of pytorch-lightning or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at pypa/pip#12063
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 36.9 MB/s eta 0:00:00
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.2/7.2 MB 105.0 MB/s eta 0:00:00
DEPRECATION: pytorch-lightning 1.7.7 has a non-standard dependency specifier torch>=1.9.. pip 24.1 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of pytorch-lightning or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at pypa/pip#12063
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
torchaudio 2.5.1+cu124 requires torch==2.5.1, but you have torch 2.6.0 which is incompatible.
fastai 2.7.18 requires torch<2.6,>=1.10, but you have torch 2.6.0 which is incompatible.
DEPRECATION: pytorch-lightning 1.7.7 has a non-standard dependency specifier torch>=1.9.
. pip 24.1 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of pytorch-lightning or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at pypa/pip#12063
Requirement already satisfied: gdown in /usr/local/lib/python3.11/dist-packages (5.2.0)
Requirement already satisfied: transformers in /usr/local/lib/python3.11/dist-packages (4.48.2)
Requirement already satisfied: beautifulsoup4 in /usr/local/lib/python3.11/dist-packages (from gdown) (4.12.3)
Requirement already satisfied: filelock in /usr/local/lib/python3.11/dist-packages (from gdown) (3.17.0)
Requirement already satisfied: requests[socks] in /usr/local/lib/python3.11/dist-packages (from gdown) (2.32.3)
Requirement already satisfied: tqdm in /usr/local/lib/python3.11/dist-packages (from gdown) (4.67.1)
Requirement already satisfied: huggingface-hub<1.0,>=0.24.0 in /usr/local/lib/python3.11/dist-packages (from transformers) (0.27.1)
Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.11/dist-packages (from transformers) (1.24.4)
Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.11/dist-packages (from transformers) (24.2)
Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.11/dist-packages (from transformers) (6.0.2)
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.11/dist-packages (from transformers) (2024.11.6)
Requirement already satisfied: tokenizers<0.22,>=0.21 in /usr/local/lib/python3.11/dist-packages (from transformers) (0.21.0)
Requirement already satisfied: safetensors>=0.4.1 in /usr/local/lib/python3.11/dist-packages (from transformers) (0.5.2)
Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.11/dist-packages (from huggingface-hub<1.0,>=0.24.0->transformers) (2024.10.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.11/dist-packages (from huggingface-hub<1.0,>=0.24.0->transformers) (4.12.2)
Requirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.11/dist-packages (from beautifulsoup4->gdown) (2.6)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.11/dist-packages (from requests[socks]->gdown) (3.4.1)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.11/dist-packages (from requests[socks]->gdown) (3.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.11/dist-packages (from requests[socks]->gdown) (2.3.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.11/dist-packages (from requests[socks]->gdown) (2024.12.14)
Requirement already satisfied: PySocks!=1.5.7,>=1.5.6 in /usr/local/lib/python3.11/dist-packages (from requests[socks]->gdown) (1.7.1)
DEPRECATION: pytorch-lightning 1.7.7 has a non-standard dependency specifier torch>=1.9.*. pip 24.1 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of pytorch-lightning or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at pypa/pip#12063
Done!

@is33
Copy link
Author

is33 commented Feb 1, 2025

Noting that it was updated 2 Years ago. Hope this can be addressed :)

@rajuaryan21
Copy link

I tried with many different versions of torch, torchtext, torchaudio and a few other libraries, but nothing worked. Let me knowif you foud any fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants