fix(app): improve global opt and run cmd parsing

[abn]

This change fixes an issue introduced in #10021 that led to options provided in the form -<shortcut><value> to be incorrectly handled.

Further, it also simplifies and fixes run command processing.

Resolves: #10051

Using pipx

pipx install --suffix=@10067 'poetry @ git+https://github.com/python-poetry/poetry.git@refs/pull/10067/head'
alias poetry=poetry@10067
poetry new foobar
# run poetry in verbose mode and output hi
poetry -C ./foobar run -v echo hi
# shows echo help
poetry -C ./foobar run echo --help
# shows poetry help
poetry -P ./foobar --help run echo --help

Using a container (podman | docker)

podman run --rm -i --entrypoint bash python:latest <<EOF
set -xe
python -m pip install --disable-pip-version-check -q git+https://github.com/python-poetry/poetry.git@refs/pull/10067/head
poetry new foobar
poetry -C /foobar run -v echo hi
poetry -C /foobar run echo --help
poetry -P /foobar --help run echo --help
EOF

Summary by Sourcery

Bug Fixes:

• Fix incorrect parsing of global options when combined with the "run" command.

[sourcery-ai]

Reviewer's Guide by Sourcery

This pull request simplifies the argument parsing for the run command and fixes a bug where arguments passed after the run command were not being handled correctly. The changes involve refactoring the way options are handled and ensuring that the run command correctly interprets arguments, including those passed before and after the command itself.

Sequence diagram for the new run command argument parsing

sequenceDiagram
    participant User
    participant Application
    participant ArgParser
    participant IO

    User->>Application: poetry run command
    Application->>Application: _configure_io()
    Application->>Application: _configure_run_command()
    Application->>ArgParser: parse_known_args(tokens)
    ArgParser-->>Application: parsed_args, remaining_args
    Application->>IO: set_input(sorted_input)
    Note over Application: Reorders tokens in format:<br/>pre_run + options + run + -- + subcommand

Loading

Flow diagram for run command argument processing

flowchart TD
    A[Start] --> B[Get command tokens]
    B --> C{Has '--' separator?}
    C -->|Yes| D[Skip processing]
    C -->|No| E[Find run command index]
    E --> F[Parse tokens before run]
    F --> G[Extract subcommand]
    G --> H[Reorder tokens]
    H --> I[Create new input]
    I --> J[End]

    style C decision

Loading

File-Level Changes

Change	Details	Files
Refactor global option configuration.	Replaced _configure_custom_application_options with _configure_global_options. Removed _option_get_value method. Options are now directly accessed from the input object. Added _sort_global_options to sort and reorder global options.	src/poetry/console/application.py
Fix argument parsing for the run command.	Added _configure_run_command to handle arguments passed to the run command. The run command now correctly parses arguments passed before and after the command name. Reorders the input tokens to ensure correct parsing of the run command and its arguments.	src/poetry/console/application.py
Add tests for the run command.	Added tests to verify that the run command is not eager. Added tests to verify that the run command correctly parses arguments passed after the command name. Added tests to verify that the run command correctly parses arguments passed before the command name. Added tests to verify that the run command keeps options passed before the command.	tests/console/commands/test_run.py
Add tests for input configuration and sorting.	Added tests to verify that the application correctly configures and sorts input tokens.	tests/console/test_application.py
Remove unused file.	Removed run_argv_input.py as it is no longer needed.	src/poetry/console/io/inputs/run_argv_input.py

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time. You can also use
  this command to specify where the summary should be inserted.

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey @abn - I've reviewed your changes and they look great!

Here's what I looked at during the review

• 🟢 General issues: all looks good
• 🟢 Security: all looks good
• 🟡 Testing: 1 issue found
• 🟢 Complexity: all looks good
• 🟢 Documentation: all looks good

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[latk]

Thank you for tackling this bug – it's a really thorny issue. I'm still seeing a lot of problems, and provide detailed reproduction steps below. I don't fully understand what's going on, but hope that the test cases help.

I think this means that any command line parameter munging based on heuristics like "there is a token called run" or "we try to parse a substring of tokens" or "we can ignore all tokens after this index" is doomed to fail one way or another. The key problem is that an option like -? or --??? may or may not consume the next token as its value. It is not possible to guess correctly, one must know that specific option. Skipping over options is exactly as hard as actually parsing them, so either Cleo implements the necessary feature to stop option processing when the first argument is found, or Poetry has to faithfully reproduce the Cleo parsing logic to insert the -- in the right place.

Reproduction steps

I tried to exhaustively check different styles of passing the --project parameter, so I installed the development version of Poetry as poetry@9d9cf303, saved the following script as reproducer.sh, and invoked it like bash reproducer.sh poetry@9d9cf303:

#!/bin/bash
set -euo pipefail

poetry="${1:-poetry}"
workdir="$(mktemp -d)"
cd "$workdir"

"$poetry" new run  # create a new package called "run"
ln -sT run somepackage
eza -aT  # show directory structure

experiment() {
  if "$@"; then
    echo PASS: "$*"
  else
    echo FAIL: "$*"
    # exit 1  # uncomment this line to turn into automated test
  fi
}

# start experiments, ALL of these should just echo "hi"
for package in run somepackage; do
  experiment env --chdir=$package "$poetry" run echo hi   # test without -P/--project option

  # option before "run" command
  experiment "$poetry" -P $package run echo hi
  experiment "$poetry" -P$package run echo hi
  experiment "$poetry" --package $package run echo hi
  experiment "$poetry" --package=$package run echo hi

  # option after "run" command
  experiment "$poetry" run -P $package echo hi
  experiment "$poetry" run -P$package echo hi
  experiment "$poetry" run --package $package echo hi
  experiment "$poetry" run --package=$package echo hi
done

cd -
rm -r "$workdir"

The eza command is useful for illustrating the resulting directory layout, but not essential. I've added a somepackage → run symlink in order to easily add examples that don't run into issues with the token run.

Output of my tests:

Created package run in run
.
├── run
│  ├── pyproject.toml
│  ├── README.md
│  ├── run
│  │  └── __init__.py
│  └── tests
│     └── __init__.py
└── somepackage -> run
Creating virtualenv run-OAWMfzkB-py3.12 in /home/user/.cache/pypoetry/virtualenvs
hi
PASS: env --chdir=run poetry@9d9cf303 run echo hi
hi
PASS: poetry@9d9cf303 -P run run echo hi

Specified path '/tmp/tmp.Mq1hAQGOLf/r' is not a valid directory.
FAIL: poetry@9d9cf303 -Prun run echo hi

Poetry could not find a pyproject.toml file in /tmp/tmp.Mq1hAQGOLf or its parents
FAIL: poetry@9d9cf303 --package run run echo hi

Poetry could not find a pyproject.toml file in /tmp/tmp.Mq1hAQGOLf or its parents
FAIL: poetry@9d9cf303 --package=run run echo hi
hi
PASS: poetry@9d9cf303 run -P run echo hi

Specified path '/tmp/tmp.Mq1hAQGOLf/r' is not a valid directory.
FAIL: poetry@9d9cf303 run -Prun echo hi

Poetry could not find a pyproject.toml file in /tmp/tmp.Mq1hAQGOLf or its parents
FAIL: poetry@9d9cf303 run --package run echo hi

Poetry could not find a pyproject.toml file in /tmp/tmp.Mq1hAQGOLf or its parents
FAIL: poetry@9d9cf303 run --package=run echo hi
hi
PASS: env --chdir=somepackage poetry@9d9cf303 run echo hi
hi
PASS: poetry@9d9cf303 -P somepackage run echo hi

Specified path '/tmp/tmp.Mq1hAQGOLf/s' is not a valid directory.
FAIL: poetry@9d9cf303 -Psomepackage run echo hi

The command "somepackage" does not exist.
FAIL: poetry@9d9cf303 --package somepackage run echo hi

Poetry could not find a pyproject.toml file in /tmp/tmp.Mq1hAQGOLf or its parents
FAIL: poetry@9d9cf303 --package=somepackage run echo hi

The command "somepackage" does not exist.
FAIL: poetry@9d9cf303 run -P somepackage echo hi

Specified path '/tmp/tmp.Mq1hAQGOLf/s' is not a valid directory.
FAIL: poetry@9d9cf303 run -Psomepackage echo hi

Poetry could not find a pyproject.toml file in /tmp/tmp.Mq1hAQGOLf or its parents
FAIL: poetry@9d9cf303 run --package somepackage echo hi

Poetry could not find a pyproject.toml file in /tmp/tmp.Mq1hAQGOLf or its parents
FAIL: poetry@9d9cf303 run --package=somepackage echo hi
/tmp

As we can see, most of the invocation styles still fail. It is easier to enumerate what did work:

• env --chdir=run poetry@9d9cf303 run echo hi
• poetry@9d9cf303 -P run run echo hi
• poetry@9d9cf303 run -P run echo hi
• env --chdir=somepackage poetry@9d9cf303 run echo hi
• poetry@9d9cf303 -P somepackage run echo hi

For the failures, some seem to truncate the package path to the first letter (before turning it into an absolute path):

Specified path '/tmp/tmp.AZHrkoKFx5/r' is not a valid directory.

• poetry@9d9cf303 -Prun run echo hi
• poetry@9d9cf303 run -Prun echo hi
• poetry@9d9cf303 -Psomepackage run echo hi
• poetry@9d9cf303 run -Psomepackage echo hi

Other failures seem to ignore the --package option an stay in the $workdir, which is not a Python package:

Poetry could not find a pyproject.toml file in /tmp/tmp.AZHrkoKFx5 or its parents

• poetry@9d9cf303 --package run run echo hi
• poetry@9d9cf303 --package=run run echo hi
• poetry@9d9cf303 run --package run echo hi
• poetry@9d9cf303 run --package=run echo hi
• poetry@9d9cf303 --package=somepackage run echo hi
• poetry@9d9cf303 run --package somepackage echo hi
• poetry@9d9cf303 run --package=somepackage echo hi

On a few cases, the subcommand detection seems to fail:

The command "somepackage" does not exist.

• poetry@9d9cf303 --package somepackage run echo hi
• poetry@9d9cf303 run -P somepackage echo hi

I also repeated the test with Poetry 2.0.1, which passes 6 test cases, which is one more than in 9d9cf30. Specifically, the example poetry run -P somepackage echo hi was lost by the changes in this PR. For the examples where both versions fail, the "command not found" errors are formatted slightly differently, and examples have moved between error categories, but I'm not sure if this is material. The truncated package path error was introduced by this PR and is not present in Poetry 2.0.1.

[abn]

I'll look into it. But quite honestly I'll be happy if things are better than it was, and we should be careful of optimizing for edge cases too much. :)

[abn]

@latk I am not sure what --package=somepackage is.

[abn]

@sourcery-ai review

[sourcery-ai]

Hey @abn - I've reviewed your changes and they look great!

Here's what I looked at during the review

• 🟢 General issues: all looks good
• 🟢 Security: all looks good
• 🟢 Testing: all looks good
• 🟢 Complexity: all looks good
• 🟢 Documentation: all looks good

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[abn]

@latk thank you for detailing the issues you faced. Most of the issues you encountered where one of two things,
a bad option (--package) and a bug introduced in #10021. I have updated the branch to fix most of the issues. And I have also added some additional test coverage to to keep things happy.

@python-poetry/core I would like some eyes on this, as I had to do a first parse of the input tokens using argparse before cleo gets to process the inputs. There unfortunately were multiple issues that made global option retrieval unreliable. Hopefully this improves the status quo, particularly of the run command rather than make things worse.

[latk]

Thank you for fixing this! The idea to create an on-the-fly argparse instance is clever, and tests have been greatly improved.

I re-ran my reproducer script with commit a1cdb6e and am delighted to see that everything works now!! (Yes, I meant to use --project, not --package – thank you for spotting this mistake.)

[github-actions]

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.