Add --preserve-pip-download-log flag

[telekid]

Resolves #21979.

[sureshjoshi]

Note: As per #21979 - telekid is @jake-normal

[tgolsson]

A small note in the release notes since this might be a "good to know" despite the small size.

[sureshjoshi]

Oh yeah, I completely forgot about this one - thought it was merged.
@jake-normal do you know how we can test this locally? Where does the file end up if I want to see it?

If I can confirm it works locally, I can write some release notes and push it forward.

[benjyw]

Thanks @jake-normal ! Seconding the question of documenting where the download log ends up when this is run in a process sandbox, and maybe we could debug-log its location.

[sureshjoshi]

Per John in the accompanying issue (me asking about performance):

No perf hit. Pex needs to capture the Pip log in full already to create a universal lock. The flag with no arg just skips a tempdir teardown.

So, this makes me think it's part of the pex created folders - I'll take a look shortly when I get to my computer.

[sureshjoshi]

pex-tool/pex#2536

Although --pip-log is just an alias for the pre-existing
--preserve-pip-download-log option, the option gains the ability to
accept an optional log path value. In addition to this pro-active means
of starting a debuggable Pex session with Pip, the log is also made more
useful in the face of a multi-target resolve by serializing the log on
targets and prefixing log lines per target. The log now also includes
the compatibility tags calculated via pip -v debug ... for any
abbreviated --platforms used in the resolve.

Sounds like stderr by default?

[sureshjoshi]

I'm not positive, but I think we should also add a log location, so that it ends up in the sandbox.

  "--preserve-pip-download-log",
  "pip-sj.log",

⏺ T/pants-sandbox-RknX7O % tree
.
|____pex
|____lock.json
|____.python-build-standalone
|____pip-sj.log
|______run.sh
|____.tmp
|____.cache
| |____pex_root

[sureshjoshi]

Turns out I have absolutely no idea how to use Github to update a PR - instead of going to this PR, using gh pushed it to a new branch in this repo: main...preserve-pip-download-log

I hate Github sometimes.

[tgolsson]

@sureshjoshi might be easier to make a change suggestion? :)