Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reuse_sandbox_directories, experimental_sandbox_async_tree_delete_idle_threads may lead to Permission denied when test case creates a 0-bit directory #25351

Open
rbeasley-avgo opened this issue Feb 21, 2025 · 2 comments
Labels
awaiting-user-response Awaiting a response from the author team-Local-Exec Issues and PRs for the Execution (Local) team type: bug untriaged

Comments

@rbeasley-avgo
Copy link

Description of the bug:

I'm upgrading our workspace from Bazel 7.4.0 to Bazel 8.1.0. My bazel test commands are quick to fail at random. Error messages mention Permission denied when trying to asynchronously clean up (or reuse?) sandbox trees.

It looks like one of our tests is misbehaving by creating a directory named TestDir-new at the root of its working directory (as opposed to beneath TEST_TMPDIR) w/ a mode of 0 (no read/write/execute bits).

ERROR: /path/to/WORKSPACE/package/subpackage/BUILD.bazel:88:10: Testing //package/subpackage:MyTest failed: Could not copy inputs into sandbox: /dev/shm/bazel-sandbox.9bc548904a956f24e976880d4b8012926278461cedcf42d6e9bf90d26f22ef55/linux-sandbox/15352/execroot/WORKSPACE/bazel-out/k8-dbg-obj/bin/package/subpackage/MyTest.runfiles/WORKSPACE/TestDir-new -> /dev/shm/bazel-sandbox.9bc548904a956f24e976880d4b8012926278461cedcf42d6e9bf90d26f22ef55/_moved_trash_dir/20057 (Permission denied)

Before I discovered that TestDir-new came from our code, I was flailing around toggling flags, and I was able to mitigate this with either of the following:

  • --experimental_sandbox_async_tree_delete_idle_threads=0
  • --noreuse_sandbox_directories

While this can be chalked up to user error, I'm still filing this only because we only encountered this failure mode when both of these features were enabled together. (At the very least, any other user encountering the same problem may find this report and infer an appropriate local fix.)

Which category does this issue belong to?

Local Execution

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

I'm not entirely sure. Best guess:

  1. Start with a large project where reuse_sandbox_directories and experimental_sandbox_async_tree_delete_idle_threads are both exercised.
  2. Add a test case which does the dumb thing of creating a directory foo beneath its current working directory w/ permissions of 0o000.
  3. Invoke Bazel to test "everything" in the workspace and watch the fireworks fly?

Which operating system are you running Bazel on?

Linux

What is the output of bazel info release?

release 8.1.0-vmware

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?

7e086c215c3cbaea66a1b795c5393311fea74bd6

If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

@meisterT
Copy link
Member

Is there a legitimate use case for this?

@oquenchil oquenchil added the awaiting-user-response Awaiting a response from the author label Feb 25, 2025
@rbeasley-avgo
Copy link
Author

Is there a legitimate use case for this?

@meisterT Probably not. :)

I filed this not because the user should be able to create such directories, but rather that the user would expect Bazel to succeed/fail consistently regardless of the reuse_sandbox_directories and experimental_sandbox_async_tree_delete_idle_threads features. It could suggest that this feature combination could fail in other, similar situations. But there's no need to worry about it until those other cases emerge.

For now, since (a) I'm the only person to hit it and (b) it's a silly edge case, closing as a "wont fix" / "user error" seems reasonable to me.

Thanks for your consideration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
awaiting-user-response Awaiting a response from the author team-Local-Exec Issues and PRs for the Execution (Local) team type: bug untriaged
Projects
None yet
Development

No branches or pull requests

6 participants