I'm looking at caffeinating more long-running depot_tools commands on
Mac, which use a variety of `subprocess` APIs; one implementation
strategy I'm considering is to wrap more of them, including
`subprocess.run`, but the most obvious name to wrap `subprocess.run`
into is already taken! I think it makes the most sense to name the
`caffeinate` function(s) after the `subprocess` call it wraps.
The callers to other parts of `subprocess` use more of the features
exposed by those APIs. For consistency, I am plumbing through more of
`subprocess.call`'s identifiable behaviors here, too:
* command-to-invoke parameter is named "args"
* as explicitly checked in cpython's subprocess.py's `_posix_spawn`:
77cb39e0c7/Lib/subprocess.py (L1816)
also allow `str`, `bytes`, and `os.PathLike` for the type of `args`,
converting the non-list-like types into a single-element list
* use kwargs packing and unpacking to forward other parameters into
`subprocess.call`
kwargs packing does not provide perfect emulation as implemented:
77cb39e0c7/Lib/subprocess.py (L386)
`timeout` could be provided as a positional parameter. However, Python
documents all subprocess.call parameters as keyword only:
https://docs.python.org/3.12/library/subprocess.html#older-high-level-api
...specifically because they don't want to bother distinguishing
between parameters "local" to a higher-level `subprocess` helper and
those that get forwarded (via `**kwargs`) to the underlying `Popen`,
so I think it is reasonable to follow their lead on that.
Bug: 462507017
Change-Id: Ia520ced7f8188c23c38826d22ccf20a3c52ddfc5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7181621
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Adam Norberg <norberg@google.com>
This adds an optional turboci_check_id parameter to
bot_update.ensure_checkout. If set, it will cause a SOURCE check to be
emitted based on the gclient config being used and any gitiles commit or
gerrit change that is used.
Bug: b:443496677
Change-Id: Ic7fc55511b9b698cffb85fe872221e534fee742d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7147310
Commit-Queue: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Robbie Iannucci <iannucci@google.com>
The json output for bot_update reports the value that are passed as
--revision flags to bot_update.py in the fixed_revisions field in the
json output. In the case of a revision of 'HEAD' for a project that has
a CL, the 'HEAD' gets resolved to the target ref of the CL. In
simulation tests, the --revision flag in the command correctly shows the
target ref, but the fixed_revisions in the json output shows 'HEAD'.
This updates the code so that the target ref resolution happens before
setting the value in the dict that is used as the fixed_revisions field
in the output json. This results in a change to the revision since the
input to produce the hash will incorporate the ref.
Change-Id: I4fc0594039faf0725ef9c293b00f31bc7b2692a2
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_internal
Recipe-Nontrivial-Roll: infra
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7148746
Auto-Submit: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Scott Lee <ddoman@chromium.org>
Commit-Queue: Scott Lee <ddoman@chromium.org>
Makes the following changes related to the exception made for fake
Chromium checkouts when checking git submodules:
1. Adds an instance of this being used in Dawn to the comment
2. Adds third_party/placeholder_chromium as an allowed dependency name
since the "dummy" from third_party/dummy_chromium runs into the
inclusive language checks.
Bug: 452840620
Change-Id: I65099d150cf9c4971e3183f1cb208df668582e4c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7170740
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Commit-Queue: Scott Lee <ddoman@chromium.org>
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Reviewed-by: Scott Lee <ddoman@chromium.org>
Currently, in order to specify a revision for a project in the
bot_update output, one has to either set the fixed_revisions or the
revision_mapping values. Both of these have additional effects on the
output json (fixed_revisions or properties) and setting them requires
fully overriding the json output. This can easily lead to output data
that doesn't match the gclient config being used.
This change overhauls the output_json test API function by adding a
revisions parameter that can be used to override the revisions used for
a project without affecting other portions of the json output. A
mod_test_data-decorated revisions method was added that enables
overriding the revisions without having to specify the full output json,
so fixed_revisions and properties will be set automatically according to
the gclient config being used. The revision_mapping parameter was
renamed to revision_properties to better distinguish it from revisions
and more accurately indicate its contents.
The following changes in behavior occur for the default test data:
* The got_revision property will only be added if the gclient config being used doesn't specify a got_revision_reverse_mapping
* If a got_revision property is added because the gclient config doesn't specify a got_revision_reverse_mapping and commit_positions is True, a got_revision_cp property will also be added
Additionally a post_check_output_json method was added that simplifies
the process of making post checks on the contents of the bot_update json
output.
Change-Id: I012b440107fa3323564fa3e1e3024c038e9944a2
Recipe-Manual-Change: build
Recipe-Manual-Change: infra
Recipe-Nontrivial-Roll: build_internal
Recipe-Nontrivial-Roll: chromiumos
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7152830
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Garrett Beaty <gbeaty@google.com>
The diff hunk can omit the number of lines after the line position.
For example, the following hunks are all valid
- @@ -1,2 +1,3 @@
- @@ -1 +1,3 @@
- @@ -1,2 +1 @@
- @@ -1 +1 @@
However, the existing regex assumes that the number of lines is always
given. It wasn't a problem before crrev.com/c/6952890, because -U3 was
applied always. However, since it changed the context lines to -U0,
git diff started generating hunks without the number of lines, if
a given changed section contains only a single line.
This CL fixes the regex.
Bug: 453641840
Change-Id: I67f680bc1d8fa046e373b065e8ccbb6ce652eb3f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7158599
Auto-Submit: Scott Lee <ddoman@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Scott Lee <ddoman@chromium.org>
Using `args[1]` to get the subcommand name doesn't work when the user
calls siso without any arguments, or when they specify global flags such
as `siso -logtostderr ninja ...`.
We can use the existing `subcmd` variable instead.
This fixes the following traceback:
```
$ siso
Traceback (most recent call last):
File "/home/philwo/depot_tools/siso.py", line 313, in <module>
sys.exit(main(sys.argv))
^^^^^^^^^^^^^^
File "/home/philwo/depot_tools/siso.py", line 292, in main
if args[1] == "ninja":
~~~~^^^
IndexError: list index out of range
```
Change-Id: Id319bdf099a2e87e049aee474734495424617a5a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7155649
Auto-Submit: Philipp Wollermann <philwo@google.com>
Commit-Queue: Philipp Wollermann <philwo@google.com>
Reviewed-by: Alex Ovsienko <ovsienko@google.com>
This updates the CheckInfraFreeze/CheckChromiumInfraFreeze functions to
operate on the _CHROMIUM_FREEZES list of tuples with start, end and
details that define freeze periods rather than a single freeze period
defined by the _CHROMIUM_FREEZE_START, _CHROMIUM_FREEZE_END and
_CHROMIUM_FREEZE_DETAILS values. This will enable defining the freeze
periods for end-of-year all at once rather than requiring an update
between the November and December/January periods.
The periods have been set to match the announced end-of-year freeze
dates.
Change-Id: I7ea9d19fad71d086cef3de0e02ea40fda35460ea
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7090808
Commit-Queue: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Scott Lee <ddoman@chromium.org>
This allows client code to check if a license is in a given allowlist
e.g. 'WITH_PERMISSION_ONLY' without exporting the lists. This is done
because we now process licenses (e.g. .lower()) when checking against
the lists.
Also fix a type annotation.
Change-Id: I7c52dd19493f72ba3fb927f1654ee1affde320dd
Bug: 452151523, 460076179
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7149079
Reviewed-by: Anne Redulla <aredulla@google.com>
Auto-Submit: Jordan Brown <rop@google.com>
Commit-Queue: Anne Redulla <aredulla@google.com>
This CL re-epxorts normalized license list consts temporarily so we
unbreak downstream clients relying on their import names.
Future CL will migrate downstream clients to use the exported functions
(which handles normalization correctly), then remove this temporary
export.
Bug: 452151523
Change-Id: I05a96fb35c3decda77890808d8f78ae867f47b09
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7148918
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Jiewei Qian <qjw@chromium.org>
Reviewed-by: Jordan Brown <rop@google.com>
This reverts commit 55f06f3e06.
Reason for revert: https://b.corp.google.com/issues/458283880#comment6. Once and a while we discover a tool which uses the vendored depot tools. In this case, we weren't passing the user's depot tools path for debugging. When that happens, the workspace will break, with no visible files in the change. Instead, we'll reintroduce this, and add have the cipd root live in the out directory. That will prevent the issue from the second bullet point.
Original change's description:
> cipd: ignore .cipd_client_root
>
> It was introduced to override the cipd bin path so that the downloaded
> files wouldn't be listed in the CiderG workspace file tree when
> the bundled depot_tools is used.
>
> However,
> - chromium no longer uses the bundled depot_tools
> - the cipd_client_root, pointing to a folder in /tmp, causes issues
> when a temp cleanup cron removes files based on the last access ts.
>
> We no longer need .cipd_client_root.
>
> Bug: 415080055
> Change-Id: Ic966699cce3173a8ce4033752edec5e53aded870
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6614519
> Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
> Commit-Queue: Scott Lee <ddoman@chromium.org>
Bug: 415080055
Change-Id: If3610a3660386f50a1567c2b7bae8437eb4860f2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7138561
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Terrence Reilly <treilly@google.com>
The function prints a warning message when it finds that
a given directory contains .gclient only without .gclient_entries.
It can possibly happen if gclient sync execution is halted
in the middle, and it causes format function commands to mess up
the format output with the warning message.
This CL is to bypass the warning in Cog env. Find more context
visit the linked bug.
Bug: 450901048
Change-Id: I87e376e9a64c8e2ce2231dc9549e0683a9f393ef
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7133882
Commit-Queue: Scott Lee <ddoman@chromium.org>
Auto-Submit: Scott Lee <ddoman@chromium.org>
Reviewed-by: Terrence Reilly <treilly@google.com>
1.46.1 adds the lucicfg.main_package() function, which enables custom
generators to know the path to the main package being executed without
having to explicitly duplicate the location within custom starlark code.
Bug: 458607662
Change-Id: Id2c173f9d7ab3162a864311e6fe86768de2dfd57
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7137242
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
