In https://crrev.com/c/6442501, apply_patch_ref() was updated to
refresh the git cache when applying patches in gclient. However,
it causes an excessive number of logs to be created, particularly
for src/v8, as the repo has a huge number of branches.
This CL simply sets the verbose option with False,
ignoring the value in options.verbose.
Bug: 407795715
Change-Id: Ibf32ff67d23f41b398cca82372c17d7ca331db26
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6486489
Commit-Queue: Scott Lee <ddoman@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Refine the post-mirror-update check in `_UpdateMirrorIfNotContains` to
verify only when the hash revision is a SHA-1 hash. This fixes a
regression where syncing refs under refs/changes/* fails because the
check was incorrectly applied to non-hash revision strings.
Bug: 407864212, 341208163
Change-Id: I07dfe29fa7f27f6c69fa281762779e305e83b91f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6469936
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
This causes more trouble than it's worth (it's breaking dawn builds).
That does mean for a fresh depot_tools install, users will need to
explicitly run git cl creds-check once, which means tracking down all
of the docs that need to say that.
In the meantime, it's probably better to delete this now.
Bug: 408427309
Change-Id: I5595756441500c43c543b47d97ff6b1bd9ddab30
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6434734
Commit-Queue: Allen Li <ayatane@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
The previous logging output didn't show which commits were actually
being cherry-picked. Here's what it used to look like:
Will cherrypick 'refs/remotes/origin/main' .. <hash> on top of <hash>.
Where it's unclear what commit `main` refers to. This changes logging
to 1. show the actual commit hash for `main`, and 2. display a list
of picked commits. The new output:
Will cherrypick <hash> .. <hash> on top of <hash>:
<git log --oneline output>
Bug: 407795715
Change-Id: I45341fed65f692a9e1d7d7b807bd40680bf162b2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6424742
Commit-Queue: Tony Seaward <seawardt@google.com>
Auto-Submit: Jakob Linke <jgruber@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Tony Seaward <seawardt@google.com>
blame.ignoreRevsFile was set unconditionally in
https://crrev.com/c/5838110, and that broke blame for repositories
without .git-blame-ignore-revs file. This change explicitly checks for
that file before it sets git config. It also removes this config if file
is not present, and effectively fixing broken git blame operations.
The check should remain there forever since .git-blame-ignore-revs can
be removed from a repository.
Bug: 368562244
Change-Id: Id8365045635eb9623b0712b14bb3e7e3206b8795
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5888125
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
[1] was originally added by an [ancient CL] in 2013.
It runs `git config ...` and checks if if the output is not "False".
It has two problems.
1) `git config` returns False only if the value was set as a text value,
AND the value is read as a text value. If "False" is set with --bool,
the value is "false", even if it's read as a text value.
from https://git-scm.com/docs/git-config
bool: canonicalize values as either "true" or "false".
In other words, the existing check would work only if someone sets
it as a text value with "False".
2) the inline comment, describing the intention, is confusing.
"Skip url auto-correction if gclient-auto-fix-url is set"
It's not clear whether it's set to any value? false? true?
Nevertheless, what the code actually does is to auto-correct the URL,
unless the config is set to False, which would unlikely be true, due
to (1). To make it work, someone needs to set it with "False" as
a text value, but "False" is not even a valid string value for
the corresponding legit boolean value that git config --bool defines.
Therefore, depot_tools had auto-corrected the URL, if the config is
- set to "False" as a boolean value
- set to true, false, bla, foo, bar, or any value
other than "False" as a text value.
OR
- unset
That's how the code had behaved from 2013 to 2024.
In Feb 2024, https://crrev.com/c/5280779 changed the behaviour.
It replaces the dubious check with GetConfigBool(), which may sound
correct by itself.
i.e., auto-correct if "remote.origin.gclient-auto-fix-url" is set to
"true".
However, it doesn't match the existing behaviour, which
auto-correct the URL regardless of the config. Depot tools had the
behaviour for 11 years. Fixing it is probably not the best decision.
This CL doesn't exactly recover the existing behaviour. Instead,
it removes the check. As a result, even if someone sets the config
with "False" as a text value, depot_tools will auto-correct the URL.
[1]: 3b9212b7ee:gclient_scm.py;l=764-768
[ancient CL]: https://chromiumcodereview.appspot.com/15325003
Bug: 342445995
Change-Id: Idc62f803f2ef1beb11a93a58432867ce16226da7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5572721
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Scott Lee <ddoman@chromium.org>
In preparation for some Windows optimizations to how git_common calls
git it is important to use git_common more widely, specifically from
scm.py and gclient_scm.py. This change updates scm.py and gclient_scm.py
and updates the associated tests:
Test command lines used when updating the tests include:
vpython3 tests/gclient_scm_test.py ManagedGitWrapperTestCaseMock.testUpdateConflict
vpython3 tests/gclient_scm_test.py GerritChangesTest.testRecoversAfterPatchFailure
vpython3 tests/gerrit_util_test.py CookiesAuthenticatorTest.testGetGitcookiesPath
Bug: 332982922
Change-Id: I7aacb110b2888c164259815385cd77e26942adc7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5478509
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
When installing a gcs dep entry, record the getnames() output to
.<object_name>_content_names. When updating the object(s) of a gcs
entry, it'll clear all files related to that gcs entry including:
.<object_name>_content_names and the paths inside it,
.<object_name>_hash, and .<object_name>_is_first_class_gcs
Bug: b/324418194
Change-Id: Ieb56623ce929b715ed103af9560efcf66b46a9c4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5454646
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Now that multiple objects can share a directory, when objects are
removed, the directory should also remove the extracted contents
from that specific object. Since those exact contents are unknown, the
whole directory will be cleared.
If an entire GCS dep is added or removed, the corresponding directory
path will be cleared as well.
.gcs_entries holds a record of which GCS deps and objects
have been downloaded, per checkout. Example:
```
{
"src": {
"src/third_party/llvm-build/Release+Asserts": [
"Linux_x64/llvmobjdump-llvmorg-19-init-2941-ga0b3dbaf-22.tar.xz",
"Linux_x64/clang-llvmorg-19-init-2941-ga0b3dbaf-22.tar.xz"
],
"src/third_party/node/linux": [
"46795170ff5df9831955f163f6966abde581c8af"
]
}
}
```
Bug: b/324418194
Change-Id: Icac113572523b61c83450880615418bf7df8bba7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5407888
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
Also take out GCS calling logic from download_google_storage and
into call_google_storage.
GCS deps look like:
'src/third_party/node/linux': {
'dep_type': 'gcs',
'condition': 'checkout_linux',
'bucket': 'chromium-nodejs/20.11.0',
'object_name': '46795170ff5df9831955f163f6966abde581c8af',
'sha256sum': '887504c37404898ca41b896f448ee6d7fc24179d8fb6a4b79d028ab7e1b7153d',
},
'src/third_party/llvm-build/Release+Asserts': {
'dep_type': 'gcs',
'condition': 'checkout_linux',
'bucket': 'chromium-browser-clang',
'object_name': 'Linux_x64/clang-llvmorg-18-init-17730-gf670112a-2.tar.xz',
'sha256sum': '1e46df9b4e63c074064d75646310cb76be2f19815997a8486987189d80f991e8',
},
Example directory for src/third_party/node/linux after gclient sync:
- tar_file.gz is the downloaded file from GCS.
- node_linux_x64/ is extracted in its path.
- `hash` contains the sha of GCS filename.
```
chromium/src/ ->
third_party/node/linux/ ->
hash, tar_file.gz, node_linux_x64/
```
Bug: b/324418194
Change-Id: Ibcbbff27e211f194ddb8a08494af56570a84a12b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5299722
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
This reverts commit 3569608028.
Reason for revert: This includes a fix for crbug.com/324358728.
The rebase-update command has logic which tries to specifically set a key to an empty string and this has been intentionally set this way[1]. The new SetConfig implementation does treats empty string as None and hence tries to unset the config, resulting in error code 5. The patchset 2 fixes this bug and adds a test to ensure SetConfig can set an empty string to be backward compatible.
[1] https://codereview.chromium.org/228353003
Original change's description:
> Revert "Update gclient to use git config caching"
>
> This reverts commit 3edda8d185.
>
> Reason for revert: Breaks rebase-update; crbug.com/324358728
>
> Original change's description:
> > Update gclient to use git config caching
> >
> > This change updates all the modules used by gclient to use `scm.GIT` for git config calls over directly invoking the subprocess.
> >
> > This change currently doesn't modify git_cache since the config reads and writes within it are done on bare repository. A follow-up CL will update git_cache.
> >
> > A follow-up CL will also update git_cl and git_map_branches since they have shown performance improvements too: https://crrev.com/c/4697786.
> >
> > Benchmarking
> > ============
> > With chromium/src as the baseline super project, this change reduces about 380 git config calls out of 507 total calls on cache hits during no-op. The below numbers are benchmarked with `update_depot_tools` turned off.
> >
> > Windows Benchmark
> > =================
> > Baseline (gpaste/6360045736951808): ~1min 12 sec.
> > With Caching (gpaste/6480065209040896): ~1min 3sec.
> > ~12.5% decrease in gclient sync noop runtime.
> >
> > Linux Benchmark
> > ===============
> > Baseline (gpaste/4730436763254784): ~3.739 sec.
> > With Caching (gpaste/4849870978940928): ~3.534 sec.
> > ~5.5% decrease in gclient sync noop runtime.
> >
> > Bug: 1501984
> > Change-Id: Ib48df2d26a0c742a9b555a1e2ed6366221c7db17
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5252498
> > Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
> > Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
>
> Bug: 1501984
> Change-Id: I4a603238d9ed43edafc8e574493800670520a1d9
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5279198
> Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Bug: 1501984
Change-Id: I405abc16c2ef6f0689031c82c61af71aad302122
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5280779
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
This reverts commit 3edda8d185.
Reason for revert: Breaks rebase-update; crbug.com/324358728
Original change's description:
> Update gclient to use git config caching
>
> This change updates all the modules used by gclient to use `scm.GIT` for git config calls over directly invoking the subprocess.
>
> This change currently doesn't modify git_cache since the config reads and writes within it are done on bare repository. A follow-up CL will update git_cache.
>
> A follow-up CL will also update git_cl and git_map_branches since they have shown performance improvements too: https://crrev.com/c/4697786.
>
> Benchmarking
> ============
> With chromium/src as the baseline super project, this change reduces about 380 git config calls out of 507 total calls on cache hits during no-op. The below numbers are benchmarked with `update_depot_tools` turned off.
>
> Windows Benchmark
> =================
> Baseline (gpaste/6360045736951808): ~1min 12 sec.
> With Caching (gpaste/6480065209040896): ~1min 3sec.
> ~12.5% decrease in gclient sync noop runtime.
>
> Linux Benchmark
> ===============
> Baseline (gpaste/4730436763254784): ~3.739 sec.
> With Caching (gpaste/4849870978940928): ~3.534 sec.
> ~5.5% decrease in gclient sync noop runtime.
>
> Bug: 1501984
> Change-Id: Ib48df2d26a0c742a9b555a1e2ed6366221c7db17
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5252498
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Bug: 1501984
Change-Id: I4a603238d9ed43edafc8e574493800670520a1d9
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5279198
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
This change updates all the modules used by gclient to use `scm.GIT` for git config calls over directly invoking the subprocess.
This change currently doesn't modify git_cache since the config reads and writes within it are done on bare repository. A follow-up CL will update git_cache.
A follow-up CL will also update git_cl and git_map_branches since they have shown performance improvements too: https://crrev.com/c/4697786.
Benchmarking
============
With chromium/src as the baseline super project, this change reduces about 380 git config calls out of 507 total calls on cache hits during no-op. The below numbers are benchmarked with `update_depot_tools` turned off.
Windows Benchmark
=================
Baseline (gpaste/6360045736951808): ~1min 12 sec.
With Caching (gpaste/6480065209040896): ~1min 3sec.
~12.5% decrease in gclient sync noop runtime.
Linux Benchmark
===============
Baseline (gpaste/4730436763254784): ~3.739 sec.
With Caching (gpaste/4849870978940928): ~3.534 sec.
~5.5% decrease in gclient sync noop runtime.
Bug: 1501984
Change-Id: Ib48df2d26a0c742a9b555a1e2ed6366221c7db17
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5252498
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Other places in gclient_scm (GitWrapper._Clone, GitWrapper.update) do
the same.
Note that we call _AutoFetchRef even after running _Fetch, because
_Fetch in not guaranteed to fetch a specific revision. For example, a
checkout might be configured without branch heads and without tags, yet
a revision might point to a tag.
Change-Id: I5cfd8ebafa4b45afd72b0251f04e5d713199f2fd
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4715239
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
