blame.ignoreRevsFile was set unconditionally in
https://crrev.com/c/5838110, and that broke blame for repositories
without .git-blame-ignore-revs file. This change explicitly checks for
that file before it sets git config. It also removes this config if file
is not present, and effectively fixing broken git blame operations.
The check should remain there forever since .git-blame-ignore-revs can
be removed from a repository.
Bug: 368562244
Change-Id: Id8365045635eb9623b0712b14bb3e7e3206b8795
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5888125
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
If git config set is executed, it writes the new content into
a temp file (but called lock-file) and then replace .gitconfig
with it.
However, if it cannot create the lock file, it returns an error.
With this CL, if git config fails with the lock file error,
depot_tools will retry it at most 5 times with 0.2s interval.
It's found that there are applications, such as vscode extensions,
executes `git config set` frequently, and those could often cause
unexpected interruptions to ongoing `git rebase-updates`
by the lock failure.
Bug: 351950514
Change-Id: I985af0d8b7458dbf47cd6baa857dc5adccf15031
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5705561
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Scott Lee <ddoman@chromium.org>
In order for git to be run on Windows and ultimately run git.exe it is
normally necessary to pass shell=True to Popen or else invoke git.bat,
or both. However this means invoking cmd.exe every time we want to
invoke git.exe, which can end up being over 1,500 invocations of cmd.exe
when running the "update" stage of gclient sync. This makes the "update"
stage take about 1.45x as long to run - it's a significant overhead.
Even with optimizations to the update stage of gclient sync in the NOP
case this repeated invocation of cmd.exe still costs about 0.8 s of a
4.9 s "update" stage.
This change teaches git_common how to find git.exe instead of just
git.bat by parsing the .exe name out of the .bat file.
Bug: 332982922
Change-Id: I4e052444276d55c0496144b9816fe365a95a56d6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5462405
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
This reverts commit 3569608028.
Reason for revert: This includes a fix for crbug.com/324358728.
The rebase-update command has logic which tries to specifically set a key to an empty string and this has been intentionally set this way[1]. The new SetConfig implementation does treats empty string as None and hence tries to unset the config, resulting in error code 5. The patchset 2 fixes this bug and adds a test to ensure SetConfig can set an empty string to be backward compatible.
[1] https://codereview.chromium.org/228353003
Original change's description:
> Revert "Update gclient to use git config caching"
>
> This reverts commit 3edda8d185.
>
> Reason for revert: Breaks rebase-update; crbug.com/324358728
>
> Original change's description:
> > Update gclient to use git config caching
> >
> > This change updates all the modules used by gclient to use `scm.GIT` for git config calls over directly invoking the subprocess.
> >
> > This change currently doesn't modify git_cache since the config reads and writes within it are done on bare repository. A follow-up CL will update git_cache.
> >
> > A follow-up CL will also update git_cl and git_map_branches since they have shown performance improvements too: https://crrev.com/c/4697786.
> >
> > Benchmarking
> > ============
> > With chromium/src as the baseline super project, this change reduces about 380 git config calls out of 507 total calls on cache hits during no-op. The below numbers are benchmarked with `update_depot_tools` turned off.
> >
> > Windows Benchmark
> > =================
> > Baseline (gpaste/6360045736951808): ~1min 12 sec.
> > With Caching (gpaste/6480065209040896): ~1min 3sec.
> > ~12.5% decrease in gclient sync noop runtime.
> >
> > Linux Benchmark
> > ===============
> > Baseline (gpaste/4730436763254784): ~3.739 sec.
> > With Caching (gpaste/4849870978940928): ~3.534 sec.
> > ~5.5% decrease in gclient sync noop runtime.
> >
> > Bug: 1501984
> > Change-Id: Ib48df2d26a0c742a9b555a1e2ed6366221c7db17
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5252498
> > Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
> > Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
>
> Bug: 1501984
> Change-Id: I4a603238d9ed43edafc8e574493800670520a1d9
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5279198
> Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Bug: 1501984
Change-Id: I405abc16c2ef6f0689031c82c61af71aad302122
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5280779
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
This reverts commit 3edda8d185.
Reason for revert: Breaks rebase-update; crbug.com/324358728
Original change's description:
> Update gclient to use git config caching
>
> This change updates all the modules used by gclient to use `scm.GIT` for git config calls over directly invoking the subprocess.
>
> This change currently doesn't modify git_cache since the config reads and writes within it are done on bare repository. A follow-up CL will update git_cache.
>
> A follow-up CL will also update git_cl and git_map_branches since they have shown performance improvements too: https://crrev.com/c/4697786.
>
> Benchmarking
> ============
> With chromium/src as the baseline super project, this change reduces about 380 git config calls out of 507 total calls on cache hits during no-op. The below numbers are benchmarked with `update_depot_tools` turned off.
>
> Windows Benchmark
> =================
> Baseline (gpaste/6360045736951808): ~1min 12 sec.
> With Caching (gpaste/6480065209040896): ~1min 3sec.
> ~12.5% decrease in gclient sync noop runtime.
>
> Linux Benchmark
> ===============
> Baseline (gpaste/4730436763254784): ~3.739 sec.
> With Caching (gpaste/4849870978940928): ~3.534 sec.
> ~5.5% decrease in gclient sync noop runtime.
>
> Bug: 1501984
> Change-Id: Ib48df2d26a0c742a9b555a1e2ed6366221c7db17
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5252498
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Bug: 1501984
Change-Id: I4a603238d9ed43edafc8e574493800670520a1d9
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5279198
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
This change updates all the modules used by gclient to use `scm.GIT` for git config calls over directly invoking the subprocess.
This change currently doesn't modify git_cache since the config reads and writes within it are done on bare repository. A follow-up CL will update git_cache.
A follow-up CL will also update git_cl and git_map_branches since they have shown performance improvements too: https://crrev.com/c/4697786.
Benchmarking
============
With chromium/src as the baseline super project, this change reduces about 380 git config calls out of 507 total calls on cache hits during no-op. The below numbers are benchmarked with `update_depot_tools` turned off.
Windows Benchmark
=================
Baseline (gpaste/6360045736951808): ~1min 12 sec.
With Caching (gpaste/6480065209040896): ~1min 3sec.
~12.5% decrease in gclient sync noop runtime.
Linux Benchmark
===============
Baseline (gpaste/4730436763254784): ~3.739 sec.
With Caching (gpaste/4849870978940928): ~3.534 sec.
~5.5% decrease in gclient sync noop runtime.
Bug: 1501984
Change-Id: Ib48df2d26a0c742a9b555a1e2ed6366221c7db17
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5252498
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
This patch makes git rebase-update only try squashing when rebasing
fails if there is more than one commit present in the branch it is
rebasing.
Squashing adds an extra step for the user when rebasing fails, and when
there is only one commit, squashing is guaranteed not to help.
Fixed: 1434450
Change-Id: I1952c1172dbf6a44060ed2da2d79ec07d67893d4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5068315
Commit-Queue: Joey Arhar <jarhar@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
If there are dirty submodules, git commit -a will commit those which is
not desired. Turns out we don't need to use -a, since reset --soft
stages all changes anyways and we can just use git commit. Note that
there is a check prior to commit to ensure we are not creating an empty
commit.
R=jojwang
Fixed: 1478668
Change-Id: Iaa1ff8e638b7431511e6e194ad59e3c4adb39deb
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4858836
Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Leave the recipes/ code at 2 space to match the rest of the recipes
project in other repos.
Reformatted using:
files=( $(
git ls-tree -r --name-only HEAD | \
grep -Ev -e '^(third_party|recipes)/' | \
grep '\.py$';
git grep -l '#!/usr/bin/env.*python' | grep -v '\.py$'
) )
parallel ./yapf -i -- "${files[@]}"
~/chromiumos/chromite/contrib/reflow_overlong_comments "${files[@]}"
The files that still had strings that were too long were manually
reformatted because they were easy and only a few issues.
autoninja.py
clang_format.py
download_from_google_storage.py
fix_encoding.py
gclient_utils.py
git_cache.py
git_common.py
git_map_branches.py
git_reparent_branch.py
gn.py
my_activity.py
owners_finder.py
presubmit_canned_checks.py
reclient_helper.py
reclientreport.py
roll_dep.py
rustfmt.py
siso.py
split_cl.py
subcommand.py
subprocess2.py
swift_format.py
upload_to_google_storage.py
These files still had lines (strings) that were too long, so the pylint
warnings were suppressed with a TODO.
auth.py
gclient.py
gclient_eval.py
gclient_paths.py
gclient_scm.py
gerrit_util.py
git_cl.py
presubmit_canned_checks.py
presubmit_support.py
scm.py
Change-Id: Ia6535c4f2c48d46b589ec1e791dde6c6b2ea858f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4836379
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
The chromium build instructions for MacOS includes enabling `fsmonitor`
within the repository. With our warning message suggesting to disable it
globally, the fsmonitor still stays enabled within the repo. This CL
fixes this by disabling `fsmonitor` within the repository.
Change-Id: If261af8bc0d0a89bc193c178c8d5b5f80125d7b7
Fixed: 1476695
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4823891
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Aravind Vasudevan <aravindvasudev@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
To avoid having users deal with merge conflicts between unstaged
gitlinks modified before and after a git rebase-update we can just
not commit unstaged gitlinks in our freeze index.
This change results in whatever gitlinks are unstaged before
git rebase-update to be just rebased on top of whatever the latest
gitlinks are from remote.
Bug: 1472148
Change-Id: If7dcb82778a0bdd14c408f0a0287780a7bdc5358
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4771356
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Gregory Nisbet <gregorynisbet@google.com>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
The previous change to git-freeze added an accidental regression
where running `git freeze` from somewhere other than the repo root
would return a mysterious error:
"Failed to index some unindexed files. Nothing to freeze."
This is because `git add` always treats pathspecs as relative to the
current working directory. Fix this by changing `git add` to always
run from the repo root.
R=ajp@google.com
Change-Id: I451f26fe35a5c6e9f3b917a1d90bdadc7065244c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4703128
Commit-Queue: Andy Perelson <ajp@google.com>
Reviewed-by: Andy Perelson <ajp@google.com>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
This adds support in two ways:
1. Displays a "(dormant)" marker as part of each branch's line when the
verbosity level is four or above. ('git bmap -v -v -v -v ...'). It's
worth noting that, with verbosity level 4, each line is now 119
characters long; verbosity level 3 is 107.
2. Enables callers to hide dormant branches using a new '--hide-dormant'
option. If the dormant branch is the parent of non-dormant branches,
it's still shown, despite being dormant, so that it can continue to
serve as the parent line for non-dormant branches.
Change-Id: I0504419fd12357563288b5d53bc49ca68a876e8f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4654849
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Orr Bernstein <orrb@google.com>
It turns out we are already parsing the full output of `git status`,
so we may as well use that. This output already ignores submodules.
This also updates the freeze implementation to avoid unnecessary
calls to `git commit` when we already know if they are needed or
not.
Note that `git stash` (which `git freeze` is emulating) already
ignores submodules as well.
Ignoring submodules while freezing should make `git rebase-update`
work again inside of repos which have dirty submodules.
R=gavinmak,jojwang
Change-Id: Ib3a7784b6e7e7c19687203c1f4c32422a49bf8e3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4680288
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Now that the presubmit system no longer supports Python 2 we can remove
support for it from the presubmit system tests. This change removes all
references to sys.version_info from this file.
This also fixes an unclosed file warning that suddenly appeared and
was stopping this change from landing.
Bug: 1207012
Change-Id: Ib74e9009b9d5cea0caf37c88e3258e9dd818f269
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4522416
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Previously, git thaw would read the whole output of git rev-list HEAD
via readlines(). This was unnecessary, because we almost always only
need to look a few of the most recent commits. Most of the runtime of git-thaw was spend on this.
After this commit, we only read the lines we actually need. This makes git thaw run much faster.
Bug: 1378479
Change-Id: I6f6c06e1df55b4943a94642aa414fc11aeea5718
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3981233
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Martin Bidlingmaier <mbid@google.com>
In theory this should prevent `git rebase-update` from invoking
potentially expensive `git gc` operations after every `git rebase`
(ideally once per branch, but sometimes as many as three per branch (!)).
This has outsized effects as the size of the repo or number of branches
increases, and additionally has penalties on Windows due to git's
POSIXey assumptions around how disk I/O works.
After doing all rebase stuff, we run `git gc --auto` from rebase-update
once (assuming we were successful; if the command failed then we don't
run gc at all).
R=apolito@google.com
Bug: 617218
Change-Id: I37a8f7953779f15b320e4ed7bb85119cb4bb665a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2868766
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
If origin/HEAD resolves to `master`, depot_tools can check if
origin/main exists. If so, it's very likely that origin/HEAD should be
updated to point to origin/main.
In the worst case, this will result in extra git remote call while not
changing local state (i.e origin/HEAD remained unchanged).
R=gavinmak@google.com
Bug: 1190702
Change-Id: I51e69d7a95d3534e1820099b4b6983d38b5e2763
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2795075
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
