This will be useful for writing tests which rely on shared 'global'
config between different working directories.
This also adds support for mocking 'system' (global, immutable) and
'workspace' (local, mutable). The workspace scope I think is a bit iffy
though, given how `git cl` is actually built - currently scm.GIT doesn't
really know about clone vs. workspace, and afaik no config adjustements
actually apply to the workspace scope.
Adds tests for mocked git config implementation, including bug fixes
to the current implementation revealed by the tests.
R=ayatane, yiwzhang
Change-Id: Ia56d2a81d8df6ae75d9f8d0497be0d67bdc03651
Bug: 355505750
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5759163
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Previously there was an error where you could have:
global: {"key": ["something"]}
local: {}
And called SetConfig("key", None, missing_ok=False) - this would end
up raising CalledProcessError(returncode=5) because the caching
layer would see that "key" was not missing, then call into the
storage layer which would do `git config --local ...` which was
missing "key" (and therefore fail).
R=ayatane, yiwzhang
Change-Id: Idc9dc63a1b1c0a4a203c403dbb3c2ea521099b26
Bug: 342644760
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5744333
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Allen Li <ayatane@chromium.org>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Previously little bits of the scm.GIT were mocked (like SetConfig,
and GetConfig, but not the other config related methods).
This changes things so that the git config state is a class whose
logic is shared between prod and test, with a 'real' and 'test'
implementation which know how to load and save configuration at
a low level.
R=yiwzhang
Change-Id: I1dc11b550908862607ea539de7fa60fbce30e700
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5660891
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Allen Li <ayatane@chromium.org>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
For a ProvidedDiffChange, the AllFiles method naively returns all files
with rglob("*"). The returned list includes files in nested submodules.
This does not match the behavior of GitChange's AllFiles which uses
git ls-files to find all files.
Implement a new SCM that stops iterating recursively when it
sees a submodule from .gitmodules in the repo root.
Bug: b/323243527
Change-Id: I170d0f1bc4a838acea04779dee3df7fca0bce359
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5648616
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
If symbolic-ref remote HEAD is not available, we skip setting it and
query remote Git using ls-remote. Such information is not stored, and
gclient will need to repeat it on the next invocation.
Instead, we can call set-head on symbolic-ref failure. While that's
slower operation than ls-remote, it saved in internal Git database and
can be reused on next gclient invocation.
We may be okay with hardcoding 'main' as default remote branch today,
but it's possible that some projects still use old default.
R=gavinmak@google.com
Change-Id: Ic4c826b888d96e367039bfc4b9bd2ba0d8b58b52
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5492789
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
This reverts commit 3569608028.
Reason for revert: This includes a fix for crbug.com/324358728.
The rebase-update command has logic which tries to specifically set a key to an empty string and this has been intentionally set this way[1]. The new SetConfig implementation does treats empty string as None and hence tries to unset the config, resulting in error code 5. The patchset 2 fixes this bug and adds a test to ensure SetConfig can set an empty string to be backward compatible.
[1] https://codereview.chromium.org/228353003
Original change's description:
> Revert "Update gclient to use git config caching"
>
> This reverts commit 3edda8d185.
>
> Reason for revert: Breaks rebase-update; crbug.com/324358728
>
> Original change's description:
> > Update gclient to use git config caching
> >
> > This change updates all the modules used by gclient to use `scm.GIT` for git config calls over directly invoking the subprocess.
> >
> > This change currently doesn't modify git_cache since the config reads and writes within it are done on bare repository. A follow-up CL will update git_cache.
> >
> > A follow-up CL will also update git_cl and git_map_branches since they have shown performance improvements too: https://crrev.com/c/4697786.
> >
> > Benchmarking
> > ============
> > With chromium/src as the baseline super project, this change reduces about 380 git config calls out of 507 total calls on cache hits during no-op. The below numbers are benchmarked with `update_depot_tools` turned off.
> >
> > Windows Benchmark
> > =================
> > Baseline (gpaste/6360045736951808): ~1min 12 sec.
> > With Caching (gpaste/6480065209040896): ~1min 3sec.
> > ~12.5% decrease in gclient sync noop runtime.
> >
> > Linux Benchmark
> > ===============
> > Baseline (gpaste/4730436763254784): ~3.739 sec.
> > With Caching (gpaste/4849870978940928): ~3.534 sec.
> > ~5.5% decrease in gclient sync noop runtime.
> >
> > Bug: 1501984
> > Change-Id: Ib48df2d26a0c742a9b555a1e2ed6366221c7db17
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5252498
> > Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
> > Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
>
> Bug: 1501984
> Change-Id: I4a603238d9ed43edafc8e574493800670520a1d9
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5279198
> Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Bug: 1501984
Change-Id: I405abc16c2ef6f0689031c82c61af71aad302122
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5280779
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Reformat this dir by itself to help merging with conflicts with other CLs.
Reformatted using:
parallel ./yapf -i -- tests/*.py
~/chromiumos/chromite/contrib/reflow_overlong_comments tests/*.py
These files still had lines (strings) that were too long, so the pylint
warnings were suppressed with a TODO.
tests/bot_update_coverage_test.py
tests/cipd_bootstrap_test.py
tests/gclient_eval_unittest.py
tests/gclient_git_smoketest.py
tests/gclient_scm_test.py
tests/gclient_smoketest.py
tests/gclient_test.py
tests/gclient_transitions_smoketest.py
tests/gclient_utils_test.py
tests/git_cl_test.py
tests/git_hyper_blame_test.py
tests/git_rebase_update_test.py
tests/lockfile_test.py
tests/metrics_test.py
tests/presubmit_canned_checks_test.py
tests/presubmit_unittest.py
tests/roll_dep_test.py
Change-Id: I8fed04b4ba81d54b8f45da612213aad27a9e1a2c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4842592
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Mike Frysinger <vapier@chromium.org>
Currently, gclient sync assumes the default branch is master, and
it doesn't work at all if such branch doesn't exist. This change queries
local git copy to get remote HEAD. If local git version is not
available, it queries remote git server using ls-remote.
This change requires git version 2.28 (depot_tools comes with 2.29).
R=ehmaldonado@chromium.org
Bug: 1156318
Change-Id: Id348e0f1004093f395139e8f4d62adb66b94ca9c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2628359
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
scm:
- Add methods to deal with git configs and tests for them.
Will make it easier to mock git config calls as opposed
to all git calls.
- Add tests to FetchUpstreamTuple
git-cl:
- Mock out settings.GetChangeRoot()
- Use scm.GIT.FetchUpstreamTuple() to reduce code duplication,
and mock it out on tests.
Bug: 1051631
Change-Id: I1a3220b4c0f3e6221b3c00550259a433c2775798
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2052552
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Anthony Polito <apolito@google.com>
Abort any cherry-picks before applying the patch, so that if the bots are in a
bad state, we don't fail.
Original change's description:
> Reland "gclient_scm: Use cherry-picking instead of rebasing."
>
> The failures were caused by:
> 1 - When one change (call it #2) has been uploaded on top of another (#1),
> and (#1) has already landed, git cherry-pick complains that the range
> '<merge-base>..<change #2>' contains empty commits, since the contents
> of (#1) are already present in the tree.
> 2 - We did not abort the cherry-picking when 'git cherry-pick' failed,
> so a failure made all further CLs in that bot fail.
>
> This CL fixes it and prevents further regressions.
>
> Original change's description:
> > gclient_scm: Use cherry-picking instead of rebasing.
> >
> > Currently gclient might include extra commits when applying patches.
> > For example, in this case we checkout |patch| and rebase it on top of |base|,
> > thus including an |extra commit| that we shouldn't.
> >
> > o master
> > |
> > . o patch
> > |/
> > o extra commit
> > |
> > o base (what gclient synced src at)
> >
> > This change uses the merge-base between |patch| and |master| to cherry-pick only
> > the changes belonging to the patch.
> >
> > Bug: 850812
> > Change-Id: I138192f96bc62b1bb19b0e1ad952c8f8c67631c4
> > Reviewed-on: https://chromium-review.googlesource.com/1137052
> > Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> > Reviewed-by: Aaron Gable <agable@chromium.org>
>
> Bug: 850812
> Change-Id: I83f38d0a258df3f5cd89e277f0d648badff29a22
> Reviewed-on: https://chromium-review.googlesource.com/1139554
> Reviewed-by: Aaron Gable <agable@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Bug: 850812
Change-Id: Ic65bda67c792bd7af5ec013a62d9615d1498eb3a
Reviewed-on: https://chromium-review.googlesource.com/1142805
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
This reverts commit f9afc77f28.
Reason for revert: https://bugs.chromium.org/p/chromium/issues/detail?id=864301#c10
Original change's description:
> Reland "gclient_scm: Use cherry-picking instead of rebasing."
>
> The failures were caused by:
> 1 - When one change (call it #2) has been uploaded on top of another (#1),
> and (#1) has already landed, git cherry-pick complains that the range
> '<merge-base>..<change #2>' contains empty commits, since the contents
> of (#1) are already present in the tree.
> 2 - We did not abort the cherry-picking when 'git cherry-pick' failed,
> so a failure made all further CLs in that bot fail.
>
> This CL fixes it and prevents further regressions.
>
> Original change's description:
> > gclient_scm: Use cherry-picking instead of rebasing.
> >
> > Currently gclient might include extra commits when applying patches.
> > For example, in this case we checkout |patch| and rebase it on top of |base|,
> > thus including an |extra commit| that we shouldn't.
> >
> > o master
> > |
> > . o patch
> > |/
> > o extra commit
> > |
> > o base (what gclient synced src at)
> >
> > This change uses the merge-base between |patch| and |master| to cherry-pick only
> > the changes belonging to the patch.
> >
> > Bug: 850812
> > Change-Id: I138192f96bc62b1bb19b0e1ad952c8f8c67631c4
> > Reviewed-on: https://chromium-review.googlesource.com/1137052
> > Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> > Reviewed-by: Aaron Gable <agable@chromium.org>
>
> Bug: 850812
> Change-Id: I83f38d0a258df3f5cd89e277f0d648badff29a22
> Reviewed-on: https://chromium-review.googlesource.com/1139554
> Reviewed-by: Aaron Gable <agable@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
TBR=agable@chromium.org,ehmaldonado@chromium.org,jbudorick@chromium.org
Change-Id: I2138757899abd6eb9bece2dac3a25676effafdf3
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 850812
Reviewed-on: https://chromium-review.googlesource.com/1142624
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>
The failures were caused by:
1 - When one change (call it #2) has been uploaded on top of another (#1),
and (#1) has already landed, git cherry-pick complains that the range
'<merge-base>..<change #2>' contains empty commits, since the contents
of (#1) are already present in the tree.
2 - We did not abort the cherry-picking when 'git cherry-pick' failed,
so a failure made all further CLs in that bot fail.
This CL fixes it and prevents further regressions.
Original change's description:
> gclient_scm: Use cherry-picking instead of rebasing.
>
> Currently gclient might include extra commits when applying patches.
> For example, in this case we checkout |patch| and rebase it on top of |base|,
> thus including an |extra commit| that we shouldn't.
>
> o master
> |
> . o patch
> |/
> o extra commit
> |
> o base (what gclient synced src at)
>
> This change uses the merge-base between |patch| and |master| to cherry-pick only
> the changes belonging to the patch.
>
> Bug: 850812
> Change-Id: I138192f96bc62b1bb19b0e1ad952c8f8c67631c4
> Reviewed-on: https://chromium-review.googlesource.com/1137052
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Aaron Gable <agable@chromium.org>
Bug: 850812
Change-Id: I83f38d0a258df3f5cd89e277f0d648badff29a22
Reviewed-on: https://chromium-review.googlesource.com/1139554
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
