Mohamed Heikal
9b4d1e485d
Integrate autoninja.py with fast_local_dev_server.py
...
- Starts the build server when the build starts.
- Writes tty filename to env variable.
- Tells the build server about the current build so it does not exit
until autoninja does, even if idle.
- Cancels pending tasks on Ctrl+c.
Change-Id: I86bb9852bd0975f381b049b9ff21c38eef7cef9d
Bug: 370589852
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5917985
Reviewed-by: Junji Watanabe <jwata@google.com >
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org >
Reviewed-by: Fumitoshi Ukai <ukai@google.com >
Auto-Submit: Mohamed Heikal <mheikal@chromium.org >
Commit-Queue: Mohamed Heikal <mheikal@chromium.org >
2024-12-12 10:28:46 -08:00
Fumitoshi Ukai
f5e1092339
siso: print error if siso binary doesn't exist in third_party/siso
...
If build/config/siso/.sisoenv exists, but third_party/siso/cipd/siso
doesn't exist, it prints error
depot_tools/siso.py: Could not find .sisoenv under build/config/siso of the current project. Did you run gclient sync?
But it should print error as siso binary not in third_party/siso
instead.
Change-Id: I6a6a920b0e6d877551f0fa7c650ab7c8165180bf
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5802659
Auto-Submit: Fumitoshi Ukai <ukai@google.com >
Commit-Queue: Fumitoshi Ukai <ukai@google.com >
Reviewed-by: Takuto Ikuta <tikuta@chromium.org >
2024-08-23 07:49:25 +00:00
Fumitoshi Ukai
06ab4cbb26
siso: call checkOutdir to prevent from running siso in ninja's out
...
checkOutdir should be called before run `siso ninja`
to not run `siso ninja` where `ninja` is used.
Change-Id: I55453b21d35396b59924be5398b47f67b592eca1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5802658
Reviewed-by: Philipp Wollermann <philwo@google.com >
Reviewed-by: Takuto Ikuta <tikuta@chromium.org >
Auto-Submit: Fumitoshi Ukai <ukai@google.com >
Commit-Queue: Fumitoshi Ukai <ukai@google.com >
2024-08-23 06:22:39 +00:00
Fumitoshi Ukai
4eb5cc887d
siso: allow run $SISO_PATH even if .sisoenv not found
...
When $SISO_PATH is set, allow to run it even if .sisoenv not found.
Useful for `siso fetch`, `siso recall` etc, which would not be
needed to run in chromium checkout dir.
Change-Id: If3bfcf879cc67dc41bedf17cd771c94de836c949
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5804427
Commit-Queue: Richard Wang <richardwa@google.com >
Auto-Submit: Fumitoshi Ukai <ukai@google.com >
Commit-Queue: Fumitoshi Ukai <ukai@google.com >
Reviewed-by: Philipp Wollermann <philwo@google.com >
Reviewed-by: Richard Wang <richardwa@google.com >
2024-08-23 05:52:46 +00:00
Josip Sokcevic
43c6415bce
Add a new possible siso path
...
We are relocating siso binary from third_party/siso/siso to
third_party/siso/cipd/siso to make it compatible with Cog, which
requires cipd packages to be placed in directories with no Git content.
Bug: 346837573
Change-Id: If40e6033e341dcedfee5670f07d68c14bd923f7b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5625091
Reviewed-by: Philipp Wollermann <philwo@google.com >
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org >
2024-06-13 00:31:10 +00:00
Fumitoshi Ukai
951578030e
siso: set PYTHONDONTWRITEBYTECODE=1 if not set
...
during build, lots of python scripts run and generates
lots of pycache *.pyc in workspace.
To keep workspace clean, set PYTHONDONTWRITEBYTECODE=1
in siso.py
Change-Id: I725c6745c44410eef4a780b8ffc30fdafd899334
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5606824
Reviewed-by: Philipp Wollermann <philwo@google.com >
Reviewed-by: Erik Staab <estaab@chromium.org >
Commit-Queue: Fumitoshi Ukai <ukai@google.com >
Auto-Submit: Fumitoshi Ukai <ukai@google.com >
2024-06-07 01:33:06 +00:00
Fumitoshi Ukai
08d5eb6d68
Reland "siso.py check ninja marker"
...
This reverts siso.py in commit 6ed0d7b86c
siso.py doesn't affect `ninja -t commands`
If out dir was built by Ninja, don't build by Siso.
(still allow `siso query` etc)
Bug: b/338414465
Change-Id: I3b317ca40153cee3987390d8b62ac070b6f85243
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5524213
Commit-Queue: Andrew Grieve <agrieve@chromium.org >
Reviewed-by: Andrew Grieve <agrieve@chromium.org >
Auto-Submit: Fumitoshi Ukai <ukai@google.com >
2024-05-08 01:57:35 +00:00
Andrew Grieve
6ed0d7b86c
Revert "Reland "siso.py: check ninja marker""
...
This reverts commit c9fb389f4b .
Reason for revert: Breaks ability to do "ninja -t commands"
Original change's description:
> Reland "siso.py: check ninja marker"
>
> This reverts commit b1c6497e19 .
>
> Reason for revert: Fix var name
>
> ```
> ukai@ukai ...src/chromium/src % ~/src/depot_tools/depot_tools/siso ninja -C out/ninja
> depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
> depot_tools/siso.py: out/ninja contains Ninja state file.
> Use `autoninja` to use reclient,
> or run `gn clean out/ninja` to switch from ninja to siso
>
> ukai@ukai ...src/chromium/src % ~/src/depot_tools/depot_tools/siso ninja -C out/siso base
> depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
> 4.63s init credentials
> reapi instance: projects/rbe-chrome-untrusted/instances/default_instance
> 1.63s loading fs state
> 3.42s load build.ninja
> 15.32s Regenerating ninja files
> 3.64s reloading
> build finished
> local:3 remote:0 cache:0 fallback:0 skip:3026
> reapi: ops: 17(err:1) / r:0(err:0) 0B / w:0(err:0) 0B
> fs: ops: 12092(err:192) / r:350(err:0) 189.04MiB / w:1(err:0) 13.96KiB
> 27.29s Build Succeeded: 3 steps - 0.11/s
>
> ```
>
> Original change's description:
> > Revert "siso.py: check ninja marker"
> >
> > This reverts commit 41bbfb8b90 .
> >
> > Reason for revert:
> > ukai@ukai ...src/chromium/src % siso ninja -C out/siso
> > depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
> > Traceback (most recent call last):
> > File "/usr/local/google/home/ukai/depot_tools/siso.py", line 115, in <module>
> > sys.exit(main(sys.argv))
> > ^^^^^^^^^^^^^^
> > File "/usr/local/google/home/ukai/depot_tools/siso.py", line 104, in main
> > checkOutdir(args[1:])
> > File "/usr/local/google/home/ukai/depot_tools/siso.py", line 21, in checkOutdir
> > for i, arg in enumerate(ninja_args):
> > ^^^^^^^^^^
> > NameError: name 'ninja_args' is not defined. Did you mean: 'ninja_marker'?
> >
> >
> > Original change's description:
> > > siso.py: check ninja marker
> > >
> > > If out dir was built by Ninja, don't build by Siso.
> > >
> > > (still allow `siso query` etc)
> > >
> > > Bug: b/338414465
> > > Change-Id: I48f8fbb5e83714fd3d27ff3a2bfffe929e4ef121
> > > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5518396
> > > Reviewed-by: Junji Watanabe <jwata@google.com >
> > > Commit-Queue: Junji Watanabe <jwata@google.com >
> > > Auto-Submit: Fumitoshi Ukai <ukai@google.com >
> > > Commit-Queue: Fumitoshi Ukai <ukai@google.com >
> >
> > Bug: b/338414465
> > Change-Id: Ie69ca1b6c1d3dc47475d9356beab29a917356479
> > No-Presubmit: true
> > No-Tree-Checks: true
> > No-Try: true
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5514902
> > Reviewed-by: Philipp Wollermann <philwo@chromium.org >
> > Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com >
> > Commit-Queue: Fumitoshi Ukai <ukai@google.com >
>
> Bug: b/338414465
> Change-Id: Idb18ad9191a062f83b538c1e2a76ee77200c4df0
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5518397
> Reviewed-by: Philipp Wollermann <philwo@chromium.org >
> Auto-Submit: Fumitoshi Ukai <ukai@google.com >
> Commit-Queue: Fumitoshi Ukai <ukai@google.com >
Bug: b/338414465
Change-Id: I0a24db3675762b92e96ab37b629fa12b4af89240
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5522633
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org >
Auto-Submit: Andrew Grieve <agrieve@chromium.org >
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org >
2024-05-07 16:10:27 +00:00
Fumitoshi Ukai
c9fb389f4b
Reland "siso.py: check ninja marker"
...
This reverts commit b1c6497e19 .
Reason for revert: Fix var name
```
ukai@ukai ...src/chromium/src % ~/src/depot_tools/depot_tools/siso ninja -C out/ninja
depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
depot_tools/siso.py: out/ninja contains Ninja state file.
Use `autoninja` to use reclient,
or run `gn clean out/ninja` to switch from ninja to siso
ukai@ukai ...src/chromium/src % ~/src/depot_tools/depot_tools/siso ninja -C out/siso base
depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
4.63s init credentials
reapi instance: projects/rbe-chrome-untrusted/instances/default_instance
1.63s loading fs state
3.42s load build.ninja
15.32s Regenerating ninja files
3.64s reloading
build finished
local:3 remote:0 cache:0 fallback:0 skip:3026
reapi: ops: 17(err:1) / r:0(err:0) 0B / w:0(err:0) 0B
fs: ops: 12092(err:192) / r:350(err:0) 189.04MiB / w:1(err:0) 13.96KiB
27.29s Build Succeeded: 3 steps - 0.11/s
```
Original change's description:
> Revert "siso.py: check ninja marker"
>
> This reverts commit 41bbfb8b90 .
>
> Reason for revert:
> ukai@ukai ...src/chromium/src % siso ninja -C out/siso
> depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
> Traceback (most recent call last):
> File "/usr/local/google/home/ukai/depot_tools/siso.py", line 115, in <module>
> sys.exit(main(sys.argv))
> ^^^^^^^^^^^^^^
> File "/usr/local/google/home/ukai/depot_tools/siso.py", line 104, in main
> checkOutdir(args[1:])
> File "/usr/local/google/home/ukai/depot_tools/siso.py", line 21, in checkOutdir
> for i, arg in enumerate(ninja_args):
> ^^^^^^^^^^
> NameError: name 'ninja_args' is not defined. Did you mean: 'ninja_marker'?
>
>
> Original change's description:
> > siso.py: check ninja marker
> >
> > If out dir was built by Ninja, don't build by Siso.
> >
> > (still allow `siso query` etc)
> >
> > Bug: b/338414465
> > Change-Id: I48f8fbb5e83714fd3d27ff3a2bfffe929e4ef121
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5518396
> > Reviewed-by: Junji Watanabe <jwata@google.com >
> > Commit-Queue: Junji Watanabe <jwata@google.com >
> > Auto-Submit: Fumitoshi Ukai <ukai@google.com >
> > Commit-Queue: Fumitoshi Ukai <ukai@google.com >
>
> Bug: b/338414465
> Change-Id: Ie69ca1b6c1d3dc47475d9356beab29a917356479
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5514902
> Reviewed-by: Philipp Wollermann <philwo@chromium.org >
> Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com >
> Commit-Queue: Fumitoshi Ukai <ukai@google.com >
Bug: b/338414465
Change-Id: Idb18ad9191a062f83b538c1e2a76ee77200c4df0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5518397
Reviewed-by: Philipp Wollermann <philwo@chromium.org >
Auto-Submit: Fumitoshi Ukai <ukai@google.com >
Commit-Queue: Fumitoshi Ukai <ukai@google.com >
2024-05-07 08:55:37 +00:00
Fumitoshi Ukai
b1c6497e19
Revert "siso.py: check ninja marker"
...
This reverts commit 41bbfb8b90 .
Reason for revert:
ukai@ukai ...src/chromium/src % siso ninja -C out/siso
depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
Traceback (most recent call last):
File "/usr/local/google/home/ukai/depot_tools/siso.py", line 115, in <module>
sys.exit(main(sys.argv))
^^^^^^^^^^^^^^
File "/usr/local/google/home/ukai/depot_tools/siso.py", line 104, in main
checkOutdir(args[1:])
File "/usr/local/google/home/ukai/depot_tools/siso.py", line 21, in checkOutdir
for i, arg in enumerate(ninja_args):
^^^^^^^^^^
NameError: name 'ninja_args' is not defined. Did you mean: 'ninja_marker'?
Original change's description:
> siso.py: check ninja marker
>
> If out dir was built by Ninja, don't build by Siso.
>
> (still allow `siso query` etc)
>
> Bug: b/338414465
> Change-Id: I48f8fbb5e83714fd3d27ff3a2bfffe929e4ef121
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5518396
> Reviewed-by: Junji Watanabe <jwata@google.com >
> Commit-Queue: Junji Watanabe <jwata@google.com >
> Auto-Submit: Fumitoshi Ukai <ukai@google.com >
> Commit-Queue: Fumitoshi Ukai <ukai@google.com >
Bug: b/338414465
Change-Id: Ie69ca1b6c1d3dc47475d9356beab29a917356479
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5514902
Reviewed-by: Philipp Wollermann <philwo@chromium.org >
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com >
Commit-Queue: Fumitoshi Ukai <ukai@google.com >
2024-05-07 08:31:16 +00:00
Fumitoshi Ukai
41bbfb8b90
siso.py: check ninja marker
...
If out dir was built by Ninja, don't build by Siso.
(still allow `siso query` etc)
Bug: b/338414465
Change-Id: I48f8fbb5e83714fd3d27ff3a2bfffe929e4ef121
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5518396
Reviewed-by: Junji Watanabe <jwata@google.com >
Commit-Queue: Junji Watanabe <jwata@google.com >
Auto-Submit: Fumitoshi Ukai <ukai@google.com >
Commit-Queue: Fumitoshi Ukai <ukai@google.com >
2024-05-07 08:23:35 +00:00
Fumitoshi Ukai
9f668bbff9
siso: use stderr to print message from siso.py
...
siso binary will print json etc (e.g. `siso fs export`)
and usually used to pipe to jq or so.
but siso.py print message when $SISO_PATH is set
and corrupt json message.
use stderr for such message.
Change-Id: Ic35ecabc159fb2053a8a29816c276d914f8d2468
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5197566
Auto-Submit: Fumitoshi Ukai <ukai@google.com >
Commit-Queue: Fumitoshi Ukai <ukai@google.com >
Reviewed-by: Takuto Ikuta <tikuta@chromium.org >
Commit-Queue: Takuto Ikuta <tikuta@chromium.org >
2024-01-15 08:53:04 +00:00
Junji Watanabe
9ed72f91a9
[siso] Propagte SIGINT to siso process
...
siso's signal handler doesn't run as expected with siso/autosiso
wrappers.
This CL sets signal handlers for the Python wrappers to propagate
signals to siso.
Note that Siso immediately exists at the 2nd Ctrl-C.
So developers can stop Siso forcibly when they want.
Bug: b/308734805
Change-Id: Idd8cc1d99931d609cfc3bedfb4c0bb4319cd20b3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4996768
Reviewed-by: Fumitoshi Ukai <ukai@google.com >
Commit-Queue: Junji Watanabe <jwata@google.com >
2023-11-01 09:13:25 +00:00
Mike Frysinger
124bb8e53c
switch to 4 space indent
...
Leave the recipes/ code at 2 space to match the rest of the recipes
project in other repos.
Reformatted using:
files=( $(
git ls-tree -r --name-only HEAD | \
grep -Ev -e '^(third_party|recipes)/' | \
grep '\.py$';
git grep -l '#!/usr/bin/env.*python' | grep -v '\.py$'
) )
parallel ./yapf -i -- "${files[@]}"
~/chromiumos/chromite/contrib/reflow_overlong_comments "${files[@]}"
The files that still had strings that were too long were manually
reformatted because they were easy and only a few issues.
autoninja.py
clang_format.py
download_from_google_storage.py
fix_encoding.py
gclient_utils.py
git_cache.py
git_common.py
git_map_branches.py
git_reparent_branch.py
gn.py
my_activity.py
owners_finder.py
presubmit_canned_checks.py
reclient_helper.py
reclientreport.py
roll_dep.py
rustfmt.py
siso.py
split_cl.py
subcommand.py
subprocess2.py
swift_format.py
upload_to_google_storage.py
These files still had lines (strings) that were too long, so the pylint
warnings were suppressed with a TODO.
auth.py
gclient.py
gclient_eval.py
gclient_paths.py
gclient_scm.py
gerrit_util.py
git_cl.py
presubmit_canned_checks.py
presubmit_support.py
scm.py
Change-Id: Ia6535c4f2c48d46b589ec1e791dde6c6b2ea858f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4836379
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org >
Auto-Submit: Mike Frysinger <vapier@chromium.org >
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org >
2023-09-06 05:48:55 +00:00
Junji Watanabe
4480f863c5
[siso] check presence of .sisoenv file
...
`set([primary_solution_path, gclient_root_path, gclient_src_root_path])` doesn't guarantee order.
This is problematic when `SISO_PATH` is used because it doesn't check the presence of `third_party/siso/siso` binary.
This CL changes siso.py to check the presence of `.sisoenv` to detect the correct `base_dir`.
Change-Id: Ieb1acc44269037d43c793eb28cd2f12ad0179b17
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4625315
Auto-Submit: Junji Watanabe <jwata@google.com >
Commit-Queue: Junji Watanabe <jwata@google.com >
Reviewed-by: Philipp Wollermann <philwo@google.com >
Reviewed-by: Fumitoshi Ukai <ukai@google.com >
2023-06-20 05:14:16 +00:00
Richard Wang
4992184cd1
[autosiso] add SISO_PATH env var
...
Bug: b/287895730
Change-Id: Ibbf658f802d46a14fc82a622f7f853dcadc25e5f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4624217
Reviewed-by: Philipp Wollermann <philwo@google.com >
Commit-Queue: Richard Wang <richardwa@google.com >
2023-06-19 07:34:02 +00:00
Fumitoshi Ukai
3ca8d0d0ca
add siso wrapper
...
Bug: b/277855544, b/277859568
Change-Id: I520834647881745db7a42fd357bc829a7c6779d8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4413517
Reviewed-by: Philipp Wollermann <philwo@google.com >
Reviewed-by: Joanna Wang <jojwang@chromium.org >
Reviewed-by: Takuto Ikuta <tikuta@chromium.org >
Reviewed-by: Junji Watanabe <jwata@google.com >
Commit-Queue: Joanna Wang <jojwang@chromium.org >
Commit-Queue: Junji Watanabe <jwata@google.com >
Auto-Submit: Fumitoshi Ukai <ukai@google.com >
2023-04-13 05:24:08 +00:00