Commit Graph

32 Commits

Author SHA1 Message Date
Alex Ovsienko
2bc167bbca siso: add default metrics project in telemetry flags if none is specified.
Bug: b/348530235
Change-Id: Iba012ffa79c4d13033234c3afc4f18b36a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7141142
Reviewed-by: Junji Watanabe <jwata@google.com>
Auto-Submit: Alex Ovsienko <ovsienko@google.com>
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
2025-11-12 00:02:23 -08:00
Alex Ovsienko
f107245af8 siso: set GOOGLE_API_USE_CLIENT_CERTIFICATE for googlers that run siso.py without running autoninja.py
apply_telemetry_flags runs only for googlers, so we don't check if it's the googler running the code again.

Additionally amend the comment.

Bug: b/348530235
Change-Id: I3b1ef1edc8c03a4aba11bdc474560de66a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7143899
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
2025-11-11 22:33:43 -08:00
Alex Ovsienko
345a14adf0 siso: enable system limits fix when we run only siso.py.
Change-Id: I355e2d12833fefdc0e9d53b53a5583fc6a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7144919
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
2025-11-11 21:34:48 -08:00
Alex Ovsienko
32a8dcc842 siso: Make it so that we check the second parameter of siso whether it is ninja or not.
The first one is always `siso`.

Bug: b/348530235
Change-Id: I2135d87cb676156b4e14ace31928e9de6a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7140206
Reviewed-by: Junji Watanabe <jwata@google.com>
Auto-Submit: Alex Ovsienko <ovsienko@google.com>
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
2025-11-11 00:08:06 -08:00
Alex Ovsienko
b26ab6e9d5 Have ninja flags be applied in more obvious manner.
Bug: b/348530235
Change-Id: Idfac0789c4881a376678f749d0d20b3309fd240c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7136527
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
2025-11-09 21:37:15 -08:00
Alex Ovsienko
2b4898e327 Reland "Enable telemetry flags for Google corp machines only."
This reverts commit 6f83fce865.

Reason for revert: Make sure that the flags are applied only for ninja runs.

Original change's description:
> Revert "Enable telemetry flags for Google corp machines only."
>
> This reverts commit 4e2be179a1.
>
> Reason for revert: 
> This caused exception
>
> ```
> + autoninja -strict_remote -d keepdepfile -d keeprsp -C out/Default chrome
> siso ninja -strict_remote -d keepdepfile -d keeprsp -C out/Default chrome
> ninja: Entering directory `out/Default'
>  4.51s init credentials by "credhelper"
> https://console.cloud.google.com/logs/viewer?project=rbe-chrome-untrusted&resource=generic_task/task_id/25d8cbb8-4fa1-486d-ba78-7865d0eea5ac
> use RBE instance "projects/rbe-chrome-untrusted/instances/default_instance"
>  2.27s loading/recompacting deps log
>  3.47s Regenerating ninja files
>  1.17s load siso config
> build finished
> local:3636 remote:33214 cache:3317 cache-write:0(err:0) fallback:0 retry:0 skip:40752
> reapi: ops: 102977(err:33221) / r:46503(err:8) 3.13GiB / w:72424(err:0) 152.35MiB
> fs: ops: 2790971(err:182693) / r:227138(err:0) 6.68GiB / w:13508(err:4) 1.87GiB
>     resource/capa used(err)  wait-avg |   s m |  serv-avg |   s m |
>   pool=action/128   2372(0)     0.00s |█      |     0.30s | █▅▂▂▂ |
>         rbe:sched  33214(0)     0.01s |▃█▂▂   |     6.43s |  ▂█▃▂ |
>        rbe:worker  33214(0)     0.87s |    █  |     5.56s |  ▂█▃▂ |
>  remoteexec/10240  33214(0)     0.10s |█▂▂▂   |    15.37s |   ██▂ |
>  1.00s shutdown cloud logging/monitoring
> 4m08.96s Build Succeeded: 40167 steps - 161.34/s
> flag provided but not defined: -enable_cloud_monitoring
> summarize siso_metrics.json
>
>  $ siso metrics summary -C <dir> \
>     [--step_types <types>] \
>     [--elapsed_time_sorting] \
>     [--elapsed_time=run|step] \
>     [--input siso_metrics.json]
>
> summarize <dir>/.siso_metrics.json (--input)
> as depot_tools/post_ninja_build_summary.py does.
>   -C string
>         ninja running directory, where siso_metrics.json exists (default ".")
>   -elapsed_time string
>         metrics to use for elapsed time. "run" or "step". "run": time to run local command or call remote execution.  "step": full duration for the step, including preproc, waiting resource to run command etc. (default "run")
>   -elapsed_time_sorting
>         Sort output by elapsed time instead of weighted time
>   -input string
>         filename of siso_metrics.json to summarize (default "siso_metrics.json")
>   -step_types string
>         semicolon separated glob patterns (go filepath.Match) for build-step grouping
> Traceback (most recent call last):
>   File "/usr/local/google/home/tikuta/depot_tools/post_build_ninja_summary.py", line 390, in <module>
>     sys.exit(main())
>              ^^^^^^
>   File "/usr/local/google/home/tikuta/depot_tools/post_build_ninja_summary.py", line 375, in main
>     subprocess.run(cmd, check=True)
>   File "/usr/local/google/home/tikuta/depot_tools/bootstrap-2@3.11.8.chromium.35_bin/python3/lib/python3.11/subprocess.py", line 571, in run
>     raise CalledProcessError(retcode, process.args,
> subprocess.CalledProcessError: Command '['/usr/local/google/home/tikuta/depot_tools/bootstrap-2@3.11.8.chromium.35_bin/python3/bin/python3', '/usr/local/google/home/tikuta/depot_tools/siso.py', 'metrics', 'summary', '-C', 'out/Default', '--step_types', 'trict_remote']' returned non-zero exit status 2.
> ```
>
> Original change's description:
> > Enable telemetry flags for Google corp machines only.
> >
> > Bug: b/348530235
> > Change-Id: I4ac32c60b806f44920bd06e31ec6c3b86a6a6964
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7104218
> > Commit-Queue: Alex Ovsienko <ovsienko@google.com>
> > Reviewed-by: Junji Watanabe <jwata@google.com>
>
> Bug: b/348530235
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Change-Id: Iff13fa73745e09d2dab4278e9571f7ed77bf3d0c
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7136522
> Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
> Reviewed-by: Alex Ovsienko <ovsienko@google.com>
> Commit-Queue: Alex Ovsienko <ovsienko@google.com>

Bug: b/348530235
Change-Id: I6307ea67de0045c705e087e1711b295c8cce4703
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7136644
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
2025-11-09 20:17:56 -08:00
Takuto Ikuta
6f83fce865 Revert "Enable telemetry flags for Google corp machines only."
This reverts commit 4e2be179a1.

Reason for revert: 
This caused exception

```
+ autoninja -strict_remote -d keepdepfile -d keeprsp -C out/Default chrome
siso ninja -strict_remote -d keepdepfile -d keeprsp -C out/Default chrome
ninja: Entering directory `out/Default'
 4.51s init credentials by "credhelper"
https://console.cloud.google.com/logs/viewer?project=rbe-chrome-untrusted&resource=generic_task/task_id/25d8cbb8-4fa1-486d-ba78-7865d0eea5ac
use RBE instance "projects/rbe-chrome-untrusted/instances/default_instance"
 2.27s loading/recompacting deps log
 3.47s Regenerating ninja files
 1.17s load siso config
build finished
local:3636 remote:33214 cache:3317 cache-write:0(err:0) fallback:0 retry:0 skip:40752
reapi: ops: 102977(err:33221) / r:46503(err:8) 3.13GiB / w:72424(err:0) 152.35MiB
fs: ops: 2790971(err:182693) / r:227138(err:0) 6.68GiB / w:13508(err:4) 1.87GiB
    resource/capa used(err)  wait-avg |   s m |  serv-avg |   s m |
  pool=action/128   2372(0)     0.00s |█      |     0.30s | █▅▂▂▂ |
        rbe:sched  33214(0)     0.01s |▃█▂▂   |     6.43s |  ▂█▃▂ |
       rbe:worker  33214(0)     0.87s |    █  |     5.56s |  ▂█▃▂ |
 remoteexec/10240  33214(0)     0.10s |█▂▂▂   |    15.37s |   ██▂ |
 1.00s shutdown cloud logging/monitoring
4m08.96s Build Succeeded: 40167 steps - 161.34/s
flag provided but not defined: -enable_cloud_monitoring
summarize siso_metrics.json

 $ siso metrics summary -C <dir> \
    [--step_types <types>] \
    [--elapsed_time_sorting] \
    [--elapsed_time=run|step] \
    [--input siso_metrics.json]

summarize <dir>/.siso_metrics.json (--input)
as depot_tools/post_ninja_build_summary.py does.
  -C string
        ninja running directory, where siso_metrics.json exists (default ".")
  -elapsed_time string
        metrics to use for elapsed time. "run" or "step". "run": time to run local command or call remote execution.  "step": full duration for the step, including preproc, waiting resource to run command etc. (default "run")
  -elapsed_time_sorting
        Sort output by elapsed time instead of weighted time
  -input string
        filename of siso_metrics.json to summarize (default "siso_metrics.json")
  -step_types string
        semicolon separated glob patterns (go filepath.Match) for build-step grouping
Traceback (most recent call last):
  File "/usr/local/google/home/tikuta/depot_tools/post_build_ninja_summary.py", line 390, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/local/google/home/tikuta/depot_tools/post_build_ninja_summary.py", line 375, in main
    subprocess.run(cmd, check=True)
  File "/usr/local/google/home/tikuta/depot_tools/bootstrap-2@3.11.8.chromium.35_bin/python3/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/local/google/home/tikuta/depot_tools/bootstrap-2@3.11.8.chromium.35_bin/python3/bin/python3', '/usr/local/google/home/tikuta/depot_tools/siso.py', 'metrics', 'summary', '-C', 'out/Default', '--step_types', 'trict_remote']' returned non-zero exit status 2.
```

Original change's description:
> Enable telemetry flags for Google corp machines only.
>
> Bug: b/348530235
> Change-Id: I4ac32c60b806f44920bd06e31ec6c3b86a6a6964
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7104218
> Commit-Queue: Alex Ovsienko <ovsienko@google.com>
> Reviewed-by: Junji Watanabe <jwata@google.com>

Bug: b/348530235
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Change-Id: Iff13fa73745e09d2dab4278e9571f7ed77bf3d0c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7136522
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Alex Ovsienko <ovsienko@google.com>
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
2025-11-09 18:39:02 -08:00
Alex Ovsienko
4e2be179a1 Enable telemetry flags for Google corp machines only.
Bug: b/348530235
Change-Id: I4ac32c60b806f44920bd06e31ec6c3b86a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7104218
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
Reviewed-by: Junji Watanabe <jwata@google.com>
2025-11-09 17:13:31 -08:00
Alex Ovsienko
36ab0d8d35 siso: add metadata labels fetching and parsing to siso for developer builds.
Bug: b/449906174
Change-Id: I218c156c89b98f09d81c4bea6ba1b5ed921d71ec
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7044008
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
2025-10-16 23:01:37 -07:00
Fumitoshi Ukai
0f565fa858 siso: add .sisorc support
if build/config/siso/.sisorc exists, it will add global flags
or subcmd flags.

global flags is line starting with "-"
e.g.
--credential_helper=gcloud

subcmd flags are line starting with subcmd name.
e.g.
ninja --verbose_failures=false -k=0

Bug: b/269554009
Change-Id: I4691b9e17571721dd5b70f6ffb063e2d2f0ac4e3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6910278
Reviewed-by: Scott Lee <ddoman@chromium.org>
Reviewed-by: Philipp Wollermann <philwo@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
2025-09-03 19:03:18 -07:00
Fumitoshi Ukai
f80f5bc99e autoninja: support running from outside of workspace
gclient_paths.GetPrimarySolutionPath() finds primary
solution path from current directory, so it would fail
if it is invoked from outside of workspace.

Pass directory to gclient_paths.GetPrimarySolutionPath
so it could find workspace correctly if it is invoked
ninja outside of workspace.

Bug: 441240584
Change-Id: I873f7883873e143ec8a64ee0e636042ac2336a2a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6898614
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Reviewed-by: Scott Lee <ddoman@chromium.org>
Commit-Queue: Scott Lee <ddoman@chromium.org>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
2025-09-02 13:08:00 -07:00
Junji Watanabe
e68bbfc572 Prepend caffeinate to build commands on macOS
When building on macOS, it's better to avoid sleeping with caffeinate by
default. This CL prepends `caffeinate` to the build commands, which can
be opted out with `--no-caffeinate`.

Change-Id: Iacb72acb6523576305b38fe45bfef12f87121a93
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6596109
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Junji Watanabe <jwata@google.com>
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
2025-05-28 18:32:34 -07:00
Arthur Milchior
cd6685b6c0 [siso]Adding import
`shutil` was used in
https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6420348
but not imported, leading to error message when executing
_is_google_corp_machine

Bug: None
Change-Id: I0bec8986f531a31905dabaadd20af68c1cd5536f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6449462
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Junji Watanabe <jwata@google.com>
Auto-Submit: Arthur Milchior <arthurmilchior@chromium.org>
2025-04-11 07:15:47 -07:00
Fumitoshi Ukai
e7df27f40f siso: don't check backend.star if backend_config dir doesn't exist
Bug: 408114984
Change-Id: I354bcd7e3c0f0e6af6496054f5f295c927cfc8f0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6431647
Commit-Queue: Philipp Wollermann <philwo@google.com>
Reviewed-by: Philipp Wollermann <philwo@google.com>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
2025-04-04 06:58:15 -07:00
Fumitoshi Ukai
e1f6feb7ac siso: better error message when backend.star doesn't exist
Change-Id: I0c87b2c371f71b9777baaf2a4acc2111b30f3633
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6420348
Feels: Gary Tong <gatong@chromium.org>
Reviewed-by: Gary Tong <gatong@chromium.org>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
2025-04-02 19:30:12 -07:00
Mohamed Heikal
9b4d1e485d Integrate autoninja.py with fast_local_dev_server.py
- Starts the build server when the build starts.
- Writes tty filename to env variable.
- Tells the build server about the current build so it does not exit
  until autoninja does, even if idle.
- Cancels pending tasks on Ctrl+c.

Change-Id: I86bb9852bd0975f381b049b9ff21c38eef7cef9d
Bug: 370589852
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5917985
Reviewed-by: Junji Watanabe <jwata@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Auto-Submit: Mohamed Heikal <mheikal@chromium.org>
Commit-Queue: Mohamed Heikal <mheikal@chromium.org>
2024-12-12 10:28:46 -08:00
Fumitoshi Ukai
f5e1092339 siso: print error if siso binary doesn't exist in third_party/siso
If build/config/siso/.sisoenv exists, but third_party/siso/cipd/siso
doesn't exist, it prints error

depot_tools/siso.py: Could not find .sisoenv under build/config/siso of the current project. Did you run gclient sync?

But it should print error as siso binary not in third_party/siso
instead.

Change-Id: I6a6a920b0e6d877551f0fa7c650ab7c8165180bf
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5802659
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
2024-08-23 07:49:25 +00:00
Fumitoshi Ukai
06ab4cbb26 siso: call checkOutdir to prevent from running siso in ninja's out
checkOutdir should be called before run `siso ninja`
to not run `siso ninja` where `ninja` is used.

Change-Id: I55453b21d35396b59924be5398b47f67b592eca1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5802658
Reviewed-by: Philipp Wollermann <philwo@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
2024-08-23 06:22:39 +00:00
Fumitoshi Ukai
4eb5cc887d siso: allow run $SISO_PATH even if .sisoenv not found
When $SISO_PATH is set, allow to run it even if .sisoenv not found.
Useful for `siso fetch`, `siso recall` etc, which would not be
needed to run in chromium checkout dir.

Change-Id: If3bfcf879cc67dc41bedf17cd771c94de836c949
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5804427
Commit-Queue: Richard Wang <richardwa@google.com>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
Reviewed-by: Philipp Wollermann <philwo@google.com>
Reviewed-by: Richard Wang <richardwa@google.com>
2024-08-23 05:52:46 +00:00
Josip Sokcevic
43c6415bce Add a new possible siso path
We are relocating siso binary from third_party/siso/siso to
third_party/siso/cipd/siso to make it compatible with Cog, which
requires cipd packages to be placed in directories with no Git content.

Bug: 346837573
Change-Id: If40e6033e341dcedfee5670f07d68c14bd923f7b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5625091
Reviewed-by: Philipp Wollermann <philwo@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
2024-06-13 00:31:10 +00:00
Fumitoshi Ukai
951578030e siso: set PYTHONDONTWRITEBYTECODE=1 if not set
during build, lots of python scripts run and generates
lots of pycache *.pyc in workspace.
To keep workspace clean, set PYTHONDONTWRITEBYTECODE=1
in siso.py

Change-Id: I725c6745c44410eef4a780b8ffc30fdafd899334
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5606824
Reviewed-by: Philipp Wollermann <philwo@google.com>
Reviewed-by: Erik Staab <estaab@chromium.org>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
2024-06-07 01:33:06 +00:00
Fumitoshi Ukai
08d5eb6d68 Reland "siso.py check ninja marker"
This reverts siso.py in commit 6ed0d7b86c
siso.py doesn't affect `ninja -t commands`

If out dir was built by Ninja, don't build by Siso.
(still allow `siso query` etc)

Bug: b/338414465
Change-Id: I3b317ca40153cee3987390d8b62ac070b6f85243
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5524213
Commit-Queue: Andrew Grieve <agrieve@chromium.org>
Reviewed-by: Andrew Grieve <agrieve@chromium.org>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
2024-05-08 01:57:35 +00:00
Andrew Grieve
6ed0d7b86c Revert "Reland "siso.py: check ninja marker""
This reverts commit c9fb389f4b.

Reason for revert: Breaks ability to do "ninja -t commands"

Original change's description:
> Reland "siso.py: check ninja marker"
>
> This reverts commit b1c6497e19.
>
> Reason for revert: Fix var name
>
> ```
> ukai@ukai ...src/chromium/src % ~/src/depot_tools/depot_tools/siso ninja -C out/ninja
> depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
> depot_tools/siso.py: out/ninja contains Ninja state file.
> Use `autoninja` to use reclient,
> or run `gn clean out/ninja` to switch from ninja to siso
>
> ukai@ukai ...src/chromium/src % ~/src/depot_tools/depot_tools/siso ninja -C out/siso base
> depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
>  4.63s init credentials
> reapi instance: projects/rbe-chrome-untrusted/instances/default_instance
>  1.63s loading fs state
>  3.42s load build.ninja
> 15.32s Regenerating ninja files
>  3.64s reloading
> build finished
> local:3 remote:0 cache:0 fallback:0 skip:3026
> reapi: ops: 17(err:1) / r:0(err:0) 0B / w:0(err:0) 0B
> fs: ops: 12092(err:192) / r:350(err:0) 189.04MiB / w:1(err:0) 13.96KiB
> 27.29s Build Succeeded: 3 steps - 0.11/s
>
> ```
>
> Original change's description:
> > Revert "siso.py: check ninja marker"
> >
> > This reverts commit 41bbfb8b90.
> >
> > Reason for revert:
> > ukai@ukai ...src/chromium/src % siso ninja -C out/siso
> > depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
> > Traceback (most recent call last):
> >   File "/usr/local/google/home/ukai/depot_tools/siso.py", line 115, in <module>
> >     sys.exit(main(sys.argv))
> >              ^^^^^^^^^^^^^^
> >   File "/usr/local/google/home/ukai/depot_tools/siso.py", line 104, in main
> >     checkOutdir(args[1:])
> >   File "/usr/local/google/home/ukai/depot_tools/siso.py", line 21, in checkOutdir
> >     for i, arg in enumerate(ninja_args):
> >                             ^^^^^^^^^^
> > NameError: name 'ninja_args' is not defined. Did you mean: 'ninja_marker'?
> >
> >
> > Original change's description:
> > > siso.py: check ninja marker
> > >
> > > If out dir was built by Ninja, don't build by Siso.
> > >
> > > (still allow `siso query` etc)
> > >
> > > Bug: b/338414465
> > > Change-Id: I48f8fbb5e83714fd3d27ff3a2bfffe929e4ef121
> > > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5518396
> > > Reviewed-by: Junji Watanabe <jwata@google.com>
> > > Commit-Queue: Junji Watanabe <jwata@google.com>
> > > Auto-Submit: Fumitoshi Ukai <ukai@google.com>
> > > Commit-Queue: Fumitoshi Ukai <ukai@google.com>
> >
> > Bug: b/338414465
> > Change-Id: Ie69ca1b6c1d3dc47475d9356beab29a917356479
> > No-Presubmit: true
> > No-Tree-Checks: true
> > No-Try: true
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5514902
> > Reviewed-by: Philipp Wollermann <philwo@chromium.org>
> > Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> > Commit-Queue: Fumitoshi Ukai <ukai@google.com>
>
> Bug: b/338414465
> Change-Id: Idb18ad9191a062f83b538c1e2a76ee77200c4df0
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5518397
> Reviewed-by: Philipp Wollermann <philwo@chromium.org>
> Auto-Submit: Fumitoshi Ukai <ukai@google.com>
> Commit-Queue: Fumitoshi Ukai <ukai@google.com>

Bug: b/338414465
Change-Id: I0a24db3675762b92e96ab37b629fa12b4af89240
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5522633
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Andrew Grieve <agrieve@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
2024-05-07 16:10:27 +00:00
Fumitoshi Ukai
c9fb389f4b Reland "siso.py: check ninja marker"
This reverts commit b1c6497e19.

Reason for revert: Fix var name

```
ukai@ukai ...src/chromium/src % ~/src/depot_tools/depot_tools/siso ninja -C out/ninja
depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
depot_tools/siso.py: out/ninja contains Ninja state file.
Use `autoninja` to use reclient,
or run `gn clean out/ninja` to switch from ninja to siso

ukai@ukai ...src/chromium/src % ~/src/depot_tools/depot_tools/siso ninja -C out/siso base
depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
 4.63s init credentials
reapi instance: projects/rbe-chrome-untrusted/instances/default_instance
 1.63s loading fs state
 3.42s load build.ninja
15.32s Regenerating ninja files
 3.64s reloading
build finished
local:3 remote:0 cache:0 fallback:0 skip:3026
reapi: ops: 17(err:1) / r:0(err:0) 0B / w:0(err:0) 0B
fs: ops: 12092(err:192) / r:350(err:0) 189.04MiB / w:1(err:0) 13.96KiB
27.29s Build Succeeded: 3 steps - 0.11/s

```

Original change's description:
> Revert "siso.py: check ninja marker"
>
> This reverts commit 41bbfb8b90.
>
> Reason for revert:
> ukai@ukai ...src/chromium/src % siso ninja -C out/siso
> depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
> Traceback (most recent call last):
>   File "/usr/local/google/home/ukai/depot_tools/siso.py", line 115, in <module>
>     sys.exit(main(sys.argv))
>              ^^^^^^^^^^^^^^
>   File "/usr/local/google/home/ukai/depot_tools/siso.py", line 104, in main
>     checkOutdir(args[1:])
>   File "/usr/local/google/home/ukai/depot_tools/siso.py", line 21, in checkOutdir
>     for i, arg in enumerate(ninja_args):
>                             ^^^^^^^^^^
> NameError: name 'ninja_args' is not defined. Did you mean: 'ninja_marker'?
>
>
> Original change's description:
> > siso.py: check ninja marker
> >
> > If out dir was built by Ninja, don't build by Siso.
> >
> > (still allow `siso query` etc)
> >
> > Bug: b/338414465
> > Change-Id: I48f8fbb5e83714fd3d27ff3a2bfffe929e4ef121
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5518396
> > Reviewed-by: Junji Watanabe <jwata@google.com>
> > Commit-Queue: Junji Watanabe <jwata@google.com>
> > Auto-Submit: Fumitoshi Ukai <ukai@google.com>
> > Commit-Queue: Fumitoshi Ukai <ukai@google.com>
>
> Bug: b/338414465
> Change-Id: Ie69ca1b6c1d3dc47475d9356beab29a917356479
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5514902
> Reviewed-by: Philipp Wollermann <philwo@chromium.org>
> Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> Commit-Queue: Fumitoshi Ukai <ukai@google.com>

Bug: b/338414465
Change-Id: Idb18ad9191a062f83b538c1e2a76ee77200c4df0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5518397
Reviewed-by: Philipp Wollermann <philwo@chromium.org>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
2024-05-07 08:55:37 +00:00
Fumitoshi Ukai
b1c6497e19 Revert "siso.py: check ninja marker"
This reverts commit 41bbfb8b90.

Reason for revert: 
ukai@ukai ...src/chromium/src % siso ninja -C out/siso
depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
Traceback (most recent call last):
  File "/usr/local/google/home/ukai/depot_tools/siso.py", line 115, in <module>
    sys.exit(main(sys.argv))
             ^^^^^^^^^^^^^^
  File "/usr/local/google/home/ukai/depot_tools/siso.py", line 104, in main
    checkOutdir(args[1:])
  File "/usr/local/google/home/ukai/depot_tools/siso.py", line 21, in checkOutdir   
    for i, arg in enumerate(ninja_args):
                            ^^^^^^^^^^
NameError: name 'ninja_args' is not defined. Did you mean: 'ninja_marker'?


Original change's description:
> siso.py: check ninja marker
>
> If out dir was built by Ninja, don't build by Siso.
>
> (still allow `siso query` etc)
>
> Bug: b/338414465
> Change-Id: I48f8fbb5e83714fd3d27ff3a2bfffe929e4ef121
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5518396
> Reviewed-by: Junji Watanabe <jwata@google.com>
> Commit-Queue: Junji Watanabe <jwata@google.com>
> Auto-Submit: Fumitoshi Ukai <ukai@google.com>
> Commit-Queue: Fumitoshi Ukai <ukai@google.com>

Bug: b/338414465
Change-Id: Ie69ca1b6c1d3dc47475d9356beab29a917356479
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5514902
Reviewed-by: Philipp Wollermann <philwo@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
2024-05-07 08:31:16 +00:00
Fumitoshi Ukai
41bbfb8b90 siso.py: check ninja marker
If out dir was built by Ninja, don't build by Siso.

(still allow `siso query` etc)

Bug: b/338414465
Change-Id: I48f8fbb5e83714fd3d27ff3a2bfffe929e4ef121
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5518396
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Junji Watanabe <jwata@google.com>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
2024-05-07 08:23:35 +00:00
Fumitoshi Ukai
9f668bbff9 siso: use stderr to print message from siso.py
siso binary will print json etc (e.g. `siso fs export`)
and usually used to pipe to jq or so.

but siso.py print message when $SISO_PATH is set
and corrupt json message.

use stderr for such message.

Change-Id: Ic35ecabc159fb2053a8a29816c276d914f8d2468
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5197566
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
2024-01-15 08:53:04 +00:00
Junji Watanabe
9ed72f91a9 [siso] Propagte SIGINT to siso process
siso's signal handler doesn't run as expected with siso/autosiso
wrappers.
This CL sets signal handlers for the Python wrappers to propagate
signals to siso.

Note that Siso immediately exists at the 2nd Ctrl-C.
So developers can stop Siso forcibly when they want.

Bug: b/308734805
Change-Id: Idd8cc1d99931d609cfc3bedfb4c0bb4319cd20b3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4996768
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Junji Watanabe <jwata@google.com>
2023-11-01 09:13:25 +00:00
Mike Frysinger
124bb8e53c switch to 4 space indent
Leave the recipes/ code at 2 space to match the rest of the recipes
project in other repos.

Reformatted using:
files=( $(
	git ls-tree -r --name-only HEAD | \
		grep -Ev -e '^(third_party|recipes)/' | \
		grep '\.py$';
	git grep -l '#!/usr/bin/env.*python' | grep -v '\.py$'
) )
parallel ./yapf -i -- "${files[@]}"
~/chromiumos/chromite/contrib/reflow_overlong_comments "${files[@]}"

The files that still had strings that were too long were manually
reformatted because they were easy and only a few issues.
autoninja.py
clang_format.py
download_from_google_storage.py
fix_encoding.py
gclient_utils.py
git_cache.py
git_common.py
git_map_branches.py
git_reparent_branch.py
gn.py
my_activity.py
owners_finder.py
presubmit_canned_checks.py
reclient_helper.py
reclientreport.py
roll_dep.py
rustfmt.py
siso.py
split_cl.py
subcommand.py
subprocess2.py
swift_format.py
upload_to_google_storage.py

These files still had lines (strings) that were too long, so the pylint
warnings were suppressed with a TODO.
auth.py
gclient.py
gclient_eval.py
gclient_paths.py
gclient_scm.py
gerrit_util.py
git_cl.py
presubmit_canned_checks.py
presubmit_support.py
scm.py

Change-Id: Ia6535c4f2c48d46b589ec1e791dde6c6b2ea858f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4836379
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
2023-09-06 05:48:55 +00:00
Junji Watanabe
4480f863c5 [siso] check presence of .sisoenv file
`set([primary_solution_path, gclient_root_path, gclient_src_root_path])` doesn't guarantee order.
This is problematic when `SISO_PATH` is used because it doesn't check the presence of `third_party/siso/siso` binary.

This CL changes siso.py to check the presence of `.sisoenv` to detect the correct `base_dir`.

Change-Id: Ieb1acc44269037d43c793eb28cd2f12ad0179b17
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4625315
Auto-Submit: Junji Watanabe <jwata@google.com>
Commit-Queue: Junji Watanabe <jwata@google.com>
Reviewed-by: Philipp Wollermann <philwo@google.com>
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
2023-06-20 05:14:16 +00:00
Richard Wang
4992184cd1 [autosiso] add SISO_PATH env var
Bug: b/287895730
Change-Id: Ibbf658f802d46a14fc82a622f7f853dcadc25e5f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4624217
Reviewed-by: Philipp Wollermann <philwo@google.com>
Commit-Queue: Richard Wang <richardwa@google.com>
2023-06-19 07:34:02 +00:00
Fumitoshi Ukai
3ca8d0d0ca add siso wrapper
Bug: b/277855544, b/277859568
Change-Id: I520834647881745db7a42fd357bc829a7c6779d8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4413517
Reviewed-by: Philipp Wollermann <philwo@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Junji Watanabe <jwata@google.com>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
2023-04-13 05:24:08 +00:00