InTheForest/chromium_depot_tools
1
0
Fork 0
mirror of https://chromium.googlesource.com/chromium/tools/depot_tools.git synced 2026-01-11 02:31:29 +00:00
Code Issues Packages Projects Releases Wiki Activity
11,990 Commits 6 Branches 0 Tags
main
Commit Graph

61 Commits

Author SHA1 Message Date
Alex Ovsienko
7c837b993a siso: have siso.py use env variables to start the collector. 
Bug: b/474306065
Change-Id: I7a443531192a2b20609c43e7ad33e4426a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7414344
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
 2026-01-08 19:57:05 -08:00
Junji Watanabe
9414627328 Revert "siso: enable autoninja to use env variables instead of flags for collector." 
This reverts commit cf876cddd6.

Reason for revert: Speculatively reverting for http://b/474306065.

Original change's description:
> siso: enable autoninja to use env variables instead of flags for collector.
>
> For clients with old siso this will not cause any breakage as we kept the flags, and did not introduce any old ones. Eventually everyone should get their collector running.
>
> The collector startup will be triggered only for clients with enabled logs collection.
>
> Bug: b/455433899
> Change-Id: I962bd551c5a1cc6a5f8904e65d24939b6a6a6964
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7373289
> Commit-Queue: Alex Ovsienko <ovsienko@google.com>
> Reviewed-by: Junji Watanabe <jwata@google.com>

Bug: b/455433899
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Change-Id: If5d592499c1bbc5ea24c5b2c8ef72aa83d500972
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7407116
Auto-Submit: Junji Watanabe <jwata@google.com>
Commit-Queue: Junji Watanabe <jwata@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
 2026-01-08 05:52:45 -08:00
Junji Watanabe
6f577a0fc5 Revert "siso: clean up _start_collector function to not modify the env dict." 
This reverts commit 02da7602fb.

Reason for revert: Speculatively reverting for http://b/474306065.

Original change's description:
> siso: clean up _start_collector function to not modify the env dict.
>
> Also do not have special status for missing socks file. Clean up socks file before starting the collector.
>
> Bug: b/455433899
> Change-Id: Ib393f00e7a6098fca1b7d1d70f5aa4616a6a6964
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7408697
> Commit-Queue: Alex Ovsienko <ovsienko@google.com>
> Reviewed-by: Junji Watanabe <jwata@google.com>

Bug: b/455433899
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Change-Id: Ie095f91c72bca65102a96fa33e77b87e79cab9ac
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7407117
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Auto-Submit: Junji Watanabe <jwata@google.com>
 2026-01-08 05:52:13 -08:00
Alex Ovsienko
02da7602fb siso: clean up _start_collector function to not modify the env dict. 
Also do not have special status for missing socks file. Clean up socks file before starting the collector.

Bug: b/455433899
Change-Id: Ib393f00e7a6098fca1b7d1d70f5aa4616a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7408697
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
Reviewed-by: Junji Watanabe <jwata@google.com>
 2026-01-08 00:27:33 -08:00
Alex Ovsienko
cf876cddd6 siso: enable autoninja to use env variables instead of flags for collector. 
For clients with old siso this will not cause any breakage as we kept the flags, and did not introduce any old ones. Eventually everyone should get their collector running.

The collector startup will be triggered only for clients with enabled logs collection.

Bug: b/455433899
Change-Id: I962bd551c5a1cc6a5f8904e65d24939b6a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7373289
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
Reviewed-by: Junji Watanabe <jwata@google.com>
 2026-01-07 22:36:25 -08:00
Alex Ovsienko
72926f8e78 siso: rarely, call to getresponse can also throw. 
Cover it in try so that clients don't have to deal with fatal error.

Bug: b/455433899
Change-Id: Icd07e2057da19650a11981a8672c0e4c6a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7409014
Auto-Submit: Alex Ovsienko <ovsienko@google.com>
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
 2026-01-07 21:17:41 -08:00
Alex Ovsienko
e04940f5e8 siso: always set the collector address, do not assume default behavior. 
Bug: b/455433899
Change-Id: Ib8c0c3041cac5491e70986c0e7a60d386a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7367697
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
Reviewed-by: Junji Watanabe <jwata@google.com>
 2026-01-06 00:14:32 -08:00
Alex Ovsienko
daf2602023 siso: introduce flag check for collector by checking its help page. 
Bug: b/473530378, b/455433899
Change-Id: I09f53cd181cc3cd516e2fa035616efb06a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7365204
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
 2026-01-05 22:01:04 -08:00
Alex Ovsienko
d80f3ba240 siso: have checks for collector presence in the flags verification. 
Bug: b/455433899
Change-Id: Ia366d343e14f10d0cebf32c5a12bedd06a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7387808
Commit-Queue: Junji Watanabe <jwata@google.com>
Auto-Submit: Alex Ovsienko <ovsienko@google.com>
Reviewed-by: Junji Watanabe <jwata@google.com>
 2026-01-05 01:43:39 -08:00
Junji Watanabe
0847ade509 Revert "siso: enable collector by default when metrics collection is on." 
This reverts commit a4318057fe.

Reason for revert: This seems to break builds with old Siso version.

Original change's description:
> siso: enable collector by default when metrics collection is on.
>
> Bug: b/455433899
> Change-Id: Icdc8f0fb516ed9782f85a935d733b9f26a6a6964
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7366313
> Commit-Queue: Alex Ovsienko <ovsienko@google.com>
> Reviewed-by: Junji Watanabe <jwata@google.com>

Bug: b/455433899
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Change-Id: I819a7271979e122d39a283dc376fc80fc741125b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7394760
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Auto-Submit: Junji Watanabe <jwata@google.com>
 2026-01-04 23:01:08 -08:00
Alex Ovsienko
a4318057fe siso: enable collector by default when metrics collection is on. 
Bug: b/455433899
Change-Id: Icdc8f0fb516ed9782f85a935d733b9f26a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7366313
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
Reviewed-by: Junji Watanabe <jwata@google.com>
 2026-01-04 17:12:01 -08:00
Alex Ovsienko
c34ff73940 siso: finish making all function type strict with type hints. 
Change-Id: I704549b7d8d27675de0100e7ab6bf9476a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7260155
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
Reviewed-by: Junji Watanabe <jwata@google.com>
 2025-12-15 02:20:54 -08:00
Alex Ovsienko
5e033c9735 siso: disable linter on staticly inferable check that original_sigint_handler is callable. 
Change-Id: I6429a0f91a52449438938e4c82df51e96a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7259440
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
 2025-12-15 02:15:21 -08:00
Alex Ovsienko
a9534daf99 Remove startswith('win') from autoninja and siso. 
Windows can only be win32, and that holds to this day: https://docs.python.org/3.15/library/sys.html#sys.platform. Using this allows us to use better vscode integration of system checking and lines greyouts: https://screenshot.googleplex.com/8gkNyfPKMB9GPeU.

Change-Id: If9ab97bc30ab80df96b1aced0e78b0706a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7259617
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
Reviewed-by: Junji Watanabe <jwata@google.com>
 2025-12-15 00:00:49 -08:00
Alex Ovsienko
ff6558d2ee siso: remove usage of platform library from siso.py 
Bug: b/455433899
Change-Id: Iff65cbfd920c93114b784bce8ded91ec6a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7237760
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Reviewed-by: Philipp Wollermann <philwo@google.com>
Commit-Queue: Philipp Wollermann <philwo@google.com>
 2025-12-09 21:48:33 -08:00
Alex Ovsienko
6cd31a3b57 siso: have siso.py control if collector creates socket file. 
Bug: b/455433899
Change-Id: Ic02cd97ef3db59263859eeab5cd70f986a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7231248
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
 2025-12-09 01:16:10 -08:00
Alex Ovsienko
fc502860ed siso: integrate collector call. 
Bug: b/455433899

Change-Id: Id72ff71e1b4269b828658c90461ba4b46a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7205886
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
 2025-12-09 01:01:29 -08:00
Alex Ovsienko
ac1b095b09 siso: do not restart the collector if projects don't match. 
The correct project is set at runtime by siso.

Bug: b/455433899
Change-Id: Ibe1d348729e53425f27812e83a5c7f466a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7234199
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
Reviewed-by: Junji Watanabe <jwata@google.com>
 2025-12-07 20:39:43 -08:00
Alex Ovsienko
f8cc59a94b siso: create _start_collector function. 
It will run only when collector subcommand is present and attempt at handling the start and restart when needed.

Bug: b/455433899
Change-Id: I9a8b8001aec29b6ca3db61ec6150deb86a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7183447
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
 2025-12-07 15:57:33 -08:00
Alex Ovsienko
f2ac99f082 siso: make sure we don't try to kill process 0 if it gets returned. 
Bug: b/455433899
Change-Id: I8d4e6df5a374e97c0d790fe608964c6f6a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7220270
Auto-Submit: Alex Ovsienko <ovsienko@google.com>
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
Commit-Queue: Junji Watanabe <jwata@google.com>
Reviewed-by: Junji Watanabe <jwata@google.com>
 2025-12-04 21:36:50 -08:00
Alex Ovsienko
db99cc40f5 siso: do not have _is_subcommand_present print to stdout. 
Bug: b/455433899
Change-Id: I20a840669538c1a48a645226b949b37f6a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7205887
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
 2025-12-01 17:09:47 -08:00
Alex Ovsienko
4d6030e6b1 siso: create function to kill potentially running collector. 
It checks running processed for what ports they occupy and kills the first process that occupies otlp_health_port that is known from config and quits.

Bug: b/455433899

Change-Id: Ib588be4c44021e55c67f1601040ac72a6a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7183397
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
 2025-12-01 16:53:57 -08:00
Alex Ovsienko
a0a276ebbf siso: super nit: remove redundant env. 
Change-Id: Ieb3b749bbb5427bb6c3c4426f8ed00f56a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7205906
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
 2025-12-01 15:38:00 -08:00
Adam Norberg
3ec220caca Rename caffeinate.run to .call and match the subprocess.call API. 
I'm looking at caffeinating more long-running depot_tools commands on
Mac, which use a variety of `subprocess` APIs; one implementation
strategy I'm considering is to wrap more of them, including
`subprocess.run`, but the most obvious name to wrap `subprocess.run`
into is already taken! I think it makes the most sense to name the
`caffeinate` function(s) after the `subprocess` call it wraps.

The callers to other parts of `subprocess` use more of the features
exposed by those APIs. For consistency, I am plumbing through more of
`subprocess.call`'s identifiable behaviors here, too:

* command-to-invoke parameter is named "args"
* as explicitly checked in cpython's subprocess.py's `_posix_spawn`:
  77cb39e0c7/Lib/subprocess.py (L1816)
  also allow `str`, `bytes`, and `os.PathLike` for the type of `args`,
  converting the non-list-like types into a single-element list
* use kwargs packing and unpacking to forward other parameters into
  `subprocess.call`

kwargs packing does not provide perfect emulation as implemented:
77cb39e0c7/Lib/subprocess.py (L386)
`timeout` could be provided as a positional parameter. However, Python
documents all subprocess.call parameters as keyword only:
https://docs.python.org/3.12/library/subprocess.html#older-high-level-api
...specifically because they don't want to bother distinguishing
between parameters "local" to a higher-level `subprocess` helper and
those that get forwarded (via `**kwargs`) to the underlying `Popen`,
so I think it is reasonable to follow their lead on that.

Bug: 462507017
Change-Id: Ia520ced7f8188c23c38826d22ccf20a3c52ddfc5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7181621
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Adam Norberg <norberg@google.com>
 2025-11-20 21:03:12 -08:00
Fumitoshi Ukai
a40c63a40b siso: fix GOOGLE_API_USE_CLIENT_CERTIFICATE 
GOOGLE_API_USE_CLIENT_CERTIFICATE=false was not passed
to siso process.

Change-Id: I44e1d63c7fbf8184822d9d888495b4818b511eac
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7171141
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
Reviewed-by: Alex Ovsienko <ovsienko@google.com>
 2025-11-18 23:49:18 -08:00
Alex Ovsienko
926832f6db siso: extract command line args handling into _process_args. 
Add tests for _process_args to make sure all flags operate as expected.

Bug: b/459690822
Change-Id: I98086cb0a274420f440b2ac3490473566a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7161336
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
 2025-11-16 23:38:15 -08:00
Alex Ovsienko
ca5576a48f siso: cover apply_metrics_labels and apply_telemetry_flags functions with tests. 
Bug: b/459690822

Change-Id: Iccb5b83e8572afa8d6a66252fc69cf766a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7141140
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
 2025-11-16 21:57:17 -08:00
Philipp Wollermann
3d559609b4 siso: Handle empty args and startup flags 
Using `args[1]` to get the subcommand name doesn't work when the user
calls siso without any arguments, or when they specify global flags such
as `siso -logtostderr ninja ...`.

We can use the existing `subcmd` variable instead.

This fixes the following traceback:

```
$ siso
Traceback (most recent call last):
File "/home/philwo/depot_tools/siso.py", line 313, in <module>
sys.exit(main(sys.argv))
^^^^^^^^^^^^^^
File "/home/philwo/depot_tools/siso.py", line 292, in main
if args[1] == "ninja":
~~~~^^^

IndexError: list index out of range
```
Change-Id: Id319bdf099a2e87e049aee474734495424617a5a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7155649
Auto-Submit: Philipp Wollermann <philwo@google.com>
Commit-Queue: Philipp Wollermann <philwo@google.com>
Reviewed-by: Alex Ovsienko <ovsienko@google.com>
 2025-11-16 19:37:24 -08:00
Alex Ovsienko
c527a62fe0 siso: create simple function to check if subcommand is present is siso using help. 
That will help with starting collector from subcommand when clients may have older siso versions installed.

Bug: b/455433899

Change-Id: I5949b46017619a1717ef87781d8430b56a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7155815
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
 2025-11-16 19:03:44 -08:00
Alex Ovsienko
2bc167bbca siso: add default metrics project in telemetry flags if none is specified. 
Bug: b/348530235
Change-Id: Iba012ffa79c4d13033234c3afc4f18b36a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7141142
Reviewed-by: Junji Watanabe <jwata@google.com>
Auto-Submit: Alex Ovsienko <ovsienko@google.com>
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
 2025-11-12 00:02:23 -08:00
Alex Ovsienko
f107245af8 siso: set GOOGLE_API_USE_CLIENT_CERTIFICATE for googlers that run siso.py without running autoninja.py 
apply_telemetry_flags runs only for googlers, so we don't check if it's the googler running the code again.

Additionally amend the comment.

Bug: b/348530235
Change-Id: I3b1ef1edc8c03a4aba11bdc474560de66a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7143899
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
 2025-11-11 22:33:43 -08:00
Alex Ovsienko
345a14adf0 siso: enable system limits fix when we run only siso.py. 
Change-Id: I355e2d12833fefdc0e9d53b53a5583fc6a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7144919
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
 2025-11-11 21:34:48 -08:00
Alex Ovsienko
32a8dcc842 siso: Make it so that we check the second parameter of siso whether it is ninja or not. 
The first one is always `siso`.

Bug: b/348530235
Change-Id: I2135d87cb676156b4e14ace31928e9de6a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7140206
Reviewed-by: Junji Watanabe <jwata@google.com>
Auto-Submit: Alex Ovsienko <ovsienko@google.com>
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
 2025-11-11 00:08:06 -08:00
Alex Ovsienko
b26ab6e9d5 Have ninja flags be applied in more obvious manner. 
Bug: b/348530235
Change-Id: Idfac0789c4881a376678f749d0d20b3309fd240c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7136527
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
 2025-11-09 21:37:15 -08:00
Alex Ovsienko
2b4898e327 Reland "Enable telemetry flags for Google corp machines only." 
This reverts commit 6f83fce865.

Reason for revert: Make sure that the flags are applied only for ninja runs.

Original change's description:
> Revert "Enable telemetry flags for Google corp machines only."
>
> This reverts commit 4e2be179a1.
>
> Reason for revert: 
> This caused exception
>
> ```
> + autoninja -strict_remote -d keepdepfile -d keeprsp -C out/Default chrome
> siso ninja -strict_remote -d keepdepfile -d keeprsp -C out/Default chrome
> ninja: Entering directory `out/Default'
>  4.51s init credentials by "credhelper"
> https://console.cloud.google.com/logs/viewer?project=rbe-chrome-untrusted&resource=generic_task/task_id/25d8cbb8-4fa1-486d-ba78-7865d0eea5ac
> use RBE instance "projects/rbe-chrome-untrusted/instances/default_instance"
>  2.27s loading/recompacting deps log
>  3.47s Regenerating ninja files
>  1.17s load siso config
> build finished
> local:3636 remote:33214 cache:3317 cache-write:0(err:0) fallback:0 retry:0 skip:40752
> reapi: ops: 102977(err:33221) / r:46503(err:8) 3.13GiB / w:72424(err:0) 152.35MiB
> fs: ops: 2790971(err:182693) / r:227138(err:0) 6.68GiB / w:13508(err:4) 1.87GiB
>     resource/capa used(err)  wait-avg |   s m |  serv-avg |   s m |
>   pool=action/128   2372(0)     0.00s |█      |     0.30s | █▅▂▂▂ |
>         rbe:sched  33214(0)     0.01s |▃█▂▂   |     6.43s |  ▂█▃▂ |
>        rbe:worker  33214(0)     0.87s |    █  |     5.56s |  ▂█▃▂ |
>  remoteexec/10240  33214(0)     0.10s |█▂▂▂   |    15.37s |   ██▂ |
>  1.00s shutdown cloud logging/monitoring
> 4m08.96s Build Succeeded: 40167 steps - 161.34/s
> flag provided but not defined: -enable_cloud_monitoring
> summarize siso_metrics.json
>
>  $ siso metrics summary -C <dir> \
>     [--step_types <types>] \
>     [--elapsed_time_sorting] \
>     [--elapsed_time=run|step] \
>     [--input siso_metrics.json]
>
> summarize <dir>/.siso_metrics.json (--input)
> as depot_tools/post_ninja_build_summary.py does.
>   -C string
>         ninja running directory, where siso_metrics.json exists (default ".")
>   -elapsed_time string
>         metrics to use for elapsed time. "run" or "step". "run": time to run local command or call remote execution.  "step": full duration for the step, including preproc, waiting resource to run command etc. (default "run")
>   -elapsed_time_sorting
>         Sort output by elapsed time instead of weighted time
>   -input string
>         filename of siso_metrics.json to summarize (default "siso_metrics.json")
>   -step_types string
>         semicolon separated glob patterns (go filepath.Match) for build-step grouping
> Traceback (most recent call last):
>   File "/usr/local/google/home/tikuta/depot_tools/post_build_ninja_summary.py", line 390, in <module>
>     sys.exit(main())
>              ^^^^^^
>   File "/usr/local/google/home/tikuta/depot_tools/post_build_ninja_summary.py", line 375, in main
>     subprocess.run(cmd, check=True)
>   File "/usr/local/google/home/tikuta/depot_tools/bootstrap-2@3.11.8.chromium.35_bin/python3/lib/python3.11/subprocess.py", line 571, in run
>     raise CalledProcessError(retcode, process.args,
> subprocess.CalledProcessError: Command '['/usr/local/google/home/tikuta/depot_tools/bootstrap-2@3.11.8.chromium.35_bin/python3/bin/python3', '/usr/local/google/home/tikuta/depot_tools/siso.py', 'metrics', 'summary', '-C', 'out/Default', '--step_types', 'trict_remote']' returned non-zero exit status 2.
> ```
>
> Original change's description:
> > Enable telemetry flags for Google corp machines only.
> >
> > Bug: b/348530235
> > Change-Id: I4ac32c60b806f44920bd06e31ec6c3b86a6a6964
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7104218
> > Commit-Queue: Alex Ovsienko <ovsienko@google.com>
> > Reviewed-by: Junji Watanabe <jwata@google.com>
>
> Bug: b/348530235
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Change-Id: Iff13fa73745e09d2dab4278e9571f7ed77bf3d0c
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7136522
> Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
> Reviewed-by: Alex Ovsienko <ovsienko@google.com>
> Commit-Queue: Alex Ovsienko <ovsienko@google.com>

Bug: b/348530235
Change-Id: I6307ea67de0045c705e087e1711b295c8cce4703
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7136644
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
 2025-11-09 20:17:56 -08:00
Takuto Ikuta
6f83fce865 Revert "Enable telemetry flags for Google corp machines only." 
This reverts commit 4e2be179a1.

Reason for revert: 
This caused exception

```
+ autoninja -strict_remote -d keepdepfile -d keeprsp -C out/Default chrome
siso ninja -strict_remote -d keepdepfile -d keeprsp -C out/Default chrome
ninja: Entering directory `out/Default'
 4.51s init credentials by "credhelper"
https://console.cloud.google.com/logs/viewer?project=rbe-chrome-untrusted&resource=generic_task/task_id/25d8cbb8-4fa1-486d-ba78-7865d0eea5ac
use RBE instance "projects/rbe-chrome-untrusted/instances/default_instance"
 2.27s loading/recompacting deps log
 3.47s Regenerating ninja files
 1.17s load siso config
build finished
local:3636 remote:33214 cache:3317 cache-write:0(err:0) fallback:0 retry:0 skip:40752
reapi: ops: 102977(err:33221) / r:46503(err:8) 3.13GiB / w:72424(err:0) 152.35MiB
fs: ops: 2790971(err:182693) / r:227138(err:0) 6.68GiB / w:13508(err:4) 1.87GiB
    resource/capa used(err)  wait-avg |   s m |  serv-avg |   s m |
  pool=action/128   2372(0)     0.00s |█      |     0.30s | █▅▂▂▂ |
        rbe:sched  33214(0)     0.01s |▃█▂▂   |     6.43s |  ▂█▃▂ |
       rbe:worker  33214(0)     0.87s |    █  |     5.56s |  ▂█▃▂ |
 remoteexec/10240  33214(0)     0.10s |█▂▂▂   |    15.37s |   ██▂ |
 1.00s shutdown cloud logging/monitoring
4m08.96s Build Succeeded: 40167 steps - 161.34/s
flag provided but not defined: -enable_cloud_monitoring
summarize siso_metrics.json

 $ siso metrics summary -C <dir> \
    [--step_types <types>] \
    [--elapsed_time_sorting] \
    [--elapsed_time=run|step] \
    [--input siso_metrics.json]

summarize <dir>/.siso_metrics.json (--input)
as depot_tools/post_ninja_build_summary.py does.
  -C string
        ninja running directory, where siso_metrics.json exists (default ".")
  -elapsed_time string
        metrics to use for elapsed time. "run" or "step". "run": time to run local command or call remote execution.  "step": full duration for the step, including preproc, waiting resource to run command etc. (default "run")
  -elapsed_time_sorting
        Sort output by elapsed time instead of weighted time
  -input string
        filename of siso_metrics.json to summarize (default "siso_metrics.json")
  -step_types string
        semicolon separated glob patterns (go filepath.Match) for build-step grouping
Traceback (most recent call last):
  File "/usr/local/google/home/tikuta/depot_tools/post_build_ninja_summary.py", line 390, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/local/google/home/tikuta/depot_tools/post_build_ninja_summary.py", line 375, in main
    subprocess.run(cmd, check=True)
  File "/usr/local/google/home/tikuta/depot_tools/bootstrap-2@3.11.8.chromium.35_bin/python3/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/local/google/home/tikuta/depot_tools/bootstrap-2@3.11.8.chromium.35_bin/python3/bin/python3', '/usr/local/google/home/tikuta/depot_tools/siso.py', 'metrics', 'summary', '-C', 'out/Default', '--step_types', 'trict_remote']' returned non-zero exit status 2.
```

Original change's description:
> Enable telemetry flags for Google corp machines only.
>
> Bug: b/348530235
> Change-Id: I4ac32c60b806f44920bd06e31ec6c3b86a6a6964
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7104218
> Commit-Queue: Alex Ovsienko <ovsienko@google.com>
> Reviewed-by: Junji Watanabe <jwata@google.com>

Bug: b/348530235
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Change-Id: Iff13fa73745e09d2dab4278e9571f7ed77bf3d0c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7136522
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Alex Ovsienko <ovsienko@google.com>
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
 2025-11-09 18:39:02 -08:00
Alex Ovsienko
4e2be179a1 Enable telemetry flags for Google corp machines only. 
Bug: b/348530235
Change-Id: I4ac32c60b806f44920bd06e31ec6c3b86a6a6964
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7104218
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
Reviewed-by: Junji Watanabe <jwata@google.com>
 2025-11-09 17:13:31 -08:00
Alex Ovsienko
36ab0d8d35 siso: add metadata labels fetching and parsing to siso for developer builds. 
Bug: b/449906174
Change-Id: I218c156c89b98f09d81c4bea6ba1b5ed921d71ec
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7044008
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Alex Ovsienko <ovsienko@google.com>
 2025-10-16 23:01:37 -07:00
Fumitoshi Ukai
0f565fa858 siso: add .sisorc support 
if build/config/siso/.sisorc exists, it will add global flags
or subcmd flags.

global flags is line starting with "-"
e.g.
--credential_helper=gcloud

subcmd flags are line starting with subcmd name.
e.g.
ninja --verbose_failures=false -k=0

Bug: b/269554009
Change-Id: I4691b9e17571721dd5b70f6ffb063e2d2f0ac4e3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6910278
Reviewed-by: Scott Lee <ddoman@chromium.org>
Reviewed-by: Philipp Wollermann <philwo@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
 2025-09-03 19:03:18 -07:00
Fumitoshi Ukai
f80f5bc99e autoninja: support running from outside of workspace 
gclient_paths.GetPrimarySolutionPath() finds primary
solution path from current directory, so it would fail
if it is invoked from outside of workspace.

Pass directory to gclient_paths.GetPrimarySolutionPath
so it could find workspace correctly if it is invoked
ninja outside of workspace.

Bug: 441240584
Change-Id: I873f7883873e143ec8a64ee0e636042ac2336a2a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6898614
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Reviewed-by: Scott Lee <ddoman@chromium.org>
Commit-Queue: Scott Lee <ddoman@chromium.org>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
 2025-09-02 13:08:00 -07:00
Junji Watanabe
e68bbfc572 Prepend caffeinate to build commands on macOS 
When building on macOS, it's better to avoid sleeping with caffeinate by
default. This CL prepends `caffeinate` to the build commands, which can
be opted out with `--no-caffeinate`.

Change-Id: Iacb72acb6523576305b38fe45bfef12f87121a93
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6596109
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Junji Watanabe <jwata@google.com>
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
 2025-05-28 18:32:34 -07:00
Arthur Milchior
cd6685b6c0 [siso]Adding import 
`shutil` was used in
https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6420348
but not imported, leading to error message when executing
_is_google_corp_machine

Bug: None
Change-Id: I0bec8986f531a31905dabaadd20af68c1cd5536f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6449462
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Junji Watanabe <jwata@google.com>
Auto-Submit: Arthur Milchior <arthurmilchior@chromium.org>
 2025-04-11 07:15:47 -07:00
Fumitoshi Ukai
e7df27f40f siso: don't check backend.star if backend_config dir doesn't exist 
Bug: 408114984
Change-Id: I354bcd7e3c0f0e6af6496054f5f295c927cfc8f0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6431647
Commit-Queue: Philipp Wollermann <philwo@google.com>
Reviewed-by: Philipp Wollermann <philwo@google.com>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
 2025-04-04 06:58:15 -07:00
Fumitoshi Ukai
e1f6feb7ac siso: better error message when backend.star doesn't exist 
Change-Id: I0c87b2c371f71b9777baaf2a4acc2111b30f3633
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6420348
Feels: Gary Tong <gatong@chromium.org>
Reviewed-by: Gary Tong <gatong@chromium.org>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
 2025-04-02 19:30:12 -07:00
Mohamed Heikal
9b4d1e485d Integrate autoninja.py with fast_local_dev_server.py 
- Starts the build server when the build starts.
- Writes tty filename to env variable.
- Tells the build server about the current build so it does not exit
  until autoninja does, even if idle.
- Cancels pending tasks on Ctrl+c.

Change-Id: I86bb9852bd0975f381b049b9ff21c38eef7cef9d
Bug: 370589852
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5917985
Reviewed-by: Junji Watanabe <jwata@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Auto-Submit: Mohamed Heikal <mheikal@chromium.org>
Commit-Queue: Mohamed Heikal <mheikal@chromium.org>
 2024-12-12 10:28:46 -08:00
Fumitoshi Ukai
f5e1092339 siso: print error if siso binary doesn't exist in third_party/siso 
If build/config/siso/.sisoenv exists, but third_party/siso/cipd/siso
doesn't exist, it prints error

depot_tools/siso.py: Could not find .sisoenv under build/config/siso of the current project. Did you run gclient sync?

But it should print error as siso binary not in third_party/siso
instead.

Change-Id: I6a6a920b0e6d877551f0fa7c650ab7c8165180bf
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5802659
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
 2024-08-23 07:49:25 +00:00
Fumitoshi Ukai
06ab4cbb26 siso: call checkOutdir to prevent from running siso in ninja's out 
checkOutdir should be called before run `siso ninja`
to not run `siso ninja` where `ninja` is used.

Change-Id: I55453b21d35396b59924be5398b47f67b592eca1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5802658
Reviewed-by: Philipp Wollermann <philwo@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
 2024-08-23 06:22:39 +00:00
Fumitoshi Ukai
4eb5cc887d siso: allow run $SISO_PATH even if .sisoenv not found 
When $SISO_PATH is set, allow to run it even if .sisoenv not found.
Useful for `siso fetch`, `siso recall` etc, which would not be
needed to run in chromium checkout dir.

Change-Id: If3bfcf879cc67dc41bedf17cd771c94de836c949
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5804427
Commit-Queue: Richard Wang <richardwa@google.com>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
Reviewed-by: Philipp Wollermann <philwo@google.com>
Reviewed-by: Richard Wang <richardwa@google.com>
 2024-08-23 05:52:46 +00:00
Josip Sokcevic
43c6415bce Add a new possible siso path 
We are relocating siso binary from third_party/siso/siso to
third_party/siso/cipd/siso to make it compatible with Cog, which
requires cipd packages to be placed in directories with no Git content.

Bug: 346837573
Change-Id: If40e6033e341dcedfee5670f07d68c14bd923f7b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5625091
Reviewed-by: Philipp Wollermann <philwo@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
 2024-06-13 00:31:10 +00:00
Fumitoshi Ukai
951578030e siso: set PYTHONDONTWRITEBYTECODE=1 if not set 
during build, lots of python scripts run and generates
lots of pycache *.pyc in workspace.
To keep workspace clean, set PYTHONDONTWRITEBYTECODE=1
in siso.py

Change-Id: I725c6745c44410eef4a780b8ffc30fdafd899334
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5606824
Reviewed-by: Philipp Wollermann <philwo@google.com>
Reviewed-by: Erik Staab <estaab@chromium.org>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
 2024-06-07 01:33:06 +00:00