Commit Graph

33 Commits

Author SHA1 Message Date
Rachael Newitt
11b0e6b805 Update the insuffient vuln scanning warning.
The previous CL changed the validation criteria, but did not update
the error message we're posting on the bugs we file, which is very
confusing. In the future we should try to write some tests to catch
things like this.

Bug: 452151523
Change-Id: I08272353a3cfada8f242355b48c4f003eec4abf4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7153383
Auto-Submit: Rachael Newitt <renewitt@google.com>
Commit-Queue: Rachael Newitt <renewitt@google.com>
Reviewed-by: Jordan Brown <rop@google.com>
Commit-Queue: Jordan Brown <rop@google.com>
2025-11-13 18:15:02 -08:00
Jordan
8acc772e9a Removing git tag as acceptable for vuln sufficiency
Change-Id: Ie611f0fe87dc0b518104588c153b7d857a696237
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7144858
Auto-Submit: Jordan Brown <rop@google.com>
Commit-Queue: Rachael Newitt <renewitt@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
2025-11-12 14:51:31 -08:00
Jordan
4f279935d6 Make license check case insensitive
I've been careful to make sure that the values in the warnings will
still have their original case so that we don't end up with all of the
licenses being in lowercase.

Bug: 452151523
Change-Id: Ifcc5e2ec38486a8e4f68fb7b066b00b8732cbc1e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7060630
Commit-Queue: Jiewei Qian <qjw@chromium.org>
Reviewed-by: Jiewei Qian <qjw@chromium.org>
Auto-Submit: Jordan Brown <rop@google.com>
2025-11-11 21:11:41 -08:00
Jordan
4734f4d95f Expand sufficient metadata warning scope
This change removes the temporary requirement that only dependencies lacking 'Update Mechanism' would have warnings about insufficient vulnerability coverage

Change-Id: I7eceea85574cee20a5a085d46607bc5a3c97df67
Bug: 448003595
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7016933
Auto-Submit: Jordan Brown <rop@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Rachael Newitt <renewitt@google.com>
2025-10-09 19:26:38 -07:00
Jordan
88b3c4e0b1 Exclude internal deps from versioning requirement
Bug:b/449859271
Change-Id: I1b1d53e4c3f6a2c70d761342b0b030a19242157a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7005408
Reviewed-by: Anne Redulla <aredulla@google.com>
Auto-Submit: Jordan Brown <rop@google.com>
Commit-Queue: Anne Redulla <aredulla@google.com>
2025-10-07 01:13:43 -07:00
Jordan
3f15de2ac3 Add Update Mechanism: Autoroll.GoogleManaged
Forked open-source projects stored in internal Google repos, which then
get pulled into ios_internal.

They have an external upstream, but we use the Google managed fork for
ios_internal

Change-Id: Ib2cd05414f31fd7e44f1222620880c2edbaac7e8
Bug: 448767146
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/7005823
Commit-Queue: Jordan Brown <rop@google.com>
Auto-Submit: Jordan Brown <rop@google.com>
Reviewed-by: Jiewei Qian <qjw@chromium.org>
2025-10-01 21:52:54 -07:00
Jordan Brown
49ffe22ecd Add a warning for insufficient metadata for vulnerability scanning.
This change introduces a `ValidationWarning` when a dependency lacks
sufficient metadata for vulnerability scanning. The warning asks
developers to provide one of the following combinations:
- 'CPEPrefix' with a version.
- A git clonable 'URL' and a 'Revision'.
- A git clonable 'URL' and a 'Version' matching the git tag.
- A package manager 'URL' and a 'Version'.

To reduce noise, this initial change is limited to dependencies that
meet the following criteria:
* Shipped; and
* Security Critical; and
* Do not provide 'Update Mechanism'

It is expected to affect about a dozen dependencies.

This change also updates some tests to be more specific to their test
cases so they don't fail for unrelated changes (like this one).

Bug: 438384123
Bug: 448003595
Change-Id: Ib1c562230b530a183e882efb1b23238b0ce0587c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6999547
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Rachael Newitt <renewitt@google.com>
Auto-Submit: Jordan Brown <rop@google.com>
2025-09-30 16:23:58 -07:00
Jordan Brown
675a3a9ccd Add Maven as identifiable package manager
These are the same identifiers used to identify Maven repositories in
internal scanners.
Bug: 438383466, 446990546
Change-Id: Ic00be8f96d28a63efaa09352007adb39a0328bae
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6976602
Commit-Queue: Jordan Brown <rop@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
2025-09-28 23:14:23 -07:00
Jordan
f916887e12 Checking for package managers in vuln sufficiency.
This improves our alignment of vuln scan sufficiency with the scanners
we are using, based on the data extracted from README.chromium files.
Other package managers are being covered based on their manifest files.

This change splits "sufficient:URL and Version" into:
* "sufficient:Git URL and Version"; and
* "sufficient:Package Manager URL and Version"

Bug: 438384047
Change-Id: Ia3262b93092cad40e60243158e437f65a04e1916
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6905113
Reviewed-by: Anne Redulla <aredulla@google.com>
Commit-Queue: Jordan Brown <rop@google.com>
2025-09-03 23:01:57 -07:00
Jordan
f01b22f782 Improving vuln scan sufficiency heuristics
* Centralised CPE/Version checking to reuse logic.
* Basic check that a url contains git, googlesource, or 'bitbucket etc to indicate it's a clonable url which is required to count as sufficient.
This brings the category closely in alignment with AutoVM, removing 100
dependencies, all of which did not have vulnerability cover.

Bug:b/438384047
Change-Id: I7483f20a177670ad1d6571ffcc2545c0faddd892
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6904943
Commit-Queue: Jordan Brown <rop@google.com>
Auto-Submit: Jordan Brown <rop@google.com>
Reviewed-by: Anne Redulla <aredulla@google.com>
2025-09-01 22:37:02 -07:00
Jordan
7e5f7cbec7 Changing CPE/Version to Warning
Bug: 441973518
Change-Id: I74ced62da409b25eed88070d473b79bd06655b82
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6903326
Reviewed-by: Dominic Battré <battre@chromium.org>
Commit-Queue: Jordan Brown <rop@google.com>
2025-09-01 00:39:30 -07:00
Jordan Brown
0699a53d84 Add validation for CPEPrefix and Version fields.
This change introduces a new validation rule: if a `CPEPrefix` is
provided but does not contain a version component, the `Version` field
must be present in the metadata. A helper function
`has_version_component` is added to `cpe_prefix.py` to check for a
version within a CPE string. Tests are added to cover the new validation
logic and the `has_version_component` function.

Bug: 438383649
Change-Id: I69938959316051d31f7fec32c5293d2c4c1a8e2a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6898421
Reviewed-by: Jiewei Qian <qjw@chromium.org>
Commit-Queue: Jordan Brown <rop@google.com>
2025-08-29 02:26:20 -07:00
Jordan Brown
dd6e72e968 Update the dependency sufficiency logic to allow for Revision:DEPS.
Change-Id: I8b54ce99829769482f356a276b3bfd3e81704708
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6773316
Commit-Queue: Jordan Brown <rop@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
2025-07-20 20:44:09 -07:00
Jordan Brown
0d6837383d Fixed bug in depot_tools presubmit
Potential to call None.lower() when doing 'vuln_scan_sufficiency'

Change-Id: I55e7f207612337ff18f5799d0366e89ecfa98b32
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6711769
Auto-Submit: Jordan Brown <rop@google.com>
Reviewed-by: Jiewei Qian <qjw@chromium.org>
Commit-Queue: Jiewei Qian <qjw@chromium.org>
2025-07-07 22:36:07 -07:00
Jordan Brown
ea4e2d4117 vuln_scan_sufficiency: Ignore internal/canonical
Change-Id: Ie0dc496abc955163364feeb198881a9b58c825d3
Bug: 429937921
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6711768
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Jordan Brown <rop@google.com>
Auto-Submit: Jordan Brown <rop@google.com>
Commit-Queue: Rachael Newitt <renewitt@google.com>
2025-07-07 21:09:22 -07:00
Jordan Brown
1b7c452940 Add Vuln scan sufficiency to metadata.
This property helps determine if the data available in metadata is sufficient to do vulnerability scanning.

Change-Id: I7cead6ca7eacf3184f6afa0a77b48fb2439f9fa9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6706867
Auto-Submit: Jordan Brown <rop@google.com>
Commit-Queue: Rachael Newitt <renewitt@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
2025-07-07 18:56:39 -07:00
Jordan
d4c627ed62 Adding Update Mechanism to Presubmit validation.
Bug: b/417367156
Change-Id: I3d12566ecf3c4d6d8d42c509fceb847fa5248776
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6606603
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Jordan Brown <rop@google.com>
2025-06-12 01:35:35 -07:00
Jordan Brown
e42fac3e9c [dependency_metadata] Allow descriptions for CVEs
This adds a new way to report CVEs that includes an accompanying
description. It also adds a new validation check that ensures that the
CVE description is present for every entry listed in the 'Mitigated:'
field.

Bug: b/392026683
Change-Id: Ie55595970b49d705ac532f1f8c41ff47d959f56c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6211644
Auto-Submit: Jordan Brown <rop@google.com>
Reviewed-by: Jiewei Qian <qjw@chromium.org>
Commit-Queue: Jiewei Qian <qjw@chromium.org>
2025-02-03 17:18:27 -08:00
Jordan Brown
98b7273c8d Add custom Mitigated field
This field stores a list of comma-separated CVE IDs that the dependency mitigates.

The field is validated to contain only valid CVE IDs.


Bug: b/392026683
Change-Id: I9578fc709086131695cfa7eee51e717c24440853
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6197756
Reviewed-by: Jiewei Qian <qjw@chromium.org>
Commit-Queue: Jordan Brown <rop@google.com>
2025-02-02 16:58:24 -08:00
Jordan Brown
fd65288d4a Add is_open_source_project to metadata validation
Reciprocal licenses can only be used in open source projects.
This change updates the presubmit validation checks to accept an
optional flag `allow_reciprocal_licenses`. When True, the allowlist is
extended to include reciprocal licenses.

Bug: 385020146
Change-Id: I0374658207bc87ffd74e033762ee4973c6e83b3b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6107863
Reviewed-by: Jordan Brown <rop@google.com>
Auto-Submit: Jordan Brown <rop@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Rachael Newitt <renewitt@google.com>
2025-01-12 14:07:21 -08:00
Jiewei Qian
6af18461ad metadata: allow a special "Revision: DEPS" syntax
This CL introduces a validation rule for "Revision: DEPS" syntax
so dependencies managed by DEPS and autorolled can use it to declare
their versioning metadata.

Bug: b/335761679
Change-Id: I0b4f99d281543f9295b122ac71036b06205a6168
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5904321
Commit-Queue: Jiewei Qian <qjw@chromium.org>
Reviewed-by: Rachael Newitt <renewitt@google.com>
2024-10-04 00:42:00 +00:00
Jiewei Qian
75932421da metadata: relax versioning info check if Chromium is the canonical repo of a dependency
If Chromium is the canonical repository, the version of the dependency
is essentially Chromium.

Change-Id: Ifcb80dbeee0d36bf71234f8f48423e8f7aa9dcf0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5772151
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Jiewei Qian <qjw@chromium.org>
2024-08-08 04:18:34 +00:00
Jiewei Qian
68c038603f metadata: add line number reporting
Adds support to report line numbers when validation fails.

Change-Id: Iba94c5b3582d7e51f15d266d188909d3a82b75cb
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5740963
Reviewed-by: Jordan Brown <rop@google.com>
Commit-Queue: Jiewei Qian <qjw@chromium.org>
Reviewed-by: Anne Redulla <aredulla@google.com>
2024-07-30 02:06:14 +00:00
Jiewei Qian
b7ed76a09d metadata: define a clear DependencyMetadata interface
This CL adds a typed interface that exposes parsed metadata for
downstream consumption.

Conventionally:
- A validated field should be retrieved by the property of the same name
- A validated field returns "None" if said field is not provided, or is
  clearly invalid (e.g. "Unknown" values)
- Raw values can still be retrieved with get_entries()

When using the properties accessor, fields are normalized and/or coerced to a suitable type (e.g. list of str, str of a particular format).

Bug: b/321154076
Change-Id: Ia56969a838e682a7b7eb1dc0781d48e1e38a2ff0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5446637
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Jiewei Qian <qjw@chromium.org>
2024-04-19 06:22:07 +00:00
Anne Redulla
36bd52621f [ssci] Support alias for Shipped field
Bug: b:297823626
Change-Id: Ib4be88567040d147f6cdba4f6c7d2b37a0f3898b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4871939
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Anne Redulla <aredulla@google.com>
2023-09-19 02:55:56 +00:00
Anne Redulla
80226254ea [ssci] Modify metadata versioning info validation
This CL changes what is considered valid versioning info. Instead of
both Date and Revision being required if Version was unknown, now only
one of Date or Revision has to be specified.

Bug: b:277147404
Change-Id: Iedb06e2d55f0cd0ef0a2931013a2a52b15befd75
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4852699
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Anne Redulla <aredulla@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
2023-09-10 22:52:26 +00:00
Anne Redulla
6715758ed9 [ssci] PEP8 formatting for metadata directory
All files in metadata/ are new, so they should follow the PEP-8 style.

Change-Id: I5d8424536c3d7b703e6b8087e0e2d70c06a1549c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4834909
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Rachael Newitt <renewitt@google.com>
2023-09-04 22:02:36 +00:00
Anne Redulla
427f0f43ad [ssci] Script to run validation on all metadata files
Adds script metadata/scan.py which can be used to search for and
validate Chromium dependency metadata files, given a repository
root directory.

Bug: b:277147404
Change-Id: Ibde0eeb7babe0b1e3f9c7f887bece629d390974a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4823596
Commit-Queue: Anne Redulla <aredulla@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
2023-08-31 03:46:17 +00:00
Anne Redulla
c7aca34c8e Define main metadata validation functions
This is a reland of commit a1cfc693af

The original commit was reverted do to `ModuleNotFoundError`s. I believe this was due to not specifying `metadata` to be part of the `depot_tools` recipe bundle. I have updated `.gitattributes` for this, and also added `__init__.py` files.

I will put the changes to `presubmit_canned_checks.py` in a later CL, once I can confirm `metadata` is being bundled.

Original change's description:
> [ssci] Added CheckChromiumMetadataFiles in presubmit_canned_checks
>
> Bug: b:277147404
> Change-Id: I14a2f11b256bc85fdfe225443ef533c38463ca3e
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4796694
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Reviewed-by: Rachael Newitt <renewitt@google.com>
> Commit-Queue: Anne Redulla <aredulla@google.com>

Bug: b:277147404
Change-Id: Ibd9efd5970a5393c157ca8763f97064d7c167803
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4803385
Reviewed-by: Rachael Newitt <renewitt@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Anne Redulla <aredulla@google.com>
2023-08-25 01:09:08 +00:00
Anne Redulla
760f8bcfb9 Revert "[ssci] Added CheckChromiumMetadataFiles in presubmit_canned_checks"
This reverts commit a1cfc693af.

Reason for revert: causing presubmit errors downstream

Original change's description:
> [ssci] Added CheckChromiumMetadataFiles in presubmit_canned_checks
>
> Bug: b:277147404
> Change-Id: I14a2f11b256bc85fdfe225443ef533c38463ca3e
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4796694
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Reviewed-by: Rachael Newitt <renewitt@google.com>
> Commit-Queue: Anne Redulla <aredulla@google.com>

Bug: b:277147404
Change-Id: I83f52494bc1a3a786505b8b74b2053269baa6e8e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4803286
Commit-Queue: Anne Redulla <aredulla@google.com>
Auto-Submit: Anne Redulla <aredulla@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Owners-Override: Anne Redulla <aredulla@google.com>
2023-08-23 02:55:35 +00:00
Anne Redulla
a1cfc693af [ssci] Added CheckChromiumMetadataFiles in presubmit_canned_checks
Bug: b:277147404
Change-Id: I14a2f11b256bc85fdfe225443ef533c38463ca3e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4796694
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Anne Redulla <aredulla@google.com>
2023-08-23 00:13:23 +00:00
Anne Redulla
3aeb682373 [ssci] Added validate method for single dependencies
Bug: b:277147404
Change-Id: I54c9c82d093cb11813e1c224da125b8d555f1b29
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4797050
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Anne Redulla <aredulla@google.com>
2023-08-21 03:33:17 +00:00
Anne Redulla
2b583af7e1 [ssci] Added parser for README validator
Bug: b:277147404
Change-Id: I7ee0fe35e1017eb477255f12045d00e855f7dfb4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4787830
Reviewed-by: Rachael Newitt <renewitt@google.com>
Auto-Submit: Anne Redulla <aredulla@google.com>
Commit-Queue: Rachael Newitt <renewitt@google.com>
2023-08-17 05:05:13 +00:00