gha: add guardrails timeouts on all jobs

We had a few "runaway jobs" recently, where the job got stuck, and kept
running for 6 hours (in one case even 24 hours, probably due some github
outage). Some of those jobs could not be terminated.

While running these actions on public repositories doesn't cost us, it's
still not desirable to have jobs running for that long (as they can still
hold up the queue).

This patch adds a blanket "2 hours" time-limit to all jobs that didn't
have a limit set. We should look at tweaking those limits to actually
expected duration, but having a default at least is a start.

Also changed the position of some existing timeouts so that we have a
consistent order in which it's set; making it easier to spot locations
where no limit is defined.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This commit is contained in:
Sebastiaan van Stijn
2024-10-10 13:43:59 +02:00
parent eacb99db41
commit 6b7e2783d1
9 changed files with 35 additions and 12 deletions

View File

@@ -12,6 +12,7 @@ name: .windows
permissions:
contents: read
on:
workflow_call:
inputs:
@@ -42,6 +43,7 @@ env:
jobs:
build:
runs-on: ${{ inputs.os }}
timeout-minutes: 120 # guardrails timeout for the whole job
env:
GOPATH: ${{ github.workspace }}\go
GOBIN: ${{ github.workspace }}\go\bin
@@ -121,7 +123,7 @@ jobs:
unit-test:
runs-on: ${{ inputs.os }}
timeout-minutes: 120
timeout-minutes: 120 # guardrails timeout for the whole job
env:
GOPATH: ${{ github.workspace }}\go
GOBIN: ${{ github.workspace }}\go\bin
@@ -203,6 +205,7 @@ jobs:
unit-test-report:
runs-on: ubuntu-latest
timeout-minutes: 120 # guardrails timeout for the whole job
if: always()
needs:
- unit-test
@@ -229,6 +232,7 @@ jobs:
integration-test-prepare:
runs-on: ubuntu-latest
timeout-minutes: 120 # guardrails timeout for the whole job
outputs:
matrix: ${{ steps.tests.outputs.matrix }}
steps:
@@ -262,8 +266,8 @@ jobs:
integration-test:
runs-on: ${{ inputs.os }}
timeout-minutes: 120 # guardrails timeout for the whole job
continue-on-error: ${{ inputs.storage == 'snapshotter' && github.event_name != 'pull_request' }}
timeout-minutes: 120
needs:
- build
- integration-test-prepare
@@ -522,6 +526,7 @@ jobs:
integration-test-report:
runs-on: ubuntu-latest
timeout-minutes: 120 # guardrails timeout for the whole job
continue-on-error: ${{ inputs.storage == 'snapshotter' && github.event_name != 'pull_request' }}
if: always()
needs: