dc
b41402b1d3
3.12.2 is the current release and carries the packaged-executable fix for /ddc-host (the page this job's va-ddc-host spec drives), so the version the tests run against is the version that ships. 3.12.0 remains the floor for `timing=early`.
418 lines
19 KiB
YAML
418 lines
19 KiB
YAML
name: Build
|
|
run-name: Running Lint Check and Licence checker on Pull Request
|
|
on: [pull_request]
|
|
|
|
env:
|
|
# the server the mocked services and app tests run against - bump deliberately.
|
|
# 3.12.0 is the floor: /ddc-host carries `timing=early` from there, which
|
|
# client/cypress/e2e/va-ddc-host.cy.ts needs. 3.12.2 is the current release and
|
|
# also carries the packaged-executable fix for the same page, so the version
|
|
# the tests run against is the one that ships.
|
|
SASJS_SERVER_VERSION: '3.12.2'
|
|
NODE_VERSION: '24.15.0'
|
|
|
|
jobs:
|
|
Build-and-ng-test:
|
|
runs-on: ubuntu-latest
|
|
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- name: Check runner size
|
|
# An undersized runner dies inside `ng build` (exitcode 137) or as a
|
|
# Karma disconnect twenty minutes in. Fail here instead, in seconds,
|
|
# with the measured numbers in the log - see the script.
|
|
run: bash .gitea/scripts/check-runner-memory.sh
|
|
- uses: actions/setup-node@v4
|
|
with:
|
|
node-version: ${{ env.NODE_VERSION }}
|
|
|
|
- name: Scan for secrets (gitleaks)
|
|
# The local pre-commit hook is suppressed entirely when .npmrc sets
|
|
# ignore-scripts=true (the prepare script that would activate the
|
|
# hooks never runs), so CI is the enforcement point: a leaked
|
|
# credential must fail the build, not just a commit.
|
|
# Uses the same @nogoo9/gitleaks binary the pre-commit hook pins - the
|
|
# gitleaks-action requires a license key for organization repos.
|
|
# Runs before the registry config is written, so the scan cannot see
|
|
# the token this job injects.
|
|
# --no-git scans the working tree, which is what a pull request gate
|
|
# is about.
|
|
run: |
|
|
npm ci
|
|
node_modules/.bin/gitleaks detect --no-git --source . --redact --no-banner -v
|
|
|
|
- name: Install Google Chrome
|
|
run: |
|
|
apt-get update
|
|
wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
|
|
apt install -y ./google-chrome*.deb
|
|
|
|
- name: Write .npmrc file
|
|
run: |
|
|
# the registry secret stays outside the repo: client/.npmrc is
|
|
# tracked now, and one git add -A would commit a live token
|
|
echo "$NPMRC" > "$RUNNER_TEMP/npmrc"
|
|
echo "NPM_CONFIG_USERCONFIG=$RUNNER_TEMP/npmrc" >> "$GITHUB_ENV"
|
|
shell: bash
|
|
env:
|
|
NPMRC: ${{ secrets.NPMRC}}
|
|
|
|
- name: Install dependencies
|
|
run: |
|
|
cd client
|
|
# Decrypt and Install sheet
|
|
echo "${{ secrets.SHEET_PWD }}" | \
|
|
gpg --batch --yes --passphrase-fd 0 \
|
|
--output ./libraries/sheet-crypto.tgz \
|
|
--decrypt ./libraries/sheet-crypto.tgz.gpg
|
|
# dependency hooks stay off (client/.npmrc); the client's own
|
|
# generator runs explicitly, because the app imports the file it makes
|
|
npm ci --ignore-scripts
|
|
npm run postinstall
|
|
# cypress downloads its binary in a dependency install hook, which the
|
|
# policy skips, so it is installed explicitly here
|
|
npx cypress install
|
|
|
|
- name: Check audit
|
|
# Audit should fail and stop the CI on any vulnerability in root and sas, and on low+ in client
|
|
run: |
|
|
npm audit --omit=dev
|
|
cd ./sas
|
|
npm audit --omit=dev
|
|
cd ../client
|
|
npm audit --omit=dev
|
|
|
|
- name: Check archiver directory walk
|
|
# the readdir-glob minimatch override in client/package.json is a major
|
|
# bump; archiver's directory() is the only runtime caller of that path,
|
|
# so exercise it rather than trusting that the install succeeded
|
|
run: |
|
|
cd client
|
|
npm run check:archiver
|
|
|
|
- name: Lint check
|
|
run: npm run lint:check
|
|
|
|
- name: SAS lint check
|
|
# sas/.sasjslint promotes each rule to "error" as its findings are
|
|
# cleared, and only errors change the exit code - so this gate fails on
|
|
# a regression in a cleared rule and passes while other rules still
|
|
# carry warnings.
|
|
run: |
|
|
cd sas
|
|
npm ci --ignore-scripts
|
|
npm run lint:sas
|
|
|
|
- name: Licence checker
|
|
run: |
|
|
cd client
|
|
npm run license-checker
|
|
|
|
- name: Angular Tests
|
|
run: |
|
|
cd client
|
|
npm run test:headless
|
|
|
|
- name: Angular Tests in other timezones
|
|
# The VA filter clauses are built from local calendar dates, so a
|
|
# UTC-only run cannot see a date that slips a day west of UTC or east of
|
|
# it. Only the VA specs carry that property, so both scripts run that
|
|
# subset rather than the whole suite a second and third time.
|
|
run: |
|
|
cd client
|
|
npm run test:tz-west
|
|
npm run test:tz-east
|
|
|
|
- name: Every Cypress spec is in the CI list
|
|
# The spec list used to be spelled out inline in --spec, and five files
|
|
# quietly fell out of it - among them the VA embed suite. Fails when a
|
|
# spec on disk is not listed.
|
|
run: bash .gitea/scripts/cypress-specs.sh coverage .
|
|
|
|
- name: Production Build
|
|
run: |
|
|
cd client
|
|
npm run build
|
|
|
|
- name: Save the production build for the Cypress job
|
|
# The Cypress job needs client/dist: the server-ci target streams it as
|
|
# the 'clickme' web app (see CONTRIBUTING.md). Handing it the build this
|
|
# job has already made saves a second cold `npm run build` (1.6 minutes,
|
|
# measured over thirteen runs) and guarantees both jobs test the same
|
|
# bytes.
|
|
#
|
|
# if-no-files-found: error, because an empty artifact is worse than a
|
|
# failed upload: the Cypress job would download nothing, deploy a service
|
|
# pack with no frontend, and the symptom would reappear there as the
|
|
# "webSourcePath ... doesn't exist" line this artefact exists to remove.
|
|
uses: actions/upload-artifact@v3
|
|
with:
|
|
name: client-dist
|
|
path: client/dist
|
|
if-no-files-found: error
|
|
|
|
- name: Remove registry config
|
|
# written earlier in this job; nothing deletes it on the way out
|
|
if: always()
|
|
run: rm -f "$RUNNER_TEMP/npmrc"
|
|
|
|
Build-and-test-development:
|
|
runs-on: ubuntu-latest
|
|
needs: Build-and-ng-test
|
|
env:
|
|
CHROME_BIN: /usr/bin/google-chrome
|
|
# Pin OS locale
|
|
LANG: en_GB.UTF-8
|
|
LC_ALL: en_GB.UTF-8
|
|
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- name: Detect client changes
|
|
id: changes
|
|
# This job runs the mocked estate (sas/mocks) against the client, so a
|
|
# pull request that touches none of what it exercises cannot change its
|
|
# outcome: a sas/ service change is not deployed here, a documentation
|
|
# change is not read, and neither is .gitea/workflows/lighthouse.yaml.
|
|
# Skipping saves the twenty-six minute suite plus its installs.
|
|
#
|
|
# The condition on every gated step is `!= 'false'`, not `== 'true'`, and
|
|
# every failure below exits 0 without writing `client=false`: if the
|
|
# decision cannot be made, the suite runs. A CI change whose worst case
|
|
# is "slow" is safe; one whose worst case is "silently tests nothing" is
|
|
# not, and one whose worst case is "every pull request goes red because a
|
|
# fetch failed" is not acceptable either.
|
|
run: |
|
|
set -euo pipefail
|
|
BASE="${{ github.base_ref }}"
|
|
if [ -z "$BASE" ]; then
|
|
echo "no base ref (not a pull request) - running the suite"
|
|
exit 0
|
|
fi
|
|
|
|
if ! git fetch --no-tags --depth=1 origin "+refs/heads/${BASE}:refs/remotes/origin/${BASE}"; then
|
|
echo "::warning::could not fetch ${BASE} - running the suite rather than deciding"
|
|
exit 0
|
|
fi
|
|
|
|
if ! CHANGED="$(git diff --name-only "refs/remotes/origin/${BASE}" HEAD)"; then
|
|
echo "::warning::could not diff against ${BASE} - running the suite rather than deciding"
|
|
exit 0
|
|
fi
|
|
|
|
# .gitea/scripts/ is in the set because the suite's spec list
|
|
# (cypress-specs.sh) and the memory gate (check-runner-memory.sh) live
|
|
# there: a pull request that changes what the suite runs, or the gate
|
|
# that lets it run, has to run it.
|
|
if printf '%s\n' "$CHANGED" | grep -qE '^(client/|sas/mocks/|\.gitea/scripts/|package-lock\.json|\.gitea/workflows/build\.yaml)'; then
|
|
echo "client=true" >> "$GITHUB_OUTPUT"
|
|
else
|
|
echo "client=false" >> "$GITHUB_OUTPUT"
|
|
echo "::notice::Cypress suite skipped - nothing under client/, sas/mocks/ or .gitea/scripts/ changed"
|
|
printf 'Skipping the Cypress suite - nothing it exercises changed:\n%s\n' "$CHANGED"
|
|
fi
|
|
|
|
- name: Cypress suite not run - no client changes
|
|
# A named step rather than a run of greyed-out ones, so the job summary
|
|
# says why it took a minute: this job reports success having tested
|
|
# nothing, which is the intended behaviour for a pull request that
|
|
# changes no client code. If this job is a required check, "green" for
|
|
# such a pull request means "not applicable", and this step is the
|
|
# evidence for it.
|
|
if: steps.changes.outputs.client == 'false'
|
|
run: |
|
|
echo "No client, sas/mocks or .gitea/scripts change in this pull request - the Cypress suite was skipped by design."
|
|
echo "The detection step above lists the files it saw, and warns instead if it could not decide (in which case the suite runs)."
|
|
- name: Check runner size
|
|
if: steps.changes.outputs.client != 'false'
|
|
# Each Cypress job compiles the frontend and runs a browser on top; a
|
|
# runner too small for that fails as a flaky-looking disconnect - see
|
|
# the script.
|
|
run: bash .gitea/scripts/check-runner-memory.sh
|
|
- uses: actions/setup-node@v4
|
|
if: steps.changes.outputs.client != 'false'
|
|
with:
|
|
node-version: ${{ env.NODE_VERSION }}
|
|
|
|
- name: Write .npmrc file
|
|
if: steps.changes.outputs.client != 'false'
|
|
run: |
|
|
# the secret arrives through the environment, not interpolated into
|
|
# this script: a quote or $( in the value would otherwise break the
|
|
# step or execute
|
|
printf '%s\n' "$NPMRC" > "$RUNNER_TEMP/npmrc"
|
|
echo "NPM_CONFIG_USERCONFIG=$RUNNER_TEMP/npmrc" >> "$GITHUB_ENV"
|
|
env:
|
|
NPMRC: ${{ secrets.NPMRC}}
|
|
|
|
- name: Install system dependencies
|
|
if: steps.changes.outputs.client != 'false'
|
|
run: |
|
|
apt-get update
|
|
wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
|
|
apt install -y ./google-chrome*.deb
|
|
apt-get -y install libgtk2.0-0 libgtk-3-0 libgbm-dev libnotify-dev libnss3 libxss1 libasound2t64 libxtst6 xauth xvfb jq zip locales
|
|
# Generate the en_GB.UTF-8 locale referenced by LANG/LC_ALL
|
|
locale-gen en_GB.UTF-8
|
|
update-locale LANG=en_GB.UTF-8 LC_ALL=en_GB.UTF-8
|
|
|
|
- name: Write cypress credentials
|
|
if: steps.changes.outputs.client != 'false'
|
|
run: echo "$CYPRESS_CREDS" > ./client/cypress.env.json
|
|
shell: bash
|
|
env:
|
|
CYPRESS_CREDS: ${{ secrets.CYPRESS_CREDS }}
|
|
|
|
- name: Install dependencies
|
|
if: steps.changes.outputs.client != 'false'
|
|
run: |
|
|
cd client
|
|
# Decrypt and Install sheet
|
|
echo ${{ secrets.SHEET_PWD }} | gpg --batch --yes --passphrase-fd 0 ./libraries/sheet-crypto.tgz.gpg
|
|
# dependency hooks stay off (client/.npmrc); the client's own
|
|
# generator runs explicitly, because the app imports the file it makes
|
|
npm ci --ignore-scripts
|
|
npm run postinstall
|
|
# cypress downloads its binary in a dependency install hook, which the
|
|
# policy skips, so it is installed explicitly here
|
|
npx cypress install
|
|
|
|
- name: Fetch the production build
|
|
if: steps.changes.outputs.client != 'false'
|
|
# The server-ci target streams client/dist as the 'clickme' web app (see
|
|
# CONTRIBUTING.md), so the production build has to exist before the mock
|
|
# services are deployed. Without it the deploy logs
|
|
# "webSourcePath: .../client/dist ... doesn't exist" and streams no
|
|
# frontend. It is the artefact Build-and-ng-test published, so this job
|
|
# does not compile the frontend a second time.
|
|
uses: actions/download-artifact@v3
|
|
with:
|
|
name: client-dist
|
|
path: client/dist
|
|
|
|
- name: Cache the npm downloads
|
|
if: steps.changes.outputs.client != 'false'
|
|
# The server is installed from npm at a pinned version, so the tarball is
|
|
# immutable and the key never needs invalidating by hand: a hit skips
|
|
# the 10.7MB download, a miss costs exactly what the zip cost before.
|
|
# continue-on-error, because a runner without cache support should still
|
|
# run the pipeline - this step only ever saves time.
|
|
continue-on-error: true
|
|
uses: actions/cache@v4
|
|
with:
|
|
path: ~/.npm
|
|
key: npm-sasjs-server-${{ env.SASJS_SERVER_VERSION }}
|
|
|
|
- name: Setup and start SASjs server
|
|
if: steps.changes.outputs.client != 'false'
|
|
run: |
|
|
npm i -g pm2
|
|
touch .env
|
|
echo RUN_TIMES=js >> .env
|
|
echo NODE_PATH=node >> .env
|
|
echo CORS=enable >> .env
|
|
echo WHITELIST=http://localhost:4200 >> .env
|
|
echo HTTP_PORT=5000 >> .env
|
|
# /ddc-host is a testing tool, so the server answers 404 for it unless
|
|
# this is set. va-ddc-host.cy.ts drives the app through that page with
|
|
# timing=early.
|
|
echo DDC_HOST_ENABLED=true >> .env
|
|
# The server refuses to run as root unless RUN_AS names an account
|
|
# (a security change in the recent releases). This job runs as root
|
|
# in a throwaway container, so opt in explicitly.
|
|
echo RUN_AS=root >> .env
|
|
cat .env
|
|
pm2 start npx --name sasjs-server -- --yes @sasjs/server@${{ env.SASJS_SERVER_VERSION }}
|
|
# `pm2 start` returns before the app is listening, and the next step
|
|
# hits :5000 immediately. Wait for the server to answer, and dump its
|
|
# log if it never does.
|
|
for i in $(seq 1 60); do
|
|
if curl -sf -o /dev/null http://localhost:5000/; then
|
|
echo "SASjs server answered on attempt ${i}"
|
|
exit 0
|
|
fi
|
|
sleep 2
|
|
done
|
|
echo "SASjs server did not come up within 120s"
|
|
pm2 list
|
|
pm2 logs sasjs-server --lines 100 --nostream
|
|
exit 1
|
|
|
|
- name: Deploy mocked services
|
|
if: steps.changes.outputs.client != 'false'
|
|
run: |
|
|
cd ./sas/mocks/sasjs
|
|
npm install -g @sasjs/cli
|
|
npm install -g replace-in-files-cli
|
|
# Remove any previous deployment (drive folder delete API) so the
|
|
# deploy and makedata start from a clean appLoc
|
|
curl -sS -X DELETE "http://localhost:5000/SASjsApi/drive/folder/?_folderPath=/Public/app/dc"
|
|
sasjs cbd -t server-ci
|
|
# Seed the DC database (mock data files on the drive).
|
|
# makedata replies with HTML (it is normally called as a URL redirect),
|
|
# which the CLI reports as "invalid Json string" - ignore that.
|
|
# NOTE: -d paths resolve from the sasjs project root (sas/mocks),
|
|
# not the cwd (sas/mocks/sasjs).
|
|
sasjs request services/admin/makedata -t server-ci -d deploy/makedata.json -o ./makedata_out.json || true
|
|
|
|
- name: Prepare and run frontend and cypress
|
|
if: steps.changes.outputs.client != 'false'
|
|
# One job, one SASjs server, one ng serve, one Chrome: four of each on one
|
|
# runner starved the app under test and every failure in that run was a
|
|
# timeout, not an assertion.
|
|
timeout-minutes: 50
|
|
run: |
|
|
cd ./client
|
|
# The spec list lives in one place, and an empty list is a broken
|
|
# workflow rather than a green run with no tests.
|
|
SPECS="$(bash ../.gitea/scripts/cypress-specs.sh all)"
|
|
if [ -z "$SPECS" ]; then
|
|
echo "the spec list resolved to nothing"
|
|
exit 1
|
|
fi
|
|
echo "specs: $SPECS"
|
|
mv ./cypress.env.example.json ./cypress.env.json
|
|
replace-in-files --regex='"username".*' --replacement='"username":"'${{ secrets.CYPRESS_USERNAME_SASJS }}'",' ./cypress.env.json
|
|
replace-in-files --regex='"password".*' --replacement='"password":"'${{ secrets.CYPRESS_PWD_SASJS }}'" ' ./cypress.env.json
|
|
cat ./cypress.env.json
|
|
npm run postinstall
|
|
# Prepare index.html to SASJS local
|
|
replace-in-files --regex='serverUrl=".*?"' --replacement='serverUrl="http://localhost:5000"' ./src/index.html
|
|
replace-in-files --regex='appLoc=".*?"' --replacement='appLoc="/Public/app/dc"' ./src/index.html
|
|
replace-in-files --regex='serverType=".*?"' --replacement='serverType="SASJS"' ./src/index.html
|
|
replace-in-files --regex='"hosturl".*' --replacement='hosturl:"http://localhost:4200",' ./cypress.config.ts
|
|
cat ./cypress.config.ts
|
|
# Start frontend and run cypress
|
|
# timeout 2700: SIGTERM after 45 min so Cypress can flush screenshots
|
|
# before the outer timeout-minutes hard-kills the step (avoids silent multi-hour hangs)
|
|
#
|
|
# video=false, screenshotOnRunFailure=true: recording twenty-four
|
|
# specs costs encode time on every run, and the recording is only ever
|
|
# opened after a failure. A screenshot of the failing step carries the
|
|
# same information for a fraction of the cost - and cypress.config.ts
|
|
# sets screenshotOnRunFailure false by default, so both are passed
|
|
# here, as CLI overrides, leaving local runs (and the demo recorder,
|
|
# which sets RECORD_WINDOW_SIZE) on video.
|
|
npx ng serve --host 0.0.0.0 --port 4200 & npx wait-on http://localhost:4200 && timeout 2700 npx cypress run --browser chrome --config video=false,screenshotOnRunFailure=true --spec "$SPECS"
|
|
|
|
- name: Zip Cypress failure evidence
|
|
# Only on a failure: a green run has nothing to look at.
|
|
if: failure() && steps.changes.outputs.client != 'false'
|
|
run: |
|
|
mkdir -p ./client/cypress/screenshots ./client/cypress/videos
|
|
zip -r cypress-artifacts.zip ./client/cypress/screenshots ./client/cypress/videos
|
|
|
|
- name: Add cypress failure evidence as an artifact
|
|
if: failure() && steps.changes.outputs.client != 'false'
|
|
# v3, not v4: this runner reports "GHESNotSupportedError:
|
|
# @actions/artifact v2.0.0+, upload-artifact@v4+ ... not supported on
|
|
# GHES", so the current major cannot upload here at all.
|
|
uses: actions/upload-artifact@v3
|
|
with:
|
|
name: cypress-artifacts.zip
|
|
path: cypress-artifacts.zip
|
|
|
|
- name: Remove registry config
|
|
# written earlier in this job; nothing deletes it on the way out
|
|
if: always()
|
|
run: rm -f "$RUNNER_TEMP/npmrc"
|