dc
d84a1d50f4
The four shards made the release-branch run red, and the reasons are in the
logs rather than in the theory:
- every failure was a timeout, not an assertion - "cy.type() failed because
the center of this element is hidden from view", "Expected
<div.app-loading> not to exist ... continuously found". Four jobs on one
runner means four SASjs servers, four ng serve and four Chromes competing
for the same CPU, and the app under test starves;
- upload-artifact@v4 does not work on this runner at all:
"GHESNotSupportedError: @actions/artifact v2.0.0+, upload-artifact@v4+ ...
not supported on GHES", so the v3 -> v4 move I made on review advice broke
the artifact step outright.
Measured, from the one sharded run that did complete: 20m34s wall against 33-35
minutes for the single job, with peak concurrency 3 - so a third off, not the
four-fold the split suggests. That is not worth a flaky pipeline while a release
sits on it.
Kept from the sharding work, because those parts were the real gain:
- the spec list has one home, .gitea/scripts/cypress-specs.sh;
- Build-and-ng-test fails when a spec on disk is not listed, so the five that
had never run in CI cannot go missing again - and they do run now;
- videos are zipped and uploaded only on failure;
- one job means one server, which is also what the guards and the balance
arguments were protecting.
The shard and fan-in jobs are gone with the branch that carried them.
255 lines
10 KiB
YAML
255 lines
10 KiB
YAML
name: Build
|
|
run-name: Running Lint Check and Licence checker on Pull Request
|
|
on: [pull_request]
|
|
|
|
env:
|
|
NODE_VERSION: '24.15.0'
|
|
|
|
jobs:
|
|
Build-and-ng-test:
|
|
runs-on: ubuntu-latest
|
|
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- uses: actions/setup-node@v4
|
|
with:
|
|
node-version: ${{ env.NODE_VERSION }}
|
|
|
|
- name: Scan for secrets (gitleaks)
|
|
# The local pre-commit hook is suppressed entirely when .npmrc sets
|
|
# ignore-scripts=true (the prepare script that would activate the
|
|
# hooks never runs), so CI is the enforcement point: a leaked
|
|
# credential must fail the build, not just a commit.
|
|
# Uses the same @nogoo9/gitleaks binary the pre-commit hook pins - the
|
|
# gitleaks-action requires a license key for organization repos.
|
|
# Runs before client/.npmrc is written below, because that file is
|
|
# created from the NPMRC secret and a later scan would flag the
|
|
# registry token this job injects itself.
|
|
# --no-git scans the working tree, which is what a pull request gate
|
|
# is about.
|
|
run: |
|
|
npm ci
|
|
node_modules/.bin/gitleaks detect --no-git --source . --redact --no-banner -v
|
|
|
|
- name: Install Google Chrome
|
|
run: |
|
|
apt-get update
|
|
wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
|
|
apt install -y ./google-chrome*.deb
|
|
|
|
- name: Write .npmrc file
|
|
run: echo "$NPMRC" >> client/.npmrc
|
|
shell: bash
|
|
env:
|
|
NPMRC: ${{ secrets.NPMRC}}
|
|
|
|
- name: Install dependencies
|
|
run: |
|
|
cd client
|
|
# Decrypt and Install sheet
|
|
echo "${{ secrets.SHEET_PWD }}" | \
|
|
gpg --batch --yes --passphrase-fd 0 \
|
|
--output ./libraries/sheet-crypto.tgz \
|
|
--decrypt ./libraries/sheet-crypto.tgz.gpg
|
|
npm ci
|
|
|
|
- name: Check audit
|
|
# Audit should fail and stop the CI on any vulnerability in root and sas, and on low+ in client
|
|
run: |
|
|
npm audit --omit=dev
|
|
cd ./sas
|
|
npm audit --omit=dev
|
|
cd ../client
|
|
npm audit --omit=dev
|
|
|
|
- name: Lint check
|
|
run: npm run lint:check
|
|
|
|
- name: Licence checker
|
|
run: |
|
|
cd client
|
|
npm run license-checker
|
|
|
|
- name: Angular Tests
|
|
run: |
|
|
cd client
|
|
npm run test:headless
|
|
|
|
- name: Angular Tests in other timezones
|
|
# The VA filter clauses are built from local calendar dates, so a
|
|
# UTC-only run cannot see a date that slips a day west of UTC or east of
|
|
# it. Only the VA specs carry that property, so both scripts run that
|
|
# subset rather than the whole suite a second and third time.
|
|
run: |
|
|
cd client
|
|
npm run test:tz-west
|
|
npm run test:tz-east
|
|
|
|
- name: Every Cypress spec is in the CI list
|
|
# The spec list used to be spelled out inline in --spec, and five files
|
|
# quietly fell out of it - among them the VA embed suite. Fails when a
|
|
# spec on disk is not listed.
|
|
run: bash .gitea/scripts/cypress-specs.sh coverage .
|
|
|
|
- name: Production Build
|
|
run: |
|
|
cd client
|
|
npm run build
|
|
|
|
|
|
# Cypress shard 1 of 4. The setup repeats in every shard because each
|
|
# one gets its own runner and its own SASjs server; that fixed cost is what
|
|
# limits how far sharding helps. The specs it runs come from
|
|
# .gitea/scripts/cypress-specs.sh.
|
|
|
|
Build-and-test-development:
|
|
runs-on: ubuntu-latest
|
|
needs: Build-and-ng-test
|
|
env:
|
|
CHROME_BIN: /usr/bin/google-chrome
|
|
# Pin OS locale
|
|
LANG: en_GB.UTF-8
|
|
LC_ALL: en_GB.UTF-8
|
|
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- uses: actions/setup-node@v4
|
|
with:
|
|
node-version: ${{ env.NODE_VERSION }}
|
|
|
|
- name: Write .npmrc file
|
|
run: |
|
|
touch client/.npmrc
|
|
echo '${{ secrets.NPMRC}}' > client/.npmrc
|
|
|
|
- name: Install system dependencies
|
|
run: |
|
|
apt-get update
|
|
wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
|
|
apt install -y ./google-chrome*.deb
|
|
apt-get -y install libgtk2.0-0 libgtk-3-0 libgbm-dev libnotify-dev libnss3 libxss1 libasound2t64 libxtst6 xauth xvfb jq zip locales
|
|
# Generate the en_GB.UTF-8 locale referenced by LANG/LC_ALL
|
|
locale-gen en_GB.UTF-8
|
|
update-locale LANG=en_GB.UTF-8 LC_ALL=en_GB.UTF-8
|
|
|
|
- name: Write cypress credentials
|
|
run: echo "$CYPRESS_CREDS" > ./client/cypress.env.json
|
|
shell: bash
|
|
env:
|
|
CYPRESS_CREDS: ${{ secrets.CYPRESS_CREDS }}
|
|
|
|
- name: Install dependencies
|
|
run: |
|
|
cd client
|
|
# Decrypt and Install sheet
|
|
echo ${{ secrets.SHEET_PWD }} | gpg --batch --yes --passphrase-fd 0 ./libraries/sheet-crypto.tgz.gpg
|
|
npm ci
|
|
|
|
- name: Build frontend for web streaming
|
|
# The server-ci target streams client/dist as the 'clickme' web app (see
|
|
# CONTRIBUTING.md), so the production build has to exist before the mock
|
|
# services are deployed. Without it the deploy logs
|
|
# "webSourcePath: .../client/dist ... doesn't exist" and streams no frontend.
|
|
run: |
|
|
cd client
|
|
npm run build
|
|
|
|
- name: Setup and start SASjs server
|
|
run: |
|
|
npm i -g pm2
|
|
curl -L https://github.com/sasjs/server/releases/latest/download/linux.zip > linux.zip
|
|
unzip linux.zip
|
|
touch .env
|
|
echo RUN_TIMES=js >> .env
|
|
echo NODE_PATH=node >> .env
|
|
echo CORS=enable >> .env
|
|
echo WHITELIST=http://localhost:4200 >> .env
|
|
echo HTTP_PORT=5000 >> .env
|
|
# The server refuses to run as root unless RUN_AS names an account
|
|
# (a security change in the recent releases). This job runs as root
|
|
# in a throwaway container, so opt in explicitly.
|
|
echo RUN_AS=root >> .env
|
|
cat .env
|
|
pm2 start api-linux --wait-ready
|
|
# `pm2 start --wait-ready` can return while the app is still
|
|
# launching (it does not always wait for the ready signal), and the
|
|
# next step hits :5000 immediately. Wait for the server to answer,
|
|
# and dump its log if it never does.
|
|
for i in $(seq 1 60); do
|
|
if curl -sf -o /dev/null http://localhost:5000/; then
|
|
echo "SASjs server answered on attempt ${i}"
|
|
exit 0
|
|
fi
|
|
sleep 2
|
|
done
|
|
echo "SASjs server did not come up within 120s"
|
|
pm2 list
|
|
pm2 logs api-linux --lines 100 --nostream
|
|
exit 1
|
|
|
|
- name: Deploy mocked services
|
|
run: |
|
|
cd ./sas/mocks/sasjs
|
|
npm install -g @sasjs/cli
|
|
npm install -g replace-in-files-cli
|
|
# Remove any previous deployment (drive folder delete API) so the
|
|
# deploy and makedata start from a clean appLoc
|
|
curl -sS -X DELETE "http://localhost:5000/SASjsApi/drive/folder/?_folderPath=/Public/app/dc"
|
|
sasjs cbd -t server-ci
|
|
# Seed the DC database (mock data files on the drive).
|
|
# makedata replies with HTML (it is normally called as a URL redirect),
|
|
# which the CLI reports as "invalid Json string" - ignore that.
|
|
# NOTE: -d paths resolve from the sasjs project root (sas/mocks),
|
|
# not the cwd (sas/mocks/sasjs).
|
|
sasjs request services/admin/makedata -t server-ci -d deploy/makedata.json -o ./makedata_out.json || true
|
|
|
|
- name: Prepare and run frontend and cypress
|
|
# One job, one SASjs server, one ng serve, one Chrome: four of each on one
|
|
# runner starved the app under test and every failure in that run was a
|
|
# timeout, not an assertion.
|
|
timeout-minutes: 50
|
|
run: |
|
|
cd ./client
|
|
# The spec list lives in one place, and an empty list is a broken
|
|
# workflow rather than a green run with no tests.
|
|
SPECS="$(bash ../.gitea/scripts/cypress-specs.sh all)"
|
|
if [ -z "$SPECS" ]; then
|
|
echo "the spec list resolved to nothing"
|
|
exit 1
|
|
fi
|
|
echo "specs: $SPECS"
|
|
mv ./cypress.env.example.json ./cypress.env.json
|
|
replace-in-files --regex='"username".*' --replacement='"username":"'${{ secrets.CYPRESS_USERNAME_SASJS }}'",' ./cypress.env.json
|
|
replace-in-files --regex='"password".*' --replacement='"password":"'${{ secrets.CYPRESS_PWD_SASJS }}'" ' ./cypress.env.json
|
|
cat ./cypress.env.json
|
|
npm run postinstall
|
|
# Prepare index.html to SASJS local
|
|
replace-in-files --regex='serverUrl=".*?"' --replacement='serverUrl="http://localhost:5000"' ./src/index.html
|
|
replace-in-files --regex='appLoc=".*?"' --replacement='appLoc="/Public/app/dc"' ./src/index.html
|
|
replace-in-files --regex='serverType=".*?"' --replacement='serverType="SASJS"' ./src/index.html
|
|
replace-in-files --regex='"hosturl".*' --replacement='hosturl:"http://localhost:4200",' ./cypress.config.ts
|
|
cat ./cypress.config.ts
|
|
# Start frontend and run cypress
|
|
# timeout 2700: SIGTERM after 45 min so Cypress can flush video/screenshots
|
|
# before the outer timeout-minutes hard-kills the step (avoids silent multi-hour hangs)
|
|
npx ng serve --host 0.0.0.0 --port 4200 & npx wait-on http://localhost:4200 && timeout 2700 npx cypress run --browser chrome --spec "$SPECS"
|
|
|
|
- name: Zip Cypress videos
|
|
# Only on a failure: a green run has nothing to watch, and zipping and
|
|
# uploading every run's videos is the part of the video cost that lands
|
|
# on the clock.
|
|
if: failure()
|
|
run: |
|
|
mkdir -p ./client/cypress/videos
|
|
zip -r cypress-videos ./client/cypress/videos
|
|
|
|
- name: Add cypress videos artifacts
|
|
if: failure()
|
|
# v3, not v4: this runner reports "GHESNotSupportedError:
|
|
# @actions/artifact v2.0.0+, upload-artifact@v4+ ... not supported on
|
|
# GHES", so the current major cannot upload here at all.
|
|
uses: actions/upload-artifact@v3
|
|
with:
|
|
name: cypress-videos.zip
|
|
path: cypress-videos.zip
|