Compare commits

...

28 Commits

Author SHA1 Message Date
s
2c2901b537 chore: rever upload artifacts actions version
All checks were successful
Build / Build-and-ng-test (pull_request) Successful in 4m3s
Build / Build-and-test-development (pull_request) Successful in 10m0s
Lighthouse Checks / lighthouse (pull_request) Successful in 18m24s
2026-04-03 10:32:28 +02:00
s
2cae7ea638 chore: improve CI workflows
Some checks failed
Build / Build-and-ng-test (pull_request) Successful in 4m3s
Build / Build-and-test-development (pull_request) Failing after 10m17s
Lighthouse Checks / lighthouse (pull_request) Failing after 18m31s
2026-04-03 09:36:39 +02:00
s
66e98a96cb fix: add workflow audits, update deps
All checks were successful
Build / Build-and-ng-test (pull_request) Successful in 4m2s
Build / Build-and-test-development (pull_request) Successful in 10m19s
Lighthouse Checks / lighthouse (24.5.0) (pull_request) Successful in 18m53s
2026-04-03 09:10:49 +02:00
e7b2ead0e2 Merge pull request 'fix: allow CSV uploads with licence row limit' (#219) from fix/213-csv-license-row-limit into main
Some checks failed
Release / Build-production-and-ng-test (push) Failing after 1m25s
Release / Build-and-test-development (push) Has been skipped
Release / release (push) Has been skipped
Reviewed-on: #219
2026-04-02 19:08:18 +00:00
s
a89657b0b8 feat: add target libref input to config download
All checks were successful
Build / Build-and-ng-test (pull_request) Successful in 4m5s
Build / Build-and-test-development (pull_request) Successful in 10m16s
Lighthouse Checks / lighthouse (24.5.0) (pull_request) Successful in 18m30s
Closes #212
2026-04-02 19:37:55 +02:00
s
4ee15e1b6e fix: resolve outer promise in parseCsvFile for non-WLATIN1 path
All checks were successful
Build / Build-and-ng-test (pull_request) Successful in 3m55s
Build / Build-and-test-development (pull_request) Successful in 10m21s
Lighthouse Checks / lighthouse (24.5.0) (pull_request) Successful in 18m49s
2026-04-02 18:48:27 +02:00
s
ed40df6295 fix: guard CSV upload with fileUpload licence flag
Some checks failed
Build / Build-and-ng-test (pull_request) Successful in 4m3s
Build / Build-and-test-development (pull_request) Failing after 11m54s
Lighthouse Checks / lighthouse (24.5.0) (pull_request) Successful in 18m35s
2026-04-02 17:40:16 +02:00
s
6d590c050d fix: use XLSX for CSV row truncation to handle new lines in values
All checks were successful
Build / Build-and-ng-test (pull_request) Successful in 3m53s
Build / Build-and-test-development (pull_request) Successful in 10m25s
Lighthouse Checks / lighthouse (24.5.0) (pull_request) Successful in 18m29s
2026-04-02 17:03:16 +02:00
47f9a54f97 Merge pull request 'feat: add embed URL parameter to hide header and back button' (#218) from feat/214-hide-titlebar-embed into fix/213-csv-license-row-limit
All checks were successful
Build / Build-and-ng-test (pull_request) Successful in 4m0s
Build / Build-and-test-development (pull_request) Successful in 10m18s
Lighthouse Checks / lighthouse (24.5.0) (pull_request) Successful in 18m56s
Reviewed-on: #218
2026-04-02 14:37:06 +00:00
s
17b0d72fbf test: add csv-limited spec to cypress workflow
All checks were successful
Build / Build-and-ng-test (pull_request) Successful in 4m1s
Build / Build-and-test-development (pull_request) Successful in 10m23s
Lighthouse Checks / lighthouse (24.5.0) (pull_request) Successful in 19m1s
2026-04-02 16:13:35 +02:00
s
0269c2421d fix: parse embed param from window.location.hash for hash router compatibility
All checks were successful
Build / Build-and-ng-test (pull_request) Successful in 4m9s
Build / Build-and-test-development (pull_request) Successful in 10m9s
Lighthouse Checks / lighthouse (24.5.0) (pull_request) Successful in 19m8s
2026-04-02 14:57:16 +02:00
s
5b260e4915 fix: allow CSV uploads with licence row limit
All checks were successful
Build / Build-and-ng-test (pull_request) Successful in 3m56s
Build / Build-and-test-development (pull_request) Successful in 10m3s
Lighthouse Checks / lighthouse (24.5.0) (pull_request) Successful in 18m49s
Fixes #213
2026-04-02 14:34:58 +02:00
5290410a17 Merge branch 'main' into feat/214-hide-titlebar-embed
All checks were successful
Build / Build-and-ng-test (pull_request) Successful in 3m56s
Build / Build-and-test-development (pull_request) Successful in 10m11s
Lighthouse Checks / lighthouse (24.5.0) (pull_request) Successful in 18m31s
2026-04-02 11:13:49 +00:00
dc9041aaec Merge pull request 'fix: quote CSV char values. Closes #215' (#216) from issue215 into main
Some checks failed
Release / Build-production-and-ng-test (push) Failing after 1m25s
Release / Build-and-test-development (push) Has been skipped
Release / release (push) Has been skipped
Reviewed-on: #216
2026-04-02 11:12:38 +00:00
s
b0dc441d68 feat: add embed URL parameter to hide header and back button
All checks were successful
Build / Build-and-ng-test (pull_request) Successful in 4m3s
Build / Build-and-test-development (pull_request) Successful in 10m6s
Lighthouse Checks / lighthouse (24.5.0) (pull_request) Successful in 18m39s
Closes #214
2026-04-02 11:26:28 +02:00
b0fc3eb5af chore: update comment
All checks were successful
Build / Build-and-ng-test (pull_request) Successful in 4m23s
Build / Build-and-test-development (pull_request) Successful in 10m7s
Lighthouse Checks / lighthouse (24.5.0) (pull_request) Successful in 19m13s
2026-03-31 17:09:17 +01:00
d9980e866d fix: quote CSV char values. Closes #215
Some checks failed
Build / Build-and-ng-test (pull_request) Successful in 4m7s
Build / Build-and-test-development (pull_request) Has been cancelled
Lighthouse Checks / lighthouse (24.5.0) (pull_request) Has been cancelled
2026-03-31 17:04:46 +01:00
semantic-release-bot
52ae3404ee chore(release): 7.4.1 [skip ci]
## [7.4.1](https://git.datacontroller.io/dc/dc/compare/v7.4.0...v7.4.1) (2026-03-12)

### Bug Fixes

* support for SASIOSNF engine (SNOW alias) plus meta assignment ([7694d1b](7694d1b0fb))
2026-03-12 00:52:17 +00:00
eecb4f4f53 Merge pull request 'fix: support for SASIOSNF engine (SNOW alias) plus meta assignment' (#209) from snowfixes into main
All checks were successful
Release / Build-production-and-ng-test (push) Successful in 3m41s
Release / Build-and-test-development (push) Successful in 9m52s
Release / release (push) Successful in 7m57s
Reviewed-on: #209
2026-03-12 00:35:16 +00:00
744345af81 chore: bump sasjs/cli
All checks were successful
Build / Build-and-ng-test (pull_request) Successful in 3m51s
Build / Build-and-test-development (pull_request) Successful in 9m58s
Lighthouse Checks / lighthouse (24.5.0) (pull_request) Successful in 18m48s
2026-03-12 00:16:10 +00:00
_
7694d1b0fb fix: support for SASIOSNF engine (SNOW alias) plus meta assignment
All checks were successful
Build / Build-and-ng-test (pull_request) Successful in 3m46s
Build / Build-and-test-development (pull_request) Successful in 9m38s
Lighthouse Checks / lighthouse (24.5.0) (pull_request) Successful in 18m18s
2026-03-10 23:50:57 +00:00
semantic-release-bot
d8010d4c0c chore(release): 7.4.0 [skip ci]
# [7.4.0](https://git.datacontroller.io/dc/dc/compare/v7.3.0...v7.4.0) (2026-02-20)

### Bug Fixes

* cli bump for mf_getscheme support ([a84ba41](a84ba41ea9))
* missing upcase on SNOW section, plus local sasjs target ([dc20064](dc200646f7))

### Features

* SAS code changes for snowflake support ([e273e87](e273e870ef))
2026-02-20 18:53:31 +00:00
a57b49c936 Merge pull request 'feat: SAS code changes for snowflake support' (#208) from sf into main
All checks were successful
Release / Build-production-and-ng-test (push) Successful in 4m0s
Release / Build-and-test-development (push) Successful in 9m57s
Release / release (push) Successful in 7m57s
Reviewed-on: #208
2026-02-20 18:36:00 +00:00
a84ba41ea9 fix: cli bump for mf_getscheme support
All checks were successful
Build / Build-and-ng-test (pull_request) Successful in 4m10s
Build / Build-and-test-development (pull_request) Successful in 10m4s
Lighthouse Checks / lighthouse (24.5.0) (pull_request) Successful in 18m58s
2026-02-20 18:15:31 +00:00
dc200646f7 fix: missing upcase on SNOW section, plus local sasjs target
All checks were successful
Build / Build-and-ng-test (pull_request) Successful in 4m11s
Build / Build-and-test-development (pull_request) Successful in 9m55s
Lighthouse Checks / lighthouse (24.5.0) (pull_request) Successful in 18m25s
2026-02-20 01:08:55 +00:00
e273e870ef feat: SAS code changes for snowflake support
All checks were successful
Build / Build-and-ng-test (pull_request) Successful in 4m9s
Build / Build-and-test-development (pull_request) Successful in 10m1s
Lighthouse Checks / lighthouse (24.5.0) (pull_request) Successful in 18m57s
2026-02-20 00:15:23 +00:00
semantic-release-bot
6fc34aca00 chore(release): 7.3.0 [skip ci]
# [7.3.0](https://git.datacontroller.io/dc/dc/compare/v7.2.8...v7.3.0) (2026-02-10)

### Bug Fixes

* bump xlsx, add crypto-shim ([8dc18b1](8dc18b155a))
* correctly applying deletes on viya, also ([46cdeb0](46cdeb0bab))
* crypto module requirement for sheetjs/crypto package ([505d0af](505d0af2b3))
* disable parsing excel in web worker beacuse it breaks in the stream apps ([280bdee](280bdeeb1b))
* Display all contexts when installing DC on Viya ([d41f88f](d41f88f8bf))
* **edit:** use cellValidation keys and hotDataSchema to fill in defaults on add row ([4957548](495754816c))
* enabling closeouts for UPDATE in CAS tables ([8b8e8ae](8b8e8aec15))
* enabling rollback when the table has formatted values ([815d6e9](815d6e97a8))
* improvements to validations ([6ceb681](6ceb681463))
* remove IE checks and conditions ([ece6bd1](ece6bd1d78))
* updates to demodata to enable auto CAS promote ([7740d2a](7740d2ac86))
* upgrade angular core and compiler ([aecd597](aecd597687))
* using fcopy instead of binary copy for file upload, for Viya 2026 compatibility ([716ee6e](716ee6eba0))
* **viewer:** search causing blank Handsontable ([338c7a2](338c7a2e41)), closes [#206](#206)

### Features

* adding demo data job ([8c2aeac](8c2aeacc85))
* **dq rules:** notnull validation when invalid cell, will auto populate a default value ([96f2518](96f2518af9))
2026-02-10 19:17:29 +00:00
f97ac70678 Merge pull request 'demodata' (#203) from demodata into main
All checks were successful
Release / Build-production-and-ng-test (push) Successful in 3m27s
Release / Build-and-test-development (push) Successful in 9m22s
Release / release (push) Successful in 7m53s
Reviewed-on: #203
2026-02-10 19:01:17 +00:00
25 changed files with 1081 additions and 479 deletions

View File

@@ -2,22 +2,24 @@ name: Build
run-name: Running Lint Check and Licence checker on Pull Request
on: [pull_request]
env:
NODE_VERSION: '24.5.0'
jobs:
Build-and-ng-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 24.5.0
node-version: ${{ env.NODE_VERSION }}
- name: Install Google Chrome
run: |
wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
echo "deb http://dl.google.com/linux/chrome/deb/ stable main" > /etc/apt/sources.list.d/google.list
apt-get update
apt-get install -y google-chrome-stable xvfb
wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
apt install -y ./google-chrome*.deb
- name: Write .npmrc file
run: echo "$NPMRC" > client/.npmrc
@@ -25,6 +27,15 @@ jobs:
env:
NPMRC: ${{ secrets.NPMRC}}
- name: Check audit
# Audit should fail and stop the CI if critical vulnerability found
run: |
npm audit --omit=dev
cd ./sas
npm audit --omit=dev
cd ../client
npm audit --audit-level=critical --omit=dev
- name: Lint check
run: npm run lint:check
@@ -52,26 +63,27 @@ jobs:
Build-and-test-development:
runs-on: ubuntu-latest
needs: Build-production-and-ng-test
needs: Build-and-ng-test
env:
CHROME_BIN: /usr/bin/google-chrome
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 24.5.0
node-version: ${{ env.NODE_VERSION }}
- name: Write .npmrc file
run: |
touch client/.npmrc
echo '${{ secrets.NPMRC}}' > client/.npmrc
- run: apt-get update
- run: wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
- run: apt install -y ./google-chrome*.deb;
- run: export CHROME_BIN=/usr/bin/google-chrome
- run: apt-get update -y
- run: apt-get -y install libgtk2.0-0 libgtk-3-0 libgbm-dev libnotify-dev libnss3 libxss1 libasound2t64 libxtst6 xauth xvfb
- run: apt -y install jq
- name: Install system dependencies
run: |
apt-get update
wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
apt install -y ./google-chrome*.deb
apt-get -y install libgtk2.0-0 libgtk-3-0 libgbm-dev libnotify-dev libnss3 libxss1 libasound2t64 libxtst6 xauth xvfb jq zip
- name: Write cypress credentials
run: echo "$CYPRESS_CREDS" > ./client/cypress.env.json
@@ -86,17 +98,18 @@ jobs:
echo ${{ secrets.SHEET_PWD }} | gpg --batch --yes --passphrase-fd 0 ./libraries/sheet-crypto.tgz.gpg
npm ci
# Install pm2 and prepare SASJS server
- run: npm i -g pm2
- run: curl -L https://github.com/sasjs/server/releases/latest/download/linux.zip > linux.zip
- run: unzip linux.zip
- run: touch .env
- run: echo RUN_TIMES=js >> .env
- run: echo NODE_PATH=node >> .env
- run: echo CORS=enable >> .env
- run: echo WHITELIST=http://localhost:4200 >> .env
- run: cat .env
- run: pm2 start api-linux --wait-ready
- name: Setup and start SASjs server
run: |
npm i -g pm2
curl -L https://github.com/sasjs/server/releases/latest/download/linux.zip > linux.zip
unzip linux.zip
touch .env
echo RUN_TIMES=js >> .env
echo NODE_PATH=node >> .env
echo CORS=enable >> .env
echo WHITELIST=http://localhost:4200 >> .env
cat .env
pm2 start api-linux --wait-ready
- name: Deploy mocked services
run: |
@@ -106,11 +119,6 @@ jobs:
sasjs cbd -t server-ci
# sasjs request services/admin/makedata -t server-ci -d ./deploy/makeData4GL.json -c ./deploy/requestConfig.json -o ./output.json
- name: Install ZIP
run: |
apt-get update
apt-get install zip
- name: Prepare and run frontend and cypress
run: |
cd ./client
@@ -126,7 +134,7 @@ jobs:
replace-in-files --regex='"hosturl".*' --replacement='hosturl:"http://localhost:4200",' ./cypress.config.ts
cat ./cypress.config.ts
# Start frontend and run cypress
npx ng serve --host 0.0.0.0 --port 4200 & npx wait-on http://localhost:4200 && npx cypress run --browser chrome --spec "cypress/e2e/liveness.cy.ts,cypress/e2e/editor.cy.ts,cypress/e2e/excel-multi-load.cy.ts,cypress/e2e/excel.cy.ts,cypress/e2e/csv.cy.ts,cypress/e2e/filtering.cy.ts,cypress/e2e/licensing.cy.ts"
npx ng serve --host 0.0.0.0 --port 4200 & npx wait-on http://localhost:4200 && npx cypress run --browser chrome --spec "cypress/e2e/csv-limited.cy.ts,cypress/e2e/liveness.cy.ts,cypress/e2e/editor.cy.ts,cypress/e2e/excel-multi-load.cy.ts,cypress/e2e/excel.cy.ts,cypress/e2e/csv.cy.ts,cypress/e2e/filtering.cy.ts,cypress/e2e/licensing.cy.ts"
- name: Zip Cypress videos
if: always()

View File

@@ -2,38 +2,31 @@ name: Lighthouse Checks
run-name: Running Lighthouse Performance and Accessibility Checks on Pull Request
on: [pull_request]
env:
NODE_VERSION: '24.5.0'
jobs:
lighthouse:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [24.5.0]
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Use Node.js ${{ matrix.node-version }}
- name: Use Node.js ${{ env.NODE_VERSION }}
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
node-version: ${{ env.NODE_VERSION }}
- name: Install Google Chrome
run: |
wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
echo "deb http://dl.google.com/linux/chrome/deb/ stable main" > /etc/apt/sources.list.d/google.list
apt-get update
apt-get install -y google-chrome-stable xvfb
wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
apt install -y ./google-chrome*.deb
- name: Install pm2 for process management
run: npm i -g pm2
- name: Install global packages
run: npm i -g pm2 @sasjs/cli wait-on
- name: Install @sasjs/cli
run: npm i -g @sasjs/cli
- name: Install wait-on globally
run: npm install -g wait-on
- name: Create .env file for sasjs/server
- name: Setup and start SASjs server
run: |
touch .env
echo RUN_TIMES=js >> .env
@@ -41,15 +34,9 @@ jobs:
echo CORS=enable >> .env
echo WHITELIST=http://localhost:4200 >> .env
cat .env
- name: Download sasjs/server package from github using curl
run: curl -L https://github.com/sasjs/server/releases/latest/download/linux.zip > linux.zip
- name: Unzip downloaded package
run: unzip linux.zip
- name: Run sasjs server
run: pm2 start api-linux --wait-ready
curl -L https://github.com/sasjs/server/releases/latest/download/linux.zip > linux.zip
unzip linux.zip
pm2 start api-linux --wait-ready
- name: Write .npmrc file
run: echo "$NPMRC" > client/.npmrc

View File

@@ -5,15 +5,20 @@ on:
branches:
- main
env:
NODE_VERSION: '24.5.0'
jobs:
Build-production-and-ng-test:
runs-on: ubuntu-latest
env:
CHROME_BIN: /usr/bin/google-chrome
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 24.5.0
node-version: ${{ env.NODE_VERSION }}
- name: Write .npmrc file
run: |
@@ -24,8 +29,7 @@ jobs:
run: |
apt-get update
wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
apt install -y ./google-chrome*.deb;
export CHROME_BIN=/usr/bin/google-chrome
apt install -y ./google-chrome*.deb
- name: Write cypress credentials
run: echo "$CYPRESS_CREDS" > ./client/cypress.env.json
@@ -43,9 +47,9 @@ jobs:
- name: Check audit
# Audit should fail and stop the CI if critical vulnerability found
run: |
npm audit --audit-level=critical --omit=dev
npm audit --omit=dev
cd ./sas
npm audit --audit-level=critical --omit=dev
npm audit --omit=dev
cd ../client
npm audit --audit-level=critical --omit=dev
@@ -63,25 +67,26 @@ jobs:
Build-and-test-development:
runs-on: ubuntu-latest
needs: Build-production-and-ng-test
env:
CHROME_BIN: /usr/bin/google-chrome
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 24.5.0
node-version: ${{ env.NODE_VERSION }}
- name: Write .npmrc file
run: |
touch client/.npmrc
echo '${{ secrets.NPMRC}}' > client/.npmrc
- run: apt-get update
- run: wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
- run: apt install -y ./google-chrome*.deb;
- run: export CHROME_BIN=/usr/bin/google-chrome
- run: apt-get update -y
- run: apt-get -y install libgtk2.0-0 libgtk-3-0 libgbm-dev libnotify-dev libnss3 libxss1 libasound2t64 libxtst6 xauth xvfb
- run: apt -y install jq
- name: Install system dependencies
run: |
apt-get update
wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
apt install -y ./google-chrome*.deb
apt-get -y install libgtk2.0-0 libgtk-3-0 libgbm-dev libnotify-dev libnss3 libxss1 libasound2t64 libxtst6 xauth xvfb jq zip
- name: Write cypress credentials
run: echo "$CYPRESS_CREDS" > ./client/cypress.env.json
@@ -96,17 +101,18 @@ jobs:
echo ${{ secrets.SHEET_PWD }} | gpg --batch --yes --passphrase-fd 0 ./libraries/sheet-crypto.tgz.gpg
npm ci
# Install pm2 and prepare SASJS server
- run: npm i -g pm2
- run: curl -L https://github.com/sasjs/server/releases/latest/download/linux.zip > linux.zip
- run: unzip linux.zip
- run: touch .env
- run: echo RUN_TIMES=js >> .env
- run: echo NODE_PATH=node >> .env
- run: echo CORS=enable >> .env
- run: echo WHITELIST=http://localhost:4200 >> .env
- run: cat .env
- run: pm2 start api-linux --wait-ready
- name: Setup and start SASjs server
run: |
npm i -g pm2
curl -L https://github.com/sasjs/server/releases/latest/download/linux.zip > linux.zip
unzip linux.zip
touch .env
echo RUN_TIMES=js >> .env
echo NODE_PATH=node >> .env
echo CORS=enable >> .env
echo WHITELIST=http://localhost:4200 >> .env
cat .env
pm2 start api-linux --wait-ready
- name: Deploy mocked services
run: |
@@ -116,11 +122,6 @@ jobs:
sasjs cbd -t server-ci
# sasjs request services/admin/makedata -t server-ci -d ./deploy/makeData4GL.json -c ./deploy/requestConfig.json -o ./output.json
- name: Install ZIP
run: |
apt-get update
apt-get install zip
- name: Prepare and run frontend and cypress
run: |
cd ./client
@@ -136,7 +137,7 @@ jobs:
replace-in-files --regex='"hosturl".*' --replacement='hosturl:"http://localhost:4200",' ./cypress.config.ts
cat ./cypress.config.ts
# Start frontend and run cypress
npx ng serve --host 0.0.0.0 --port 4200 & npx wait-on http://localhost:4200 && npx cypress run --browser chrome --spec "cypress/e2e/liveness.cy.ts,cypress/e2e/editor.cy.ts,cypress/e2e/excel-multi-load.cy.ts,cypress/e2e/excel.cy.ts,cypress/e2e/csv.cy.ts,cypress/e2e/filtering.cy.ts,cypress/e2e/licensing.cy.ts"
npx ng serve --host 0.0.0.0 --port 4200 & npx wait-on http://localhost:4200 && npx cypress run --browser chrome --spec "cypress/e2e/csv-limited.cy.ts,cypress/e2e/liveness.cy.ts,cypress/e2e/editor.cy.ts,cypress/e2e/excel-multi-load.cy.ts,cypress/e2e/excel.cy.ts,cypress/e2e/csv.cy.ts,cypress/e2e/filtering.cy.ts,cypress/e2e/licensing.cy.ts"
- name: Zip Cypress videos
if: always()
@@ -155,10 +156,10 @@ jobs:
needs: [Build-production-and-ng-test, Build-and-test-development]
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 24.5.0
node-version: ${{ env.NODE_VERSION }}
- name: Write .npmrc file
run: |
@@ -168,17 +169,11 @@ jobs:
env:
NPMRC: ${{ secrets.NPMRC}}
- name: Install packages
- name: Install system packages
run: |
apt-get update
apt-get install zip -y
# sasjs cli is used to compile & build the SAS services
apt-get install -y zip jq doxygen
npm i -g @sasjs/cli
# jq is used to parse the release JSON
apt-get install jq -y
# doxygen is used for the SASJS docs
apt-get update
apt-get install doxygen -y
- name: Frontend Preliminary Build
description: We want to prevent creating empty release if frontend fails

View File

@@ -1,3 +1,49 @@
## [7.4.1](https://git.datacontroller.io/dc/dc/compare/v7.4.0...v7.4.1) (2026-03-12)
### Bug Fixes
* support for SASIOSNF engine (SNOW alias) plus meta assignment ([7694d1b](https://git.datacontroller.io/dc/dc/commit/7694d1b0fb2bd0407c8598147fbae87a00d889a8))
# [7.4.0](https://git.datacontroller.io/dc/dc/compare/v7.3.0...v7.4.0) (2026-02-20)
### Bug Fixes
* cli bump for mf_getscheme support ([a84ba41](https://git.datacontroller.io/dc/dc/commit/a84ba41ea9f0c97ae24f0a572b8cf5ec200f2132))
* missing upcase on SNOW section, plus local sasjs target ([dc20064](https://git.datacontroller.io/dc/dc/commit/dc200646f7df2fd1910841f392c314532aae7581))
### Features
* SAS code changes for snowflake support ([e273e87](https://git.datacontroller.io/dc/dc/commit/e273e870efbf7875db869b760f2c7b1f39d571ae))
# [7.3.0](https://git.datacontroller.io/dc/dc/compare/v7.2.8...v7.3.0) (2026-02-10)
### Bug Fixes
* bump xlsx, add crypto-shim ([8dc18b1](https://git.datacontroller.io/dc/dc/commit/8dc18b155abfc20fd0b043e0d70bbbc17e6b6811))
* correctly applying deletes on viya, also ([46cdeb0](https://git.datacontroller.io/dc/dc/commit/46cdeb0babee6870553a41877cbe85204e7099c4))
* crypto module requirement for sheetjs/crypto package ([505d0af](https://git.datacontroller.io/dc/dc/commit/505d0af2b3b3c1c79c65045dcaffc263e0f8e796))
* disable parsing excel in web worker beacuse it breaks in the stream apps ([280bdee](https://git.datacontroller.io/dc/dc/commit/280bdeeb1b82f00689f46c68a3cde3f2d24bc18f))
* Display all contexts when installing DC on Viya ([d41f88f](https://git.datacontroller.io/dc/dc/commit/d41f88f8bf5bb2c725ee3edba085e8961ab8c727))
* **edit:** use cellValidation keys and hotDataSchema to fill in defaults on add row ([4957548](https://git.datacontroller.io/dc/dc/commit/495754816c0e757b8f8b1c0ad51246dc7b65d957))
* enabling closeouts for UPDATE in CAS tables ([8b8e8ae](https://git.datacontroller.io/dc/dc/commit/8b8e8aec159ff2f50cfa4683bcd7a25aabb75bf8))
* enabling rollback when the table has formatted values ([815d6e9](https://git.datacontroller.io/dc/dc/commit/815d6e97a8e304d79d48cc949ba126e02a318dc1))
* improvements to validations ([6ceb681](https://git.datacontroller.io/dc/dc/commit/6ceb6814633691b6d4ac2cb898cfb75e9d609102))
* remove IE checks and conditions ([ece6bd1](https://git.datacontroller.io/dc/dc/commit/ece6bd1d787d722531334fc4f1396a94cf6d92ec))
* updates to demodata to enable auto CAS promote ([7740d2a](https://git.datacontroller.io/dc/dc/commit/7740d2ac8694295b33b40a30603d8239818896f5))
* upgrade angular core and compiler ([aecd597](https://git.datacontroller.io/dc/dc/commit/aecd5976875a7c01189248c5f5aa3478b28c1ab2))
* using fcopy instead of binary copy for file upload, for Viya 2026 compatibility ([716ee6e](https://git.datacontroller.io/dc/dc/commit/716ee6eba0a28f4f0a7a96b0719caf15da9b6e78))
* **viewer:** search causing blank Handsontable ([338c7a2](https://git.datacontroller.io/dc/dc/commit/338c7a2e418c47e34331bd04718cd816f978837c)), closes [#206](https://git.datacontroller.io/dc/dc/issues/206)
### Features
* adding demo data job ([8c2aeac](https://git.datacontroller.io/dc/dc/commit/8c2aeacc85da5c106c356709cefcb412ed0a71db))
* **dq rules:** notnull validation when invalid cell, will auto populate a default value ([96f2518](https://git.datacontroller.io/dc/dc/commit/96f2518af9e547956be5862a1322d9ab8e07369b))
## [7.2.8](https://git.datacontroller.io/dc/dc/compare/v7.2.7...v7.2.8) (2026-02-06)

View File

@@ -0,0 +1,95 @@
const username = Cypress.env('username')
const password = Cypress.env('password')
const hostUrl = Cypress.env('hosturl')
const appLocation = Cypress.env('appLocation')
const longerCommandTimeout = Cypress.env('longerCommandTimeout')
const serverType = Cypress.env('serverType')
const libraryToOpenIncludes = Cypress.env(`libraryToOpenIncludes_${serverType}`)
const fixturePath = 'csvs/'
context('csv file upload restriction (free tier): ', function () {
this.beforeEach(() => {
cy.visit(hostUrl + appLocation)
cy.get('body').then(($body) => {
const usernameInput = $body.find('input.username')[0]
if (usernameInput && !Cypress.dom.isHidden(usernameInput)) {
cy.get('input.username').type(username)
cy.get('input.password').type(password)
cy.get('.login-group button').click()
}
})
cy.get('.app-loading', { timeout: longerCommandTimeout }).should(
'not.exist'
)
// Skip licensing page if presented - continue with free tier
cy.url().then((url) => {
if (url.includes('licensing')) {
cy.get('button').contains('Continue with free tier').click()
}
})
visitPage('home')
})
it('1 | File upload is restricted on free tier', () => {
openTableFromTree(libraryToOpenIncludes, 'mpe_x_test')
// Click upload button - should show feature locked modal
cy.get('.buttonBar button:last-child').should('exist').click()
cy.get('.modal-title').should('contain', 'Locked Feature (File Upload)')
})
})
const openTableFromTree = (libNameIncludes: string, tablename: string) => {
cy.get('.app-loading', { timeout: longerCommandTimeout })
.should('not.exist')
.then(() => {
cy.get('.nav-tree clr-tree > clr-tree-node', {
timeout: longerCommandTimeout
}).then((treeNodes: any) => {
let targetLib
for (let node of treeNodes) {
if (node.innerText.toLowerCase().includes(libNameIncludes)) {
targetLib = node
break
}
}
cy.get(targetLib).within(() => {
cy.get('.clr-tree-node-content-container > button').click()
cy.get('.clr-treenode-link').then((innerNodes: any) => {
for (let innerNode of innerNodes) {
if (innerNode.innerText.toLowerCase().includes(tablename)) {
innerNode.click()
break
}
}
})
})
})
})
}
const attachFile = (filename: string, callback?: any) => {
cy.get('.buttonBar button:last-child')
.should('exist')
.click()
.then(() => {
cy.get('input[type="file"]#file-upload')
.attachFile(`/${fixturePath}/${filename}`)
.then(() => {
if (callback) callback()
})
})
}
const visitPage = (url: string) => {
cy.visit(`${hostUrl}${appLocation}/#/${url}`)
}

View File

@@ -4,7 +4,11 @@ PRIMARY_KEY_FIELD,SOME_CHAR,SOME_DROPDOWN,SOME_NUM,SOME_DATE,SOME_DATETIME,SOME_
2,even more dummy data,Option 3,42,12FEB1960,01JAN1960:00:00:42,0:02:22,3,44
3,"It was a dark and stormy night. The wind was blowing a gale! The captain said to his mate - mate, tell us a tale. And this, is the tale he told: It was a dark and stormy night. The wind was blowing a gale! The captain said to his mate - mate, tell us a tale. And this, is the tale he told: It was a dark and stormy night. The wind was blowing a gale! The captain said to his mate - mate, tell us a tale. And this, is the tale he told: It was a dark and stormy night. The wind was blowing a gale! The captain said to his mate - mate, tell us a tale. And this, is the tale he told:",Option 2,1613.001,27FEB1961,01JAN1960:00:07:03,0:00:44,3,44
4,if you can fill the unforgiving minute,Option 1,1613.0011235,02AUG1971,29MAY1973:06:12:03,0:06:52,3,44
1010,10 bottles of beer on the wall,Option 1,0.9153696885,04MAR1962,01JAN1960:12:47:55,0:01:40,92,76
1010,"10 bottles of beer
on the wall",Option 1,0.9153696885,04MAR1962,01JAN1960:12:47:55,0:01:40,92,76
1011,11 bottles of beer on the wall,Option 1,0.3531217558,29MAR1960,01JAN1960:03:33:24,0:01:03,80,29
1012,12 bottles of beer on the wall,Option 1,0.6743748717,02AUG1962,01JAN1960:07:25:59,0:00:10,16,98
1013,13 bottles of beer on the wall,Option 1,0.1305445992,11SEP1960,01JAN1960:13:51:32,0:00:35,73,15
1 PRIMARY_KEY_FIELD SOME_CHAR SOME_DROPDOWN SOME_NUM SOME_DATE SOME_DATETIME SOME_TIME SOME_SHORTNUM SOME_BESTNUM
4 2 even more dummy data Option 3 42 12FEB1960 01JAN1960:00:00:42 0:02:22 3 44
5 3 It was a dark and stormy night. The wind was blowing a gale! The captain said to his mate - mate, tell us a tale. And this, is the tale he told: It was a dark and stormy night. The wind was blowing a gale! The captain said to his mate - mate, tell us a tale. And this, is the tale he told: It was a dark and stormy night. The wind was blowing a gale! The captain said to his mate - mate, tell us a tale. And this, is the tale he told: It was a dark and stormy night. The wind was blowing a gale! The captain said to his mate - mate, tell us a tale. And this, is the tale he told: Option 2 1613.001 27FEB1961 01JAN1960:00:07:03 0:00:44 3 44
6 4 if you can fill the unforgiving minute Option 1 1613.0011235 02AUG1971 29MAY1973:06:12:03 0:06:52 3 44
7 1010 10 bottles of beer on the wall 10 bottles of beer on the wall Option 1 0.9153696885 04MAR1962 01JAN1960:12:47:55 0:01:40 92 76
8 1011 11 bottles of beer on the wall Option 1 0.3531217558 29MAR1960 01JAN1960:03:33:24 0:01:03 80 29
9 1012 12 bottles of beer on the wall Option 1 0.6743748717 02AUG1962 01JAN1960:07:25:59 0:00:10 16 98
10 1013 13 bottles of beer on the wall Option 1 0.1305445992 11SEP1960 01JAN1960:13:51:32 0:00:35 73 15
11 1014 14 bottles of beer on the wall Option 1 0.7409067949 26JUL1960 01JAN1960:05:18:10 0:00:41 30 89
12 1011 1015 11 bottles of beer on the wall 15 bottles of beer on the wall Option 1 0.3531217558 0.0869016028 29MAR1960 28FEB1961 01JAN1960:03:33:24 01JAN1960:13:23:45 0:01:03 0:00:44 80 29 3
13 1012 1016 12 bottles of beer on the wall 16 bottles of beer on the wall Option 1 0.6743748717 0.0462121419 02AUG1962 09AUG1962 01JAN1960:07:25:59 01JAN1960:07:42:38 0:00:10 0:01:17 16 62 98 2
14 1013 1017 13 bottles of beer on the wall 17 bottles of beer on the wall Option 1 0.1305445992 0.7501918947 11SEP1960 14MAY1962 01JAN1960:13:51:32 01JAN1960:04:40:20 0:00:35 0:00:15 73 53 15 65

View File

@@ -37,7 +37,7 @@
"hyperformula": "^2.5.0",
"iconv-lite": "^0.5.0",
"jquery-datetimepicker": "^2.5.21",
"jsrsasign": "^11.1.0",
"jsrsasign": "11.1.1",
"marked": "^5.0.0",
"moment": "^2.30.1",
"ngx-clipboard": "^16.0.0",
@@ -18234,9 +18234,9 @@
}
},
"node_modules/jsrsasign": {
"version": "11.1.0",
"resolved": "https://registry.npmjs.org/jsrsasign/-/jsrsasign-11.1.0.tgz",
"integrity": "sha512-Ov74K9GihaK9/9WncTe1mPmvrO7Py665TUfUKvraXBpu+xcTWitrtuOwcjf4KMU9maPaYn0OuaWy0HOzy/GBXg==",
"version": "11.1.1",
"resolved": "https://registry.npmjs.org/jsrsasign/-/jsrsasign-11.1.1.tgz",
"integrity": "sha512-6w95OOXH8DNeGxakqLndBEqqwQ6A70zGaky1oxfg8WVLWOnghTfJsc5Tknx+Z88MHSb1bGLcqQHImOF8Lk22XA==",
"license": "MIT",
"funding": {
"url": "https://github.com/kjur/jsrsasign#donations"
@@ -25444,9 +25444,9 @@
}
},
"node_modules/undici-types": {
"version": "7.16.0",
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-7.16.0.tgz",
"integrity": "sha512-Zz+aZWSj8LE6zoxD+xrjh4VfkIG8Ya6LvYkZqtUQGJPZjYl53ypCaUwWqo7eI0x66KBGeRo+mlBEkMSeSZ38Nw==",
"version": "7.18.2",
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-7.18.2.tgz",
"integrity": "sha512-AsuCzffGHJybSaRrmr5eHr81mwJU3kjw6M+uprWvCXiNeN9SOGwQ3Jn8jb8m3Z6izVgknn1R0FTCEAP2QrLY/w==",
"dev": true,
"license": "MIT",
"optional": true

View File

@@ -67,7 +67,7 @@
"hyperformula": "^2.5.0",
"iconv-lite": "^0.5.0",
"jquery-datetimepicker": "^2.5.21",
"jsrsasign": "^11.1.0",
"jsrsasign": "11.1.1",
"marked": "^5.0.0",
"moment": "^2.30.1",
"ngx-clipboard": "^16.0.0",

View File

@@ -55,6 +55,7 @@ export interface HandsontableStaticConfig {
* Cached viyaApi collections, search and selected endpoint
*/
export const globals: {
embed: boolean
rootParam: string
dcLib: string
xlmaps: XLMapListItem[]
@@ -69,6 +70,7 @@ export const globals: {
handsontable: HandsontableStaticConfig
[key: string]: any
} = {
embed: false,
rootParam: <string>'',
dcLib: '',
xlmaps: [],

View File

@@ -107,7 +107,7 @@
</div>
</ng-container>
<header class="app-header">
<header class="app-header" *ngIf="!embed">
<!-- <button
*ngIf="
isMainRoute('view') ||
@@ -213,9 +213,10 @@
</header>
<nav
*ngIf="
router.url.includes('submitted') ||
router.url.includes('approve') ||
router.url.includes('history')
!embed &&
(router.url.includes('submitted') ||
router.url.includes('approve') ||
router.url.includes('history'))
"
class="subnav"
>

View File

@@ -70,6 +70,7 @@ export class AppComponent {
public syssite = this.appService.syssite
public licenceState = this.licenceService.licenceState
public embed = globals.embed
constructor(
private appService: AppService,
@@ -143,6 +144,16 @@ export class AppComponent {
}
})
const hashQuery = window.location.hash.split('?')[1]
if (hashQuery) {
const embedParam = new URLSearchParams(hashQuery).get('embed')
if (embedParam !== null) {
const isEmbed = embedParam !== 'false'
globals.embed = isEmbed
this.embed = isEmbed
}
}
this.subscribeToShowAbortModal()
this.subscribeToRequestsModal()
this.subscribeToStartupData()

View File

@@ -165,7 +165,7 @@
class="card-header clr-row buttonBar headerBar clr-flex-md-row clr-justify-content-center clr-justify-content-lg-end"
>
<div
*ngIf="tableTrue"
*ngIf="tableTrue && !embed"
class="clr-col-12 clr-col-md-3 clr-col-lg-4 backBtn"
>
<span

View File

@@ -264,6 +264,9 @@ export class EditorComponent implements OnInit, AfterViewInit, OnDestroy {
public badEdit = false
public badEditCause: string | undefined
public badEditTitle: string | undefined
get embed() {
return globals.embed
}
public tableTrue: boolean | undefined
public saveLoading = false
public approvers: string[] = []

View File

@@ -30,7 +30,7 @@ export const freeTierConfig: LicenceState = {
lineage_daily_limit: 3,
tables_in_library_limit: 35,
viewbox: true,
fileUpload: true,
fileUpload: false,
editRecord: true,
addRecord: true
}

View File

@@ -375,38 +375,30 @@ export class SpreadsheetUtil {
fileType: string
): Promise<ParseResult> {
return new Promise((resolve, reject) => {
if (this.licenceState.value.submit_rows_limit !== Infinity) {
if (!this.licenceState.value.fileUpload) {
uploader.queue.pop()
return reject(
'Excel files only. To unlock CSV uploads, please contact support@datacontroller.io'
'File uploads are not enabled for this licence. Please contact support@datacontroller.io'
)
}
if (parseParams.encoding === 'WLATIN1') {
let reader = new FileReader()
const self = this
// Closure to capture the file information.
reader.onload = (theFile: any) => {
let encoded = iconv.decode(
Buffer.from(theFile.target.result),
'CP-1252'
)
let blob = new Blob([encoded], { type: fileType })
let encodedFile: File = blobToFile(blob, parseParams.file.name)
uploader.queue.pop()
uploader.addToQueue([encodedFile])
if (parseParams.encoding !== 'WLATIN1') return resolve({ uploader })
return resolve({
uploader
})
}
const reader = new FileReader()
reader.onload = (theFile) => {
if (!theFile.target?.result) return resolve({ uploader })
reader.readAsArrayBuffer(parseParams.file)
} else {
return resolve({
uploader
})
const text = theFile.target.result as string
const encoded = iconv.encode(text, 'CP-1252')
const blob = new Blob([encoded], { type: fileType })
const encodedFile: File = blobToFile(blob, parseParams.file.name)
uploader.queue.pop()
uploader.addToQueue([encodedFile])
return resolve({ uploader })
}
reader.readAsText(parseParams.file)
})
}

View File

@@ -236,7 +236,19 @@
<div class="admin-action">
Download Configuration
<button (click)="downloadConfiguration()" class="btn btn-info btn-sm">
<input
type="text"
class="clr-input libref-input"
maxlength="8"
[ngModel]="dcLib"
(ngModelChange)="targetLibref = $event.toUpperCase()"
placeholder="Target Libref"
/>
<button
(click)="downloadConfiguration()"
[disabled]="targetLibref !== dcLib && !isValidLibref(targetLibref)"
class="btn btn-info btn-sm"
>
DOWNLOAD
</button>
</div>

View File

@@ -0,0 +1,5 @@
.libref-input {
width: 100px;
margin: 0 8px;
text-transform: uppercase;
}

View File

@@ -10,6 +10,7 @@ import { EnvironmentInfo } from './models/environment-info.model'
import { AppSettingsService } from '../services/app-settings.service'
import { AppSettings } from '../models/AppSettings'
import { RequestWrapperResponse } from '../models/request-wrapper/RequestWrapperResponse'
import { globals } from '../_globals'
@Component({
selector: 'app-system',
@@ -39,6 +40,8 @@ export class SystemComponent implements OnInit {
responseModal: boolean = false
Infinity = Infinity
dcLib: string = globals.dcLib
targetLibref: string = globals.dcLib
licenceState = this.licenceService.licenceState
settings: AppSettings
@@ -71,13 +74,21 @@ export class SystemComponent implements OnInit {
this.appSettingsService.setAppSettings(this.settings)
}
isValidLibref(value: string): boolean {
return /^[A-Za-z_]\w{0,7}$/.test(value.trim())
}
downloadConfiguration() {
let sasjsConfig = this.sasService.getSasjsConfig()
let storage = sasjsConfig.serverUrl
let metaData = sasjsConfig.appLoc
let path = this.sasService.getExecutionPath()
let lib = this.targetLibref.toUpperCase().trim()
let downUrl =
storage + path + '/?_program=' + metaData + '/services/admin/exportconfig'
if (lib && lib !== this.dcLib && this.isValidLibref(lib)) {
downUrl += '&dclib=' + encodeURIComponent(lib)
}
window.open(downUrl)
}

View File

@@ -1,6 +1,6 @@
{
"name": "dcfrontend",
"version": "7.2.8",
"version": "7.4.1",
"description": "Data Controller",
"devDependencies": {
"@saithodev/semantic-release-gitea": "^2.1.0",

994
sas/package-lock.json generated

File diff suppressed because it is too large Load Diff

View File

@@ -28,7 +28,7 @@
},
"private": true,
"dependencies": {
"@sasjs/cli": "^4.13.1",
"@sasjs/core": "^4.60.0"
"@sasjs/cli": "^4.15.0",
"@sasjs/core": "^4.62.0"
}
}

View File

@@ -26,8 +26,7 @@ NOTES:
One cannot use BETWEEN
One cannot use &xx_from LE [tstamp] LE &xx_from (equivalent to above).
Background:
http://stackoverflow.com/questions/20005950/best-practice-for-scd-date-pairs-closing-opening-timestamps
Background: https://stackoverflow.com/questions/20005950
Areas for optimisation
- loading temporal history (currently experimental)
@@ -220,7 +219,8 @@ Areas for optimisation
%local engine_type;
%let engine_type=%mf_getengine(&base_lib);
%if (&engine_type=REDSHIFT or &engine_type=POSTGRES) and %length(&CLOSE_VARS)>0
%if %length(&CLOSE_VARS)>0 and (&engine_type=REDSHIFT or &engine_type=POSTGRES
or &engine_type=SNOW or &engine_type=SASIOSNF)
%then %do;
%put NOTE:; %put NOTE-;%put NOTE-;%put NOTE-;
%put NOTE- CLOSE_VARS functionality not yet supported in &engine_type;
@@ -636,7 +636,9 @@ data work.bitemp0_append &keepvars &outds_del(drop=&md5_col )
%dc_assignlib(WRITE,&base_lib,passthru=myAlias)
create table work.bitemp0_base as select * from connection to myAlias(
%end;
%else %if &engine_type=REDSHIFT or &engine_type=POSTGRES %then %do;
%else %if &engine_type=REDSHIFT or &engine_type=POSTGRES or &engine_type=SNOW
or &engine_type=SASIOSNF
%then %do;
/* grab schema */
%let baselib_schema=%mf_getschema(&base_lib);
%if &baselib_schema.X ne X %then %let baselib_schema=&baselib_schema..;
@@ -652,18 +654,24 @@ data work.bitemp0_append &keepvars &outds_del(drop=&md5_col )
call symputx('redcnt',x,'l');
run;
%end;
/* cannot persist temp tables so must create a temporary permanent table */
%let temp_table=%mf_getuniquename(prefix=XDCTEMP);
%let temp_table=%upcase(%mf_getuniquename(prefix=XDCTEMP));
%if &loadtype=BITEMPORAL or &loadtype=TXTEMPORAL %then
%let base_table=(select * from &baselib_schema.&base_dsn
where timestamp &sqlnow < &tech_to );
%else %let base_table=&baselib_schema.&base_dsn;
/* make empty table first - must clone & drop extra cols as autoload is bad */
/* make in-db empty table with PK + MD5 only */
%dc_assignlib(WRITE,&base_lib,passthru=myAlias)
exec (create table &temp_table (like &baselib_schema.&base_dsn)) by myAlias;
%if &engine_type=REDSHIFT %then %do;
exec (alter table &temp_table alter sortkey none) by myAlias;
%if &engine_type=SNOW or &engine_type=SASIOSNF %then %do;
exec (create transient table &baselib_schema.&temp_table
like &baselib_schema.&base_dsn
) by myAlias;
%end;
%else %do;
/* cannot persist temp tables so must create a temporary permanent table */
exec (create table &temp_table (like &baselib_schema.&base_dsn)) by myAlias;
%if &engine_type=REDSHIFT %then %do;
exec (alter table &temp_table alter sortkey none) by myAlias;
%end;
%end;
%local dropcols;
%let dropcols=%mf_wordsinstr1butnotstr2(
@@ -678,9 +686,12 @@ data work.bitemp0_append &keepvars &outds_del(drop=&md5_col )
exec (alter table &temp_table add column &md5_col varchar(32);) by myAlias;
/* create view to strip formats and avoid warns in log */
data work.vw_bitemp0/view=work.vw_bitemp0;
/* inherit remote length to handle byte expansion */
if 0 then set &base_lib..&temp_table(keep=&md5_col);
set work.bitemp0_append(keep=&pk &md5_col);
format _all_;
run;
proc append base=&base_lib..&temp_table
%if &engine_type=REDSHIFT %then %do;
(
@@ -733,6 +744,7 @@ data work.bitemp0_append &keepvars &outds_del(drop=&md5_col )
%if &engine_type=OLEDB or &engine_type=REDSHIFT or &engine_type=POSTGRES
or &engine_type=SNOW or &engine_type=SASIOSNF
%then %do;
); proc sql; drop table &base_lib.."&temp_table"n;
%end;
@@ -1187,7 +1199,7 @@ run;
%else %if (&loadtype=BITEMPORAL or &loadtype=TXTEMPORAL or &loadtype=UPDATE)
%then %do;
data _null_;
putlog "&sysmacroname: &loadtype operation using &engine_type engine";
putlog "&sysmacroname: &loadtype operation using *&engine_type* engine";
run;
%local flexinow;
proc sql;
@@ -1203,16 +1215,27 @@ run;
%dc_assignlib(WRITE,&base_lib,passthru=myAlias)
execute(
%end;
%else %if &engine_type=REDSHIFT or &engine_type=POSTGRES %then %do;
%let innertable=%mf_getuniquename(prefix=XDCTEMP);
%else %if &engine_type=REDSHIFT or &engine_type=POSTGRES or &engine_type=SNOW
or &engine_type=SASIOSNF
%then %do;
%let innertable=%upcase(%mf_getuniquename(prefix=XDCTEMP));
%let top_table=&baselib_schema.&base_dsn;
%let flexinow=timestamp &SQLNOW;
/* make empty table first - must clone & drop extra cols
as autoload is bad */
%dc_assignlib(WRITE,&base_lib,passthru=myAlias)
exec (create table &innertable (like &baselib_schema.&base_dsn)) by myAlias;
%if &engine_type=REDSHIFT %then %do;
exec (alter table &innertable alter sortkey none) by myAlias;
%if &engine_type=SNOW or &engine_type=SASIOSNF %then %do;
exec (create transient table &baselib_schema.&innertable
like &baselib_schema.&base_dsn
) by myAlias;
%end;
%else %do;
exec (create table &innertable
(like &baselib_schema.&base_dsn)
) by myAlias;
%if &engine_type=REDSHIFT %then %do;
exec (alter table &innertable alter sortkey none) by myAlias;
%end;
%end;
%let dropcols=%mf_wordsinstr1butnotstr2(
str1=%upcase(%mf_getvarlist(&basecopy))
@@ -1240,6 +1263,7 @@ run;
execute(
%end;
%else %do;
%put Not using passthrough for *&engine_type* engine;
%let innertable=bitemp5d_subquery;
%let top_table=&base_lib..&base_dsn;
%let flexinow=&now;
@@ -1292,6 +1316,7 @@ run;
1=1);
%if &engine_type=OLEDB or &engine_type=REDSHIFT or &engine_type=POSTGRES
or &engine_type=SNOW or &engine_type=SASIOSNF
%then %do;
) by myAlias;
execute (drop table &baselib_schema.&innertable) by myAlias;

View File

@@ -1,5 +1,5 @@
/**
@file mpe_refreshtables.sas
@file
@brief Refreshes the data catalog
@details Assumes library is already assigned.
Usage:
@@ -11,6 +11,7 @@
@version 9.3
@author 4GL Apps Ltd
**/
%macro mpe_refreshcatalogs(lib,cat=#all);

View File

@@ -238,6 +238,43 @@
"assetPaths": []
}
},
{
"name": "local",
"serverUrl": "http://localhost:5000",
"serverType": "SASJS",
"httpsAgentOptions": {
"rejectUnauthorized": false,
"allowInsecureRequests": true
},
"appLoc": "/Public/app/dc",
"deployConfig": {
"deployServicePack": true,
"deployScripts": []
},
"macroFolders": [
"sasjs/targets/server/macros_server"
],
"programFolders": [
"sasjs/db/datactrl"
],
"serviceConfig": {
"serviceFolders": [
"sasjs/targets/server/services_server/admin",
"sasjs/targets/server/services_server/usernav"
],
"initProgram": "sasjs/utils/serviceinitserver.sas",
"termProgram": "",
"macroVars": {}
},
"streamConfig": {
"streamWeb": true,
"streamWebFolder": "web",
"webSourcePath": "../client/dist",
"streamServiceName": "DataController",
"streamLogo": "favicon.ico",
"assetPaths": []
}
},
{
"name": "server-mihajlo",
"serverUrl": "https://sas9.4gl.io",

View File

@@ -16,6 +16,7 @@
@li mf_existfeature.sas
@li dc_assignlib.sas
@li mp_ds2cards.sas
@li mp_ds2csv.sas
@li mp_abort.sas
@li mp_binarycopy.sas
@li mp_cntlout.sas
@@ -117,10 +118,8 @@ options validvarname=upcase;
/* cannot proc export excel if PC Files is not licensed */
%then %do;
%let outfile=%sysfunc(pathname(work))/&table..csv;
PROC EXPORT DATA= staged
OUTFILE= "&outfile"
DBMS=csv REPLACE;
RUN;
/* cannot use PROC EXPORT as we need to wrap all char values in quotes */
%mp_ds2csv(work.staged,outfile="&outfile",headerformat=NAME)
%let ext=csv;
%let mimetype=csv;
%end;