blog(derived-variables): computed columns you can filter on #27

2026-10-21T14:38:32Z

hermes commented

2026-10-21 14:38:32 +00:00

A feature post for derived variables: computed columns that do not exist in the table, and that can be filtered on.

The post explains the design driver first - a WHERE cannot see a variable created in the same DATA step, which is why the statements go into a view - then the MPE_DERIVED configuration, dependency ordering through VAR_DEPS, and the limits (expression only, no shadowing or self-reference, never stored, EDIT screen only, not usable in row level security).

Three screenshots, captured from a live EDIT session against the demo tables:

the grid, showing LINE_VALUE, REGION and REGION_LABEL as read-only derived columns
REGION = 'West', a filter on a character derived column
LINE_VALUE > 50, a filter on a numeric one

Publishes with the release that carries the feature; it is written against the branch rather than main, so it should not merge before that release.

A feature post for derived variables: computed columns that do not exist in the table, and that can be filtered on. The post explains the design driver first - a `WHERE` cannot see a variable created in the same DATA step, which is why the statements go into a view - then the `MPE_DERIVED` configuration, dependency ordering through `VAR_DEPS`, and the limits (expression only, no shadowing or self-reference, never stored, EDIT screen only, not usable in row level security). Three screenshots, captured from a live EDIT session against the demo tables: - the grid, showing `LINE_VALUE`, `REGION` and `REGION_LABEL` as read-only derived columns - `REGION = 'West'`, a filter on a character derived column - `LINE_VALUE > 50`, a filter on a numeric one Publishes with the release that carries the feature; it is written against the branch rather than main, so it should not merge before that release.

hermes added 1 commit 2026-10-21 14:38:32 +00:00

blog(derived-variables): computed columns you can filter on 67cf263ec5

The feature post: why a computed column is not filterable in a single DATA
step, the view that makes it so, the MPE_DERIVED configuration, dependency
ordering by VAR_DEPS, and what the feature will not do. Three screenshots from
the EDIT screen - the grid, a filter on a character derived column, and a
filter on a numeric one.

hermes reviewed

2026-10-21 14:57:37 +00:00

hermes left a comment

Reviewed the full diff (new blog post plus three screenshots) against the feature implementation in dc/dc#348 at head ccd468b3 - the WHERE/view design explanation, the MPE_DERIVED configuration rows, the generated-statement example (label doubling and quote-escaped expression match mp_derivedvars), the VAR_DEPS dependency ordering and both-directions check, the cycle message format (A -> B -> C -> A), the expression limits (no ; & % /* */), the read-only grid claim (CLS_RULE resolves to READ, cypress spec asserts the values), the RLS rejection, the discard-before-security-check in stagedata, the saved-filter behaviour in VIEW/downloads, and the demo-data claims (two western states, one row over 50) all verify against the code and the derived-variables.cy.ts spec. No release tooling in this repo, so the subject format is not release-driving.

content/blog/derived-variables/index.md:133 - "The configuration table, the settings and the post-edit validation are created by the usual upgrade" - the release migration (sas/sasjs/db/migrations/20261019_v7.19_release.sas in dc/dc#348) creates the MPE_DERIVED table, its MPE_TABLES registration, the VAR_TYPE selectbox values and the MPE_VALIDATIONS rules; it creates no settings (no MPE_CONFIG / DC_* option is touched anywhere in the PR). Drop "the settings" or name what is actually created, e.g. "The configuration table, its dropdown and validation rules, and the post-edit validation are created by the usual upgrade".
Missing gitleaks pre-commit hook: no .pre-commit-config.yaml at the repo root. Standing all-repo requirement - add one with a gitleaks hook (id: gitleaks) pinned to an exact release tag (current release v8.30.1). Re-flagged from the #25 review; still not landed.
No npm audit step in either publish workflow (.gitea/workflows/publish.yml, .github/workflows/publish.yml). Both run npm ci, then build and deploy with nothing that fails the build on a vulnerability. Add npm audit (default level, or --audit-level=low) as a gating step to at least one pipeline. Re-flagged from the #25 review; still not landed.

3 findings above for review.

Reviewed the full diff (new blog post plus three screenshots) against the feature implementation in dc/dc#348 at head ccd468b3 - the WHERE/view design explanation, the MPE_DERIVED configuration rows, the generated-statement example (label doubling and quote-escaped expression match mp_derivedvars), the VAR_DEPS dependency ordering and both-directions check, the cycle message format (A -> B -> C -> A), the expression limits (no ; & % /* */), the read-only grid claim (CLS_RULE resolves to READ, cypress spec asserts the values), the RLS rejection, the discard-before-security-check in stagedata, the saved-filter behaviour in VIEW/downloads, and the demo-data claims (two western states, one row over 50) all verify against the code and the derived-variables.cy.ts spec. No release tooling in this repo, so the subject format is not release-driving. - content/blog/derived-variables/index.md:133 - "The configuration table, the settings and the post-edit validation are created by the usual upgrade" - the release migration (sas/sasjs/db/migrations/20261019_v7.19_release.sas in dc/dc#348) creates the MPE_DERIVED table, its MPE_TABLES registration, the VAR_TYPE selectbox values and the MPE_VALIDATIONS rules; it creates no settings (no MPE_CONFIG / DC_* option is touched anywhere in the PR). Drop "the settings" or name what is actually created, e.g. "The configuration table, its dropdown and validation rules, and the post-edit validation are created by the usual upgrade". - Missing gitleaks pre-commit hook: no .pre-commit-config.yaml at the repo root. Standing all-repo requirement - add one with a gitleaks hook (id: gitleaks) pinned to an exact release tag (current release v8.30.1). Re-flagged from the #25 review; still not landed. - No npm audit step in either publish workflow (.gitea/workflows/publish.yml, .github/workflows/publish.yml). Both run npm ci, then build and deploy with nothing that fails the build on a vulnerability. Add npm audit (default level, or --audit-level=low) as a gating step to at least one pipeline. Re-flagged from the #25 review; still not landed. 3 findings above for review.

hermes added 1 commit 2026-10-21 15:10:23 +00:00

blog(derived-variables): include the configuration table, full-page screenshots f38b5d795c

Adds the MPE_DERIVED rows behind the three demo variables, and recaptures the
grid images at full page so the whole screen is in frame.

hermes referenced this pull request from dc/docs.datacontroller.io

2026-10-21 22:14:01 +00:00

docs(config): derived variables in the EDIT screen #21

You are not authorized to merge this pull request.

This pull request can be merged automatically.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin site/derived-variables:site/derived-variables

git checkout site/derived-variables

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: dc/datacontroller.io#27