Docs for #347 (dc/dc#348) - derived (computed) variables in the EDIT screen.
New Configuration page, docs/derived-variables.md: what a derived variable is, how it is configured in MPE_DERIVED, how filtering works, the rules and limits, the data-exposure warning and the performance note.
New Table Guide page, docs/tables/mpe_derived.md.
Both registered in mkdocs.yml (Configuration and Table Guide sections).
Verification
mkdocs build completes with no warnings for the new pages (the only INFO lines are pre-existing absolute-link notes on the mpe_xlmap_* pages).
Notes
No UI change in this PR, so no screenshots. The product PR is dc/dc#348 - it should merge first, or together with this one.
## Summary
Docs for #347 (dc/dc#348) - derived (computed) variables in the EDIT screen.
- New **Configuration** page, `docs/derived-variables.md`: what a derived variable is, how it is configured in MPE_DERIVED, how filtering works, the rules and limits, the data-exposure warning and the performance note.
- New **Table Guide** page, `docs/tables/mpe_derived.md`.
- Both registered in `mkdocs.yml` (Configuration and Table Guide sections).
## Verification
`mkdocs build` completes with no warnings for the new pages (the only INFO lines are pre-existing absolute-link notes on the `mpe_xlmap_*` pages).
## Notes
No UI change in this PR, so no screenshots. The product PR is dc/dc#348 - it should merge first, or together with this one.
New Configuration page explaining derived (computed) variables: what they
are, how to configure them in MPE_DERIVED, how filtering works, the rules
and limits (expressions only, self-contained, read-only, not stored), and
the data-exposure and performance notes.
New MPE_DERIVED table page, registered in the Table Guide nav.
Reviewed the full diff (2 new pages, mkdocs.yml nav registration) against the dc/dc#348 implementation and a local mkdocs build with the CI-pinned versions (mkdocs 1.6.1, material 9.7.7 - build is clean, only the pre-existing mpe_xlmap_* absolute-link INFO notes).
docs/derived-variables.md:9 - the link EDIT screen is a dead anchor: the built dc-userguide page has no id="edit" - the section heading is "### Editor", which renders as id="editor" (the page ToC itself links #editor). Change to /dc-userguide/#editor.
docs/derived-variables.md:59 - the reserved-name list (TX_FROM, TX_TO, PROCESSED_DTTM, _____DELETE__THIS__RECORD_____) is incomplete against the validation in dc/dc#348: %mpe_derived also rejects VAR_TXFROM, VAR_TXTO and VAR_PROCESSED. Add them, so an admin who names a derived variable VAR_TXFROM is not stopped by an error the docs do not mention.
2 findings above for review.
Reviewed the full diff (2 new pages, mkdocs.yml nav registration) against the dc/dc#348 implementation and a local mkdocs build with the CI-pinned versions (mkdocs 1.6.1, material 9.7.7 - build is clean, only the pre-existing mpe_xlmap_* absolute-link INFO notes).
- docs/derived-variables.md:9 - the link [EDIT screen](/dc-userguide/#edit) is a dead anchor: the built dc-userguide page has no id="edit" - the section heading is "### Editor", which renders as id="editor" (the page ToC itself links #editor). Change to /dc-userguide/#editor.
- docs/derived-variables.md:59 - the reserved-name list (`TX_FROM`, `TX_TO`, `PROCESSED_DTTM`, `_____DELETE__THIS__RECORD_____`) is incomplete against the validation in dc/dc#348: %mpe_derived also rejects `VAR_TXFROM`, `VAR_TXTO` and `VAR_PROCESSED`. Add them, so an admin who names a derived variable VAR_TXFROM is not stopped by an error the docs do not mention.
2 findings above for review.
Review follow-up: /dc-userguide/#editor (the heading is 'Editor'), the
reserved names include VAR_TXFROM/VAR_TXTO/VAR_PROCESSED, and derived
columns are not shown in VIEW or downloads though a saved filter on one
still applies there.
docs/derived-variables.md:9 - link corrected to /dc-userguide/#editor.
docs/derived-variables.md:59 - the reserved-name list now includes VAR_TXFROM, VAR_TXTO and VAR_PROCESSED, matching %mpe_derived.
I also added a line to the Filtering section, since the dc/dc side changed: derived columns are an EDIT screen feature - the VIEW screen and downloads do not show them, but a saved filter that references one still applies there (dc/dc#348 now builds the derived view in viewdata/getrawdata for exactly that case, and drops the columns from the output).
Both fixed in 2272f55.
- `docs/derived-variables.md:9` - link corrected to `/dc-userguide/#editor`.
- `docs/derived-variables.md:59` - the reserved-name list now includes `VAR_TXFROM`, `VAR_TXTO` and `VAR_PROCESSED`, matching `%mpe_derived`.
I also added a line to the Filtering section, since the dc/dc side changed: derived columns are an EDIT screen feature - the VIEW screen and downloads do not show them, but a saved filter that references one still applies there (dc/dc#348 now builds the derived view in viewdata/getrawdata for exactly that case, and drops the columns from the output).
Reviewed the full diff at 2272f55 - re-review after the follow-up commit; both prior findings (the dead /dc-userguide/#edit anchor and the missing VAR_TXFROM/VAR_TXTO/VAR_PROCESSED reserved names) are addressed, and the new VIEW/download paragraph matches the dc/dc#348 implementation (saved filters apply against the derived view, columns dropped from output). No CI runs on PRs in this repo (publish.yml triggers on push to main only); the mkdocs build with the CI-pinned versions was verified clean at the previous head and this delta is prose-only (nav, anchors and page set unchanged).
docs/derived-variables.md:65 - "A configuration error - a bad expression, an unknown column, a name collision - stops the EDIT screen with a message naming the variable" overstates the implementation for two of the three named cases: only the configuration checks name the variable (the abort message is built from name + reason in mpe_derived.sas), while a bad expression or an unknown column fails later in the view build and the user-facing abort is "Unable to build derived view work.vw_derived for " - the table is named, not the variable. Scope the claim to the configuration checks, or drop "with a message naming the variable".
1 finding above for review.
Reviewed the full diff at 2272f55 - re-review after the follow-up commit; both prior findings (the dead /dc-userguide/#edit anchor and the missing VAR_TXFROM/VAR_TXTO/VAR_PROCESSED reserved names) are addressed, and the new VIEW/download paragraph matches the dc/dc#348 implementation (saved filters apply against the derived view, columns dropped from output). No CI runs on PRs in this repo (publish.yml triggers on push to main only); the mkdocs build with the CI-pinned versions was verified clean at the previous head and this delta is prose-only (nav, anchors and page set unchanged).
- docs/derived-variables.md:65 - "A configuration error - a bad expression, an unknown column, a name collision - stops the EDIT screen with a message naming the variable" overstates the implementation for two of the three named cases: only the configuration checks name the variable (the abort message is built from name + reason in mpe_derived.sas), while a bad expression or an unknown column fails later in the view build and the user-facing abort is "Unable to build derived view work.vw_derived for <libds>" - the table is named, not the variable. Scope the claim to the configuration checks, or drop "with a message naming the variable".
1 finding above for review.
A derived variable can now be built on one with a lower VARIABLE_ORDER, so
the pages describe the order column, the ordered rule (replacing the
self-contained one) and a chained example.
Fixed in e233f50. The claim now separates the two cases: the configuration checks (name, type, length, collision, forward reference) name the variable, and a syntax error or unknown column is caught when the view is compiled and names the table.
Fixed in e233f50. The claim now separates the two cases: the configuration checks (name, type, length, collision, forward reference) name the variable, and a syntax error or unknown column is caught when the view is compiled and names the table.
Re-review at e233f508 - the delta from the 2272f55 review is prose-only: the VARIABLE_ORDER documentation and the scoped configuration-error sentence. Both are accurate against the dc/dc#348 head (VARIABLE_ORDER required positive integer, references to strictly-lower-order variables allowed, same-or-higher rejected with the variable named; syntax errors and unknown columns surface as the view-build abort, which names the table). No new pages or links, nav and anchors unchanged, code fences balanced. No blocking issues.
Re-review at e233f508 - the delta from the 2272f55 review is prose-only: the VARIABLE_ORDER documentation and the scoped configuration-error sentence. Both are accurate against the dc/dc#348 head (VARIABLE_ORDER required positive integer, references to strictly-lower-order variables allowed, same-or-higher rejected with the variable named; syntax errors and unknown columns surface as the view-build abort, which names the table). No new pages or links, nav and anchors unchanged, code fences balanced. No blocking issues.
Re-review at b9f7300 - the delta from the e233f508 review is the single docs(config) rename commit aligning the MPE_DERIVED column names to the product's VAR_* rename (dc/dc#348 head 5280322); both pages and every example now carry VAR_NM/VAR_ORDER/VAR_TYPE/VAR_LEN/VAR_FMT/VAR_LOGIC, matching mpe_derived.ddl, the 20261019_v7.19_release.sas migration and the mpe_derived macro. The prior reviews' findings are all addressed; nav, anchors and page set are unchanged, so the verified-clean mkdocs build carries over. One note tied to the product PR, not a docs defect in itself.
docs/tables/mpe_derived.md:22 - the documented column set matches the shipped product only at the dc/dc#348 head: the product's fresh-install writer (sas/sasjs/macros/mpe_makedatamodel.sas:702-707) still creates the pre-rename VARIABLE_* columns while its index and readers use VAR_*, so the page documents a schema a fresh deployment does not yet expose (flagged on dc/dc#348). Merge after that lands, or note the dependency.
1 finding above for review.
Re-review at b9f7300 - the delta from the e233f508 review is the single docs(config) rename commit aligning the MPE_DERIVED column names to the product's VAR_* rename (dc/dc#348 head 5280322); both pages and every example now carry VAR_NM/VAR_ORDER/VAR_TYPE/VAR_LEN/VAR_FMT/VAR_LOGIC, matching mpe_derived.ddl, the 20261019_v7.19_release.sas migration and the mpe_derived macro. The prior reviews' findings are all addressed; nav, anchors and page set are unchanged, so the verified-clean mkdocs build carries over. One note tied to the product PR, not a docs defect in itself.
- docs/tables/mpe_derived.md:22 - the documented column set matches the shipped product only at the dc/dc#348 head: the product's fresh-install writer (sas/sasjs/macros/mpe_makedatamodel.sas:702-707) still creates the pre-rename VARIABLE_* columns while its index and readers use VAR_*, so the page documents a schema a fresh deployment does not yet expose (flagged on dc/dc#348). Merge after that lands, or note the dependency.
1 finding above for review.
Re-review at 9d27f37 - the delta from the b9f7300 review is the single docs(config) paragraph (docs/derived-variables.md:81) describing the submit-time validation hook; it matches the shipped hook at dc/dc#348 head 0b010f2 (name, type, length, format and expression checks plus the duplicate-name and earlier-reference rules at submit time, with the target-library collision correctly documented as read-time only), and the rest of the page set is unchanged, so the verified-clean mkdocs build carries over. One note, carried and still tied to the product PR.
docs/tables/mpe_derived.md:19-25 - the documented VAR_NM/VAR_ORDER/VAR_TYPE/VAR_LEN/VAR_FMT/VAR_LOGIC column set matches the shipped product only at the dc/dc#348 head: the product's fresh-install writer (sas/sasjs/macros/mpe_makedatamodel.sas:702-707) still creates the pre-rename VARIABLE_* columns while its index and readers use VAR_*, so the page documents a schema a fresh deployment does not yet expose (flagged on dc/dc#348). Merge after that lands, or note the dependency.
1 finding above for review.
Re-review at 9d27f37 - the delta from the b9f7300 review is the single docs(config) paragraph (docs/derived-variables.md:81) describing the submit-time validation hook; it matches the shipped hook at dc/dc#348 head 0b010f2 (name, type, length, format and expression checks plus the duplicate-name and earlier-reference rules at submit time, with the target-library collision correctly documented as read-time only), and the rest of the page set is unchanged, so the verified-clean mkdocs build carries over. One note, carried and still tied to the product PR.
- docs/tables/mpe_derived.md:19-25 - the documented VAR_NM/VAR_ORDER/VAR_TYPE/VAR_LEN/VAR_FMT/VAR_LOGIC column set matches the shipped product only at the dc/dc#348 head: the product's fresh-install writer (sas/sasjs/macros/mpe_makedatamodel.sas:702-707) still creates the pre-rename VARIABLE_* columns while its index and readers use VAR_*, so the page documents a schema a fresh deployment does not yet expose (flagged on dc/dc#348). Merge after that lands, or note the dependency.
1 finding above for review.
Re-review at 3d5090d - the delta from the 9d27f37 review is the single docs(config) paragraph (docs/derived-variables.md:83) on the browser-side validation rules on MPE_DERIVED. Checked against the product at dc/dc#348 head 27ddcfb: the dropdown (MPE_SELECTBOX C/N for VAR_TYPE), the VAR_ORDER and VAR_LEN min/max rules, the pattern checks on VAR_NM, VAR_FMT and VAR_LOGIC, and the browser-evaluated / hook-authoritative split it states all match what shipped. The dependency carried from the previous review (docs/tables/mpe_derived.md documenting the VAR_* schema a fresh install did not yet create) is resolved - mpe_makedatamodel.sas now creates the VAR_* columns.
One note, carried and tied to the product PR: the VAR_FMT grid pattern shipped at dc/dc#348 rejects the bare w.d forms the hook and the read-time macro accept (flagged there); if it is widened rather than reverted, no docs change is needed - this page only says "pattern checked".
No blocking issues.
Re-review at 3d5090d - the delta from the 9d27f37 review is the single docs(config) paragraph (docs/derived-variables.md:83) on the browser-side validation rules on MPE_DERIVED. Checked against the product at dc/dc#348 head 27ddcfb: the dropdown (MPE_SELECTBOX C/N for VAR_TYPE), the VAR_ORDER and VAR_LEN min/max rules, the pattern checks on VAR_NM, VAR_FMT and VAR_LOGIC, and the browser-evaluated / hook-authoritative split it states all match what shipped. The dependency carried from the previous review (docs/tables/mpe_derived.md documenting the VAR_* schema a fresh install did not yet create) is resolved - mpe_makedatamodel.sas now creates the VAR_* columns.
One note, carried and tied to the product PR: the VAR_FMT grid pattern shipped at dc/dc#348 rejects the bare w.d forms the hook and the read-time macro accept (flagged there); if it is widened rather than reverted, no docs change is needed - this page only says "pattern checked".
No blocking issues.
Re-review at 98e41db - the delta from the 3d5090d review is the docs/derived-variables.md:81 edit dropping "or a duplicate name" from the submit-time rejection list, tracking the hook change at dc/dc#348. Checked against the product at dc/dc#348 head 8bae212: the remaining submit-time list matches the hook's checks exactly, and the page's other claims still hold. One finding.
docs/derived-variables.md:81 - an exact duplicate name is still rejected at submit time, by the loader's primary key check (mpe_loader.sas:414, "duplicates on the Primary Key columns (LIBREF DSN VAR_NM)") rather than by the hook, but after this edit no part of the page documents duplicate-name rejection at all - the Rules and limits section does not mention it either. Restore the clause with the corrected attribution, e.g. "A bad name, type, length, format or expression - or an exact duplicate name, rejected by the primary key check rather than the hook, or a reference to a variable that is not defined earlier - is rejected at submit time". Worth qualifying that the loader check compares raw values, so a case-variant duplicate (Region vs REGION) is not caught there and surfaces at approve time instead.
1 finding above for review.
Re-review at 98e41db - the delta from the 3d5090d review is the docs/derived-variables.md:81 edit dropping "or a duplicate name" from the submit-time rejection list, tracking the hook change at dc/dc#348. Checked against the product at dc/dc#348 head 8bae212: the remaining submit-time list matches the hook's checks exactly, and the page's other claims still hold. One finding.
- docs/derived-variables.md:81 - an exact duplicate name is still rejected at submit time, by the loader's primary key check (mpe_loader.sas:414, "duplicates on the Primary Key columns (LIBREF DSN VAR_NM)") rather than by the hook, but after this edit no part of the page documents duplicate-name rejection at all - the Rules and limits section does not mention it either. Restore the clause with the corrected attribution, e.g. "A bad name, type, length, format or expression - or an exact duplicate name, rejected by the primary key check rather than the hook, or a reference to a variable that is not defined earlier - is rejected at submit time". Worth qualifying that the loader check compares raw values, so a case-variant duplicate (Region vs REGION) is not caught there and surfaces at approve time instead.
1 finding above for review.
Re-review at 926299f - the delta from the 98e41db review is the docs/derived-variables.md:57 example statement and the :61 prose switching the documented label to <VAR_NM>=, tracking the product change at dc/dc#348 head 4d9ad38 (mpe_derived.sas:245 builds the label as cats(name,'=',expression), single quotes doubled, truncated at 256 - the new example matches exactly). mkdocs.yml nav and the table page are unchanged in this push. One finding, carried and unaddressed.
docs/derived-variables.md:81 - carried from the 98e41db review, unaddressed: with "or a duplicate name" dropped, no part of the page documents duplicate-name rejection any more. An exact duplicate VAR_NM for the same table is still rejected at submit time - by the loader's primary key check on (LIBREF DSN VAR_NM) (mpe_loader.sas:414), not by the hook - and that check compares the raw submitted values, so a case-variant duplicate (Region and REGION) is not caught there and only surfaces at approve time. Restore the clause with the corrected attribution, e.g. "or an exact duplicate name, rejected by the primary key check rather than the hook", and consider noting the case-variant behaviour.
1 finding above for review.
Re-review at 926299f - the delta from the 98e41db review is the docs/derived-variables.md:57 example statement and the :61 prose switching the documented label to <VAR_NM>=<expression>, tracking the product change at dc/dc#348 head 4d9ad38 (mpe_derived.sas:245 builds the label as cats(name,'=',expression), single quotes doubled, truncated at 256 - the new example matches exactly). mkdocs.yml nav and the table page are unchanged in this push. One finding, carried and unaddressed.
- docs/derived-variables.md:81 - carried from the 98e41db review, unaddressed: with "or a duplicate name" dropped, no part of the page documents duplicate-name rejection any more. An exact duplicate VAR_NM for the same table is still rejected at submit time - by the loader's primary key check on (LIBREF DSN VAR_NM) (mpe_loader.sas:414), not by the hook - and that check compares the raw submitted values, so a case-variant duplicate (Region and REGION) is not caught there and only surfaces at approve time. Restore the clause with the corrected attribution, e.g. "or an exact duplicate name, rejected by the primary key check rather than the hook", and consider noting the case-variant behaviour.
1 finding above for review.
Each row lists every input the expression uses, physical columns and derived
variables alike. No sequence number to maintain; statements are emitted in
dependency order and a cycle is rejected.
Re-review at 710463c - the delta from the 926299f review is the VAR_DEPENDENCIES rework commit; docs claims were checked against the dc/dc PR head (5f6569b9) implementation. mkdocs nav placement, table column list vs DDL, the statement example (label quoting, default formats), the reserved-name list, the EDIT-only scoping (viewdata/getrawdata drop derived columns while keeping them as filter targets) and the VAR_TYPE dropdown claim (MPE_SELECTBOX rows for MPE_DERIVED.VAR_TYPE exist, mpe_makedata.sas:683-699) all verify against the code. One finding.
docs/derived-variables.md:82 and :84 - "a reference to a derived variable that is not defined earlier" is the retired VAR_ORDER wording; there is no "earlier" any more. At read time the rejection is now "VAR_LOGIC references X, which is not in VAR_DEPENDENCIES" (mpe_derived.sas derived_undeclared check), and at submit time the hook rejects undeclared references and dependency cycles (mpe_derived_postedit.sas). Reword both to match, e.g. "a reference to a derived variable that is not declared in VAR_DEPENDENCIES".
1 finding above for review.
Re-review at 710463c - the delta from the 926299f review is the VAR_DEPENDENCIES rework commit; docs claims were checked against the dc/dc PR head (5f6569b9) implementation. mkdocs nav placement, table column list vs DDL, the statement example (label quoting, default formats), the reserved-name list, the EDIT-only scoping (viewdata/getrawdata drop derived columns while keeping them as filter targets) and the VAR_TYPE dropdown claim (MPE_SELECTBOX rows for MPE_DERIVED.VAR_TYPE exist, mpe_makedata.sas:683-699) all verify against the code. One finding.
- docs/derived-variables.md:82 and :84 - "a reference to a derived variable that is not defined earlier" is the retired VAR_ORDER wording; there is no "earlier" any more. At read time the rejection is now "VAR_LOGIC references X, which is not in VAR_DEPENDENCIES" (mpe_derived.sas derived_undeclared check), and at submit time the hook rejects undeclared references and dependency cycles (mpe_derived_postedit.sas). Reword both to match, e.g. "a reference to a derived variable that is not declared in VAR_DEPENDENCIES".
1 finding above for review.
Re-review at cddf39e - the delta from the 710463c review is the VAR_DEPENDENCIES -> VAR_DEPS rename (column list on the table page, examples, the rules paragraph and the validation-rules paragraph). Checked against the product at dc/dc#348 head ee5d7c6, which carries the same rename (mpe_derived.ddl, mpe_derived.sas, mpe_derived_postedit.sas, the MPE_VALIDATIONS rows in mpe_makedata.sas): the rename is complete in the docs with no leftover VAR_DEPENDENCIES, and the column list, examples, statement form and rule claims still match the implementation. One finding, carried from the 710463c review and unaddressed.
docs/derived-variables.md:81 and :83 - "a reference to a derived variable that is not defined earlier" is the retired VAR_ORDER wording; there is no "earlier" since the dependency rework. At read time the rejection is "VAR_LOGIC references X, which is not in VAR_DEPS" (mpe_derived.sas, the undeclared-reference check) and at submit time the hook rejects undeclared references and cycles (mpe_derived_postedit.sas). Reword both, e.g. "a reference to a derived variable that is not declared in VAR_DEPS".
1 finding above for review.
Re-review at cddf39e - the delta from the 710463c review is the VAR_DEPENDENCIES -> VAR_DEPS rename (column list on the table page, examples, the rules paragraph and the validation-rules paragraph). Checked against the product at dc/dc#348 head ee5d7c6, which carries the same rename (mpe_derived.ddl, mpe_derived.sas, mpe_derived_postedit.sas, the MPE_VALIDATIONS rows in mpe_makedata.sas): the rename is complete in the docs with no leftover VAR_DEPENDENCIES, and the column list, examples, statement form and rule claims still match the implementation. One finding, carried from the 710463c review and unaddressed.
- docs/derived-variables.md:81 and :83 - "a reference to a derived variable that is not defined earlier" is the retired VAR_ORDER wording; there is no "earlier" since the dependency rework. At read time the rejection is "VAR_LOGIC references X, which is not in VAR_DEPS" (mpe_derived.sas, the undeclared-reference check) and at submit time the hook rejects undeclared references and cycles (mpe_derived_postedit.sas). Reword both, e.g. "a reference to a derived variable that is not declared in VAR_DEPS".
1 finding above for review.
Re-review at 617007b - the queued head d96453a moved once during this pass: the author added the shadowing paragraph and then reverted it in the same window, so the net delta from the cddf39e review is nil on the substance - the "No collisions" bullet is back, matching the reverted product code in dc/dc#348 head 9f70c56 (VAR_NM collides with a column is an error again) and sasjs/core head ea279cd. Checked the full diff against the product: the column list, examples, statement form, reserved names, dependency rules and validation-rules paragraph all match the implementation at 9f70c56. One finding, carried from the cddf39e review and unaddressed.
docs/derived-variables.md:81 and :83 - "a reference to a derived variable that is not defined earlier" is the retired VAR_ORDER wording; there is no "earlier" since the dependency rework. At read time the rejection is "VAR_LOGIC references X, which is not in VAR_DEPS" (mpe_derived.sas, the undeclared-reference check) and at submit time the hook rejects undeclared references and cycles (mpe_derived_postedit.sas). Reword both, e.g. "a reference to a derived variable that is not declared in VAR_DEPS".
1 finding above for review.
Re-review at 617007b - the queued head d96453a moved once during this pass: the author added the shadowing paragraph and then reverted it in the same window, so the net delta from the cddf39e review is nil on the substance - the "No collisions" bullet is back, matching the reverted product code in dc/dc#348 head 9f70c56 (VAR_NM collides with a column is an error again) and sasjs/core head ea279cd. Checked the full diff against the product: the column list, examples, statement form, reserved names, dependency rules and validation-rules paragraph all match the implementation at 9f70c56. One finding, carried from the cddf39e review and unaddressed.
- docs/derived-variables.md:81 and :83 - "a reference to a derived variable that is not defined earlier" is the retired VAR_ORDER wording; there is no "earlier" since the dependency rework. At read time the rejection is "VAR_LOGIC references X, which is not in VAR_DEPS" (mpe_derived.sas, the undeclared-reference check) and at submit time the hook rejects undeclared references and cycles (mpe_derived_postedit.sas). Reword both, e.g. "a reference to a derived variable that is not declared in VAR_DEPS".
1 finding above for review.
Re-review at ac48b91 - the queued head 617007b moved once during this pass (the VAR_LABEL docs commit); the delta is the VAR_LABEL row in the configuration list, the two example blocks, the label paragraph and the MPE_DERIVED column entry, all verified against the product code at dc/dc#348 head ace4a66 - the default <VAR_NM>= label with the configured-label override, the even-quote truncation claim, and var_label varchar(256) in the ddl, makedatamodel, migration and mock schema all match. The mkdocs.yml registrations are in place in the Configuration and Table Guide sections, and the /dc-userguide/#editor, /tables/mpe_derived/, /filter/ and /column-level-security/ link targets all resolve. One finding, carried unaddressed.
docs/derived-variables.md:84 and :86 - carried, unaddressed: "a reference to a derived variable that is not defined earlier" is the retired VAR_ORDER wording; there is no "earlier" since the dependency rework. At read time the rejection is "VAR_LOGIC references X, which is not in VAR_DEPS" (the undeclared-reference check in mpe_derived.sas) and at submit time the hook rejects undeclared references and cycles. Reword both, e.g. "a reference to a derived variable that is not declared in VAR_DEPS".
1 finding above for review.
Re-review at ac48b91 - the queued head 617007b moved once during this pass (the VAR_LABEL docs commit); the delta is the VAR_LABEL row in the configuration list, the two example blocks, the label paragraph and the MPE_DERIVED column entry, all verified against the product code at dc/dc#348 head ace4a66 - the default <VAR_NM>=<expression> label with the configured-label override, the even-quote truncation claim, and var_label varchar(256) in the ddl, makedatamodel, migration and mock schema all match. The mkdocs.yml registrations are in place in the Configuration and Table Guide sections, and the /dc-userguide/#editor, /tables/mpe_derived/, /filter/ and /column-level-security/ link targets all resolve. One finding, carried unaddressed.
- docs/derived-variables.md:84 and :86 - carried, unaddressed: "a reference to a derived variable that is not defined earlier" is the retired VAR_ORDER wording; there is no "earlier" since the dependency rework. At read time the rejection is "VAR_LOGIC references X, which is not in VAR_DEPS" (the undeclared-reference check in mpe_derived.sas) and at submit time the hook rejects undeclared references and cycles. Reword both, e.g. "a reference to a derived variable that is not declared in VAR_DEPS".
1 finding above for review.
Three screenshots from the EDIT screen: the derived columns in the grid, a
filter on a character derived column, and a filter on a numeric one. Plus a
short 'how it works' explaining that the statements go into a DATA step view
because a WHERE cannot see a variable created in the same step.
Re-review at a70daf17 - the delta from the ac48b91 review is the single docs(derived) commit: the new "What it looks like" and "How it works" sections plus the three screenshots. The new claims verify against the product at dc/dc#348 head ccd468b3: the view-and-filter mechanism matches getdata.sas, the demo-table claims match the DERIVED_DEMO data and derived-variables.cy.ts (REGION = 'West' selects the two western states, LINE_VALUE > 50 leaves the one row, values 21/99), and the read-only claim matches the CLS_RULE=READ assertion in getdata.test.sas. One finding, carried unaddressed.
docs/derived-variables.md:117 and :119 - carried, unaddressed: "a reference to a derived variable that is not defined earlier" is the retired VAR_ORDER wording; there is no "earlier" since the dependency rework. At read time the rejection is "VAR_LOGIC references X, which is not in VAR_DEPS" (the undeclared-reference check in mpe_derived.sas / mp_derivedvars) and at submit time the hook rejects undeclared references and cycles (mpe_derived_postedit.sas). Reword both, e.g. "a reference to a derived variable that is not declared in VAR_DEPS".
1 finding above for review.
Re-review at a70daf17 - the delta from the ac48b91 review is the single docs(derived) commit: the new "What it looks like" and "How it works" sections plus the three screenshots. The new claims verify against the product at dc/dc#348 head ccd468b3: the view-and-filter mechanism matches getdata.sas, the demo-table claims match the DERIVED_DEMO data and derived-variables.cy.ts (REGION = 'West' selects the two western states, LINE_VALUE > 50 leaves the one row, values 21/99), and the read-only claim matches the CLS_RULE=READ assertion in getdata.test.sas. One finding, carried unaddressed.
- docs/derived-variables.md:117 and :119 - carried, unaddressed: "a reference to a derived variable that is not defined earlier" is the retired VAR_ORDER wording; there is no "earlier" since the dependency rework. At read time the rejection is "VAR_LOGIC references X, which is not in VAR_DEPS" (the undeclared-reference check in mpe_derived.sas / mp_derivedvars) and at submit time the hook rejects undeclared references and cycles (mpe_derived_postedit.sas). Reword both, e.g. "a reference to a derived variable that is not declared in VAR_DEPS".
1 finding above for review.
The screenshots now show the whole page rather than a cropped viewport, and the
configuration table is included so the three variables in the grid can be
traced to the three rows that define them.
A warning banner on both pages, and '(draft)' in the navigation title, so a
reader is not surprised when the feature is not in their installed version.
Remove both when the release carrying derived variables goes out.
mkdocs build passes; the banner and all four images render.
Marked as draft in c25686c so this can merge ahead of the release.
a warning banner on both derived-variables.md and tables/mpe_derived.md, stating the feature is in development and ships in the next release
(draft) in the navigation titles in mkdocs.yml
mkdocs build passes and the rendered page carries the banner and all four images. At release, remove the two banners and the two (draft) suffixes.
The blog post (dc/datacontroller.io#27) stays with the release rather than merging now, since a post announcing a feature that is not yet installable would be worse than a late post.
Marked as draft in c25686c so this can merge ahead of the release.
- a warning banner on both `derived-variables.md` and `tables/mpe_derived.md`, stating the feature is in development and ships in the next release
- `(draft)` in the navigation titles in `mkdocs.yml`
`mkdocs build` passes and the rendered page carries the banner and all four images. At release, remove the two banners and the two `(draft)` suffixes.
The blog post (dc/datacontroller.io#27) stays with the release rather than merging now, since a post announcing a feature that is not yet installable would be worse than a late post.
There is no earlier: dependencies are declared in VAR_DEPS and checked in both
directions, so the rejection is an undeclared reference. Both sentences now say
so.
Fixed in 3e4e741. Both sentences now say a reference to a derived variable that is not declared in VAR_DEPS, which is what the read-time engine reports (VAR_LOGIC references X, which is not in VAR_DEPS) and what the post edit hook rejects. Grepped the page, the MPE_DERIVED table page and the blog post for the same retired wording: no other occurrences. mkdocs build passes.
Fixed in 3e4e741. Both sentences now say a reference to a derived variable that is not declared in VAR_DEPS, which is what the read-time engine reports (VAR_LOGIC references X, which is not in VAR_DEPS) and what the post edit hook rejects. Grepped the page, the MPE_DERIVED table page and the blog post for the same retired wording: no other occurrences. mkdocs build passes.
TX_FROM, TX_TO and PROCESSED_DTTM are the default names of the metadata
columns rather than reserved names - they are physical columns, refused by the
collision check whatever MPE_TABLES calls them. The only name refused outright
is the delete marker. The code now matches: the hard-coded list is gone.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
hermes
allan
Summary
Docs for #347 (dc/dc#348) - derived (computed) variables in the EDIT screen.
docs/derived-variables.md: what a derived variable is, how it is configured in MPE_DERIVED, how filtering works, the rules and limits, the data-exposure warning and the performance note.docs/tables/mpe_derived.md.mkdocs.yml(Configuration and Table Guide sections).Verification
mkdocs buildcompletes with no warnings for the new pages (the only INFO lines are pre-existing absolute-link notes on thempe_xlmap_*pages).Notes
No UI change in this PR, so no screenshots. The product PR is dc/dc#348 - it should merge first, or together with this one.
Reviewed the full diff (2 new pages, mkdocs.yml nav registration) against the dc/dc#348 implementation and a local mkdocs build with the CI-pinned versions (mkdocs 1.6.1, material 9.7.7 - build is clean, only the pre-existing mpe_xlmap_* absolute-link INFO notes).
TX_FROM,TX_TO,PROCESSED_DTTM,_____DELETE__THIS__RECORD_____) is incomplete against the validation in dc/dc#348: %mpe_derived also rejectsVAR_TXFROM,VAR_TXTOandVAR_PROCESSED. Add them, so an admin who names a derived variable VAR_TXFROM is not stopped by an error the docs do not mention.2 findings above for review.
Both fixed in
2272f55.docs/derived-variables.md:9- link corrected to/dc-userguide/#editor.docs/derived-variables.md:59- the reserved-name list now includesVAR_TXFROM,VAR_TXTOandVAR_PROCESSED, matching%mpe_derived.I also added a line to the Filtering section, since the dc/dc side changed: derived columns are an EDIT screen feature - the VIEW screen and downloads do not show them, but a saved filter that references one still applies there (dc/dc#348 now builds the derived view in viewdata/getrawdata for exactly that case, and drops the columns from the output).
Reviewed the full diff at
2272f55- re-review after the follow-up commit; both prior findings (the dead /dc-userguide/#edit anchor and the missing VAR_TXFROM/VAR_TXTO/VAR_PROCESSED reserved names) are addressed, and the new VIEW/download paragraph matches the dc/dc#348 implementation (saved filters apply against the derived view, columns dropped from output). No CI runs on PRs in this repo (publish.yml triggers on push to main only); the mkdocs build with the CI-pinned versions was verified clean at the previous head and this delta is prose-only (nav, anchors and page set unchanged).1 finding above for review.
Fixed in
e233f50. The claim now separates the two cases: the configuration checks (name, type, length, collision, forward reference) name the variable, and a syntax error or unknown column is caught when the view is compiled and names the table.Re-review at
e233f508- the delta from the2272f55review is prose-only: the VARIABLE_ORDER documentation and the scoped configuration-error sentence. Both are accurate against the dc/dc#348 head (VARIABLE_ORDER required positive integer, references to strictly-lower-order variables allowed, same-or-higher rejected with the variable named; syntax errors and unknown columns surface as the view-build abort, which names the table). No new pages or links, nav and anchors unchanged, code fences balanced. No blocking issues.Re-review at
b9f7300- the delta from thee233f508review is the single docs(config) rename commit aligning the MPE_DERIVED column names to the product's VAR_* rename (dc/dc#348 head 5280322); both pages and every example now carry VAR_NM/VAR_ORDER/VAR_TYPE/VAR_LEN/VAR_FMT/VAR_LOGIC, matching mpe_derived.ddl, the 20261019_v7.19_release.sas migration and the mpe_derived macro. The prior reviews' findings are all addressed; nav, anchors and page set are unchanged, so the verified-clean mkdocs build carries over. One note tied to the product PR, not a docs defect in itself.1 finding above for review.
Re-review at
9d27f37- the delta from theb9f7300review is the single docs(config) paragraph (docs/derived-variables.md:81) describing the submit-time validation hook; it matches the shipped hook at dc/dc#348 head 0b010f2 (name, type, length, format and expression checks plus the duplicate-name and earlier-reference rules at submit time, with the target-library collision correctly documented as read-time only), and the rest of the page set is unchanged, so the verified-clean mkdocs build carries over. One note, carried and still tied to the product PR.1 finding above for review.
Re-review at
3d5090d- the delta from the9d27f37review is the single docs(config) paragraph (docs/derived-variables.md:83) on the browser-side validation rules on MPE_DERIVED. Checked against the product at dc/dc#348 head 27ddcfb: the dropdown (MPE_SELECTBOX C/N for VAR_TYPE), the VAR_ORDER and VAR_LEN min/max rules, the pattern checks on VAR_NM, VAR_FMT and VAR_LOGIC, and the browser-evaluated / hook-authoritative split it states all match what shipped. The dependency carried from the previous review (docs/tables/mpe_derived.md documenting the VAR_* schema a fresh install did not yet create) is resolved - mpe_makedatamodel.sas now creates the VAR_* columns.One note, carried and tied to the product PR: the VAR_FMT grid pattern shipped at dc/dc#348 rejects the bare w.d forms the hook and the read-time macro accept (flagged there); if it is widened rather than reverted, no docs change is needed - this page only says "pattern checked".
No blocking issues.
Re-review at
98e41db- the delta from the3d5090dreview is the docs/derived-variables.md:81 edit dropping "or a duplicate name" from the submit-time rejection list, tracking the hook change at dc/dc#348. Checked against the product at dc/dc#348 head 8bae212: the remaining submit-time list matches the hook's checks exactly, and the page's other claims still hold. One finding.1 finding above for review.
Re-review at
926299f- the delta from the98e41dbreview is the docs/derived-variables.md:57 example statement and the :61 prose switching the documented label to <VAR_NM>=, tracking the product change at dc/dc#348 head 4d9ad38 (mpe_derived.sas:245 builds the label as cats(name,'=',expression), single quotes doubled, truncated at 256 - the new example matches exactly). mkdocs.yml nav and the table page are unchanged in this push. One finding, carried and unaddressed.98e41dbreview, unaddressed: with "or a duplicate name" dropped, no part of the page documents duplicate-name rejection any more. An exact duplicate VAR_NM for the same table is still rejected at submit time - by the loader's primary key check on (LIBREF DSN VAR_NM) (mpe_loader.sas:414), not by the hook - and that check compares the raw submitted values, so a case-variant duplicate (Region and REGION) is not caught there and only surfaces at approve time. Restore the clause with the corrected attribution, e.g. "or an exact duplicate name, rejected by the primary key check rather than the hook", and consider noting the case-variant behaviour.1 finding above for review.
Re-review at
710463c- the delta from the926299freview is the VAR_DEPENDENCIES rework commit; docs claims were checked against the dc/dc PR head (5f6569b9) implementation. mkdocs nav placement, table column list vs DDL, the statement example (label quoting, default formats), the reserved-name list, the EDIT-only scoping (viewdata/getrawdata drop derived columns while keeping them as filter targets) and the VAR_TYPE dropdown claim (MPE_SELECTBOX rows for MPE_DERIVED.VAR_TYPE exist, mpe_makedata.sas:683-699) all verify against the code. One finding.1 finding above for review.
Re-review at
cddf39e- the delta from the710463creview is the VAR_DEPENDENCIES -> VAR_DEPS rename (column list on the table page, examples, the rules paragraph and the validation-rules paragraph). Checked against the product at dc/dc#348 head ee5d7c6, which carries the same rename (mpe_derived.ddl, mpe_derived.sas, mpe_derived_postedit.sas, the MPE_VALIDATIONS rows in mpe_makedata.sas): the rename is complete in the docs with no leftover VAR_DEPENDENCIES, and the column list, examples, statement form and rule claims still match the implementation. One finding, carried from the710463creview and unaddressed.1 finding above for review.
Re-review at
617007b- the queued headd96453amoved once during this pass: the author added the shadowing paragraph and then reverted it in the same window, so the net delta from thecddf39ereview is nil on the substance - the "No collisions" bullet is back, matching the reverted product code in dc/dc#348 head 9f70c56 (VAR_NM collides with a column is an error again) and sasjs/core head ea279cd. Checked the full diff against the product: the column list, examples, statement form, reserved names, dependency rules and validation-rules paragraph all match the implementation at 9f70c56. One finding, carried from thecddf39ereview and unaddressed.1 finding above for review.
Re-review at
ac48b91- the queued head617007bmoved once during this pass (the VAR_LABEL docs commit); the delta is the VAR_LABEL row in the configuration list, the two example blocks, the label paragraph and the MPE_DERIVED column entry, all verified against the product code at dc/dc#348 head ace4a66 - the default <VAR_NM>= label with the configured-label override, the even-quote truncation claim, and var_label varchar(256) in the ddl, makedatamodel, migration and mock schema all match. The mkdocs.yml registrations are in place in the Configuration and Table Guide sections, and the /dc-userguide/#editor, /tables/mpe_derived/, /filter/ and /column-level-security/ link targets all resolve. One finding, carried unaddressed.1 finding above for review.
Re-review at
a70daf17- the delta from theac48b91review is the single docs(derived) commit: the new "What it looks like" and "How it works" sections plus the three screenshots. The new claims verify against the product at dc/dc#348 head ccd468b3: the view-and-filter mechanism matches getdata.sas, the demo-table claims match the DERIVED_DEMO data and derived-variables.cy.ts (REGION = 'West' selects the two western states, LINE_VALUE > 50 leaves the one row, values 21/99), and the read-only claim matches the CLS_RULE=READ assertion in getdata.test.sas. One finding, carried unaddressed.1 finding above for review.
Marked as draft in
c25686cso this can merge ahead of the release.derived-variables.mdandtables/mpe_derived.md, stating the feature is in development and ships in the next release(draft)in the navigation titles inmkdocs.ymlmkdocs buildpasses and the rendered page carries the banner and all four images. At release, remove the two banners and the two(draft)suffixes.The blog post (dc/datacontroller.io#27) stays with the release rather than merging now, since a post announcing a feature that is not yet installable would be worse than a late post.
Fixed in
3e4e741. Both sentences now say a reference to a derived variable that is not declared in VAR_DEPS, which is what the read-time engine reports (VAR_LOGIC references X, which is not in VAR_DEPS) and what the post edit hook rejects. Grepped the page, the MPE_DERIVED table page and the blog post for the same retired wording: no other occurrences. mkdocs build passes.