Adds a new Developer Guide section to the docs site, with two pages:
Filter in a Pre-Edit Hook (developer-guide/filter-pre-edit-hook.md)
One Table, Two Rule Sets (developer-guide/two-rule-sets-one-table.md)
Nav entries are added under the User Guide, and the four screenshots move to docs/img/.
Filter in a Pre-Edit Hook
A client wanted to capture the query applied in a pre-edit hook and hand back a modified version of it - a date filter widened to include the previous quarter. This page documents what is actually possible, from the service source:
editors/getdata returns the filter as sasparams.FILTER_TEXT and as the query clauses. It does not return FILTER_RK - that is an input, and the editor already holds it in the URL.
PRE_EDIT_HOOK runs in getdataafterwork.OUT is filtered and sorted, and beforeFILTER_TEXT is read back out of filtref. So a hook can rewrite the returned filter text/clauses, but cannot change which rows come back that way - to change the rows it must re-read the table itself.
The worked example takes the reference date from work.OUT (the rows already returned), widens the filter to the previous quarter, re-reads the table so the grid matches, and returns the widened clause.
Row Level Security is kept applied to both branches - the user's clause (regenerated from work.QUERY with %mp_filtergenerate) and Data Controller's own clause (the validity + RLS part, obtained by calling %mpe_filtermaster with filter_rk=-1) are combined as ( user OR previous-quarter ) AND ( validity + RLS ). Wrapping the whole existing filter instead would let the extra rows through without their RLS condition.
Notes the DC_MAXOBS_WEBEDIT cap and how to keep the filter chips (query) in step.
One Table, Two Rule Sets
Moved here from the datacontroller.io blog so the developer-facing material lives in one place. Content is unchanged apart from docs front matter, root-relative image paths, and a cross-link from the "Other things to watch" bullet to the new filter page.
The companion change on the website repo removes the blog post and redirects its URL here.
Verification
mkdocs build --clean succeeds (the only INFO lines are the pre-existing absolute-link notices).
Both pages render, with syntax-highlighted SAS blocks and all four images resolving from /img/.
## What
Adds a new **Developer Guide** section to the docs site, with two pages:
- **Filter in a Pre-Edit Hook** (`developer-guide/filter-pre-edit-hook.md`)
- **One Table, Two Rule Sets** (`developer-guide/two-rule-sets-one-table.md`)
Nav entries are added under the User Guide, and the four screenshots move to `docs/img/`.
## Filter in a Pre-Edit Hook
A client wanted to capture the query applied in a pre-edit hook and hand back a modified version of it - a date filter widened to include the previous quarter. This page documents what is actually possible, from the service source:
- `editors/getdata` returns the filter as `sasparams.FILTER_TEXT` and as the `query` clauses. It does **not** return `FILTER_RK` - that is an input, and the editor already holds it in the URL.
- `PRE_EDIT_HOOK` runs in `getdata` **after** `work.OUT` is filtered and sorted, and **before** `FILTER_TEXT` is read back out of `filtref`. So a hook can rewrite the returned filter text/clauses, but cannot change which rows come back that way - to change the rows it must re-read the table itself.
- The worked example takes the reference date from `work.OUT` (the rows already returned), widens the filter to the previous quarter, re-reads the table so the grid matches, and returns the widened clause.
- Row Level Security is kept applied to both branches - the user's clause (regenerated from `work.QUERY` with `%mp_filtergenerate`) and Data Controller's own clause (the validity + RLS part, obtained by calling `%mpe_filtermaster` with `filter_rk=-1`) are combined as `( user OR previous-quarter ) AND ( validity + RLS )`. Wrapping the whole existing filter instead would let the extra rows through without their RLS condition.
- Notes the `DC_MAXOBS_WEBEDIT` cap and how to keep the filter chips (`query`) in step.
## One Table, Two Rule Sets
Moved here from the datacontroller.io blog so the developer-facing material lives in one place. Content is unchanged apart from docs front matter, root-relative image paths, and a cross-link from the "Other things to watch" bullet to the new filter page.
The companion change on the website repo removes the blog post and redirects its URL here.
## Verification
- `mkdocs build --clean` succeeds (the only INFO lines are the pre-existing absolute-link notices).
- Both pages render, with syntax-highlighted SAS blocks and all four images resolving from `/img/`.
Two pages, both aimed at SAS developers rather than end users:
- Filter in a Pre-Edit Hook: what editors/getdata returns (sasparams.FILTER_TEXT
and the query clauses, never FILTER_RK), when PRE_EDIT_HOOK runs relative to
the filter, and a worked example that widens a date filter to the previous
quarter - re-reading the table so the rows match, and keeping Row Level
Security applied to both branches.
- One Table, Two Rule Sets: moved from the datacontroller.io blog, so the
developer-facing material lives in one place. Images moved to docs/img/.
Filter in a Pre-Edit Hook - new Developer Guide section in the nav, syntax-highlighted SAS block.
One Table, Two Rule Sets - the moved post, with all four images resolving from /img/.
Rendered pages from a local `mkdocs build`:
**Filter in a Pre-Edit Hook** - new Developer Guide section in the nav, syntax-highlighted SAS block.
**One Table, Two Rule Sets** - the moved post, with all four images resolving from `/img/`.
allan
merged commit 10ca9547b3 into main2026-10-15 16:28:21 +00:00
allan
deleted branch docs/developer-guide2026-10-15 16:28:21 +00:00
Reviewed the full diff (two new Developer Guide pages, four images, mkdocs nav) for technical accuracy against the dc product source, plus repo hardening. No release-from-commits pipeline exists in this repo, so commit-convention checking does not apply.
docs/developer-guide/two-rule-sets-one-table.md:107 - the %let vs call symputx mechanism does not hold as stated: "a %let creates a new variable in the hook's own scope and is silently discarded". Per documented SAS macro scope resolution, a %let in %inc'd code resolves through the enclosing invocation chain and updates the most local existing variable - it would find LIBREF/DS in mpe_loader's local symbol table and update them there, the same table call symputx updates (the variables exist there because mpe_loader.sas:132-133 creates them with %let inside the macro). Both forms work identically in this hook context; a %let only creates a new local if no enclosing table holds the name, which is not the case here. If a %let was observed failing in practice, the real cause is something else (for example a %local shadow inside a wrapper %macro in the hook itself) - worth re-verifying before this stands as the page's headline gotcha.
docs/developer-guide/filter-pre-edit-hook.md:128 claims the re-read at :103 mirrors "the data step getdata uses itself - including dropping the transaction and processing columns it drops", but getdata's step (dc repo, sas/sasjs/services/editors/getdata.sas:230,245) also drops &drop_rk for tables configured with RK_UNDERLYING, and its BITEMPORAL branch deduplicates to the latest record per PK (getdata.sas:235-242). The example at :103 does neither, so on an RK or bitemporal table the re-read returns the surrogate key column / every temporal version, diverging from what getdata itself returns. Add &drop_rk and the dedup, or scope the claim to plain non-RK, non-bitemporal tables.
.pre-commit-config.yaml - the repo carries no gitleaks pre-commit hook (no .pre-commit-config.yaml at the PR head). This is a standing all-repo requirement: add one with the gitleaks hook pinned to an exact release tag (repo: https://github.com/gitleaks/gitleaks, rev: vX.Y.Z, hooks: - id: gitleaks).
.gitea/workflows/publish.yml:36,40-43,46-47 - deploy tooling is installed with no version pins: npm -g install cloudron-surfer, pip install mkdocs mkdocs-material fontawesome_markdown mkdocs-redirects, and npx @marp-team/marp-cli all resolve to whatever is latest at deploy time, so a compromised or breaking upstream release ships to (or takes down) the production site on the next merge. Pin exact versions in the workflow, and add a committed .npmrc with ignore-scripts=true and save-exact=true so the npm-side installs run without package install scripts.
4 findings above for review.
Reviewed the full diff (two new Developer Guide pages, four images, mkdocs nav) for technical accuracy against the dc product source, plus repo hardening. No release-from-commits pipeline exists in this repo, so commit-convention checking does not apply.
- docs/developer-guide/two-rule-sets-one-table.md:107 - the %let vs call symputx mechanism does not hold as stated: "a `%let` creates a new variable in the hook's own scope and is silently discarded". Per documented SAS macro scope resolution, a %let in %inc'd code resolves through the enclosing invocation chain and updates the most local existing variable - it would find LIBREF/DS in mpe_loader's local symbol table and update them there, the same table call symputx updates (the variables exist there because mpe_loader.sas:132-133 creates them with %let inside the macro). Both forms work identically in this hook context; a %let only creates a new local if no enclosing table holds the name, which is not the case here. If a %let was observed failing in practice, the real cause is something else (for example a %local shadow inside a wrapper %macro in the hook itself) - worth re-verifying before this stands as the page's headline gotcha.
- docs/developer-guide/filter-pre-edit-hook.md:128 claims the re-read at :103 mirrors "the data step `getdata` uses itself - including dropping the transaction and processing columns it drops", but getdata's step (dc repo, sas/sasjs/services/editors/getdata.sas:230,245) also drops `&drop_rk` for tables configured with RK_UNDERLYING, and its BITEMPORAL branch deduplicates to the latest record per PK (getdata.sas:235-242). The example at :103 does neither, so on an RK or bitemporal table the re-read returns the surrogate key column / every temporal version, diverging from what getdata itself returns. Add `&drop_rk` and the dedup, or scope the claim to plain non-RK, non-bitemporal tables.
- .pre-commit-config.yaml - the repo carries no gitleaks pre-commit hook (no .pre-commit-config.yaml at the PR head). This is a standing all-repo requirement: add one with the gitleaks hook pinned to an exact release tag (repo: https://github.com/gitleaks/gitleaks, rev: vX.Y.Z, hooks: - id: gitleaks).
- .gitea/workflows/publish.yml:36,40-43,46-47 - deploy tooling is installed with no version pins: `npm -g install cloudron-surfer`, `pip install mkdocs mkdocs-material fontawesome_markdown mkdocs-redirects`, and `npx @marp-team/marp-cli` all resolve to whatever is latest at deploy time, so a compromised or breaking upstream release ships to (or takes down) the production site on the next merge. Pin exact versions in the workflow, and add a committed .npmrc with ignore-scripts=true and save-exact=true so the npm-side installs run without package install scripts.
4 findings above for review.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
hermes
What
Adds a new Developer Guide section to the docs site, with two pages:
developer-guide/filter-pre-edit-hook.md)developer-guide/two-rule-sets-one-table.md)Nav entries are added under the User Guide, and the four screenshots move to
docs/img/.Filter in a Pre-Edit Hook
A client wanted to capture the query applied in a pre-edit hook and hand back a modified version of it - a date filter widened to include the previous quarter. This page documents what is actually possible, from the service source:
editors/getdatareturns the filter assasparams.FILTER_TEXTand as thequeryclauses. It does not returnFILTER_RK- that is an input, and the editor already holds it in the URL.PRE_EDIT_HOOKruns ingetdataafterwork.OUTis filtered and sorted, and beforeFILTER_TEXTis read back out offiltref. So a hook can rewrite the returned filter text/clauses, but cannot change which rows come back that way - to change the rows it must re-read the table itself.work.OUT(the rows already returned), widens the filter to the previous quarter, re-reads the table so the grid matches, and returns the widened clause.work.QUERYwith%mp_filtergenerate) and Data Controller's own clause (the validity + RLS part, obtained by calling%mpe_filtermasterwithfilter_rk=-1) are combined as( user OR previous-quarter ) AND ( validity + RLS ). Wrapping the whole existing filter instead would let the extra rows through without their RLS condition.DC_MAXOBS_WEBEDITcap and how to keep the filter chips (query) in step.One Table, Two Rule Sets
Moved here from the datacontroller.io blog so the developer-facing material lives in one place. Content is unchanged apart from docs front matter, root-relative image paths, and a cross-link from the "Other things to watch" bullet to the new filter page.
The companion change on the website repo removes the blog post and redirects its URL here.
Verification
mkdocs build --cleansucceeds (the only INFO lines are the pre-existing absolute-link notices)./img/.Rendered pages from a local
mkdocs build:Filter in a Pre-Edit Hook - new Developer Guide section in the nav, syntax-highlighted SAS block.
One Table, Two Rule Sets - the moved post, with all four images resolving from
/img/.Reviewed the full diff (two new Developer Guide pages, four images, mkdocs nav) for technical accuracy against the dc product source, plus repo hardening. No release-from-commits pipeline exists in this repo, so commit-convention checking does not apply.
docs/developer-guide/two-rule-sets-one-table.md:107 - the %let vs call symputx mechanism does not hold as stated: "a
%letcreates a new variable in the hook's own scope and is silently discarded". Per documented SAS macro scope resolution, a %let in %inc'd code resolves through the enclosing invocation chain and updates the most local existing variable - it would find LIBREF/DS in mpe_loader's local symbol table and update them there, the same table call symputx updates (the variables exist there because mpe_loader.sas:132-133 creates them with %let inside the macro). Both forms work identically in this hook context; a %let only creates a new local if no enclosing table holds the name, which is not the case here. If a %let was observed failing in practice, the real cause is something else (for example a %local shadow inside a wrapper %macro in the hook itself) - worth re-verifying before this stands as the page's headline gotcha.docs/developer-guide/filter-pre-edit-hook.md:128 claims the re-read at :103 mirrors "the data step
getdatauses itself - including dropping the transaction and processing columns it drops", but getdata's step (dc repo, sas/sasjs/services/editors/getdata.sas:230,245) also drops&drop_rkfor tables configured with RK_UNDERLYING, and its BITEMPORAL branch deduplicates to the latest record per PK (getdata.sas:235-242). The example at :103 does neither, so on an RK or bitemporal table the re-read returns the surrogate key column / every temporal version, diverging from what getdata itself returns. Add&drop_rkand the dedup, or scope the claim to plain non-RK, non-bitemporal tables..pre-commit-config.yaml - the repo carries no gitleaks pre-commit hook (no .pre-commit-config.yaml at the PR head). This is a standing all-repo requirement: add one with the gitleaks hook pinned to an exact release tag (repo: https://github.com/gitleaks/gitleaks, rev: vX.Y.Z, hooks: - id: gitleaks).
.gitea/workflows/publish.yml:36,40-43,46-47 - deploy tooling is installed with no version pins:
npm -g install cloudron-surfer,pip install mkdocs mkdocs-material fontawesome_markdown mkdocs-redirects, andnpx @marp-team/marp-cliall resolve to whatever is latest at deploy time, so a compromised or breaking upstream release ships to (or takes down) the production site on the next merge. Pin exact versions in the workflow, and add a committed .npmrc with ignore-scripts=true and save-exact=true so the npm-side installs run without package install scripts.4 findings above for review.