Review 431: dc-overview.md still said "Up to 500 rows can be edited (in the web
editor) at one time" - the wrong number (DC_MAXOBS_WEBEDIT ships at 250) and, once
dc/dc#352 lands, the wrong behaviour (an over-limit selection is served truncated
with a warning, not refused or capped at a count). It now names both limits,
whichever binds first, and links to the options that set them.
Sweeping for the same stale figure turned up dcu-tableviewer.md, which says 500
twice and uses 500 in its worked example of the row count above the grid.
DC_MAXOBS_WEBVIEW has been 2000 since 20261003_row_and_cell_limits.sas (the
migration records 500 -> 2000), and mpe_makedata seeds 2000, so those three
places were two releases out of date. Fixed here, and flagged in the PR reply as
VIEW rather than EDIT in case it belongs in its own change.
Checked: no "500 rows" or stale abort wording left in docs/, and the two anchors
the overview now links to (dc_maxobs_webedit, dc_maxcells_webedit) exist on the
published options page.
browser_info and browser_url_vars were only described inside the troubleshooting
page, under Support Diagnostics. That is the right place to find them if you
already know they exist and are looking for a log line; it is not where anyone
looks when writing a hook, which is what they are mostly for.
The new Developer Guide page carries the whole subject: which services receive
the two tables, the column lists, the debug values per platform and the MPEINIT
log lines, reading them from a hook with the exist() guard, and the reminder that
they are client-supplied and so diagnostics rather than a security boundary.
Troubleshooting keeps three stub headings - browser_info, browser_url_vars,
reading-them-from-a-hook - pointing at the page, so the anchors that are already
published keep resolving, and the detail is not maintained twice.
Linked from the three places a hook author actually starts: the HOOK Scripts
section of the MPE_TABLES page, the pre-edit hook guide (where the editor's
inputs are listed), the two-rule-sets guide (post-edit hook), and the VA embed
page, where embed=va is exactly the kind of thing a hook wants to branch on.
VAR_LABEL is free text in any script and the quotes are escaped when the
statement is built, but it is a column label: control characters, line breaks and
over 256 characters are refused, the last of those because the engine would cut
the value down silently.
Two gaps found reviewing the pages against the code that is shipping:
- A derived variable cannot be named by an active row level security rule of
any scope - mpe_derived_postedit.sas rejects it, because rules are validated
against the physical table where a derived column does not exist. Neither the
derived variables page nor row-level-security.md said so.
- The startup failure modal's Show log button (#356, in this release) was not
documented anywhere. Added to troubleshooting, with why the browser request
is what gets the log on Viya.
The truncation is whichever of DC_MAXOBS_WEBEDIT and DC_MAXCELLS_WEBEDIT
binds first, and for a wide table it is the cell cap - floor(cells/columns)
rows, potentially far below the row limit. The hook page named only the row
cap and quoted its 250 default, so a hook author would check the row count
alone. Same correction in the two-rule-sets page.
getdata no longer refuses a selection over DC_MAXOBS_WEBEDIT or
DC_MAXCELLS_WEBEDIT - it serves the rows within the limit and flags the
truncation against the table name. Update the option reference and the two
developer-guide pages that claimed the edit screen stops with "Table is too
big". The hook page also carried the old 500 row default.
TX_FROM, TX_TO and PROCESSED_DTTM are the default names of the metadata
columns rather than reserved names - they are physical columns, refused by the
collision check whatever MPE_TABLES calls them. The only name refused outright
is the delete marker. The code now matches: the hard-coded list is gone.
There is no earlier: dependencies are declared in VAR_DEPS and checked in both
directions, so the rejection is an undeclared reference. Both sentences now say
so.
A warning banner on both pages, and '(draft)' in the navigation title, so a
reader is not surprised when the feature is not in their installed version.
Remove both when the release carrying derived variables goes out.
mkdocs build passes; the banner and all four images render.
The screenshots now show the whole page rather than a cropped viewport, and the
configuration table is included so the three variables in the grid can be
traced to the three rows that define them.
Three screenshots from the EDIT screen: the derived columns in the grid, a
filter on a character derived column, and a filter on a numeric one. Plus a
short 'how it works' explaining that the statements go into a DATA step view
because a WHERE cannot see a variable created in the same step.
Each row lists every input the expression uses, physical columns and derived
variables alike. No sequence number to maintain; statements are emitted in
dependency order and a cycle is rejected.
A derived variable can now be built on one with a lower VARIABLE_ORDER, so
the pages describe the order column, the ordered rule (replacing the
self-contained one) and a chained example.
Review follow-up: /dc-userguide/#editor (the heading is 'Editor'), the
reserved names include VAR_TXFROM/VAR_TXTO/VAR_PROCESSED, and derived
columns are not shown in VIEW or downloads though a saved filter on one
still applies there.
New Configuration page explaining derived (computed) variables: what they
are, how to configure them in MPE_DERIVED, how filtering works, the rules
and limits (expressions only, self-contained, read-only, not stored), and
the data-exposure and performance notes.
New MPE_DERIVED table page, registered in the Table Guide nav.
The clause shown above the grid is the resolved WHERE clause, so a date,
datetime or time filter reads as a SAS literal rather than the internal day or
second count. Note it on the filter page and on the MPE_FILTERSOURCE RAW_VALUE
column, which is where the value is stored.
The pre-edit hook example re-added _____DELETE__THIS__RECORD_____="No" when
it re-read the table, mirroring the data step getdata.sas used to run. That
column is created by the editor itself (it is the Delete? column) and is
never part of the dataset, so the example no longer adds it - and a response
that carries the name is refused by the editor, which the "What the hook
does" bullet now states.
Also notes the reserved name on the MPE_TABLES page, and says in the user
guide that the Delete? column is created by Data Controller rather than
being part of the table.
The post-edit hook section claimed a `%let` "creates a new variable in the
hook's own scope and is silently discarded" while `call symputx` updates the
existing one. That is not how SAS resolves it: LIBREF and DS already exist in
an enclosing symbol table (mpe_loader's local table), and a macro only gets a
new local when the name is absent from every enclosing table, so both forms
assign to the same variable. Reworded to state that, and to keep the practical
advice (call symputx is explicit and does not depend on the enclosing scope
still holding the name).
Raised in review of #16.
The publish workflow installed cloudron-surfer, the mkdocs pip set and
@marp-team/marp-cli with no version pins, so a breaking or compromised upstream
release would ship straight to the live site on the next merge to main.
- pin cloudron-surfer@6.4.1, mkdocs==1.6.1, mkdocs-material==9.7.7,
fontawesome_markdown==0.2.6, mkdocs-redirects==1.2.3 and
@marp-team/marp-cli@4.5.1 (the versions the site builds with today)
- add .npmrc with ignore-scripts=true and save-exact=true; neither npm package
ships an install/preinstall/postinstall hook, so nothing is skipped
- The re-read step now sorts work.out by &sortpk after rebuilding it, matching
getdata (which sorts before the hook runs and does not sort again), so the
grid keeps the order the user first saw. The bullet records why.
- Adds a .pre-commit-config.yaml with the gitleaks hook pinned to v8.30.1, so
staged content is scanned at commit time.
- Adds a "What it looks like" section to the pre-edit hook page: the editor
before (the user's own filter, one quarter of rows) and after (the hook's
widened filter, both quarters), captured from the mocked estate.
- Corrects the re-read step: it now drops &drop_rk like getdata does, and the
accompanying note records that a bitemporal table also needs the latest-record
dedup, so the example no longer claims to mirror getdata exactly while missing
both (raised in review of #16).
Two pages, both aimed at SAS developers rather than end users:
- Filter in a Pre-Edit Hook: what editors/getdata returns (sasparams.FILTER_TEXT
and the query clauses, never FILTER_RK), when PRE_EDIT_HOOK runs relative to
the filter, and a worked example that widens a date filter to the previous
quarter - re-reading the table so the rows match, and keeping Row Level
Security applied to both branches.
- One Table, Two Rule Sets: moved from the datacontroller.io blog, so the
developer-facing material lives in one place. Images moved to docs/img/.
DC_MAXOBS_WEBEDIT 100 -> 250, DC_MAXOBS_WEBVIEW 500 -> 2000, and a new
DC_MAXCELLS_WEBEDIT (200000) which is applied together with the EDIT
observation limit - whichever is reached first.
The docs site only offered Material's light scheme. This adds the dark
scheme and makes it the default.
- palette: two entries, slate first, each with a toggle. Material takes the
first entry as the default, so the site opens dark and a sun icon in the
header switches to light (and a moon switches back).
- dc-brand.css: the dark scheme's greys are derived from a hue variable, so
it is pointed at the brand slate (206deg) rather than Material's neutral
225deg, which gives the whole scheme the same cast as the header.
- dc-brand.css: the brand slate is unreadable on a dark background, so the
dark scheme uses a light tint of it (#9fb5c6) for body links. Green stays
the accent, which keeps link hover visible in both schemes.
Contrast, over the dark background rgb(30, 36, 41): body text 8.8:1, body
links 7.4:1, inline code 7.2:1, accent green 5.6:1, header text 10.2:1. The
light scheme is unchanged.
allan
dc