Three screenshots from the EDIT screen: the derived columns in the grid, a
filter on a character derived column, and a filter on a numeric one. Plus a
short 'how it works' explaining that the statements go into a DATA step view
because a WHERE cannot see a variable created in the same step.
Each row lists every input the expression uses, physical columns and derived
variables alike. No sequence number to maintain; statements are emitted in
dependency order and a cycle is rejected.
A derived variable can now be built on one with a lower VARIABLE_ORDER, so
the pages describe the order column, the ordered rule (replacing the
self-contained one) and a chained example.
Review follow-up: /dc-userguide/#editor (the heading is 'Editor'), the
reserved names include VAR_TXFROM/VAR_TXTO/VAR_PROCESSED, and derived
columns are not shown in VIEW or downloads though a saved filter on one
still applies there.
New Configuration page explaining derived (computed) variables: what they
are, how to configure them in MPE_DERIVED, how filtering works, the rules
and limits (expressions only, self-contained, read-only, not stored), and
the data-exposure and performance notes.
New MPE_DERIVED table page, registered in the Table Guide nav.
The clause shown above the grid is the resolved WHERE clause, so a date,
datetime or time filter reads as a SAS literal rather than the internal day or
second count. Note it on the filter page and on the MPE_FILTERSOURCE RAW_VALUE
column, which is where the value is stored.
The pre-edit hook example re-added _____DELETE__THIS__RECORD_____="No" when
it re-read the table, mirroring the data step getdata.sas used to run. That
column is created by the editor itself (it is the Delete? column) and is
never part of the dataset, so the example no longer adds it - and a response
that carries the name is refused by the editor, which the "What the hook
does" bullet now states.
Also notes the reserved name on the MPE_TABLES page, and says in the user
guide that the Delete? column is created by Data Controller rather than
being part of the table.
The post-edit hook section claimed a `%let` "creates a new variable in the
hook's own scope and is silently discarded" while `call symputx` updates the
existing one. That is not how SAS resolves it: LIBREF and DS already exist in
an enclosing symbol table (mpe_loader's local table), and a macro only gets a
new local when the name is absent from every enclosing table, so both forms
assign to the same variable. Reworded to state that, and to keep the practical
advice (call symputx is explicit and does not depend on the enclosing scope
still holding the name).
Raised in review of #16.
The publish workflow installed cloudron-surfer, the mkdocs pip set and
@marp-team/marp-cli with no version pins, so a breaking or compromised upstream
release would ship straight to the live site on the next merge to main.
- pin cloudron-surfer@6.4.1, mkdocs==1.6.1, mkdocs-material==9.7.7,
fontawesome_markdown==0.2.6, mkdocs-redirects==1.2.3 and
@marp-team/marp-cli@4.5.1 (the versions the site builds with today)
- add .npmrc with ignore-scripts=true and save-exact=true; neither npm package
ships an install/preinstall/postinstall hook, so nothing is skipped
- The re-read step now sorts work.out by &sortpk after rebuilding it, matching
getdata (which sorts before the hook runs and does not sort again), so the
grid keeps the order the user first saw. The bullet records why.
- Adds a .pre-commit-config.yaml with the gitleaks hook pinned to v8.30.1, so
staged content is scanned at commit time.
- Adds a "What it looks like" section to the pre-edit hook page: the editor
before (the user's own filter, one quarter of rows) and after (the hook's
widened filter, both quarters), captured from the mocked estate.
- Corrects the re-read step: it now drops &drop_rk like getdata does, and the
accompanying note records that a bitemporal table also needs the latest-record
dedup, so the example no longer claims to mirror getdata exactly while missing
both (raised in review of #16).
Two pages, both aimed at SAS developers rather than end users:
- Filter in a Pre-Edit Hook: what editors/getdata returns (sasparams.FILTER_TEXT
and the query clauses, never FILTER_RK), when PRE_EDIT_HOOK runs relative to
the filter, and a worked example that widens a date filter to the previous
quarter - re-reading the table so the rows match, and keeping Row Level
Security applied to both branches.
- One Table, Two Rule Sets: moved from the datacontroller.io blog, so the
developer-facing material lives in one place. Images moved to docs/img/.
DC_MAXOBS_WEBEDIT 100 -> 250, DC_MAXOBS_WEBVIEW 500 -> 2000, and a new
DC_MAXCELLS_WEBEDIT (200000) which is applied together with the EDIT
observation limit - whichever is reached first.
The docs site only offered Material's light scheme. This adds the dark
scheme and makes it the default.
- palette: two entries, slate first, each with a toggle. Material takes the
first entry as the default, so the site opens dark and a sun icon in the
header switches to light (and a moon switches back).
- dc-brand.css: the dark scheme's greys are derived from a hue variable, so
it is pointed at the brand slate (206deg) rather than Material's neutral
225deg, which gives the whole scheme the same cast as the header.
- dc-brand.css: the brand slate is unreadable on a dark background, so the
dark scheme uses a light tint of it (#9fb5c6) for body links. Green stays
the accent, which keeps link hover visible in both schemes.
Contrast, over the dark background rgb(30, 36, 41): body text 8.8:1, body
links 7.4:1, inline code 7.2:1, accent green 5.6:1, header text 10.2:1. The
light scheme is unchanged.
The docs site rendered as stock Material: indigo header, indigo links and
Open Sans. Neither colour nor type matched the marketing site
(datacontroller.io) or the application itself.
- Palette: the slate (#314351) and green (#79a843) already used by the
marketing navbar, hero and logo accent, wired up via a `custom` palette
and defined in the new docs/dc-brand.css.
- Type: Montserrat for text, matching the marketing site's headings.
- Logo: the application's own dc-logo.svg mark in the header.
The previous `palette:` mapping form (a dict of primary/accent) is no
longer honoured by the Material version this repo builds against, so the
theme fell through to Material's default indigo - the site was not in fact
rendering the White/Amber it asked for. The list form used here is the one
Material currently reads.
On SAS 9 and Viya the app is reached through the platform's job URL, so
work.browser_url_vars carries the platform's parameters (_FILE, _program,
_debug, _webout) alongside DC's. Say so, so a hook author does not read
_FILE as a Data Controller parameter.
The integrity section claimed SAS writes nothing to disk outside the
appLoc. The appLoc is the SAS Folder the stored processes or jobs are
deployed into (metadata), so it is not a disk location at all - the only
physical location the deployment programs write to is the Data Controller
location (dcLoc) configured at first launch.
The adapter sends _debug=128 (not 131) on the Viya web JES path when
runAsTask is enabled, which is what the frontend uses there. Document
that value alongside 131, and note that editors/loadfile is reached
through the multipart file upload, which carries no input tables.
The support diagnostics are now sent only where they can be used: the
startup service and the services that execute customer-provided code
(hook scripts, dynamic cell dropdown programs) - not with every service
call. The page's URL parameters arrive as a new browser_url_vars table,
one row per parameter, so a SAS developer reads a parameter by name
rather than parsing the url string.
Adds /downloads to the Installation section, listing what each release
asset is for and how to verify a download against the SHA256SUMS file
the release pipeline publishes (sha256sum -c, plus certutil for
Windows). The SAS 9 deployment page now links to it wherever it sends
the reader to the releases page.
mkdocs build passes; nav, sitemap and cross-links verified in the
rendered output.
Every service call carries a single-row browser_info input table, readable
by any service or hook script as work.browser_info and dumped to the job log
when debug is on. Document its columns and how to see it, so a support ticket
can be answered from the log without follow-up questions.
MINVAL and MAXVAL now say what a special missing does to them and that the bound
itself may be a special missing, pointing at the Special Missing Values section,
since that is the least obvious thing about them.
Also in that section: the framing sentence covers the range rules' ordering, the
summary sentence is rewritten (it read badly), and the arrow in the ROUND row is
plain ASCII.
A range rule keys both the cell value and its own value into the order SAS uses
for a numeric variable: every missing below every non-missing value, and the
missing values ordered ._ then the regular missing then .A through .Z. So a range
can be written in special missings and mean what SAS means by it - MINVAL .A with
MAXVAL .C accepts .B and rejects .D - and the numeric cases follow from the same
order: a missing fails a numeric MINVAL and passes a numeric MAXVAL, and a number
sits above every missing.
Replaces the "a range rule ignores a missing value" wording, which described the
previous take's behaviour rather than SAS's.
Three corrections to the special missing section:
- the period is optional: a special missing is typed as `a` or `.a`, `_` or `._`
- MINVAL and MAXVAL both accept a missing value, and both still reject a real
number that is out of range. A minimum constrains a number, and a missing is
not a number - NOTNULL is the rule for a column that must be populated
- only NOTNULL and the formulas actually reject a special missing, so the
"not supported" summary is replaced with what does reject it and what does not
dc
allan