sas/.sasjslint takes maxLineLength from 180 to 80 - the linter's own default -
and promotes it to "error".
53 of the 258 .sas files already hold a line longer than 80. Rewrapping them
would be a large, reviewable-only-in-theory diff, so each keeps its current
longest line as an exact ceiling via a per-file override, e.g.
@sasjslint {"maxLineLength": 95} in services/validations/tables_all.sas. A
file with no override is held to 80, and an overridden file cannot grow past
the length it has today: the ceiling is the file's longest line, not a round
number.
Both directions were checked: a 101 character line added to an
un-overridden file fails with "Line exceeds maximum length by 21 characters",
and a 114 character line added to macros/mpe_execute.sas (ceiling 101) fails
by 13. `npm run lint:sas` is clean with the setting in force.
refreshcatalog.test and viewtables.test failed to complete with
ERROR: could not create /tmp/DataController/<libref>/dc_staging/refreshcatalog:mpe_refreshcatalogs: + <stamp>
The staging folder a closeout writes its statements to is
&dc_staging_area/&etlsource, and etlsource defaulted to
'refreshcatalog:<step>:%mf_uid()'. A default is resolved when the macro is
compiled, not when it is called, so every refresh in a session reused the same
folder - the first step created it, the next one could not, and the abort took
the whole suite down. Whichever suite reached the staging area first decided
which others failed, which is why the count moved between runs.
etlsource now defaults to 0 and is set per call from
%mf_getuniquename(prefix=<step>): unique, a plain name rather than a path, and
resolved at the point of use.
Verified on the estate: refreshcatalog 4/4 (100%), viewtables 2/2 (100%).
getcolvals - mf_nobs returns -1 when the dataset does not exist, so the guard
tested ==0 and a request that supplied no iwant at all fell through to the SQL.
It now tests <=0, which covers both an empty and an absent input.
mpe_refreshcatalogs - etlsource defaulted to a bare uid while its two siblings
use refreshcatalog:<step>:<uid>, so a load through this path could not be traced
to the step that wrote it. The default now has the same shape, and the explicit
etlsource=0 path follows it too.
mpe_execute - the status table was read without being cleared, so a child that
died before core wrote a row left the previous call's row in place and failfast
read it as completed. The table is deleted before the call.
makedata - KEEPME is compared through cats(), so a numeric column no longer
raises in upcase().
MPE_DATALOADS is keyed on (processed_dttm, libref, dsn, etlsource), and etlsource
was the calling macro name, so a second catalogue load of the same table in the
same second was refused as a duplicate. The three refresh macros now take an
etlsource parameter, defaulting to refreshcatalog:<step>:%mf_uid(), which records
which step loaded what and keeps the audit row unique.
dc_refreshcatalog also echoes the generated statements when debug is on, since a
failure inside one of them otherwise names no libref.
BREAKING CHANGE: Introduction of 4 new tables for capturing information related to catalogs and their objects. Migration script prepared and available in the DB folder (usual place)
dc
allan