Author SHA1 Message Date
allan ca62bb391c Merge pull request 'docs(options): the EDIT limits truncate the grid and warn' (#22) from docs/editor-limit-warning into main
Publish to docs.datacontroller.io / Deploy docs (push) Successful in 1m21s
Reviewed-on: #22
2026-10-26 11:59:54 +00:00
dc 856e57173f docs(overview): the EDIT limits truncate, and the viewer default is 2000 not 500
Review 431: dc-overview.md still said "Up to 500 rows can be edited (in the web
editor) at one time" - the wrong number (DC_MAXOBS_WEBEDIT ships at 250) and, once
dc/dc#352 lands, the wrong behaviour (an over-limit selection is served truncated
with a warning, not refused or capped at a count). It now names both limits,
whichever binds first, and links to the options that set them.

Sweeping for the same stale figure turned up dcu-tableviewer.md, which says 500
twice and uses 500 in its worked example of the row count above the grid.
DC_MAXOBS_WEBVIEW has been 2000 since 20261003_row_and_cell_limits.sas (the
migration records 500 -> 2000), and mpe_makedata seeds 2000, so those three
places were two releases out of date. Fixed here, and flagged in the PR reply as
VIEW rather than EDIT in case it belongs in its own change.

Checked: no "500 rows" or stale abort wording left in docs/, and the two anchors
the overview now links to (dc_maxobs_webedit, dc_maxcells_webedit) exist on the
published options page.
2026-10-26 11:57:21 +00:00
allan c29c955e1a Merge pull request 'docs(7.19): the RLS restriction on derived variables, and the startup Show log button' (#23) from docs/7.19-readiness into main
Publish to docs.datacontroller.io / Deploy docs (push) Successful in 1m24s
Reviewed-on: #23
2026-10-26 11:53:53 +00:00
dc c7626b2797 docs(developer): a page for the browser variables, and links from where they are used
browser_info and browser_url_vars were only described inside the troubleshooting
page, under Support Diagnostics. That is the right place to find them if you
already know they exist and are looking for a log line; it is not where anyone
looks when writing a hook, which is what they are mostly for.

The new Developer Guide page carries the whole subject: which services receive
the two tables, the column lists, the debug values per platform and the MPEINIT
log lines, reading them from a hook with the exist() guard, and the reminder that
they are client-supplied and so diagnostics rather than a security boundary.

Troubleshooting keeps three stub headings - browser_info, browser_url_vars,
reading-them-from-a-hook - pointing at the page, so the anchors that are already
published keep resolving, and the detail is not maintained twice.

Linked from the three places a hook author actually starts: the HOOK Scripts
section of the MPE_TABLES page, the pre-edit hook guide (where the editor's
inputs are listed), the two-rule-sets guide (post-edit hook), and the VA embed
page, where embed=va is exactly the kind of thing a hook wants to branch on.
2026-10-26 11:42:35 +00:00
dc f08c2dff0e docs(derived): the label rule - any text, one line, 256 characters
VAR_LABEL is free text in any script and the quotes are escaped when the
statement is built, but it is a column label: control characters, line breaks and
over 256 characters are refused, the last of those because the engine would cut
the value down silently.
2026-10-26 09:53:55 +00:00
dc de47b25e11 docs(7.19): the RLS restriction on derived variables, and the startup Show log button
Two gaps found reviewing the pages against the code that is shipping:

- A derived variable cannot be named by an active row level security rule of
  any scope - mpe_derived_postedit.sas rejects it, because rules are validated
  against the physical table where a derived column does not exist. Neither the
  derived variables page nor row-level-security.md said so.

- The startup failure modal's Show log button (#356, in this release) was not
  documented anywhere. Added to troubleshooting, with why the browser request
  is what gets the log on Viya.
2026-10-26 08:44:27 +00:00
dc 561e92e090 docs(developer-guide): name both EDIT caps in the hook page
The truncation is whichever of DC_MAXOBS_WEBEDIT and DC_MAXCELLS_WEBEDIT
binds first, and for a wide table it is the cell cap - floor(cells/columns)
rows, potentially far below the row limit.  The hook page named only the row
cap and quoted its 250 default, so a hook author would check the row count
alone.  Same correction in the two-rule-sets page.
2026-10-22 19:27:51 +00:00
dc 971b007606 docs(options): the EDIT limits truncate the grid and warn
getdata no longer refuses a selection over DC_MAXOBS_WEBEDIT or
DC_MAXCELLS_WEBEDIT - it serves the rows within the limit and flags the
truncation against the table name.  Update the option reference and the two
developer-guide pages that claimed the edit screen stops with "Table is too
big".  The hook page also carried the old 500 row default.
2026-10-22 16:40:48 +00:00
14 changed files with 160 additions and 97 deletions

No files matched your search

+3 -1
View File
@@ -35,7 +35,9 @@ When importing an excel file, all rows are loaded into the web page. You have a
A CSV upload bypasses the part where the records are loaded into the web page, and ALL rows are sent to the staging area directly. This makes it suitable for larger uploads. A CSV upload bypasses the part where the records are loaded into the web page, and ALL rows are sent to the staging area directly. This makes it suitable for larger uploads.
## Edit Stage Approve Workflow ## Edit Stage Approve Workflow
Up to 500 rows can be edited (in the web editor) at one time. These edits are submitted to a staging area. After one or more approvals (acceptances) the changes are applied to the source table. The EDIT screen loads up to [DC_MAXOBS_WEBEDIT](/dcc-options/#dc_maxobs_webedit) rows (250 by default), or as many rows as fit within [DC_MAXCELLS_WEBEDIT](/dcc-options/#dc_maxcells_webedit) (200,000 cells) - whichever is reached first, so a wide table loads fewer rows than a narrow one. A larger table is not refused: it loads truncated, with a warning against the table name, and the filter is the way to reach the rest.
These edits are submitted to a staging area. After one or more approvals (acceptances) the changes are applied to the source table.
![screenshot](img/dcu_flow.png) ![screenshot](img/dcu_flow.png)
+2
View File
@@ -23,6 +23,8 @@ Set to YES or NO to enable email alerts. This requires email options to be preco
### DC_MAXCELLS_WEBEDIT ### DC_MAXCELLS_WEBEDIT
By default, a maximum of 200,000 cells (observations multiplied by variables) can be edited in the browser at one time. This limit is applied together with [DC_MAXOBS_WEBEDIT](#dc_maxobs_webedit) - whichever is reached first. By default, a maximum of 200,000 cells (observations multiplied by variables) can be edited in the browser at one time. This limit is applied together with [DC_MAXOBS_WEBEDIT](#dc_maxobs_webedit) - whichever is reached first.
A selection over either limit is served truncated: the grid carries the rows that fit, and a warning triangle next to the table name names the limit that was reached. Filter the table to see the rest.
The cell count matters because the cost of an EDIT screen load tracks the number of cells, not the number of rows. A 10 column table is comfortable at 250 rows, but 250 rows of a 1000 column table is 250,000 cells. The cell count matters because the cost of an EDIT screen load tracks the number of cells, not the number of rows. A 10 column table is comfortable at 250 rows, but 250 rows of a 1000 column table is 250,000 cells.
### DC_MAXOBS_WEBEDIT ### DC_MAXOBS_WEBEDIT
+2
View File
@@ -217,6 +217,8 @@ The code is simply `%include`'d at the relevant point during backend execution.
If the entry ends in `".sas"` it is assumed to be a physical, filesystem file. Otherwise, the source code is extracted from SAS Drive or Metadata. If the entry ends in `".sas"` it is assumed to be a physical, filesystem file. Otherwise, the source code is extracted from SAS Drive or Metadata.
The hook shares the WORK library of the service that runs it, so it can read whatever that service received. In particular the two browser tables are there whenever the app made the request - `work.browser_info` (the page URL, the embedding document, timezone, locale, versions) and `work.browser_url_vars` (the page's URL parameters, one row each). See [Browser Variables](/developer-guide/browser-variables/). A `POST_EDIT_HOOK` that runs on an uploaded file is the exception: the upload path is multipart and carries no input tables.
To illustrate: To illustrate:
* Physical filesystem (ends in .sas): `/opt/sas/code/myprogram.sas` * Physical filesystem (ends in .sas): `/opt/sas/code/myprogram.sas`
+10 -83
View File
@@ -142,97 +142,24 @@ The app version is bundled into the frontend during the release, and is visible
You can also determine the app version (and SASjs Version, and build time) by opening browser Development Tools and running `appinfo()` in the console. You can also determine the app version (and SASjs Version, and build time) by opening browser Development Tools and running `appinfo()` in the console.
## The app does not start - Show log
A startup failure shows a modal with the error and the service that failed. When that service is `public/startupservice`, the modal offers **Show log**, which opens the service in a new browser window with `_debug=131`, so the SAS log is the page body.
The button exists because of how Viya runs a request from JavaScript: it can go through a Compute execution task (`_executionTasks=true`), where the log is not returned and the request can time out before the service finishes. The browser request runs the same stored program without a task, so the log is available even when the adapter got nothing back. On SAS 9 it opens the stored process, and on SASjs Server the route returns the service output rather than a log.
## Support Diagnostics ## Support Diagnostics
The frontend sends two input tables with the startup service and with the services that execute customer-provided code - the [hook scripts](macros.md) (pre/post edit and approve hooks) and the dynamic cell dropdown programs. They are available in those services as `work.browser_info` and `work.browser_url_vars`, so a support ticket can be diagnosed from the job log without asking the user follow-up questions, and a hook can adapt to its context. Other services do not receive them. The frontend sends two input tables with the startup service and with the services that execute customer-provided code - the [hook scripts](dcc-tables.md#hook-scripts) (pre/post edit and approve hooks) and the dynamic cell dropdown programs. See [Browser Variables](/developer-guide/browser-variables/) for the column lists, the log lines, how to read them from a hook, and which services receive them.
The services that receive the tables are: `public/startupservice`, `editors/getdata`, `editors/getdynamiccolvals`, `editors/stagedata`, `editors/loadfile`, `editors/restore` and `auditors/postdata`.
`editors/loadfile` is the exception in practice: the app reaches it through the adapter's file-upload call (multipart), which carries no input tables, so the service - and the post edit hook it runs - does not receive them in normal use. It is on the list for completeness, and would receive them if the service were ever invoked through the ordinary request path.
### browser_info ### browser_info
A single-row table with these columns: The browser context: page URL, embedding document, timezone, locale, versions, and the user agent. Columns and examples: [Browser Variables](/developer-guide/browser-variables/#browser_info).
| Column | Description |
|---|---|
| `url` | The URL of the Data Controller page itself (the iframe), not the document embedding it. When the editor is embedded in a report, the embedding URL is visible here. |
| `referrer` | The embedding document, from `document.referrer`. For an embedded report this is the report URL, so the embedding report can be told apart from the editor URL. |
| `timezone` | The browser's IANA timezone name, e.g. `Europe/Berlin`. |
| `tz_offset` | The browser's UTC offset in minutes, as `Date.getTimezoneOffset()` returns it (positive west of UTC, so `Europe/Berlin` in summer is -120). |
| `locale` | The browser locale, e.g. `en-GB`. |
| `dc_version` | The Data Controller build, as shown by `appinfo()`. |
| `adapter_version` | The `@sasjs/adapter` version the client was built with. |
| `browser` | The browser family, parsed from the user agent. |
| `browser_version` | The browser version. |
| `platform` | The operating system, parsed from the user agent. |
| `user_agent` | The raw `navigator.userAgent` string. |
### browser_url_vars ### browser_url_vars
The page URL parameters as one row per parameter, so a SAS program can read a parameter by name rather than parse the `url` string: The page URL parameters, one row per parameter. Read a parameter by name rather than parsing the page URL - the platform's own parameters are in there too. Columns and examples: [Browser Variables](/developer-guide/browser-variables/#browser_url_vars).
| Column | Description |
|---|---|
| `name` | The URL parameter name. |
| `value` | The URL parameter value. |
Parameters from both the search string and the hash query string (Angular routes carry them after the `#`) are included; on a name collision the search string value wins. The table is sent only when the page URL has at least one parameter.
On SAS 9 and Viya the app is reached through the platform's own job URL, so the platform's parameters (`_FILE`, `_program`, `_debug`, `_webout` and friends) are in the table alongside DC's. Read a parameter by name rather than assuming every row belongs to Data Controller - `embed` and `labels` are DC's, `_FILE` and `_program` are the platform's.
The values in both tables are supplied by the client, so treat them as diagnostics hints rather than a security boundary.
To see the tables, run any of the services above with debug on. The debug value depends on the path: `&_debug=131` on the Compute API and SAS 9, or `&_debug=128` on the Viya web (JES) path with `runAsTask` enabled - which is what the frontend sends there, so turning debug on in the app is enough. Either value makes the session initialisation write the tables to the job log:
```
NOTE: MPEINIT: work.browser_url_vars:
name=embed value=va
NOTE: MPEINIT: work.browser_info:
url=... referrer=... timezone=Europe/Berlin tz_offset=-120 locale=en-GB
dc_version=7.15.0 adapter_version=4.19.0 browser=Chrome browser_version=120.0
platform=Linux user_agent=...
```
A service called directly (for instance from a URL, or by a script), or by an older frontend, has no `work.browser_info` - the log then records that fact instead.
### Reading the tables from a hook ### Reading the tables from a hook
A [hook script](macros.md) is `%include`d into the running service, so it shares the service's WORK library and can read the tables directly. Take an embedded editor opened at: Guard the read with `%sysfunc(exist())` - a service called directly has no `work.browser_info`. Worked examples: [Browser Variables](/developer-guide/browser-variables/#reading-them-from-a-hook).
```
https://datacontroller.io/#/editor/SASHELP.CLASS?labels=true&embed=va
```
`browser_url_vars` then holds a row per parameter - `labels=true` and `embed=va` - so a hook can pick up either by name:
```sas
%let labels=N;
%let embed=N;
%if %sysfunc(exist(work.browser_url_vars)) %then %do;
proc sql noprint;
select value into :labels
from work.browser_url_vars
where name = 'labels';
select value into :embed
from work.browser_url_vars
where name = 'embed';
quit;
%end;
```
To branch on what the client reports:
```sas
%if %sysfunc(exist(work.browser_info)) %then %do;
data _null_;
set work.browser_info;
call symputx('dc_timezone', timezone);
call symputx('dc_locale', locale);
run;
%end;
```
!!! warning
Guard with `%sysfunc(exist())` before reading. A service called directly, or by a frontend older than this feature, has no `work.browser_info` at all, so an unguarded `set` or `select` fails the job.
+4 -4
View File
@@ -1,7 +1,7 @@
# Data Controller for SAS: Viewer # Data Controller for SAS: Viewer
The viewer screen provides a raw view of the underlying table. The viewer screen provides a raw view of the underlying table.
Choose a library, then a table, and click view to see the first 500 rows. Choose a library, then a table, and click view to see the first rows - up to the `DC_MAXOBS_WEBVIEW` limit, 2,000 rows by default.
A filter option is provided should you wish to view a different section of rows. A filter option is provided should you wish to view a different section of rows.
The following libraries will be visible: The following libraries will be visible:
@@ -21,9 +21,9 @@ The search covers **every** column of the table at once, so you do not need to k
- **Character search** (the default) is a case sensitive CONTAINS match against every character column, so `Smith` finds `Smithson` and `smith` finds `Goldsmith`, but `smith` will not find `Smithson`. - **Character search** (the default) is a case sensitive CONTAINS match against every character column, so `Smith` finds `Smithson` and `smith` finds `Goldsmith`, but `smith` will not find `Smithson`.
- **Numeric search** (tick the Numeric box) is an exact match against every numeric column, so `42` will not find `420` or `4210`. - **Numeric search** (tick the Numeric box) is an exact match against every numeric column, so `42` will not find `420` or `4210`.
- If a filter is applied, the search runs within the filtered rows rather than the whole table. - If a filter is applied, the search runs within the filtered rows rather than the whole table.
- The grid is capped by the [`DC_MAXOBS_WEBVIEW`](/dcc-options/#dc_maxobs_webview) option (500 rows by default). - The grid is capped by the [`DC_MAXOBS_WEBVIEW`](/dcc-options/#dc_maxobs_webview) option (2,000 rows by default).
- For a search, the cap applies to the row count as well: a search that matches more rows than the cap returns 500 of them, and the count shown next to the table name is the number of rows returned, not the total number of matches. - For a search, the cap applies to the row count as well: a search that matches more rows than the cap returns 2,000 of them, and the count shown next to the table name is the number of rows returned, not the total number of matches.
- Without a search, the count shown next to the table name is the full number of rows in the filtered view, so it can be higher than the number of rows displayed - a table of 1,200 rows reports `1,200 rows` above a grid holding the first 500. - Without a search, the count shown next to the table name is the full number of rows in the filtered view, so it can be higher than the number of rows displayed - a table of 5,000 rows reports `5,000 rows` above a grid holding the first 2,000.
- A search that matches nothing shows a "No data found with given conditions" panel rather than an empty grid. - A search that matches nothing shows a "No data found with given conditions" panel rather than an empty grid.
![Full table search](img/full-table-search.png) ![Full table search](img/full-table-search.png)
+7
View File
@@ -6,6 +6,11 @@ description: Derived variables are computed columns that appear in the EDIT scre
# Derived Variables # Derived Variables
!!! warning "Draft - not in a release yet"
This page describes a feature that is in development. It is published early so
the wording and the screenshots can be reviewed; the behaviour described here
ships in the next release.
A derived variable is a column that does not exist in the target table, but is computed when the table is loaded into the [EDIT screen](/dc-userguide/#editor). It is read-only, it appears after the physical columns, and it can be filtered on like any other column. A derived variable is a column that does not exist in the target table, but is computed when the table is loaded into the [EDIT screen](/dc-userguide/#editor). It is read-only, it appears after the physical columns, and it can be filtered on like any other column.
Derived variables are useful when users need to work with a value that is a function of the stored data - a banding, a concatenation, a calculation - without changing the table, the load process, or the ETL. Derived variables are useful when users need to work with a value that is a function of the stored data - a banding, a concatenation, a calculation - without changing the table, the load process, or the ETL.
@@ -115,6 +120,8 @@ Derived columns themselves are an EDIT screen feature: the VIEW screen and the d
- **Dependencies declared.** `VAR_DEPS` lists every input the expression uses - physical columns and derived variables alike. Statements are generated in dependency order, so no sequence number is maintained, and the declaration is checked in both directions: every reference must be declared, and every declared name must be referenced and must exist. Circular references are rejected. - **Dependencies declared.** `VAR_DEPS` lists every input the expression uses - physical columns and derived variables alike. Statements are generated in dependency order, so no sequence number is maintained, and the declaration is checked in both directions: every reference must be declared, and every declared name must be referenced and must exist. Circular references are rejected.
- **No collisions.** `VAR_NM` must not match an existing column of the target table, or `_____DELETE__THIS__RECORD_____`. - **No collisions.** `VAR_NM` must not match an existing column of the target table, or `_____DELETE__THIS__RECORD_____`.
- **Read-only.** Derived columns cannot be edited in the grid, and any value submitted for them is discarded before the approval package is created. - **Read-only.** Derived columns cannot be edited in the grid, and any value submitted for them is discarded before the approval package is created.
- **Label is free text.** `VAR_LABEL` takes any characters, in any script - quotes are escaped when the statement is generated - but it is a column label, so it is one line: control characters and line breaks are refused, and so is a value over 256 characters, which the engine would otherwise cut down silently.
- **Not for row level security.** A derived variable cannot be named by an active [row level security](/row-level-security/) rule, of any scope. RLS rules are validated against the physical table, where a derived column does not exist, so a rule naming one is rejected when it is submitted.
- **Recomputed on every read.** The value always reflects the current data and the current definition, and there is no history of previous derived values. - **Recomputed on every read.** The value always reflects the current data and the current definition, and there is no history of previous derived values.
A configuration error - a bad name, an invalid type or length, a name collision, or a reference to a derived variable that is not declared in `VAR_DEPS` - stops the EDIT screen with a message naming the variable. A syntax error or an unknown column in the expression itself is caught when the view is compiled, and stops the screen with a message naming the table. A configuration error - a bad name, an invalid type or length, a name collision, or a reference to a derived variable that is not declared in `VAR_DEPS` - stops the EDIT screen with a message naming the variable. A syntax error or an unknown column in the expression itself is caught when the view is compiled, and stops the screen with a message naming the table.
+113
View File
@@ -0,0 +1,113 @@
---
layout: article
title: Browser Variables
description: The frontend sends the browser context to the backend as two input tables - browser_info and browser_url_vars - available to the startup service, the edit services and hook scripts.
---
# Browser Variables
Two input tables travel with most requests, so a SAS program can see the browser it is serving rather than asking the user:
| Table | Holds |
|---|---|
| `work.browser_info` | one row - the page URL, the document embedding it, the browser timezone and locale, and the versions in play |
| `work.browser_url_vars` | one row per parameter on the page URL |
They exist for two jobs: diagnosing a support ticket from the job log, and letting a [hook script](/dcc-tables/#hook-scripts) adapt to how the app was opened - embedded in a VA report, opened with `?labels=true`, or used by someone in another timezone.
## Which services receive them
`public/startupservice`, `editors/getdata`, `editors/getdynamiccolvals`, `editors/stagedata`, `editors/loadfile`, `editors/restore` and `auditors/postdata`. Other services do not receive them.
`editors/loadfile` is the exception in practice: the app reaches it through the adapter's multipart upload, which carries no input tables, so the service - and the post edit hook it runs - does not receive them in normal use. It is on the list for completeness, and would receive them if it were ever invoked through the ordinary request path.
## browser_info
A single row with these columns:
| Column | Description |
|---|---|
| `url` | The URL of the Data Controller page itself (the iframe), not the document embedding it. When the editor is embedded in a report, the embedding URL is visible here. |
| `referrer` | The embedding document, from `document.referrer`. For an embedded report this is the report URL, so the embedding report can be told apart from the editor URL. |
| `timezone` | The browser's IANA timezone name, e.g. `Europe/Berlin`. |
| `tz_offset` | The browser's UTC offset in minutes, as `Date.getTimezoneOffset()` returns it (positive west of UTC, so `Europe/Berlin` in summer is -120). |
| `locale` | The browser locale, e.g. `en-GB`. |
| `dc_version` | The Data Controller build, as shown by `appinfo()`. |
| `adapter_version` | The `@sasjs/adapter` version the client was built with. |
| `browser` | The browser family, parsed from the user agent. |
| `browser_version` | The browser version. |
| `platform` | The operating system, parsed from the user agent. |
| `user_agent` | The raw `navigator.userAgent` string. |
## browser_url_vars
The page URL parameters, one row per parameter, so a SAS program can read a parameter by name rather than parse the `url` string:
| Column | Description |
|---|---|
| `name` | The URL parameter name. |
| `value` | The URL parameter value. |
Parameters from both the search string and the hash query string (Angular routes carry them after the `#`) are included; on a name collision the search string value wins. The table is sent only when the page URL has at least one parameter.
On SAS 9 and Viya the app is reached through the platform's own job URL, so the platform's parameters (`_FILE`, `_program`, `_debug`, `_webout` and friends) are in the table alongside Data Controller's. Read a parameter by name rather than assuming every row belongs to the app - `embed` and `labels` are Data Controller's, `_FILE` and `_program` are the platform's.
## Seeing them in the job log
Run any of the services above with debug on. The value depends on the path: `&_debug=131` on the Compute API and SAS 9, or `&_debug=128` on the Viya web (JES) path with `runAsTask` enabled - which is what the frontend sends there, so turning debug on in the app is enough. Either value makes the session initialisation write both tables to the job log:
```
NOTE: MPEINIT: work.browser_url_vars:
name=embed value=va
NOTE: MPEINIT: work.browser_info:
url=... referrer=... timezone=Europe/Berlin tz_offset=-120 locale=en-GB
dc_version=7.15.0 adapter_version=4.19.0 browser=Chrome browser_version=120.0
platform=Linux user_agent=...
```
A service called directly (for instance from a URL, or by a script), or by a frontend older than this feature, has no `work.browser_info` - the log then records that fact instead.
## Reading them from a hook
A [hook script](/dcc-tables/#hook-scripts) is `%include`d into the running service, so it shares the service's WORK library and can read the tables directly. Take an embedded editor opened at:
```
https://datacontroller.io/#/editor/SASHELP.CLASS?labels=true&embed=va
```
`browser_url_vars` then holds a row per parameter - `labels=true` and `embed=va` - so a hook can pick up either by name:
```sas
%let labels=N;
%let embed=N;
%if %sysfunc(exist(work.browser_url_vars)) %then %do;
proc sql noprint;
select value into :labels
from work.browser_url_vars
where name = 'labels';
select value into :embed
from work.browser_url_vars
where name = 'embed';
quit;
%end;
```
To branch on what the client reports:
```sas
%if %sysfunc(exist(work.browser_info)) %then %do;
data _null_;
set work.browser_info;
call symputx('dc_timezone', timezone);
call symputx('dc_locale', locale);
run;
%end;
```
!!! warning
Guard with `%sysfunc(exist())` before reading. A service called directly, or a frontend older than this feature, has no `work.browser_info` at all, so an unguarded `set` or `select` fails the job.
## What they are not
Both tables are supplied by the client, so treat them as diagnostic hints rather than a security boundary. Never authorise on anything in them - a caller can send whatever it likes.
+4 -2
View File
@@ -19,6 +19,8 @@ When a filtered editor loads, `editors/getdata` returns the filter to the client
The response does not carry the filter key (`FILTER_RK`). That is an *input* to `getdata`, not an output - the editor already holds it in the URL (`/editor/<table>/<FILTER_RK>`), so there is nothing to return. The response does not carry the filter key (`FILTER_RK`). That is an *input* to `getdata`, not an output - the editor already holds it in the URL (`/editor/<table>/<FILTER_RK>`), so there is nothing to return.
The hook also receives the browser tables that came in with the request - `work.browser_info` and `work.browser_url_vars` - so it can tell an embedded editor from a standalone one, or take a value from the page URL. See [Browser Variables](/developer-guide/browser-variables/).
## When the hook runs ## When the hook runs
`getdata` builds the filter, applies it, and sorts the result, all before the pre-edit hook runs. In order: `getdata` builds the filter, applies it, and sorts the result, all before the pre-edit hook runs. In order:
@@ -144,9 +146,9 @@ The captures below are the editor for a table with an `ORDER_DATE` column, opene
The user's clause and Data Controller's clause have to be kept apart, because Row Level Security lives in the second one. Combining them as `( user OR previous-quarter ) AND ( validity + RLS )` keeps RLS applied to both branches. Wrapping the whole existing filter instead - `( existing OR previous-quarter )` - would let the previous-quarter rows through without their RLS condition, so do not take that shortcut. The user's clause and Data Controller's clause have to be kept apart, because Row Level Security lives in the second one. Combining them as `( user OR previous-quarter ) AND ( validity + RLS )` keeps RLS applied to both branches. Wrapping the whole existing filter instead - `( existing OR previous-quarter )` - would let the previous-quarter rows through without their RLS condition, so do not take that shortcut.
## The row cap ## The row and cell caps
`getdata` refuses a result larger than `DC_MAXOBS_WEBEDIT` (500 by default) with *Table is too big*. Widening a filter widens the result, so a hook that adds a quarter can push a table over the cap. Check the count before handing the data back, or raise the option for that table in `MPE_CONFIG`. `getdata` truncates a result over either `DC_MAXOBS_WEBEDIT` or `DC_MAXCELLS_WEBEDIT`, whichever binds first - a wide table runs out of cells before rows, at floor(cells / columns) rows - and warns against the table name. Widening a filter widens the result, so a hook that adds a quarter can push a table over the cap: the rows past the limit never reach the browser. Check both counts before handing the data back, or raise the options for that table in `MPE_CONFIG`.
## Matching the filter chips ## Matching the filter chips
@@ -100,6 +100,8 @@ run;
From that moment the changeset is an approval against `ORDERS.ORDERS`. The approver sees a diff against the real table, the load writes to the real table, and the mirror is never touched. From that moment the changeset is an approval against `ORDERS.ORDERS`. The approver sees a diff against the real table, the load writes to the real table, and the mirror is never touched.
While it runs, the hook can also read `work.browser_info` and `work.browser_url_vars` - the browser context that came with the submit - to tell an embedded editor from a standalone one, or to pick up a parameter from the page URL. See [Browser Variables](/developer-guide/browser-variables/).
![A change submitted against the mirror, raised against the real table](/img/approval-routed.png) ![A change submitted against the mirror, raised against the real table](/img/approval-routed.png)
### Reassigning LIBREF and DS ### Reassigning LIBREF and DS
@@ -109,7 +111,7 @@ From that moment the changeset is an approval against `ORDERS.ORDERS`. The appro
### Other things to watch ### Other things to watch
- Concurrency behaves correctly here. Both mirrors route their submit to the same base table, so every approval locks, loads and logs against one identity - unlike the two-libref route above. - Concurrency behaves correctly here. Both mirrors route their submit to the same base table, so every approval locks, loads and logs against one identity - unlike the two-libref route above.
- The filter has already been applied to the empty mirror by the time the pre-edit hook runs, so a hook that reads the real table ignores the user's filter unless it re-applies it (`where %inc filtref`). On a small table you will not notice; on a large one the `DC_MAXOBS_WEBEDIT` cap will stop the edit screen with "Table is too big". [Updating the Filter in a Pre-Edit Hook](/developer-guide/filter-pre-edit-hook/) works through extending that filter. - The filter has already been applied to the empty mirror by the time the pre-edit hook runs, so a hook that reads the real table ignores the user's filter unless it re-applies it (`where %inc filtref`). On a small table you will not notice; on a large one the `DC_MAXOBS_WEBEDIT` or `DC_MAXCELLS_WEBEDIT` cap truncates the edit screen, with a warning against the table name. [Updating the Filter in a Pre-Edit Hook](/developer-guide/filter-pre-edit-hook/) works through extending that filter.
- The hook's output must have the same columns the editor expects - the real table, minus any transaction or processing columns that Data Controller drops on load. - The hook's output must have the same columns the editor expects - the real table, minus any transaction or processing columns that Data Controller drops on load.
- The real table must itself be registered in `MPE_TABLES`. The approval screen resolves the table's audit settings from that row, so a changeset routed to a table with no registration cannot be reviewed - the submit is refused up front, naming the table. - The real table must itself be registered in `MPE_TABLES`. The approval screen resolves the table's audit settings from that row, so a changeset routed to a table with no registration cannot be reviewed - the submit is refused up front, naming the table.
- The mirror's `MPE_TABLES` row is read for the edit screen and the real table's for the load, so keep their `buskey`, `loadtype` and temporal column settings identical. - The mirror's `MPE_TABLES` row is read for the edit screen and the real table's for the load, so keep their `buskey`, `loadtype` and temporal column settings identical.
+2
View File
@@ -53,6 +53,8 @@ In the VA Report Designer, add a **Data-Driven Content** object and set the URL
## Debugging ## Debugging
A hook script can tell an embedded editor from a standalone one without any client-side inspection: `work.browser_info` carries the page URL and `referrer`, and `work.browser_url_vars` carries `embed=va`. See [Browser Variables](/developer-guide/browser-variables/).
The following snippet can be used in console to dump the values being provided to DC from VA: The following snippet can be used in console to dump the values being provided to DC from VA:
``` ```
-4
View File
@@ -45,10 +45,6 @@ Described [here](https://docs.datacontroller.io/api/).
Below are some examples of Features that have been requested (and delivered) into Data Controller. Below are some examples of Features that have been requested (and delivered) into Data Controller.
### Derived Variables
Columns that do not exist in the table - a banding, a concatenation, a calculation - can be configured per table and appear in the EDIT screen, read-only and filterable, without changing the table or the load process. Nothing derived is stored. See [docs](/derived-variables)
### Column Level Security ### Column Level Security
The columns that can be VIEWed or EDITed are now configurable. See [docs](/column-level-security) The columns that can be VIEWed or EDITed are now configurable. See [docs](/column-level-security)
+2
View File
@@ -41,6 +41,8 @@ This variable is an integer, and used to divide a complex filter into numerous s
### RLS_VARIABLE_NM ### RLS_VARIABLE_NM
This is the name of the variable against which a filter value will be applied This is the name of the variable against which a filter value will be applied
It must be a physical column of the table. A [derived variable](/derived-variables/) cannot be named here: rules are validated against the table itself, where a derived column does not exist, so such a rule is rejected when it is submitted.
### RLS_OPERATOR ### RLS_OPERATOR
The available operator will depend on whether the column is character or numeric. Example values: The available operator will depend on whether the column is character or numeric. Example values:
+5
View File
@@ -6,6 +6,11 @@ description: The MPE_DERIVED table is used to configure computed columns that ap
# MPE_DERIVED # MPE_DERIVED
!!! warning "Draft - not in a release yet"
This page describes a feature that is in development. It is published early so
the wording and the screenshots can be reviewed; the behaviour described here
ships in the next release.
The MPE_DERIVED table is used to configure derived (computed) variables - columns that are created when a table is loaded into the EDIT screen, so that they can be displayed and filtered on without being stored. More details are available in the [user guide](/derived-variables/) The MPE_DERIVED table is used to configure derived (computed) variables - columns that are created when a table is loaded into the EDIT screen, so that they can be displayed and filtered on without being stored. More details are available in the [user guide](/derived-variables/)
The table is itself registered in [MPE_TABLES](/tables/mpe_tables/), so it is maintained through the EDIT screen with the usual review and approve workflow. The table is itself registered in [MPE_TABLES](/tables/mpe_tables/), so it is maintained through the EDIT screen with the usual review and approve workflow.
+3 -2
View File
@@ -17,6 +17,7 @@ nav:
- Developer Guide: - Developer Guide:
- Filter in a Pre-Edit Hook: developer-guide/filter-pre-edit-hook.md - Filter in a Pre-Edit Hook: developer-guide/filter-pre-edit-hook.md
- One Table, Two Rule Sets: developer-guide/two-rule-sets-one-table.md - One Table, Two Rule Sets: developer-guide/two-rule-sets-one-table.md
- Browser Variables: developer-guide/browser-variables.md
- Table Guide: - Table Guide:
- MPE_AUDIT: tables/mpe_audit.md - MPE_AUDIT: tables/mpe_audit.md
- MPE_COLUMN_LEVEL_SECURITY: tables/mpe_column_level_security.md - MPE_COLUMN_LEVEL_SECURITY: tables/mpe_column_level_security.md
@@ -32,7 +33,7 @@ nav:
- MPE_DATASTATUS_TABS: tables/mpe_datastatus_tabs.md - MPE_DATASTATUS_TABS: tables/mpe_datastatus_tabs.md
- MPE_DATADICTIONARY: tables/mpe_datadictionary.md - MPE_DATADICTIONARY: tables/mpe_datadictionary.md
- MPE_DATALOADS: tables/mpe_dataloads.md - MPE_DATALOADS: tables/mpe_dataloads.md
- MPE_DERIVED: tables/mpe_derived.md - MPE_DERIVED (draft): tables/mpe_derived.md
- MPE_EMAILS: tables/mpe_emails.md - MPE_EMAILS: tables/mpe_emails.md
- MPE_EXCEL_CONFIG: tables/mpe_excel_config.md - MPE_EXCEL_CONFIG: tables/mpe_excel_config.md
- MPE_FILTERANYTABLE: tables/mpe_filteranytable.md - MPE_FILTERANYTABLE: tables/mpe_filteranytable.md
@@ -59,7 +60,7 @@ nav:
- CAS Tables: cas-tables.md - CAS Tables: cas-tables.md
- Column Level Security: column-level-security.md - Column Level Security: column-level-security.md
- Dates / Datetimes: dcc-dates.md - Dates / Datetimes: dcc-dates.md
- Derived Variables: derived-variables.md - Derived Variables (draft): derived-variables.md
- Dynamic Cell Dropdown: dynamic-cell-dropdown.md - Dynamic Cell Dropdown: dynamic-cell-dropdown.md
- Emails: emails.md - Emails: emails.md
- Excel: excel.md - Excel: excel.md