Merge pull request 'docs(options): the EDIT limits truncate the grid and warn' (#22) from docs/editor-limit-warning into main
Publish to docs.datacontroller.io / Deploy docs (push) Successful in 1m21s

Reviewed-on: #22
This commit was merged in pull request #22.
This commit is contained in:
allan committed 2026-10-26 11:59:54 +00:00
commit ca62bb391c
5 files changed
+12 -8

No files matched your search

+3 -1
View File
@@ -35,7 +35,9 @@ When importing an excel file, all rows are loaded into the web page. You have a
A CSV upload bypasses the part where the records are loaded into the web page, and ALL rows are sent to the staging area directly. This makes it suitable for larger uploads.
## Edit Stage Approve Workflow
Up to 500 rows can be edited (in the web editor) at one time. These edits are submitted to a staging area. After one or more approvals (acceptances) the changes are applied to the source table.
The EDIT screen loads up to [DC_MAXOBS_WEBEDIT](/dcc-options/#dc_maxobs_webedit) rows (250 by default), or as many rows as fit within [DC_MAXCELLS_WEBEDIT](/dcc-options/#dc_maxcells_webedit) (200,000 cells) - whichever is reached first, so a wide table loads fewer rows than a narrow one. A larger table is not refused: it loads truncated, with a warning against the table name, and the filter is the way to reach the rest.
These edits are submitted to a staging area. After one or more approvals (acceptances) the changes are applied to the source table.
![screenshot](img/dcu_flow.png)
+2
View File
@@ -23,6 +23,8 @@ Set to YES or NO to enable email alerts. This requires email options to be preco
### DC_MAXCELLS_WEBEDIT
By default, a maximum of 200,000 cells (observations multiplied by variables) can be edited in the browser at one time. This limit is applied together with [DC_MAXOBS_WEBEDIT](#dc_maxobs_webedit) - whichever is reached first.
A selection over either limit is served truncated: the grid carries the rows that fit, and a warning triangle next to the table name names the limit that was reached. Filter the table to see the rest.
The cell count matters because the cost of an EDIT screen load tracks the number of cells, not the number of rows. A 10 column table is comfortable at 250 rows, but 250 rows of a 1000 column table is 250,000 cells.
### DC_MAXOBS_WEBEDIT
+4 -4
View File
@@ -1,7 +1,7 @@
# Data Controller for SAS: Viewer
The viewer screen provides a raw view of the underlying table.
Choose a library, then a table, and click view to see the first 500 rows.
Choose a library, then a table, and click view to see the first rows - up to the `DC_MAXOBS_WEBVIEW` limit, 2,000 rows by default.
A filter option is provided should you wish to view a different section of rows.
The following libraries will be visible:
@@ -21,9 +21,9 @@ The search covers **every** column of the table at once, so you do not need to k
- **Character search** (the default) is a case sensitive CONTAINS match against every character column, so `Smith` finds `Smithson` and `smith` finds `Goldsmith`, but `smith` will not find `Smithson`.
- **Numeric search** (tick the Numeric box) is an exact match against every numeric column, so `42` will not find `420` or `4210`.
- If a filter is applied, the search runs within the filtered rows rather than the whole table.
- The grid is capped by the [`DC_MAXOBS_WEBVIEW`](/dcc-options/#dc_maxobs_webview) option (500 rows by default).
- For a search, the cap applies to the row count as well: a search that matches more rows than the cap returns 500 of them, and the count shown next to the table name is the number of rows returned, not the total number of matches.
- Without a search, the count shown next to the table name is the full number of rows in the filtered view, so it can be higher than the number of rows displayed - a table of 1,200 rows reports `1,200 rows` above a grid holding the first 500.
- The grid is capped by the [`DC_MAXOBS_WEBVIEW`](/dcc-options/#dc_maxobs_webview) option (2,000 rows by default).
- For a search, the cap applies to the row count as well: a search that matches more rows than the cap returns 2,000 of them, and the count shown next to the table name is the number of rows returned, not the total number of matches.
- Without a search, the count shown next to the table name is the full number of rows in the filtered view, so it can be higher than the number of rows displayed - a table of 5,000 rows reports `5,000 rows` above a grid holding the first 2,000.
- A search that matches nothing shows a "No data found with given conditions" panel rather than an empty grid.
![Full table search](img/full-table-search.png)
+2 -2
View File
@@ -146,9 +146,9 @@ The captures below are the editor for a table with an `ORDER_DATE` column, opene
The user's clause and Data Controller's clause have to be kept apart, because Row Level Security lives in the second one. Combining them as `( user OR previous-quarter ) AND ( validity + RLS )` keeps RLS applied to both branches. Wrapping the whole existing filter instead - `( existing OR previous-quarter )` - would let the previous-quarter rows through without their RLS condition, so do not take that shortcut.
## The row cap
## The row and cell caps
`getdata` refuses a result larger than `DC_MAXOBS_WEBEDIT` (500 by default) with *Table is too big*. Widening a filter widens the result, so a hook that adds a quarter can push a table over the cap. Check the count before handing the data back, or raise the option for that table in `MPE_CONFIG`.
`getdata` truncates a result over either `DC_MAXOBS_WEBEDIT` or `DC_MAXCELLS_WEBEDIT`, whichever binds first - a wide table runs out of cells before rows, at floor(cells / columns) rows - and warns against the table name. Widening a filter widens the result, so a hook that adds a quarter can push a table over the cap: the rows past the limit never reach the browser. Check both counts before handing the data back, or raise the options for that table in `MPE_CONFIG`.
## Matching the filter chips
@@ -111,7 +111,7 @@ While it runs, the hook can also read `work.browser_info` and `work.browser_url_
### Other things to watch
- Concurrency behaves correctly here. Both mirrors route their submit to the same base table, so every approval locks, loads and logs against one identity - unlike the two-libref route above.
- The filter has already been applied to the empty mirror by the time the pre-edit hook runs, so a hook that reads the real table ignores the user's filter unless it re-applies it (`where %inc filtref`). On a small table you will not notice; on a large one the `DC_MAXOBS_WEBEDIT` cap will stop the edit screen with "Table is too big". [Updating the Filter in a Pre-Edit Hook](/developer-guide/filter-pre-edit-hook/) works through extending that filter.
- The filter has already been applied to the empty mirror by the time the pre-edit hook runs, so a hook that reads the real table ignores the user's filter unless it re-applies it (`where %inc filtref`). On a small table you will not notice; on a large one the `DC_MAXOBS_WEBEDIT` or `DC_MAXCELLS_WEBEDIT` cap truncates the edit screen, with a warning against the table name. [Updating the Filter in a Pre-Edit Hook](/developer-guide/filter-pre-edit-hook/) works through extending that filter.
- The hook's output must have the same columns the editor expects - the real table, minus any transaction or processing columns that Data Controller drops on load.
- The real table must itself be registered in `MPE_TABLES`. The approval screen resolves the table's audit settings from that row, so a changeset routed to a table with no registration cannot be reviewed - the submit is refused up front, naming the table.
- The mirror's `MPE_TABLES` row is read for the edit screen and the real table's for the load, so keep their `buskey`, `loadtype` and temporal column settings identical.