4 Commits
Author SHA1 Message Date
allan ca62bb391c Merge pull request 'docs(options): the EDIT limits truncate the grid and warn' (#22) from docs/editor-limit-warning into main
Publish to docs.datacontroller.io / Deploy docs (push) Successful in 1m21s
Reviewed-on: #22
2026-10-26 11:59:54 +00:00
dc 856e57173f docs(overview): the EDIT limits truncate, and the viewer default is 2000 not 500
Review 431: dc-overview.md still said "Up to 500 rows can be edited (in the web
editor) at one time" - the wrong number (DC_MAXOBS_WEBEDIT ships at 250) and, once
dc/dc#352 lands, the wrong behaviour (an over-limit selection is served truncated
with a warning, not refused or capped at a count). It now names both limits,
whichever binds first, and links to the options that set them.

Sweeping for the same stale figure turned up dcu-tableviewer.md, which says 500
twice and uses 500 in its worked example of the row count above the grid.
DC_MAXOBS_WEBVIEW has been 2000 since 20261003_row_and_cell_limits.sas (the
migration records 500 -> 2000), and mpe_makedata seeds 2000, so those three
places were two releases out of date. Fixed here, and flagged in the PR reply as
VIEW rather than EDIT in case it belongs in its own change.

Checked: no "500 rows" or stale abort wording left in docs/, and the two anchors
the overview now links to (dc_maxobs_webedit, dc_maxcells_webedit) exist on the
published options page.
2026-10-26 11:57:21 +00:00
dc 561e92e090 docs(developer-guide): name both EDIT caps in the hook page
The truncation is whichever of DC_MAXOBS_WEBEDIT and DC_MAXCELLS_WEBEDIT
binds first, and for a wide table it is the cell cap - floor(cells/columns)
rows, potentially far below the row limit.  The hook page named only the row
cap and quoted its 250 default, so a hook author would check the row count
alone.  Same correction in the two-rule-sets page.
2026-10-22 19:27:51 +00:00
dc 971b007606 docs(options): the EDIT limits truncate the grid and warn
getdata no longer refuses a selection over DC_MAXOBS_WEBEDIT or
DC_MAXCELLS_WEBEDIT - it serves the rows within the limit and flags the
truncation against the table name.  Update the option reference and the two
developer-guide pages that claimed the edit screen stops with "Table is too
big".  The hook page also carried the old 500 row default.
2026-10-22 16:40:48 +00:00
5 changed files with 12 additions and 8 deletions

No files matched your search

+3 -1
View File
@@ -35,7 +35,9 @@ When importing an excel file, all rows are loaded into the web page. You have a
A CSV upload bypasses the part where the records are loaded into the web page, and ALL rows are sent to the staging area directly. This makes it suitable for larger uploads.
## Edit Stage Approve Workflow
Up to 500 rows can be edited (in the web editor) at one time. These edits are submitted to a staging area. After one or more approvals (acceptances) the changes are applied to the source table.
The EDIT screen loads up to [DC_MAXOBS_WEBEDIT](/dcc-options/#dc_maxobs_webedit) rows (250 by default), or as many rows as fit within [DC_MAXCELLS_WEBEDIT](/dcc-options/#dc_maxcells_webedit) (200,000 cells) - whichever is reached first, so a wide table loads fewer rows than a narrow one. A larger table is not refused: it loads truncated, with a warning against the table name, and the filter is the way to reach the rest.
These edits are submitted to a staging area. After one or more approvals (acceptances) the changes are applied to the source table.
![screenshot](img/dcu_flow.png)
+2
View File
@@ -23,6 +23,8 @@ Set to YES or NO to enable email alerts. This requires email options to be preco
### DC_MAXCELLS_WEBEDIT
By default, a maximum of 200,000 cells (observations multiplied by variables) can be edited in the browser at one time. This limit is applied together with [DC_MAXOBS_WEBEDIT](#dc_maxobs_webedit) - whichever is reached first.
A selection over either limit is served truncated: the grid carries the rows that fit, and a warning triangle next to the table name names the limit that was reached. Filter the table to see the rest.
The cell count matters because the cost of an EDIT screen load tracks the number of cells, not the number of rows. A 10 column table is comfortable at 250 rows, but 250 rows of a 1000 column table is 250,000 cells.
### DC_MAXOBS_WEBEDIT
+4 -4
View File
@@ -1,7 +1,7 @@
# Data Controller for SAS: Viewer
The viewer screen provides a raw view of the underlying table.
Choose a library, then a table, and click view to see the first 500 rows.
Choose a library, then a table, and click view to see the first rows - up to the `DC_MAXOBS_WEBVIEW` limit, 2,000 rows by default.
A filter option is provided should you wish to view a different section of rows.
The following libraries will be visible:
@@ -21,9 +21,9 @@ The search covers **every** column of the table at once, so you do not need to k
- **Character search** (the default) is a case sensitive CONTAINS match against every character column, so `Smith` finds `Smithson` and `smith` finds `Goldsmith`, but `smith` will not find `Smithson`.
- **Numeric search** (tick the Numeric box) is an exact match against every numeric column, so `42` will not find `420` or `4210`.
- If a filter is applied, the search runs within the filtered rows rather than the whole table.
- The grid is capped by the [`DC_MAXOBS_WEBVIEW`](/dcc-options/#dc_maxobs_webview) option (500 rows by default).
- For a search, the cap applies to the row count as well: a search that matches more rows than the cap returns 500 of them, and the count shown next to the table name is the number of rows returned, not the total number of matches.
- Without a search, the count shown next to the table name is the full number of rows in the filtered view, so it can be higher than the number of rows displayed - a table of 1,200 rows reports `1,200 rows` above a grid holding the first 500.
- The grid is capped by the [`DC_MAXOBS_WEBVIEW`](/dcc-options/#dc_maxobs_webview) option (2,000 rows by default).
- For a search, the cap applies to the row count as well: a search that matches more rows than the cap returns 2,000 of them, and the count shown next to the table name is the number of rows returned, not the total number of matches.
- Without a search, the count shown next to the table name is the full number of rows in the filtered view, so it can be higher than the number of rows displayed - a table of 5,000 rows reports `5,000 rows` above a grid holding the first 2,000.
- A search that matches nothing shows a "No data found with given conditions" panel rather than an empty grid.
![Full table search](img/full-table-search.png)
+2 -2
View File
@@ -146,9 +146,9 @@ The captures below are the editor for a table with an `ORDER_DATE` column, opene
The user's clause and Data Controller's clause have to be kept apart, because Row Level Security lives in the second one. Combining them as `( user OR previous-quarter ) AND ( validity + RLS )` keeps RLS applied to both branches. Wrapping the whole existing filter instead - `( existing OR previous-quarter )` - would let the previous-quarter rows through without their RLS condition, so do not take that shortcut.
## The row cap
## The row and cell caps
`getdata` refuses a result larger than `DC_MAXOBS_WEBEDIT` (500 by default) with *Table is too big*. Widening a filter widens the result, so a hook that adds a quarter can push a table over the cap. Check the count before handing the data back, or raise the option for that table in `MPE_CONFIG`.
`getdata` truncates a result over either `DC_MAXOBS_WEBEDIT` or `DC_MAXCELLS_WEBEDIT`, whichever binds first - a wide table runs out of cells before rows, at floor(cells / columns) rows - and warns against the table name. Widening a filter widens the result, so a hook that adds a quarter can push a table over the cap: the rows past the limit never reach the browser. Check both counts before handing the data back, or raise the options for that table in `MPE_CONFIG`.
## Matching the filter chips
@@ -111,7 +111,7 @@ While it runs, the hook can also read `work.browser_info` and `work.browser_url_
### Other things to watch
- Concurrency behaves correctly here. Both mirrors route their submit to the same base table, so every approval locks, loads and logs against one identity - unlike the two-libref route above.
- The filter has already been applied to the empty mirror by the time the pre-edit hook runs, so a hook that reads the real table ignores the user's filter unless it re-applies it (`where %inc filtref`). On a small table you will not notice; on a large one the `DC_MAXOBS_WEBEDIT` cap will stop the edit screen with "Table is too big". [Updating the Filter in a Pre-Edit Hook](/developer-guide/filter-pre-edit-hook/) works through extending that filter.
- The filter has already been applied to the empty mirror by the time the pre-edit hook runs, so a hook that reads the real table ignores the user's filter unless it re-applies it (`where %inc filtref`). On a small table you will not notice; on a large one the `DC_MAXOBS_WEBEDIT` or `DC_MAXCELLS_WEBEDIT` cap truncates the edit screen, with a warning against the table name. [Updating the Filter in a Pre-Edit Hook](/developer-guide/filter-pre-edit-hook/) works through extending that filter.
- The hook's output must have the same columns the editor expects - the real table, minus any transaction or processing columns that Data Controller drops on load.
- The real table must itself be registered in `MPE_TABLES`. The approval screen resolves the table's audit settings from that row, so a changeset routed to a table with no registration cannot be reviewed - the submit is refused up front, naming the table.
- The mirror's `MPE_TABLES` row is read for the edit screen and the real table's for the load, so keep their `buskey`, `loadtype` and temporal column settings identical.