The defaults changed in dc/dc (Data Controller PR #335):
option
was
now
DC_MAXOBS_WEBVIEW (VIEW)
500
2000
DC_MAXOBS_WEBEDIT (EDIT)
100
250
DC_MAXCELLS_WEBEDIT (EDIT)
-
200000
Documents the new DC_MAXCELLS_WEBEDIT option and explains why the EDIT screen needs a cell count as well as a row count: the cost of an EDIT load tracks cells (rows x columns), not rows, so a row limit alone cannot bound a wide table. 100 rows of a 1001 column table already takes ~17 seconds to load, which is what 2000 rows of a 9 column table costs.
Should be merged after dc/dc PR #335, which is the change these defaults describe.
The defaults changed in dc/dc (Data Controller PR #335):
| option | was | now |
|---|---|---|
| `DC_MAXOBS_WEBVIEW` (VIEW) | 500 | 2000 |
| `DC_MAXOBS_WEBEDIT` (EDIT) | 100 | 250 |
| `DC_MAXCELLS_WEBEDIT` (EDIT) | - | 200000 |
Documents the new `DC_MAXCELLS_WEBEDIT` option and explains why the EDIT screen needs a cell count as well as a row count: the cost of an EDIT load tracks cells (rows x columns), not rows, so a row limit alone cannot bound a wide table. 100 rows of a 1001 column table already takes ~17 seconds to load, which is what 2000 rows of a 9 column table costs.
Should be merged after dc/dc PR #335, which is the change these defaults describe.
DC_MAXOBS_WEBEDIT 100 -> 250, DC_MAXOBS_WEBVIEW 500 -> 2000, and a new
DC_MAXCELLS_WEBEDIT (200000) which is applied together with the EDIT
observation limit - whichever is reached first.
Reviewed the docs diff (dcc-options.md limit updates) plus repo-level hardening (pre-commit hook, CI, package files).
Repo-level: dc/docs.datacontroller.io has no secret-scanning pre-commit hook at this head - no .pre-commit-config.yaml and no .git-hooks/ directory. This is a standing requirement on every repo: add a gitleaks hook pinned to an exact release tag so staged content is scanned at commit time. dc/dc's .git-hooks/pre-commit (gitleaks protect --staged, binary from the pinned @nogoo9/gitleaks devDependency, core.hooksPath activation documented in CONTRIBUTING.md) is the in-estate pattern to copy.
docs/dcc-options.md:24 - "a maximum of 200,000 cells (observations multiplied by variables)" undercounts what the service enforces. getdata.sas computes the cell count on %mf_getvarlist(work.out), which already includes the housekeeping delete column, so the enforced count is rows x (variables + 1); the dc/dc mock mirrors this with visibleColumns.length + 1. At the boundary this misleads: 200 rows x 1000 variables reads as 200,000 = at the limit by the page's arithmetic, but the service computes 200 x 1001 = 200,200 and refuses the edit. Add a clause noting the count includes the housekeeping delete column.
2 findings above for review.
Reviewed the docs diff (dcc-options.md limit updates) plus repo-level hardening (pre-commit hook, CI, package files).
- Repo-level: dc/docs.datacontroller.io has no secret-scanning pre-commit hook at this head - no .pre-commit-config.yaml and no .git-hooks/ directory. This is a standing requirement on every repo: add a gitleaks hook pinned to an exact release tag so staged content is scanned at commit time. dc/dc's .git-hooks/pre-commit (gitleaks protect --staged, binary from the pinned @nogoo9/gitleaks devDependency, core.hooksPath activation documented in CONTRIBUTING.md) is the in-estate pattern to copy.
- docs/dcc-options.md:24 - "a maximum of 200,000 cells (observations multiplied by variables)" undercounts what the service enforces. getdata.sas computes the cell count on %mf_getvarlist(work.out), which already includes the housekeeping delete column, so the enforced count is rows x (variables + 1); the dc/dc mock mirrors this with visibleColumns.length + 1. At the boundary this misleads: 200 rows x 1000 variables reads as 200,000 = at the limit by the page's arithmetic, but the service computes 200 x 1001 = 200,200 and refuses the edit. Add a clause noting the count includes the housekeeping delete column.
2 findings above for review.
allan
merged commit b1ec8bec7c into main2026-10-11 10:28:48 +00:00
allan
deleted branch docs/row-cell-limits2026-10-11 10:28:48 +00:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
hermes
The defaults changed in dc/dc (Data Controller PR #335):
DC_MAXOBS_WEBVIEW(VIEW)DC_MAXOBS_WEBEDIT(EDIT)DC_MAXCELLS_WEBEDIT(EDIT)Documents the new
DC_MAXCELLS_WEBEDIToption and explains why the EDIT screen needs a cell count as well as a row count: the cost of an EDIT load tracks cells (rows x columns), not rows, so a row limit alone cannot bound a wide table. 100 rows of a 1001 column table already takes ~17 seconds to load, which is what 2000 rows of a 9 column table costs.Should be merged after dc/dc PR #335, which is the change these defaults describe.
Reviewed the docs diff (dcc-options.md limit updates) plus repo-level hardening (pre-commit hook, CI, package files).
Repo-level: dc/docs.datacontroller.io has no secret-scanning pre-commit hook at this head - no .pre-commit-config.yaml and no .git-hooks/ directory. This is a standing requirement on every repo: add a gitleaks hook pinned to an exact release tag so staged content is scanned at commit time. dc/dc's .git-hooks/pre-commit (gitleaks protect --staged, binary from the pinned @nogoo9/gitleaks devDependency, core.hooksPath activation documented in CONTRIBUTING.md) is the in-estate pattern to copy.
docs/dcc-options.md:24 - "a maximum of 200,000 cells (observations multiplied by variables)" undercounts what the service enforces. getdata.sas computes the cell count on %mf_getvarlist(work.out), which already includes the housekeeping delete column, so the enforced count is rows x (variables + 1); the dc/dc mock mirrors this with visibleColumns.length + 1. At the boundary this misleads: 200 rows x 1000 variables reads as 200,000 = at the limit by the page's arithmetic, but the service computes 200 x 1001 = 200,200 and refuses the edit. Add a clause noting the count includes the housekeeping delete column.
2 findings above for review.