dc/docs.datacontroller.io
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: adding more info on REPLACE loadtype
All checks were successful
Publish to docs.datacontroller.io / Deploy docs (push) Successful in 2m22s
Details

Browse Source
This commit is contained in:
Allan 2023-08-25 12:23:52 +01:00
parent 6db54d6b84
commit bc494d7e90
3 changed files with 4 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
docs/dcc-tables.md
View File

@ -41,6 +41,7 @@ The loadtype determines the nature of the update to be applied. Valid values are
- UPDATE. This is the most basic type, and any updates will happen 'in place'. Simply provide the primary key fields in the `BUSKEY` column. - UPDATE. This is the most basic type, and any updates will happen 'in place'. Simply provide the primary key fields in the `BUSKEY` column.
- TXTEMPORAL. This signifies an SCD2 type load. For this type the validity fields (valid from, valid to) should be specified in the `VAR_TXFROM` and `VAR_TXTO` fields. The table itself should include `VAR_TXFROM` in the physical key. The remainder of the primary key fields (not including `VAR_TXFROM`) should be specified in `BUSKEY`. - TXTEMPORAL. This signifies an SCD2 type load. For this type the validity fields (valid from, valid to) should be specified in the `VAR_TXFROM` and `VAR_TXTO` fields. The table itself should include `VAR_TXFROM` in the physical key. The remainder of the primary key fields (not including `VAR_TXFROM`) should be specified in `BUSKEY`.
- BITEMPORAL. These tables have two time dimensions - a version history, and a business history. The version history (SCD2) fields should be specified in `VAR_TXFROM` and `VAR_TXTO` and the business history fields should be specified in `VAR_BUSFROM` and `VAR_BUSTO`. Both the `VAR_TXFROM` and `VAR_BUSFROM` fields should be in the physical key of the actual table, but should NOT be specified in the `BUSKEY` field. - BITEMPORAL. These tables have two time dimensions - a version history, and a business history. The version history (SCD2) fields should be specified in `VAR_TXFROM` and `VAR_TXTO` and the business history fields should be specified in `VAR_BUSFROM` and `VAR_BUSTO`. Both the `VAR_TXFROM` and `VAR_BUSFROM` fields should be in the physical key of the actual table, but should NOT be specified in the `BUSKEY` field.
- REPLACE. This loadtype simply deletes all the rows and appends the staged data. Changes are NOT added to the audit table. In the diff screen, previous rows are displayed as deleted, and staged rows as new (modified values are no displayed). Can be useful for updating single-row tables.
This is a required field. This is a required field.

2
docs/index.md
View File

@ -37,7 +37,7 @@ Data Controller is regularly updated with new features. If you see something th
* DDL generator - in SAS, TSQL and PGSQL flavours * DDL generator - in SAS, TSQL and PGSQL flavours
* Workflow - run a SAS program before or after an edit or approval * Workflow - run a SAS program before or after an edit or approval
* Role based security - editors, approvers, administrators * Role based security - editors, approvers, administrators
* Data Models supported: SCD2, Bitemporal, composite keys, retained keys, update and full replace * Data Models supported: SCD2, Bitemporal, composite keys, retained keys, update, format catalogs
* Email Alerts for data changes * Email Alerts for data changes
* [Row Level Security](/row-level-security) * [Row Level Security](/row-level-security)
* Excel [formula support](excel) * Excel [formula support](excel)

2
docs/macros.md
View File

@ -36,7 +36,7 @@ The macros work dynamically, taking data types / lengths etc from the table meta
* High column volume * High column volume
* New data types in CAS (Type 6 as well as 1 & 2) * New data types in CAS (Type 6 as well as 1 & 2)
Multiple load types are supported (full REPLACE, regular UPDATE, SCD2 loads, even BITEMPORAL). Composite keys are of course possible, so are retained keys (and the max key value can be table or data driven). Keys are taken from the staging table, so they could be based on an md5 hash, a UUID, or whatever you choose. It just needs to fit within the constraints of a SAS data step (so, max 32767 width for a character variable, 8 bytes numerical precision). Multiple load types are supported (regular UPDATE, SCD2, format catalog loads, even BITEMPORAL). Composite keys are of course possible, so are retained keys (and the max key value can be table or data driven). Keys are taken from the staging table, so they could be based on an md5 hash, a UUID, or whatever you choose. It just needs to fit within the constraints of a SAS data step (so, max 32767 width for a character variable, 8 bytes numerical precision).
A column (eg PROCESSED_DTTM) may be nominated to retain the current timestamp when doing any type of upload. A column (eg PROCESSED_DTTM) may be nominated to retain the current timestamp when doing any type of upload.