dc/docs.datacontroller.io
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: adding more info on REPLACE loadtype
All checks were successful
Publish to docs.datacontroller.io / Deploy docs (push) Successful in 2m22s
Details

Browse Source
This commit is contained in:
Allan 2023-08-25 12:23:52 +01:00
parent 6db54d6b84
commit bc494d7e90
3 changed files with 4 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
docs/dcc-tables.md
View File

@ -41,6 +41,7 @@ The loadtype determines the nature of the update to be applied. Valid values are
- UPDATE. This is the most basic type, and any updates will happen 'in place'. Simply provide the primary key fields in the `BUSKEY` column. - UPDATE. This is the most basic type, and any updates will happen 'in place'. Simply provide the primary key fields in the `BUSKEY` column.
- TXTEMPORAL. This signifies an SCD2 type load. For this type the validity fields (valid from, valid to) should be specified in the `VAR_TXFROM` and `VAR_TXTO` fields. The table itself should include `VAR_TXFROM` in the physical key. The remainder of the primary key fields (not including `VAR_TXFROM`) should be specified in `BUSKEY`. - TXTEMPORAL. This signifies an SCD2 type load. For this type the validity fields (valid from, valid to) should be specified in the `VAR_TXFROM` and `VAR_TXTO` fields. The table itself should include `VAR_TXFROM` in the physical key. The remainder of the primary key fields (not including `VAR_TXFROM`) should be specified in `BUSKEY`.
- BITEMPORAL. These tables have two time dimensions - a version history, and a business history. The version history (SCD2) fields should be specified in `VAR_TXFROM` and `VAR_TXTO` and the business history fields should be specified in `VAR_BUSFROM` and `VAR_BUSTO`. Both the `VAR_TXFROM` and `VAR_BUSFROM` fields should be in the physical key of the actual table, but should NOT be specified in the `BUSKEY` field. - BITEMPORAL. These tables have two time dimensions - a version history, and a business history. The version history (SCD2) fields should be specified in `VAR_TXFROM` and `VAR_TXTO` and the business history fields should be specified in `VAR_BUSFROM` and `VAR_BUSTO`. Both the `VAR_TXFROM` and `VAR_BUSFROM` fields should be in the physical key of the actual table, but should NOT be specified in the `BUSKEY` field.
- REPLACE. This loadtype simply deletes all the rows and appends the staged data. Changes are NOT added to the audit table. In the diff screen, previous rows are displayed as deleted, and staged rows as new (modified values are no displayed). Can be useful for updating single-row tables.
This is a required field. This is a required field.

4
docs/index.md
View File

@ -29,7 +29,7 @@ The following resources contain additional information on the Data Controller:
Data Controller is regularly updated with new features. If you see something that is not listed, and we agree it would be useful, you can engage us with Developer Days to build the feature in. Data Controller is regularly updated with new features. If you see something that is not listed, and we agree it would be useful, you can engage us with Developer Days to build the feature in.
* [Excel uploads](/dcu-fileupload/#excel-uploads) - drag & drop directly into SAS. All versions of excel supported. * [Excel uploads](/dcu-fileupload/#excel-uploads) - drag & drop directly into SAS. All versions of excel supported.
* Data Lineage - at both table and column level, export as image or CSV * Data Lineage - at both table and column level, export as image or CSV
* Data Validation Rules - both automatic and user defined * Data Validation Rules - both automatic and user defined
* Data Dictionary - map data definitions and ownership * Data Dictionary - map data definitions and ownership
@ -37,7 +37,7 @@ Data Controller is regularly updated with new features. If you see something th
* DDL generator - in SAS, TSQL and PGSQL flavours * DDL generator - in SAS, TSQL and PGSQL flavours
* Workflow - run a SAS program before or after an edit or approval * Workflow - run a SAS program before or after an edit or approval
* Role based security - editors, approvers, administrators * Role based security - editors, approvers, administrators
* Data Models supported: SCD2, Bitemporal, composite keys, retained keys, update and full replace * Data Models supported: SCD2, Bitemporal, composite keys, retained keys, update, format catalogs
* Email Alerts for data changes * Email Alerts for data changes
* [Row Level Security](/row-level-security) * [Row Level Security](/row-level-security)
* Excel [formula support](excel) * Excel [formula support](excel)

2
docs/macros.md
View File

@ -36,7 +36,7 @@ The macros work dynamically, taking data types / lengths etc from the table meta
* High column volume * High column volume
* New data types in CAS (Type 6 as well as 1 & 2) * New data types in CAS (Type 6 as well as 1 & 2)
Multiple load types are supported (full REPLACE, regular UPDATE, SCD2 loads, even BITEMPORAL). Composite keys are of course possible, so are retained keys (and the max key value can be table or data driven). Keys are taken from the staging table, so they could be based on an md5 hash, a UUID, or whatever you choose. It just needs to fit within the constraints of a SAS data step (so, max 32767 width for a character variable, 8 bytes numerical precision). Multiple load types are supported (regular UPDATE, SCD2, format catalog loads, even BITEMPORAL). Composite keys are of course possible, so are retained keys (and the max key value can be table or data driven). Keys are taken from the staging table, so they could be based on an md5 hash, a UUID, or whatever you choose. It just needs to fit within the constraints of a SAS data step (so, max 32767 width for a character variable, 8 bytes numerical precision).
A column (eg PROCESSED_DTTM) may be nominated to retain the current timestamp when doing any type of upload. A column (eg PROCESSED_DTTM) may be nominated to retain the current timestamp when doing any type of upload.