dc/docs.datacontroller.io
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update macros.md

Browse Source
This commit is contained in:
Allan Bowe 2021-02-22 09:01:07 +01:00 committed by GitHub
parent 02f5311749
commit 0dbf891fb9
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 2 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/macros.md
View File

@ -37,6 +37,8 @@ Multiple load types are supported (full REPLACE, regular UPDATE, SCD2 loads, eve
A column (eg PROCESSED_DTTM) may be nominated to retain the current timestamp when doing any type of upload.
If the stage table has columns that are not in the base table, the extra columns are ignored. If the base table has columns that are not in the stage table, the process aborts.
There is also a locking mechanism to avoid conflict where multiple users (or jobs) are trying to load the same table at the same time. If the loader cannot get a lock, it will keep trying for a configurable amount of time, until it does.
In the process of data discovery and comparing the hashes of the data values there are checks at every step of the way - the macro will abort if there are any WARNINGs or ERRORs or anything else awry. Some of these checks may be turned off for performance (eg the check for uniqueness of the business key in the staging table).