dc/docs.datacontroller.io
5
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: automated commit

Browse Source
This commit is contained in:
munja
2022-04-30 18:36:00 +01:00
parent d4537dfe13
commit 971f756d9e
3 changed files with 11 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
docs/admin-services.md
View File

@@ -51,18 +51,19 @@ EXAMPLES:
## Refresh Data Catalog
In any SAS estate, it's unlikely the size & shape of data will remain static. By running a regular Catalog Scan, you can track changes such as:
- Library Properties (size, schema, path, number of tables)
- Table Properties (size, number of columns, primary keys)
- Variable Properties (attributes)
The data is stored with SCD2 so you can actually **track changes to your model over time**! Curious when that new column appeared? Just check the history in [MPE_DATACATALOG_TABS](/mpe_datacatalog_tabs.md).
The data is stored with SCD2 so you can actually **track changes to your model over time**! Curious when that new column appeared? Just check the history in [MPE_DATACATALOG_TABS](/mpe_datacatalog_tabs).
To run the refresh process, just trigger the stored process, eg below:
* `services/admin/refreshcatalog`
* `services/admin/refreshcatalog&libref=MYLIB`
The `&libref=` parameter is optional - if you want to run the process just for a single library.
The optional `&libref=` parameter allows you to run the process for a single library. Just provide the libref.
When doing a full scan, the following LIBREFS are ignored:
@@ -74,14 +75,14 @@ When doing a full scan, the following LIBREFS are ignored:
* 'TEMP'
* `WORK'
Be aware that the scan process can take a long time if you have a lot of tables. Also, note that if a library refresh crashes (due to invalid connection properties), you can exclude it from the subsequent refresh process by adding the `LIBREF` (pipe-separated) to the `DCXXXX.MPE_CONFIG` table (where `var_scope='DC_CATALOG' and var_name='DC_IGNORELIBS'`).
Be aware that the scan process can take a long time if you have a lot of tables! Also, note that if a library refresh crashes (due to invalid connection properties), you can exclude it from the subsequent refresh process by adding the `LIBREF` (pipe-separated) to the `DCXXXX.MPE_CONFIG` table (where `var_scope='DC_CATALOG' and var_name='DC_IGNORELIBS'`).
Output tables (all SCD2):
* [MPE_DATACATALOG_LIBS](/mpe_datacatalog_libs.md) - Library attributes
* [MPE_DATACATALOG_TABS](/mpe_datacatalog_tabs.md) - Table attributes
* [MPE_DATACATALOG_VARS](/mpe_datacatalog_vars.md) - Column attributes
* [MPE_DATASTATUS_LIBS](/mpe_datastatus_libs.md) - Frequently changing library attributes (such as size & number of tables)
* [MPE_DATASTATUS_TABS](/mpe_datastatus_tabs.md) - Frequently changing table attributes (such as size & number of rows)
* [MPE_DATACATALOG_LIBS](/mpe_datacatalog_libs) - Library attributes
* [MPE_DATACATALOG_TABS](/mpe_datacatalog_tabs) - Table attributes
* [MPE_DATACATALOG_VARS](/mpe_datacatalog_vars) - Column attributes
* [MPE_DATASTATUS_LIBS](/mpe_datastatus_libs) - Frequently changing library attributes (such as size & number of tables)
* [MPE_DATASTATUS_TABS](/mpe_datastatus_tabs) - Frequently changing table attributes (such as size & number of rows)

2
docs/mpe_datacatalog_libs.md
View File

@@ -8,7 +8,7 @@ description: The MPE_DATACATALOG_LIBS table catalogs library attributes such as
The `MPE_DATACATALOG_LIBS` table catalogs library attributes such as engine, paths, permissions, owners & schemas.
More frequently changing attributes (such as size and number of tables) are stored in [MPE_DATASTATUS_LIBS](/mpe_datastatus_libs.md).
More frequently changing attributes (such as size and number of tables) are stored in [MPE_DATASTATUS_LIBS](/mpe_datastatus_libs).
To ignore additional librefs, or to trigger a scan, see the instructions [here](https://docs.datacontroller.io/admin-services/#refresh-data-catalog)

2
docs/mpe_datacatalog_tabs.md
View File

@@ -8,7 +8,7 @@ description: The MPE_DATACATALOG_TABS table catalogs attributes such as number o
The `MPE_DATACATALOG_TABS` table catalogs attributes such as number of variables, compression status, and primary key fields.
More frequently changing attributes (such as size modification date and number of observations) are stored in [MPE_DATASTATUS_TABS](/mpe_datastatus_tabs.md).
More frequently changing attributes (such as size modification date and number of observations) are stored in [MPE_DATASTATUS_TABS](/mpe_datastatus_tabs).
To trigger a scan, see the instructions [here](https://docs.datacontroller.io/admin-services/#refresh-data-catalog)