dc/docs.datacontroller.io
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fixing links
All checks were successful
Publish to docs.datacontroller.io / Deploy docs (push) Successful in 1m46s
Details

Browse Source
This commit is contained in:
zmaj 2024-01-24 10:24:58 +00:00
parent 1549496248
commit 09d302c309
4 changed files with 23 additions and 18 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

22
docs/excel.md
View File

@ -14,7 +14,7 @@ Data Controller supports two approaches for importing Excel data into SAS:
Thanks to our pro license of [sheetJS](https://sheetjs.com/), we can support all versions of excel, large workbooks, and fast extracts. We also support the ingest of [password-protected workbooks](/videos#uploading-a-password-protected-excel-file). Thanks to our pro license of [sheetJS](https://sheetjs.com/), we can support all versions of excel, large workbooks, and fast extracts. We also support the ingest of [password-protected workbooks](/videos#uploading-a-password-protected-excel-file).
)
Note that data is extracted from excel from _within the browser_ - meaning there is no need for any special SAS modules / products. Note that data is extracted from excel from _within the browser_ - meaning there is no need for any special SAS modules / products.
@ -54,11 +54,11 @@ If loading very large files (eg over 10mb) it is more efficient to use CSV forma
* MAXVAL * MAXVAL
* CASE * CASE
Note that the HARDSELECT_*** hooks are not applied to the rendered Excel values (they are currently only applied when editing a cell). Note that the HARDSELECT_*** hooks are not applied to the rendered Excel values (they are only applied when actively editing a cell).
![image](https://user-images.githubusercontent.com/4420615/233036372-87b8dd02-a4cd-4f19-ac1b-bb9fdc850607.png) ![image](https://user-images.githubusercontent.com/4420615/233036372-87b8dd02-a4cd-4f19-ac1b-bb9fdc850607.png)
## Formulas ### Formulas
It is possible to configure certain columns to be extracted as formulae, rather than raw values. The target column must be character, and it should be wide enough to support the longest formula in the source data. If the order of values is important, you should include a row number in your primary key. It is possible to configure certain columns to be extracted as formulae, rather than raw values. The target column must be character, and it should be wide enough to support the longest formula in the source data. If the order of values is important, you should include a row number in your primary key.
@ -76,22 +76,22 @@ The final table will look like this:
# Complex Excel Uploads ## Complex Excel Uploads
Through the use of "Excel Maps" this feature enables a series of cells / ranges to be dynamically extracted from anywhere within a workbook - either through absolute / relative positioning, or by reference to a "matched" (search) string. Through the use of "Excel Maps" you can dynamically extract individual cells or entire ranges from anywhere within a workbook - either through absolute / relative positioning, or by reference to a "matched" (search) string.
Configuration is made in the following tables: Configuration is made in the following tables:
1. [MPE_XLMAP_RULES](tables/mpe_xlmap_rules.md) - detailed extraction rules for a particular map 1. [MPE_XLMAP_RULES](/tables/mpe_xlmap_rules.md) - detailed extraction rules for a particular map
2. [MPE_XLMAP_INFO](tables/mpe_xlmap_info.md) - optional map-level attributes 2. [MPE_XLMAP_INFO](/tables/mpe_xlmap_info.md) - optional map-level attributes
Each [rule](tables/mpe_xlmap_rules.md) will extract either a single cell or a rectangular range from the source workbook. The target will be [MPE_XLMAP_DATA](tables/mpe_xlmap_data.md), or whichever table is configured in [MPE_XLMAP_INFO](tables/mpe_xlmap_info.md). Each [rule](/tables/mpe_xlmap_rules.md) will extract either a single cell or a rectangular range from the source workbook. The target will be [MPE_XLMAP_DATA](/tables/mpe_xlmap_data.md), or whichever table is configured in [MPE_XLMAP_INFO](/tables/mpe_xlmap_info.md).
To illustrate with an example - consider the following excel. The yellow cells need to be imported. To illustrate with an example - consider the following excel. The yellow cells need to be imported.
![](img/xlmap_example.png) ![](img/xlmap_example.png)
The [MPE_XLMAP_RULES](tables/mpe_xlmap_rules.md) configuration entries _might_ (as there are multiple ways) be as follows: The [MPE_XLMAP_RULES](/tables/mpe_xlmap_rules.md) configuration entries _might_ (as there are multiple ways) be as follows:
|XLMAP_ID|XLMAP_RANGE_ID|XLMAP_SHEET|XLMAP_START|XLMAP_FINISH| |XLMAP_ID|XLMAP_RANGE_ID|XLMAP_SHEET|XLMAP_START|XLMAP_FINISH|
|---|---|---|---|---| |---|---|---|---|---|
@ -102,9 +102,9 @@ The [MPE_XLMAP_RULES](tables/mpe_xlmap_rules.md) configuration entries _might_ (
|MAP01|RENT|/1|`MATCH E R[0]C[2]: Rent/mortgage`|| |MAP01|RENT|/1|`MATCH E R[0]C[2]: Rent/mortgage`||
|MAP01|CELL|/1|`MATCH E R[0]C[2]: Cell phone`|| |MAP01|CELL|/1|`MATCH E R[0]C[2]: Cell phone`||
To import the excel, the end user simply needs to navigate to the LOAD tab, Chooses "Files", select the appropriate map (eg MAP01), and upload. This will stage the new records in [MPE_XLMAP_DATA](tables/mpe_xlmap_data.md) which will go through the usual approval process and quality checks. A copy of the source excel file will be attached to each upload. To import the excel, the end user simply needs to navigate to the LOAD tab, choose "Files", select the appropriate map (eg MAP01), and upload. This will stage the new records in [MPE_XLMAP_DATA](/tables/mpe_xlmap_data.md) which will go through the usual approval process and quality checks. A copy of the source excel file will be attached to each upload.
The corresponding [MPE_XLMAP_DATA](tables/mpe_xlmap_data.md) table will appear as follows: The corresponding [MPE_XLMAP_DATA](/tables/mpe_xlmap_data.md) table will appear as follows:
| LOAD_REF | XLMAP_ID | XLMAP_RANGE_ID | ROW_NO | COL_NO | VALUE_TXT | | LOAD_REF | XLMAP_ID | XLMAP_RANGE_ID | ROW_NO | COL_NO | VALUE_TXT |
|---------------|----------|----------------|--------|--------|-----------------| |---------------|----------|----------------|--------|--------|-----------------|

8
docs/tables/mpe_xlmap_data.md
View File

@ -8,18 +8,18 @@ og_image: ../img/mpe_xlmap_data.png
# MPE_XLMAP_DATA # MPE_XLMAP_DATA
The MPE_XLMAP_DATA table stores the data from complex excel uploads in a vertical format. It is possible to configure a different target, so long as it has the same attributes, in the [MPE_XLMAP_INFO](tables/mpe_xlmap_info.md) table. The MPE_XLMAP_DATA table stores the data from complex excel uploads in a vertical format. It is possible to configure a different target, so long as it has the same attributes, in the [MPE_XLMAP_INFO](/tables/mpe_xlmap_info.md) table.
![Screenshot](../img/mpe_xlmap_data.png) ![Screenshot](../img/mpe_xlmap_data.png)
See also: See also:
* [Excel Uploads](/excel.md) * [Excel Uploads](/excel.md)
* [MPE_XLMAP_INFO](tables/mpe_xlmap_info.md) * [MPE_XLMAP_INFO](/tables/mpe_xlmap_info.md)
* [MPE_XLMAP_RULES](tables/mpe_xlmap_rules.md) * [MPE_XLMAP_RULES](/tables/mpe_xlmap_rules.md)
!!!note !!!note
This table cannot be edited directly! Any target tables configured in [MPE_XLMAP_INFO](tables/mpe_xlmap_info.md) are also removed from the Tables list in the LOAD menu. This table cannot be edited directly! Any target tables configured in [MPE_XLMAP_INFO](/tables/mpe_xlmap_info.md) are also removed from the Tables list in the LOAD menu.
## Columns ## Columns

6
docs/tables/mpe_xlmap_info.md
View File

@ -17,8 +17,8 @@ The information is optional (unless you wish to configure a non-default target t
See also: See also:
* [Excel Uploads](/excel.md) * [Excel Uploads](/excel.md)
* [MPE_XLMAP_DATA](tables/mpe_xlmap_data.md) * [MPE_XLMAP_DATA](/tables/mpe_xlmap_data.md)
* [MPE_XLMAP_RULES](tables/mpe_xlmap_rules.md) * [MPE_XLMAP_RULES](/tables/mpe_xlmap_rules.md)
@ -29,4 +29,4 @@ See also:
- 🔑 `TX_TO num`: SCD2 close datetime - 🔑 `TX_TO num`: SCD2 close datetime
- 🔑 `XLMAP_ID char(32)`: A unique, UPPERCASE reference for the excel map. - 🔑 `XLMAP_ID char(32)`: A unique, UPPERCASE reference for the excel map.
- `XLMAP_DESCRIPTION char(1000)`: Map Description - `XLMAP_DESCRIPTION char(1000)`: Map Description
- `XLMAP_TARGETLIBDS char(41)`: An alternative target table to which to upload the data. This MUST have the same structure as the [MPE_XLMAP_DATA](tables/mpe_xlmap_data.md) table. The table must also be configured in [MPE_TABLES](tables/mpe_tables.md). - `XLMAP_TARGETLIBDS char(41)`: An alternative target table to which to upload the data. This MUST have the same structure as the [MPE_XLMAP_DATA](/tables/mpe_xlmap_data.md) table. The table must also be configured in [MPE_TABLES](/tables/mpe_tables.md).

5
docs/tables/mpe_xlmap_rules.md
View File

@ -12,6 +12,11 @@ The MPE_XLMAP_RULES table contains the rules for mapping excel cells and ranges
![Screenshot](../img/mpe_xlmap_rules.png) ![Screenshot](../img/mpe_xlmap_rules.png)
See also:
* [Excel Uploads](/excel.md)
* [MPE_XLMAP_DATA](/tables/mpe_xlmap_data.md)
* [MPE_XLMAP_INFO](/tables/mpe_xlmap_info.md)
## Columns ## Columns