|
|
|
|
@ -9,7 +9,7 @@ og_image: https://docs.datacontroller.io/img/dci_deploymentdiagramviya.png
|
|
|
|
|
|
|
|
|
|
## Overview
|
|
|
|
|
Data Controller for SAS Viya consists of a frontend, a set of Job Execution Services, a staging area, a Compute Context, and a database library. The library can be a SAS Base engine if desired, however this can cause contention (eg table locks) if end users are able to connect to the datasets directly, eg via Enterprise Guide or Base SAS.
|
|
|
|
|
A database that supports concurrent access is highly recommended.
|
|
|
|
|
A database that supports concurrent access is recommended.
|
|
|
|
|
|
|
|
|
|
## Prerequisites
|
|
|
|
|
|
|
|
|
|
@ -37,29 +37,28 @@ The below areas of the SAS Viya platform are modified when deploying Data Contro
|
|
|
|
|
|
|
|
|
|
<img src="/img/dci_deploymentdiagramviya.svg" height="350" style="border:3px solid black" >
|
|
|
|
|
|
|
|
|
|
!!! note
|
|
|
|
|
The "streaming" version of Viya uses the files API for web content, so there is no need for the web server component.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Deployment
|
|
|
|
|
Data Controller deployment is split between 3 deployment types:
|
|
|
|
|
Data Controller deployment is split between 2 deployment types:
|
|
|
|
|
|
|
|
|
|
* Demo version
|
|
|
|
|
* Full Version (manual deploy)
|
|
|
|
|
* Full Version (automated deploy)
|
|
|
|
|
* Streaming (web content served from SAS Drive)
|
|
|
|
|
* Full (web content served from dedicated web server)
|
|
|
|
|
|
|
|
|
|
<!--
|
|
|
|
|
## Full Version - Manual Deploy
|
|
|
|
|
-->
|
|
|
|
|
|
|
|
|
|
There are several parts to this proces:
|
|
|
|
|
|
|
|
|
|
1. Create the Compute Context
|
|
|
|
|
2. Deploy Frontend
|
|
|
|
|
4. Prepare the database and update settings (optional)
|
|
|
|
|
5. Update the Compute Context autoexec
|
|
|
|
|
2. Deploy Frontend (Full deploy only)
|
|
|
|
|
3. Deploy Services
|
|
|
|
|
4. First Run Configuration
|
|
|
|
|
5. Prepare the database and update settings (optional)
|
|
|
|
|
6. Update the Compute Context autoexec
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Create Compute Context
|
|
|
|
|
|
|
|
|
|
The Viya Compute context is used to spawn the Job Execution Services - such that those services may run under the specified system account, with a particular autoexec.
|
|
|
|
|
### 1. Create Shared Compute Context
|
|
|
|
|
|
|
|
|
|
We strongly recommend a dedicated compute context for running Data Controller. The setup requires an Administrator account.
|
|
|
|
|
|
|
|
|
|
@ -74,21 +73,60 @@ We strongly recommend a dedicated compute context for running Data Controller.
|
|
|
|
|
|
|
|
|
|
!!! note
|
|
|
|
|
XCMD is NOT required to use Data Controller.
|
|
|
|
|
### Deploy frontend
|
|
|
|
|
|
|
|
|
|
### 2. Deploy Frontend (Full Deploy only)
|
|
|
|
|
|
|
|
|
|
Unzip the frontend into your chosen directory (eg `/var/www/html/DataController`) on the SAS Web Server. Open `index.html` and update the following inside `dcAdapterSettings`:
|
|
|
|
|
|
|
|
|
|
- `appLoc` - this should point to the root folder on SAS Drive where you would like the Job Execution services to be created. This folder should initially, NOT exist (if it is found, the backend will not be deployed)
|
|
|
|
|
- `contextName` - here you should put the name of the compute context you created in the previous step.
|
|
|
|
|
- `dcPath` - the physical location on the filesystem to be used for staged data. This is only used at deployment time, it can be configured later in `$(appLoc)/services/settings.sas` or in the autoexec if used.
|
|
|
|
|
- `adminGroup` - the name of an existing group, which should have unrestricted access to Data Controller. This is only used at deployment time, it can be configured later in `$(appLoc)/services/settings.sas` or in the autoexec if used.
|
|
|
|
|
- `servertype` - should be SASVIYA
|
|
|
|
|
- `debug` - can stay as `false` for performance, but could be switched to `true` for debugging startup issues
|
|
|
|
|
- `useComputeApi` - use `true` for best performance.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Now, open https://YOURSERVER/DataController (using whichever subfolder you deployed to above) using an account that has the SAS privileges to write to the `appLoc` location.
|
|
|
|
|
### 3. Deploy Services
|
|
|
|
|
|
|
|
|
|
Services are deployed by running a SAS program.
|
|
|
|
|
|
|
|
|
|
#### Deploy Services: Streaming Deploy
|
|
|
|
|
|
|
|
|
|
Run the following in SAS Studio:
|
|
|
|
|
|
|
|
|
|
```sas
|
|
|
|
|
%let apploc=/Public/DataController; /* desired SAS Drive location */
|
|
|
|
|
filename dc url "https://git.datacontroller.io/dc/dc/releases/download/latest/viya.sas";
|
|
|
|
|
%inc dc;
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
#### Deploy Services: Full Deploy
|
|
|
|
|
|
|
|
|
|
Run the following in SAS Studio:
|
|
|
|
|
|
|
|
|
|
```sas
|
|
|
|
|
%let apploc=/Public/DataController; /* Per configuration in Step 2 above */
|
|
|
|
|
filename dc url "https://git.datacontroller.io/dc/dc/releases/download/latest/viya_noweb.sas";
|
|
|
|
|
%inc dc;
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
### 4. First Run Configuration
|
|
|
|
|
|
|
|
|
|
Now the services are deployed (including the service which creates the staging area) we can open the Data Controller web interface and make the necessary configurations:
|
|
|
|
|
|
|
|
|
|
#### First Run Configuration: Streaming Deploy
|
|
|
|
|
|
|
|
|
|
At the end of the SAS log from Step 3, there will be a link (`YOURSAS.SERVER/SASJobExecution?_file=/YOUR/APPLOC/services/DC.html`). Open this to perform the configuration - such as:
|
|
|
|
|
|
|
|
|
|
* dcpath - physical path for depleyment
|
|
|
|
|
* Admin Group - will have full access to DC
|
|
|
|
|
* Compute Context - should be a shared compute for a multi-user deployment
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#### First Run Configuration: Full Deploy
|
|
|
|
|
|
|
|
|
|
Open https://YOURSERVER/DataController (using whichever subfolder you deployed to above) using an account that has the SAS privileges to write to the `appLoc` location.
|
|
|
|
|
|
|
|
|
|
You will be presented with a deployment screen like the one below. Be sure to check the "Recreate Database" option and then click the "Deploy" button.
|
|
|
|
|
|
|
|
|
|
@ -96,9 +134,8 @@ You will be presented with a deployment screen like the one below. Be sure to c
|
|
|
|
|
|
|
|
|
|
Your services are deployed! And the app is operational, albeit still a little sluggish, as every single request is using the APIs to fetch the content of the `$(appLoc)/services/settings.sas` file.
|
|
|
|
|
|
|
|
|
|
To improve responsiveness by another 700ms we recommend you follow the steps in [Update Compute Context Autoexec](/dci-deploysasviya/#update-compute-context-autoexec) below.
|
|
|
|
|
|
|
|
|
|
### Deploy Database
|
|
|
|
|
### 5. Deploy Database
|
|
|
|
|
If you have a lot of users, such that concurrency (locked datasets) becomes an issue, you might consider migrating the control library to a database.
|
|
|
|
|
|
|
|
|
|
The first part to this is generating the DDL (and inserts). For this, use the DDL exporter as described [here](/admin-services/#export-database). If you need a flavour of DDL that is not yet supported, [contact us](https://datacontroller.io/contact/).
|
|
|
|
|
@ -107,7 +144,9 @@ Step 2 is simply to run this DDL in your preferred database.
|
|
|
|
|
|
|
|
|
|
Step 3 is to update the library definition in the `$(appLoc)/services/settings.sas` file using SAS Studio.
|
|
|
|
|
|
|
|
|
|
### Update Compute Context Autoexec
|
|
|
|
|
### 6.Update Compute Context Autoexec
|
|
|
|
|
|
|
|
|
|
To improve responsiveness by another 700ms we recommend you follow these steps:
|
|
|
|
|
|
|
|
|
|
First, open the `$(appLoc)/services/settings.sas` file in SAS Studio, and copy the code.
|
|
|
|
|
|
|
|
|
|
@ -132,54 +171,3 @@ To explain each of these lines:
|
|
|
|
|
* The final libname statement can also be configured to point at a database instead of a BASE engine directory (contact us for DDL)
|
|
|
|
|
|
|
|
|
|
If you have additional libraries that you would like to use in Data Controller, they should also be defined here.
|
|
|
|
|
<!--
|
|
|
|
|
## Full Version - Automated Deploy
|
|
|
|
|
|
|
|
|
|
The automated deploy makes use of the SASjs CLI to create the dependent context and job execution services. In addition to the standard prerequisites (a registered viya system account and a prepared database) you will also need:
|
|
|
|
|
|
|
|
|
|
* a local copy of the [SASjs CLI](https://sasjs.io/sasjs-cli/#installation)
|
|
|
|
|
* a Client / Secret - with an administrator group in SCOPE, and an authorization_code GRANT_TYPE. The SASjs [Viya Token Generator](https://github.com/sasjs/viyatoken) may help with this.
|
|
|
|
|
|
|
|
|
|
### Prepare the Target and Token
|
|
|
|
|
|
|
|
|
|
To configure this part (one time, manual step), we need to run a single command:
|
|
|
|
|
```
|
|
|
|
|
sasjs add
|
|
|
|
|
```
|
|
|
|
|
A sequence of command line prompts will follow for defining the target. These prompts are described [here](https://sasjs.io/sasjs-cli-add/). Note that `appLoc` is the SAS Drive location in which the Data Controller jobs will be deployed.
|
|
|
|
|
|
|
|
|
|
### Prepare the Context JSON
|
|
|
|
|
This file describes the context that the CI/CD process will generate. Save this file, eg as `myContext.json`.
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
{
|
|
|
|
|
"name": "DataControllerContext",
|
|
|
|
|
"attributes": {
|
|
|
|
|
"reuseServerProcesses": true,
|
|
|
|
|
"runServerAs": "mycasaccount"
|
|
|
|
|
},
|
|
|
|
|
"environment": {
|
|
|
|
|
"autoExecLines": [
|
|
|
|
|
"%let DC_LIBREF=DCDBVIYA;",
|
|
|
|
|
"%let DC_ADMIN_GROUP={{YOUR DC ADMIN GROUP}};",
|
|
|
|
|
"%let DC_STAGING_AREA={{YOUR DEDICATED FILE SYSTEM DRIVE}};",
|
|
|
|
|
"libname &dc_libref {{YOUR DC DATABASE}};",
|
|
|
|
|
],
|
|
|
|
|
"options": []
|
|
|
|
|
},
|
|
|
|
|
"launchContext": {
|
|
|
|
|
"contextName": "SAS Job Execution launcher context"
|
|
|
|
|
},
|
|
|
|
|
"launchType": "service",
|
|
|
|
|
}
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
### Prepare Deployment Script
|
|
|
|
|
|
|
|
|
|
The deployment script will run on a build server (or local desktop) and execute as follows:
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
# Create the SAS Viya Target
|
|
|
|
|
sasjs context create --source myContext.json --target myTarget
|
|
|
|
|
```
|
|
|
|
|
-->
|
