Merge pull request 'Update' (#10) from dcio_refactor into main

Reviewed-on: #10

.gitea/workflows/publish.yml

@@ -26,4 +26,4 @@ jobs:
 
       - name: Publish
         run: |
-          surfer put --token ${{ secrets.SURFER_TOKEN }} --server  website.datacontroller.io public/* /
+          surfer put --token ${{ secrets.SURFER_TOKEN }} --server  datacontroller.io public/* /

.npmrc

@@ -0,0 +1 @@
+legacy-peer-deps=true

.nvmrc

@@ -1 +1 @@
-v14.17.0
+v20.19.6

content/blog/2024-data-controller-for-sas/index.md

@@ -0,0 +1,70 @@
+---
+title: Data Controller - 2024 Recap
+description: A recap of the features delivered in Data Controller for SAS over the 24 releases of 2024
+date: '2024-12-31 12:00:00'
+author: 'Allan Bowe'
+authorLink: https://www.linkedin.com/in/allanbowe/
+previewImg: './darkmode.png'
+tags:
+  - Data Controller
+  - SAS
+  - Excel
+---
+
+Today marks the last day of 2024 and another period of growth for Data Controller for SAS. We made **24** [releases](https://git.datacontroller.io/dc/dc/releases) over the last 12 months, moving from [v6.3.1](https://git.datacontroller.io/dc/dc/releases/tag/v6.3.1) to [v6.12.1](https://git.datacontroller.io/dc/dc/releases/tag/v6.12.1).  We also on-boarded three new clients - two banks,  and an insurance company.  Every single one of our existing customers renewed.  And some even invested in some new features.  In this article we'll go through the major additions made during 2024.
+
+## Complex Excel Uploads
+
+A Basel III regulatory workstream necessitated ingestion of a series of Quantitative Tables as well as data from other workbooks, that were particularly large and packed with data fields situated in dynamic locations within the workbook.  
+
+To meet the requirements (and more) we built a feature to allow the ingestion of multiple ranges, from multiple worksheets, from within an arbitrary workbook.  The target range can be defined using either a fixed (A1) or dynamic (R1C1) notation.  The sheets themselves may be referenced by name, or position. 
+
+By way of example, consider the ingestion of the yellow squares below:
+
+![](./xlmap_example.png)
+
+One potential configuration (of many) to upload the above data would be as follows (extract from [MPE_XLMAP_RULES](/tables/mpe_xlmap_rules) table):
+
+
+![](./complexloads.png)
+
+
+The following video illustrates:
+
+<iframe title="Complex Excel Uploads" width="560" height="315" src="https://vid.4gl.io/videos/embed/3338f448-e92d-4822-b3ec-7f6d7530dfc8" frameborder="0" allowfullscreen="" sandbox="allow-same-origin allow-scripts allow-popups allow-forms"></iframe>
+
+
+## Filter By Variable
+
+Since [v6.5](https://git.datacontroller.io/dc/dc/releases/tag/v6.5.0) it is now possible to use variables instead of _values_ on the right hand side of a query expression.  This provides more flexibility in creating filters.
+
+![](./byvar.png)
+
+## Data Restore
+
+Since adding the [audit](https://docs.datacontroller.io/tables/mpe_audit/) data last year, it is now possible (since [v6.8](https://git.datacontroller.io/dc/dc/releases/tag/v6.8.0)) to [restore](https://docs.datacontroller.io/restore/) a table to a particular previous version!
+
+This works by navigating first to the submission of the version to be restored, and hitting the red REVERT button.  This will make a NEW submission with the changes that need to be made to revert the values to the previous state (which needs to be approved in the usual fashion).
+
+![](https://docs.datacontroller.io/img/restore.png)
+
+## Accessibility Updates
+
+It's a hard requirement that Data Controller provides the necessary accessibility adjustments (eg Dark Mode, keyboard shortcuts, contrast / font size settings etc) to accomodate users with particular needs (eg section 508 and EAA).  Thanks to our HandsOnTable licence, the DC data grid will even support for screen readers, such as NVDA, Jaws (Windows) and VoiceOver (MacOS).
+
+![](https://git.datacontroller.io/dc/dc/attachments/09bf644e-63bb-412a-aa50-1617ae08ab2c)
+
+<iframe title="Data Controller Accessibility" width="560" height="315" src="https://vid.4gl.io/videos/embed/b3d7e820-249f-4bc2-9e1a-473ce90a7a96" frameborder="0" allowfullscreen="" sandbox="allow-same-origin allow-scripts allow-popups allow-forms"></iframe>
+
+## Multiload Feature
+
+For a long time we have supported the load of a single (rectangular) table from anywhere in a workbook.  Well, as of [v6.11](https://git.datacontroller.io/dc/dc/compare/v6.10.1...v6.11.0) you can now load _multiple_ tables from a single workbook, significantly reducing the effort of manual uploads.  This piece of work also resulted in a big re-write of our excel extraction logic, which is now 20x faster than before.
+
+![](https://git.datacontroller.io/dc/dc/attachments/ebd0b47f-d912-4897-98cf-276a56dda106)
+
+
+------------
+
+We would like to thank all of our customers for their enthusiasm for and support with our product.  Our goal is to make it secure, simple, and fast to load ad-hoc data into SAS.
+
+If you'd like to kick the tyres, [reach out](https://datacontroller.io/contact/), or just go ahead and perform an [installation](https://docs.datacontroller.io/dci-requirements/) - the base product is completely free to use, with [premium features](https://datacontroller.io/pricing/) / support plan available on request.

content/blog/afa-insurance/index.md

@@ -0,0 +1,87 @@
+---
+title: AFA Insurance and Data Controller for SAS®
+description: Data Controller for SAS helps the DWh team at AFA försäkring with their steering table management
+date: '2025-02-25 09:00:00'
+author: 'Allan Bowe'
+authorLink: https://www.linkedin.com/in/allanbowe/
+previewImg: './afa_logo.png'
+tags:
+  - Use Cases
+---
+
+
+AFA försäkring is a well-known insurance provider in Sweden with a focus on collective agreements and occupational insurance.  
+
+AFA has leveraged SAS for data management and analytics for many years, and recently integrated Data Controller for SAS to enhance controls and audit reporting for a number of business processes.
+
+We caught up with [Fredrik Englund](https://www.linkedin.com/in/fredrik-englund-7402031/) (FE) and [Henrik Forsell](https://www.linkedin.com/in/henrikforsell/) (HF) to learn more about their usage of this toolset.
+
+| ![Fredrik Englund](./fredrik.avif)| 
+|:--:| 
+| *Fredrik Englund (FE)* |
+
+
+### Can you tell us a bit about yourself?
+(FE) We are the developers of our DWh, a group of around 10 people.  My role is SAS Developer and System Architect.  
+
+### What do you use Data Controller for?
+Mainly for steering (mapping, control) tables.  We use it in our handling system for insurance agents – we have a signal system, such that when an agent finishes a task, there is a signal to move forward to the next step in the process.  We map the signals to different process steps and use that in our reports to follow up.  The mapping of signals to process steps are made by end users themselves, using Data Controller, and we make the approvals directly in production without needing fixed programs or anything else.  This is a great advantage for us. 
+
+We have batches running and checking if there are new signals, and we have set up Data Controller with a filter so that end users get only the new signals in the interface, and can just do the mapping and click OK. 
+
+So it is very easy for us now to do that process!
+
+### Why are you using Data Controller, instead of another approach or tool?
+We heard about Data Controller from the community, and it seemed like a great fit for our use case - it’s SAS friendly, it’s built on SAS, and it’s easy to incorporate.  
+
+The things we really like are the EDIT and APPROVAL process, and the HISTORY of all changes that is built into the product.
+
+### Is anyone else at AFA using Data Controller?
+Not right now but we plan to use it more.  We have people in the process development team who need to change targets once a year or maybe more often.  Right now they send us the new targets and we add them in the programs / update the Visual Analytics reports, but we plan to use steering / control tables for them, so they can update targets themselves in Data Controller.  Then they can request to change them when they like, and we have control over the timing of when the changes are applied. 
+
+We also have some percentages (relating to payments) with levels that change during the year, and those users should also be able to change the limits themselves in the Data Controller interface.
+
+| ![Henrik Forsell](./henrik.jpeg)| 
+|:--:| 
+| *Henrik Forsell (HF)* |
+
+ 
+(HF)  This takes us completely out of the equation – we don’t have to do the changes, it’s all on them, they own it.
+
+(FE)  Yes – their data, their controls.  We can leave it to them from the development environment, so they are not dependent on us for any changes, so it’s really good.
+
+(HF)  So the users can own the data, all of these things, they can change it themselves, we don’t need to do anything once we set it up.
+
+### How much time does this save you, say, on a monthly basis?
+(FE)  It’s hard to say how much time/money it saves – it’s more about the quality, and the control, and the history.  If they put in wrong mappings, targets, or levels – how much will it cost us to reverse the the batches, it’s very hard to say
+
+(HF)  It’s also about the fact that even if it’s only one number in one report, if we have to do the changes, we’d almost have to setup a small project; as in, a task that has to be checked, added into our test environment, run tests, ok it, and move on into production.  So even though it’s a small thing that changed, the process around it takes a lot of work.  One change might be 3-4 hours in total.  But now with Data Controller, they can change it twice a day if they want, it’s up to them, it doesn’t affect our work at all.
+
+(FE)  If we compare it to the old solution, they did the changes directly in the base tables – in production – so if they added the wrong values it would crash.  Which happened, a few times.  So it’s hard to say but you can spend like 10 hours on finding the error, or maybe more.
+
+(HF)  Or having to do a rollback, and then you involve even more people
+
+(FE)  It’s much more about quality.  It can save us 2-10 hours a month, hard to say.
+
+### What are your favourite features of DC?
+(FE)  The UI is simple to understand (in a good way).  It’s very responsive, very quick.  For us the APPROVAL part is the big thing, together with the history of all changes.  It’s really good to see who changes what, and when.  We have a few people who can do the approval, and they can check very thoroughly that it’s correct before they approve it.  So that’s actually the big one.
+
+Then another good / big thing is the [dropdowns](https://docs.datacontroller.io/dynamic-cell-dropdown/), such that you can provide only values that are valid.  And you can do [HOOKS](https://docs.datacontroller.io/dcc-tables/#pre_edit_hook) to check the data, both after and before.  
+
+Another was the [data lineage](https://docs.datacontroller.io/dcu-lineage/), which we haven’t had before.  Now we can have analytics users check the lineage.  We didn’t expect that but it was a nice thing to have.
+
+### Is there anything you’d like to add?
+(HF)  We love the response time from the developers. 
+
+(FE)  The support is fabulous.  If we can have this from all of our vendors it would be a perfect world.
+
+(HF)  And it is very important.  When we found something – no words – you are there.  That helps a lot.
+
+(FE)  Also, we haven’t tried out all the [excel import](https://docs.datacontroller.io/excel/) features yet but we’d like to try that in some way, as there is a lot of data that exists in excel, and that can be a way for business users to import that data, and check it, and improve it.  So - we have plans.
+
+--- 
+The team behind Data Controller would like to say "STORT TACK" to Fredrik and Henrik for their kind words! 
+
+---
+
+The previous article in this series is available [here](/allianz-insurance-data-controller-sas/).

content/blog/data-controller-developer-perspective/index.md

@@ -15,7 +15,7 @@ tags:
 
 <h1>What problem does Data Controller for SAS® solve?</h1>
 
-<div class="imgHolder alignright"><a href="https://www.linkedin.com/in/rgagor/"><img src="/wp-content/uploads/2020/07/IMG-20190430-WA0049.jpg" alt="Rafal Gagor - Veteran SAS Developer" width="180" height="180" /></a><div><span>Rafal Gagor - Veteran SAS Developer</span></div></div>
+<div class="imgHolder alignright"><a target="_blank" rel="noopener" href="https://www.linkedin.com/in/rgagor/"><img src="/wp-content/uploads/2020/07/IMG-20190430-WA0049.jpg" alt="Rafal Gagor - Veteran SAS Developer" width="180" height="180" /></a><div><span>Rafal Gagor - Veteran SAS Developer</span></div></div>
 
 It's a question we get asked a lot, and so this is the first of a series of articles that explore real users and their actual use cases. We caught up with [Rafal Gagor](https://www.linkedin.com/in/rgagor/), a DI Developer with 2 decades of SAS and Financial Services experience, to get his impressions after using Data Controller for SAS on a client project.
 

content/blog/data-quality-and-the-nbb_2017_27-circular/index.md

@@ -16,9 +16,9 @@ tags:
   - Solvency II
 ---
 
-When applying financial regulations in the EU (such as Solvency II, Basel III or GDPR) it is common for Member States to maintain or introduce national provisions to further specify how such rules might be applied. The National Bank of Belgium (NBB) is no stranger to this, and releases a steady stream of circulars via their <a href="https://www.nbb.be/en/financial-oversight/general/news/circulars-and-communications">website</a>. The <a href="https://www.nbb.be/doc/cp/eng/2017/20171012_nbb_2017_27.pdf">circular</a> of 12th October 2017 (NBB_2017_27, Jan Smets) is particularly interesting as it lays out a number of concrete recommendations for Belgian financial institutions with regard to Data Quality - and stated that these should be applied to internal reporting processes as well as the prudential data submitted. This fact is well known by affected industry participants, who have already performed a self assessment for YE2017 and reviewed documentation expectations as part of the HY2018 submission. <h2>Quality of External Data</h2> The DQ requirements for reporting are described by the 6 <a href="https://www.nbb.be/doc/cp/eng/2017/20171012_nbb_2017_27_annex.pdf">dimensions</a> (Accuracy, Reliability, Completeness, Consistency, Plausibility, Timeliness), as well as the Data Quality Framework described by Patrick Hogan <a href="https://www.bankingsupervision.europa.eu/press/conferences/sup_rep_conf/shared/pdf/Item4_1_PatrickHogan.pdf">here</a> and <a href="https://www.bankingsupervision.europa.eu/press/conferences/sup_rep_conf/shared/pdf/2017/Data_quality_framework_tools_and_products.pdf">here</a>. There are a number of 'hard checks' implemented in OneGate as part of the XBRL submissions, which are kept up to date <a href="http://www.eba.europa.eu/risk-analysis-and-data/reporting-frameworks">here</a>. However, OneGate cannot be used as a validation tool - the regulators will be monitoring the <strong>reliability</strong> of submissions by comparing the magnitude of change between resubmissions! Not to mention the data <strong>plausibility</strong> (changes in submitted values over time). <h2>Data Quality Culture</h2> When it comes to internal processes, CRO's across Belgium must now demonstrate to accredited statutory auditors that they satisfy the 3 Principles of the circular (Governance, Technical Capacities, Process). A long list of action points are detailed - it's clear that a <em>lot</em> of documentation will be required to fulfil these obligations! And not only that - the documentation will need to be continually updated and maintained. It's fair to say that automated solutions have the potential to provide significant time &amp; cost savings in this regard. <h2>Data Controller for SAS®</h2> The Data Controller is a web based solution for capturing data from users. Data Quality is applied at source, changes are routed through an approval process before being applied, and all updates are captured for subsequent audit. The tool provides evidence of compliance with NBB_2017_27 in the following ways: <h4>Separation of Roles for Data Preparation and Validation (principle 1.2)</h4> Data Controller differentiates between Editors (who provide the data) and Approvers (who sign it off). Editors stage data via the web interface, or by direct file upload. Approvers are then shown the new, changed, or deleted records - and can accept or reject the update. <a href="/wp-content/uploads/2018/10/Screen-Shot-2018-10-13-at-22.50.56.png"><img class="aligncenter wp-image-962" src="/wp-content/uploads/2018/10/Screen-Shot-2018-10-13-at-22.50.56.png" alt="" width="553" height="296" /></a>
+When applying financial regulations in the EU (such as Solvency II, Basel III or GDPR) it is common for Member States to maintain or introduce national provisions to further specify how such rules might be applied. The National Bank of Belgium (NBB) is no stranger to this, and releases a steady stream of circulars via their <a target="_blank" rel="noopener" href="https://www.nbb.be/en/financial-oversight/general/news/circulars-and-communications">website</a>. The <a target="_blank" rel="noopener" href="https://www.nbb.be/doc/cp/eng/2017/20171012_nbb_2017_27.pdf">circular</a> of 12th October 2017 (NBB_2017_27, Jan Smets) is particularly interesting as it lays out a number of concrete recommendations for Belgian financial institutions with regard to Data Quality - and stated that these should be applied to internal reporting processes as well as the prudential data submitted. This fact is well known by affected industry participants, who have already performed a self assessment for YE2017 and reviewed documentation expectations as part of the HY2018 submission. <h2>Quality of External Data</h2> The DQ requirements for reporting are described by the 6 <a target="_blank" rel="noopener" href="https://www.nbb.be/doc/cp/eng/2017/20171012_nbb_2017_27_annex.pdf">dimensions</a> (Accuracy, Reliability, Completeness, Consistency, Plausibility, Timeliness), as well as the Data Quality Framework described by Patrick Hogan <a target="_blank" rel="noopener" href="https://www.bankingsupervision.europa.eu/press/conferences/sup_rep_conf/shared/pdf/Item4_1_PatrickHogan.pdf">here</a> and <a target="_blank" rel="noopener" href="https://www.bankingsupervision.europa.eu/press/conferences/sup_rep_conf/shared/pdf/2017/Data_quality_framework_tools_and_products.pdf">here</a>. There are a number of 'hard checks' implemented in OneGate as part of the XBRL submissions, which are kept up to date <a target="_blank" rel="noopener" href="http://www.eba.europa.eu/risk-analysis-and-data/reporting-frameworks">here</a>. However, OneGate cannot be used as a validation tool - the regulators will be monitoring the <strong>reliability</strong> of submissions by comparing the magnitude of change between resubmissions! Not to mention the data <strong>plausibility</strong> (changes in submitted values over time). <h2>Data Quality Culture</h2> When it comes to internal processes, CRO's across Belgium must now demonstrate to accredited statutory auditors that they satisfy the 3 Principles of the circular (Governance, Technical Capacities, Process). A long list of action points are detailed - it's clear that a <em>lot</em> of documentation will be required to fulfil these obligations! And not only that - the documentation will need to be continually updated and maintained. It's fair to say that automated solutions have the potential to provide significant time &amp; cost savings in this regard. <h2>Data Controller for SAS®</h2> The Data Controller is a web based solution for capturing data from users. Data Quality is applied at source, changes are routed through an approval process before being applied, and all updates are captured for subsequent audit. The tool provides evidence of compliance with NBB_2017_27 in the following ways: <h4>Separation of Roles for Data Preparation and Validation (principle 1.2)</h4> Data Controller differentiates between Editors (who provide the data) and Approvers (who sign it off). Editors stage data via the web interface, or by direct file upload. Approvers are then shown the new, changed, or deleted records - and can accept or reject the update. <a href="/wp-content/uploads/2018/10/Screen-Shot-2018-10-13-at-22.50.56.png"><img class="aligncenter wp-image-962" src="/wp-content/uploads/2018/10/Screen-Shot-2018-10-13-at-22.50.56.png" alt="" width="553" height="296" /></a>
 
 <h4>Capacities established should ensure compliance in times of stress (principle 2.1)</h4>
 As an Enterprise tool, the Data Controller is as scalable and resilient as your existing SAS platform.
 
-<h4>Capture of Errors and Inconsistencies (principle 2.2)</h4> Data Controller has a number of features to ensure timely detection of Data Quality issues at source (such as cell validation, post edit hook scripts, duplicate removals, rejection of data with missing columns, etc etc). Where errors do make it into the system, a full history is kept (logs, copies of files etc) for all uploads and approvals. Emails of such errors can be configured for follow up. <h4>Tools and Techniques for Information Management Should be Automated (principle 2.3)</h4> The Data Controller can be configured to execute specific .sas programs after data validation. This enables the development of a secure and <em>integrated</em> workflow, and helps companies to avoid the additional documentation penalties associated with "miscellaneous unconnected computer applications" and manual information processing. <a href="/wp-content/uploads/2018/10/Screen-Shot-2018-10-13-at-22.53.38.png"><img class="aligncenter wp-image-963" src="/wp-content/uploads/2018/10/Screen-Shot-2018-10-13-at-22.53.38.png" alt="" width="278" height="128" /></a> &nbsp; <h4>Periodic Review &amp; Improvements (principles 2.4 and 3.4)</h4> The Data Controller is actively maintained with the specific aim to reduce the cost of compliance with regulations such as NBB_2017_27. Our <a href="https://slides.com/allanbowe/datacontroller/#/">roadmap</a> includes new features such as pre-canned reports, version 'signoff', and the ability to reinstate previous versions of data. <h4>A process for correction and final validation of reporting before submission (3.1)</h4> As a primary and dedicated tool for data corrections, Data Controller can be described once and used everywhere. <h4>List of Divisions Involved in Preparing Tables (principle 3.2)</h4> By using the Data Controller in combination with knowledge of data lineage (eg from SAS metadata or manual lookup table) it becomes possible to produce an automated report to identify exactly who - and hence which division - was involved in both the preparation and the validation of the all source data per reporting table for each reporting cycle. <h4>Processes should integrate and document key controls (principle 3.3)</h4> Data Controller can be used as a staging point for verifying the quality of data, eg when data from one department must be passed to another department for processing. The user access policy will be as per the existing policy for your SAS environment. <h2>Summary</h2> Whilst the circular provides valuable clarity on the expectations of the NBB, there are significant costs involved to prepare for, and maintain, compliance with the guidance. This is especially the case where reporting processes are disparate, and make use of disconnected EUCs and manual processes. The Data Controller for SAS® addresses and automates a number of pain points as specifically described in the circular. It is a robust and easy-to-use tool, actively maintained and <a href="http://docs.datacontroller.io">documented</a>, and provides an integrated solution on a tried and trusted platform for data management. &nbsp;
+<h4>Capture of Errors and Inconsistencies (principle 2.2)</h4> Data Controller has a number of features to ensure timely detection of Data Quality issues at source (such as cell validation, post edit hook scripts, duplicate removals, rejection of data with missing columns, etc etc). Where errors do make it into the system, a full history is kept (logs, copies of files etc) for all uploads and approvals. Emails of such errors can be configured for follow up. <h4>Tools and Techniques for Information Management Should be Automated (principle 2.3)</h4> The Data Controller can be configured to execute specific .sas programs after data validation. This enables the development of a secure and <em>integrated</em> workflow, and helps companies to avoid the additional documentation penalties associated with "miscellaneous unconnected computer applications" and manual information processing. <a href="/wp-content/uploads/2018/10/Screen-Shot-2018-10-13-at-22.53.38.png"><img class="aligncenter wp-image-963" src="/wp-content/uploads/2018/10/Screen-Shot-2018-10-13-at-22.53.38.png" alt="" width="278" height="128" /></a> &nbsp; <h4>Periodic Review &amp; Improvements (principles 2.4 and 3.4)</h4> The Data Controller is actively maintained with the specific aim to reduce the cost of compliance with regulations such as NBB_2017_27. Our <a target="_blank" rel="noopener" href="https://slides.com/allanbowe/datacontroller/#/">roadmap</a> includes new features such as pre-canned reports, version 'signoff', and the ability to reinstate previous versions of data. <h4>A process for correction and final validation of reporting before submission (3.1)</h4> As a primary and dedicated tool for data corrections, Data Controller can be described once and used everywhere. <h4>List of Divisions Involved in Preparing Tables (principle 3.2)</h4> By using the Data Controller in combination with knowledge of data lineage (eg from SAS metadata or manual lookup table) it becomes possible to produce an automated report to identify exactly who - and hence which division - was involved in both the preparation and the validation of the all source data per reporting table for each reporting cycle. <h4>Processes should integrate and document key controls (principle 3.3)</h4> Data Controller can be used as a staging point for verifying the quality of data, eg when data from one department must be passed to another department for processing. The user access policy will be as per the existing policy for your SAS environment. <h2>Summary</h2> Whilst the circular provides valuable clarity on the expectations of the NBB, there are significant costs involved to prepare for, and maintain, compliance with the guidance. This is especially the case where reporting processes are disparate, and make use of disconnected EUCs and manual processes. The Data Controller for SAS® addresses and automates a number of pain points as specifically described in the circular. It is a robust and easy-to-use tool, actively maintained and <a target="_blank" rel="noopener" href="http://docs.datacontroller.io">documented</a>, and provides an integrated solution on a tried and trusted platform for data management. &nbsp;

content/blog/der-touristik/index.md

@@ -13,4 +13,4 @@ tags:
   - Use Cases
 ---
 
-We caught up with <a href="https://www.linkedin.com/in/herbert-gro%C3%9Fmann-53690517a/">Herbert Grossmann</a> of DER Touristik to understand how Data Controller for SAS is used within the BICC and the types of challenges it solves. <a href="https://www.linkedin.com/in/herbert-gro%C3%9Fmann-53690517a/"><img class=" wp-image-1137 alignright" src="/wp-content/uploads/2020/08/0-1.jpeg" alt="" width="183" height="183" /></a> The previous article in this series can be found <a href="/data-controller-developer-perspective/">here</a>. <h2>Guten Tag, Herby! Can you tell us about your role within DER Touristik?</h2> Yes, I am working here as project manager for BI and Analytics and my department is the BICC (Business Intelligence Competence Centre), and we have an absolute focus on the SAS technology stack - so that’s my daily business. <h2>Great. And, I understand you guys are using Data Controller for SAS. What do you use it for?</h2> Well, mainly for managing control tables, that we have a lot of nowadays, in the data warehouse. But we also implemented what we call an "early bird booking system". There we have used the Approval process within Data Controller, which is excellent, because users, business departments etc, can approve data that would normally only be accessible within the back-end warehouse itself. So now they have an interface, which limits their access to specific views, and this is very useful - it was also highly commended by our management. <h2>So, business users can approve modifications to secure warehouse tables without having direct write-access themselves?</h2> Exactly <h2>Fantastic. Next question. How does having Data Controller make your life easier?</h2> Well - there is the version control of course, that gives us a much better traceability of changes to see what was changed by whom, at what time. And we have the immediate constraint checking which is also very useful because some of the tables are sensitive towards, let’s say, the changes of the primary key. And in the past when we did it the "old fashioned way" it was possible that by mistake that someone could cause duplicate primary keys or stuff like that, so this is now not possible anymore, which is very good. And like the example that I mentioned before, that now we can grant access to certain sensitive tables even for business users that would normally have no access, but we can decide whether to give them at least the right to <em>view</em> these tables, or during special events <em>edit</em> tables, or approve edits of those tables. So this gives a lot of opportunities, and makes it much easier than it was in the past. <h2>Nice! And so, talking about the past, before you had Data Controller, how did you manage modifications to data in SAS?</h2> We classically used two approaches - on one hand using SAS Enterprise Guide to directly edit tables or do imports, such as imports of excel sheets for example. On the other hand, we have some batch processes that also do imports of Excel tables or CSV tables. So those were the classic and standard ways. And of course especially the batch one we are still using for some files, depending on the situation. But we do no editing of tables directly with Enterprise Guide anymore because it is much safer and easier to use the Data Controller. <h2>Understood. So on the Data Controller side, what would you say were your favourite features and why?</h2> I would say that I like the editor as a whole very much. I think that is great that in the moment you make a table editable, you can define the ways in which you would edit the tables. Like whether there is some historic logging or not, and the fact you can set the constraints. And in the editor then you have a lot of Data Quality opportunities such as defining drop-down lists for certain attributes, which really makes editing the tables easier and much more comfortable. It was a little bit of a pain in the past but now it’s almost fun. <h2>That's great feedback! Is there anything else, any comments you would like to add?</h2> Yes, I like the fact that Data Controller is really just a part of the SAS environment. It’s not a completely separate application that you have to install somewhere, but a kind of pluggable part of the SAS environment. I liked it very much because then you still have everything in your hands. I mean I am not a developer but my knowledge of SAS is already enough to match the criteria to be able to handle the Data Controller as whole, to even do the updates and/or to modify things. And also it’s easy to show others who have experience with SAS how the tool works and what is to be done when there are data issues. And yeah, I think that’s a big advantage. <img class="wp-image-1140 aligncenter" src="/wp-content/uploads/2020/08/Group-1dt-1-e1597092362693.png" alt="SAS DER Touristik" width="242" height="213" />
+We caught up with <a target="_blank" rel="noopener" href="https://www.linkedin.com/in/herbert-gro%C3%9Fmann-53690517a/">Herbert Grossmann</a> of DER Touristik to understand how Data Controller for SAS is used within the BICC and the types of challenges it solves. <a target="_blank" rel="noopener" href="https://www.linkedin.com/in/herbert-gro%C3%9Fmann-53690517a/"><img class=" wp-image-1137 alignright" src="/wp-content/uploads/2020/08/0-1.jpeg" alt="" width="183" height="183" /></a> The previous article in this series can be found <a href="/data-controller-developer-perspective/">here</a>. <h2>Guten Tag, Herby! Can you tell us about your role within DER Touristik?</h2> Yes, I am working here as project manager for BI and Analytics and my department is the BICC (Business Intelligence Competence Centre), and we have an absolute focus on the SAS technology stack - so that’s my daily business. <h2>Great. And, I understand you guys are using Data Controller for SAS. What do you use it for?</h2> Well, mainly for managing control tables, that we have a lot of nowadays, in the data warehouse. But we also implemented what we call an "early bird booking system". There we have used the Approval process within Data Controller, which is excellent, because users, business departments etc, can approve data that would normally only be accessible within the back-end warehouse itself. So now they have an interface, which limits their access to specific views, and this is very useful - it was also highly commended by our management. <h2>So, business users can approve modifications to secure warehouse tables without having direct write-access themselves?</h2> Exactly <h2>Fantastic. Next question. How does having Data Controller make your life easier?</h2> Well - there is the version control of course, that gives us a much better traceability of changes to see what was changed by whom, at what time. And we have the immediate constraint checking which is also very useful because some of the tables are sensitive towards, let’s say, the changes of the primary key. And in the past when we did it the "old fashioned way" it was possible that by mistake that someone could cause duplicate primary keys or stuff like that, so this is now not possible anymore, which is very good. And like the example that I mentioned before, that now we can grant access to certain sensitive tables even for business users that would normally have no access, but we can decide whether to give them at least the right to <em>view</em> these tables, or during special events <em>edit</em> tables, or approve edits of those tables. So this gives a lot of opportunities, and makes it much easier than it was in the past. <h2>Nice! And so, talking about the past, before you had Data Controller, how did you manage modifications to data in SAS?</h2> We classically used two approaches - on one hand using SAS Enterprise Guide to directly edit tables or do imports, such as imports of excel sheets for example. On the other hand, we have some batch processes that also do imports of Excel tables or CSV tables. So those were the classic and standard ways. And of course especially the batch one we are still using for some files, depending on the situation. But we do no editing of tables directly with Enterprise Guide anymore because it is much safer and easier to use the Data Controller. <h2>Understood. So on the Data Controller side, what would you say were your favourite features and why?</h2> I would say that I like the editor as a whole very much. I think that is great that in the moment you make a table editable, you can define the ways in which you would edit the tables. Like whether there is some historic logging or not, and the fact you can set the constraints. And in the editor then you have a lot of Data Quality opportunities such as defining drop-down lists for certain attributes, which really makes editing the tables easier and much more comfortable. It was a little bit of a pain in the past but now it’s almost fun. <h2>That's great feedback! Is there anything else, any comments you would like to add?</h2> Yes, I like the fact that Data Controller is really just a part of the SAS environment. It’s not a completely separate application that you have to install somewhere, but a kind of pluggable part of the SAS environment. I liked it very much because then you still have everything in your hands. I mean I am not a developer but my knowledge of SAS is already enough to match the criteria to be able to handle the Data Controller as whole, to even do the updates and/or to modify things. And also it’s easy to show others who have experience with SAS how the tool works and what is to be done when there are data issues. And yeah, I think that’s a big advantage. <img class="wp-image-1140 aligncenter" src="/wp-content/uploads/2020/08/Group-1dt-1-e1597092362693.png" alt="SAS DER Touristik" width="242" height="213" />

content/blog/euc-management-system/index.md

@@ -1,7 +1,7 @@
 ---
 title: EUC Management Systems need these 12 Attributes
 description: An EUC management system should automatically identify, clean, secure, backup, and integrate EUC data with full auditability, ownership, and approval.
-date: '2018-10-30 09:13:25'
+date: '2019-01-01 09:00:00'
 author: 'Allan Bowe'
 authorLink: https://www.linkedin.com/in/allanbowe/
 previewImg: './DC-UML-Activity-Diagram-2.png'
@@ -16,14 +16,16 @@ tags:
 ---
 
 End User Computing (EUC) applications are unavoidable - the challenge is not to erase them, but to embrace automated approaches to EUC management that will identify, clean, secure, backup, and integrate EUC data with full auditability, ownership, and approval.
+
 <h2>The Much-Maligned EUC</h2>
 EUC applications such as Excel, Access Databases, and locally executed programs, are often targeted as the source of a myriad of risks - such as financial misstatements, internal fraud, incorrect models, and potential for business process disruption. The rationale being that business developed / owned applications are not subject to the same access controls, development &amp; testing standards, documentation and release management processes as can be found over the "IT Fence". Whilst this is probably true, the inherent flexibility of EUCs that can be quickly updated without service desk requests, project codes, or lost arms &amp; legs - means that EUCs are, regardless, here to stay.
 
 The challenge is to find a way to shine a light onto this "Shadow IT", and provide a framework by which EUC data can be extracted in a simple, safe, secure, scalable, and auditable fashion. <a href="/wp-content/uploads/2018/10/DC-UML-Use-Case-Diagram-EUC.png"><img class="aligncenter size-large wp-image-1008" src="/wp-content/uploads/2018/10/DC-UML-Use-Case-Diagram-EUC.png" alt="EUC Use Case Diagram" /></a>
+
 <h2>EUCs can be Controlled</h2>
-The 'war on EUCs' cannot be won - it simply isn't practical to ban them, or to migrate / redevelop every closely held and highly complex legacy VBA application. Until alternative solutions for Citizen Developers to build Enterprise Apps (such as <a href="https://sasjs.io">SASjs</a>) become mainstream, simple measures / controls on the EUCs themselves must be implemented - such as version control, readonly attributes, embedded documentation, peer review etc. In the meantime, a management system for EUCs is the ideal place for capturing the requisite metadata needed to monitor, audit, and secure the data therein. Such a management system should have, as a minimum, the following attributes:
+The 'war on EUCs' cannot be won - it simply isn't practical to ban them, or to migrate / redevelop every closely held and highly complex legacy VBA application. Until alternative solutions for Citizen Developers to build Enterprise Apps (such as <a target="_blank" rel="noopener" href="https://sasjs.io">SASjs</a>) become mainstream, simple measures / controls on the EUCs themselves must be implemented - such as version control, readonly attributes, embedded documentation, peer review etc. In the meantime, a management system for EUCs is the ideal place for capturing the requisite metadata needed to monitor, audit, and secure the data therein. Such a management system should have, as a minimum, the following attributes:
 <h3>EUC Data Quality at Source</h3>
-The ability to run data quality routines at the point of data upload (from EUC to secure IT environment) provides instant feedback to EUC operators that will allow them to make corrections and avoid costly post-upload investigations, re-runs, or worse - incorrect results. As part of this process, it should be easy to create and update those Data Quality rules. A longer discussion of Data Quality can be found <a href="https://www.linkedin.com/pulse/zen-art-data-quality-allan-bowe/">here</a>.
+The ability to run data quality routines at the point of data upload (from EUC to secure IT environment) provides instant feedback to EUC operators that will allow them to make corrections and avoid costly post-upload investigations, re-runs, or worse - incorrect results. As part of this process, it should be easy to create and update those Data Quality rules. A longer discussion of Data Quality can be found <a target="_blank" rel="noopener" href="https://www.linkedin.com/pulse/zen-art-data-quality-allan-bowe/">here</a>.
 <h3>EUC Data Review (4 eyes)</h3>
 After EUC data is submitted, it should be reviewed before the target database is updated. It should be possible (but not mandatory) for this check to be performed by a different individual. When performing that check, it should only be necessary to review new / changed / deleted records. For changed records, the reviewer should also be able to see the original values. If the data is approved, the target table is updated. If rejected, the staged data can simply be archived.
 <h3>Roles &amp; Responsibilities (RACI)</h3>

content/blog/sarbanes-oxley/index.md

@@ -23,9 +23,9 @@ There are many aspects to full Sarbanes-Oxley (SOX) compliance, the [legislation
 
 Data Controller facilitates internal controls through a 4 eyes review & approve mechanism for data changes. This, combined with data validation and an integrated workflow feature, provides a mechanism to easily track and report on the number of internal controls (quality rules, signoffs, rejections), as well as the frequency they are applied, who is applying them, which data items the controls relate to, and who is performing them. Such metrics can be compared and contrasted with pre-existing and current quality measures to help determine control effectiveness. Variations in the number of submit / approve cycles between reporting teams, also provide objective and repeatable measurements to support the assessment of the effectiveness of internal controls.
 
-<div class="imgHolder"><a href="https://www.govinfo.gov/content/pkg/BILLS-107hr3763enr/pdf/BILLS-107hr3763enr.pdf"><img class="wp-image-1105 size-full aligncenter" title="Sec 404. (Sarbanes-Oxley)" src="/wp-content/uploads/2020/08/Screenshot-from-2020-08-07-17-57-01.png" alt="Sarbanes Oxley"/></a><caption>Sarbanes Oxley</caption></div>
+<div class="imgHolder"><a target="_blank" rel="noopener" href="https://www.govinfo.gov/content/pkg/BILLS-107hr3763enr/pdf/BILLS-107hr3763enr.pdf"><img class="wp-image-1105 size-full aligncenter" title="Sec 404. (Sarbanes-Oxley)" src="/wp-content/uploads/2020/08/Screenshot-from-2020-08-07-17-57-01.png" alt="Sarbanes Oxley"/></a><caption>Sarbanes Oxley</caption></div>
 
-&nbsp; Section 404 is widely considered the most onerous part of Sarbanes-Oxley, as the documentation and testing of all the controls requires significant time and effort. To address this, the <a href="https://pcaobus.org/">Public Company Accounting Oversight Board</a> (PCAOB - a US non-profit created by the Sarbanes-Oxley act itself) released<a href="https://pcaobus.org/Rulemaking/Docket%20021/2007-06-12_Release_No_2007-005A.pdf"> additional guidance</a> to assist management and auditors in producing their reports. This is officially labeled "Auditing Standard No. 5 - <em>An Audit of Internal Control Over Financial Reporting That Is Integrated with An Audit of Financial Statements"</em> A few points are highlighted by the guidance in this standard that are pertinent to users of Data Controller. <h2>PCAOB AS5 Sec24 - Controls Over Management Override</h2> Management Overrides (the freedom to simply "replace" reporting figures based on, presumably, sound judgement) are entity level controls that can be easily captured (in a centralised manner) by Data Controller. This in fact, is the "core functionality" of the tool. Data Stewards / Data Processors (Editors) make the change, then one or more Data Owners / Data Controllers (Approvers) sign it off before it is applied to the target table. A copy of the original excel file (if used) and a record of who made the change, when, what the change was, and why (if a reason is provided) is recorded. <a href="https://docs.datacontroller.io/dcc-validations/">Data Validation</a> rules can also be defined to ensure that inputs fit the desired pattern(s). <a href="https://pcaobus.org/Rulemaking/Docket%20021/2007-06-12_Release_No_2007-005A.pdf"><img class="aligncenter wp-image-1122" src="/wp-content/uploads/2020/08/Screenshot-from-2020-08-10-10-41-12.png" alt="Sarbanes Oxley sas management overrides" width="887" height="409" /></a> For fun, we made a short video for this part:
+&nbsp; Section 404 is widely considered the most onerous part of Sarbanes-Oxley, as the documentation and testing of all the controls requires significant time and effort. To address this, the <a target="_blank" rel="noopener" href="https://pcaobus.org/">Public Company Accounting Oversight Board</a> (PCAOB - a US non-profit created by the Sarbanes-Oxley act itself) released<a target="_blank" rel="noopener" href="https://pcaobus.org/Rulemaking/Docket%20021/2007-06-12_Release_No_2007-005A.pdf"> additional guidance</a> to assist management and auditors in producing their reports. This is officially labeled "Auditing Standard No. 5 - <em>An Audit of Internal Control Over Financial Reporting That Is Integrated with An Audit of Financial Statements"</em> A few points are highlighted by the guidance in this standard that are pertinent to users of Data Controller. <h2>PCAOB AS5 Sec24 - Controls Over Management Override</h2> Management Overrides (the freedom to simply "replace" reporting figures based on, presumably, sound judgement) are entity level controls that can be easily captured (in a centralised manner) by Data Controller. This in fact, is the "core functionality" of the tool. Data Stewards / Data Processors (Editors) make the change, then one or more Data Owners / Data Controllers (Approvers) sign it off before it is applied to the target table. A copy of the original excel file (if used) and a record of who made the change, when, what the change was, and why (if a reason is provided) is recorded. <a target="_blank" rel="noopener" href="https://docs.datacontroller.io/dcc-validations/">Data Validation</a> rules can also be defined to ensure that inputs fit the desired pattern(s). <a target="_blank" rel="noopener" href="https://pcaobus.org/Rulemaking/Docket%20021/2007-06-12_Release_No_2007-005A.pdf"><img class="aligncenter wp-image-1122" src="/wp-content/uploads/2020/08/Screenshot-from-2020-08-10-10-41-12.png" alt="Sarbanes Oxley sas management overrides" width="887" height="409" /></a> For fun, we made a short video for this part:
 
 `youtube: https://youtu.be/iY3KQZL4ok0`
 
@@ -39,7 +39,7 @@ Below is an example of column level lineage. Like Table Level lineage, this can
 
 &nbsp; The ability to define additional data lineages, outside of SAS (eg between spreadsheets or other reporting systems) is in the product roadmap, along with lineage from SAS Viya. <h2>PCAOB AS5 App B - Benchmarking of Automated Controls</h2> The use of IT secured financial controls can significantly reduce the cost of Sarbanes-Oxley compliance testing following the first year assessment, particularly where the source code is secured and cannot be modified by users. The core programs (services) within the Data Controller application that perform data signoffs are mature, distinct and change tracked - so it is possible for Data Controller to be upgraded in-place without affecting the benchmarking strategy. This contrasts with spreadsheet based control mechanisms, which must be revalidated in each reporting period.
 
-<div class="imgHolder"><a href="https://pcaobus.org/Rulemaking/Docket%20021/2007-06-12_Release_No_2007-005A.pdf"><img class="aligncenter" title="PCAOB Release 2007-005A, Appendix B" src="/wp-content/uploads/2020/08/Screenshot-from-2020-08-08-22-15-50.png" alt="Sarbanes Oxley SAS"/></a><caption>PCAOB Release 2007-005A, Appendix B</caption></div>
+<div class="imgHolder"><a target="_blank" rel="noopener" href="https://pcaobus.org/Rulemaking/Docket%20021/2007-06-12_Release_No_2007-005A.pdf"><img class="aligncenter" title="PCAOB Release 2007-005A, Appendix B" src="/wp-content/uploads/2020/08/Screenshot-from-2020-08-08-22-15-50.png" alt="Sarbanes Oxley SAS"/></a><caption>PCAOB Release 2007-005A, Appendix B</caption></div>
 
 ## Sarbanes-Oxley Act Section 1102 - Tampering
 

content/blog/siemens-healthineers-smart-data-catalog/index.md

@@ -16,7 +16,7 @@ tags:
   - VBA
 ---
 
-Data Controller was implemented at Siemens Healthineers to facilitate their SAS-Powered Smart Data Catalog and integrate with Data Lineage reporting. We are grateful to <a href="https://www.linkedin.com/in/helen-stark-5bb15b6a/">Helen Stark</a> (Power User) and <a href="https://www.linkedin.com/in/hans-juergen-kopperger-942634b7/">Hans-Juergen Kopperger</a> (SAS App Developer) for sharing their "before and after" experiences. The previous article in this series is available <a href="/der-touristik/">here</a>.
+Data Controller was implemented at Siemens Healthineers to facilitate their SAS-Powered Smart Data Catalog and integrate with Data Lineage reporting. We are grateful to <a target="_blank" rel="noopener" href="https://www.linkedin.com/in/helen-stark-5bb15b6a/">Helen Stark</a> (Power User) and <a target="_blank" rel="noopener" href="https://www.linkedin.com/in/hans-juergen-kopperger-942634b7/">Hans-Juergen Kopperger</a> (SAS App Developer) for sharing their "before and after" experiences. The previous article in this series is available <a href="/der-touristik/">here</a>.
 
 ---
 
@@ -76,7 +76,7 @@ In the past we had a custom Stored Process web app for uploading excel files, ba
 
 I would often receive support tickets in relation to this upload, the cause of which was often due to the diversity of our excel templates, and being unsure which was the right template...So, we would have a lot of discussions about how to bring data into the backend in a controlled manner.
 
-Then one day, I got information through <a href="https://sasusergroup.de/">SAS User Group Germany</a> that you provide a solution with Data Controller. I was initially interested in the <a href="https://docs.datacontroller.io/videos/#data-lineage">Data Lineage</a> functionality, but then I understood the main concept behind Data Controller. And for me the main benefit is that I can save a lot of time - with out of the box features like the web data editor, and the web upload facility with excel spreadsheet drag and drop. And there is the automatic workflow behind with the mandatory approval step. Since we implemented Data Controller, we no longer get those support tickets.
+Then one day, I got information through <a target="_blank" rel="noopener" href="https://sasusergroup.de/">SAS User Group Germany</a> that you provide a solution with Data Controller. I was initially interested in the <a target="_blank" rel="noopener" href="https://docs.datacontroller.io/videos/#data-lineage">Data Lineage</a> functionality, but then I understood the main concept behind Data Controller. And for me the main benefit is that I can save a lot of time - with out of the box features like the web data editor, and the web upload facility with excel spreadsheet drag and drop. And there is the automatic workflow behind with the mandatory approval step. Since we implemented Data Controller, we no longer get those support tickets.
 
 ### Fantastic. If you had to pick your top features, what would they be?
 
@@ -84,7 +84,7 @@ The main benefit is getting data controlled, and into the backend. The controlle
 
 The transparency of the history page is another benefit. I can look at every requested submit or approval - what changes have been applied, what changes have been submitted, and what changes have been approved. This helps us a lot to get data transparency.
 
-The <a href="https://docs.datacontroller.io/emails/">email alerts</a> is a great feature. For the communication of changes, we had previously created a team's collaboration chat. e.g. if someone did a change and needed to request an approval. But with email alerts, the notification of changes is now automatically sent to the responsible data owner, who can immediately click the email link and do his approval. This speeds up the whole process.
+The <a target="_blank" rel="noopener" href="https://docs.datacontroller.io/emails/">email alerts</a> is a great feature. For the communication of changes, we had previously created a team's collaboration chat. e.g. if someone did a change and needed to request an approval. But with email alerts, the notification of changes is now automatically sent to the responsible data owner, who can immediately click the email link and do his approval. This speeds up the whole process.
 
 Another advantage is the "database approach" for updates. So, someone is changing one row in a table which is connected to his use case, another guy can change other rows of the same table, nearly simultaneously. Because not everyone is changing the same rows. Everyone has their own subset of rows, their own "workspace" within one table. In the past we would have one excel template, and this would always override all values. We would have a lot of excel templates going around our colleagues, so there were always conflicts of overrides and versioning, and stuff like that. With Data Controller, it's now a simple, easy and transparent data capture process.
 

content/blog/v6-0-api-explorer/index.md

@@ -1,5 +1,5 @@
 ---
-title: "v6.0 Release: Viya API Explorer"
+title: 'v6.0 Release: Viya API Explorer'
 description: Data Controller community tier now includes an API explorer!  We've also overhauled the (in)format ingestion capability, and revamped our pricing (now with unlimited users across all tiers).
 date: '2023-06-26 09:00:00'
 author: 'Allan Bowe'
@@ -15,13 +15,12 @@ With a (Viya) API explorer, an overhauled (in)format ingestion capability, and n
 
 ## Viya API Explorer
 
-Following on from the metadata explorer (SAS 9 EBI feature) we have been looking to provide a similar capability for Viya.  And so, we built the API explorer!
+Following on from the metadata explorer (SAS 9 EBI feature) we have been looking to provide a similar capability for Viya. And so, we built the API explorer!
 
-This lets you easily trigger the (GET) APIs and explore the responses without having to break open Postman or another development toolkit.  Here's an example of opening a Job and examining the SAS code:
+This lets you easily trigger the (GET) APIs and explore the responses without having to break open Postman or another development toolkit. Here's an example of opening a Job and examining the SAS code:
 
 <iframe title="Browsing Viya API in Data Controller" width="560" height="315" src="https://vid.4gl.io/videos/embed/e284f815-a6dc-4998-80bd-152d54cb81a9?title=0" frameborder="0" allowfullscreen="" sandbox="allow-same-origin allow-scripts allow-popups"></iframe>
 
-
 Here we grab the raw JSON for pasting into VS Code:
 
 <iframe title="Grabbing JSON from Viya APIs with Data Controller" width="560" height="315" src="https://vid.4gl.io/videos/embed/18914633-342b-48f1-9021-bb01a8b33198?title=0&amp;warningTitle=0" frameborder="0" allowfullscreen="" sandbox="allow-same-origin allow-scripts allow-popups"></iframe>
@@ -32,29 +31,27 @@ And here we toggle the start / limit parameters to bring back more values:
 
 We would love YOUR feedback as to how we can extend this API explorer to make it an even more useful tool!
 
-
-
 ## Unlimited Users
 
-If you've been following us for a while you've probably heard the '5 users free' tagline.  Well - you will hear it no more, as we now offer **unlimited users for all tier levels**!
+If you've been following us for a while you've probably heard the '5 users free' tagline. Well - you will hear it no more, as we now offer **unlimited users for all tier levels**!
 
 That's right, you can download Data Controller (Community Edition) and use it across your entire enterprise TODAY, without spending a penny.
 
-If, however, you would like priority support and full access to all features, we ask that you engage us on <a href="https://datacontroller.io/pricing">paid subscription plan</a>.
+If, however, you would like priority support and full access to all features, we ask that you engage us on <a target="_blank" rel="noopener" href="https://datacontroller.io/pricing">paid subscription plan</a>.
 
 ## (IN)FORMAT Capabilities
 
-Previously we only supported ingestion of run-of-the-mill SAS formats.  Following customer feedback, we have now expanded this capability to include:
+Previously we only supported ingestion of run-of-the-mill SAS formats. Following customer feedback, we have now expanded this capability to include:
 
-* Informats
+- Informats
-* Multilabel Formats
+- Multilabel Formats
-* NotSorted Formats
+- NotSorted Formats
 
-The addition of these format types broke the data model we were using previously for holding format data.  We had incorrectly assumed that the CNTLOUT dataset could be keyed on TYPE, FMTNAME and START.
+The addition of these format types broke the data model we were using previously for holding format data. We had incorrectly assumed that the CNTLOUT dataset could be keyed on TYPE, FMTNAME and START.
 
-In fact, START can be null, and the format data can have complete duplicates (multilabel). Furthermore, the _order_ of records is important (notsorted).  Therefore we have applied a new key (TYPE, FMTNAME, FMTROW) where FMTROW is the index of the record of the format in question.
+In fact, START can be null, and the format data can have complete duplicates (multilabel). Furthermore, the _order_ of records is important (notsorted). Therefore we have applied a new key (TYPE, FMTNAME, FMTROW) where FMTROW is the index of the record of the format in question.
 
-This means if you insert a row in a format, Data Controller will see this as a CHANGE to all the rows underneath (if they are not duplicates).  This difference in behaviour, as well as the the change in the model, is the "breaking change" in this release (hence major version bump).  It will likely only affect you though if you are using Excel or CSV to upload (in)format data.
+This means if you insert a row in a format, Data Controller will see this as a CHANGE to all the rows underneath (if they are not duplicates). This difference in behaviour, as well as the the change in the model, is the "breaking change" in this release (hence major version bump). It will likely only affect you though if you are using Excel or CSV to upload (in)format data.
 
 This primary key (TYPE, FMTNAME, FMTROW) is now also indicated in VIEW mode.
 
@@ -68,7 +65,6 @@ We've added a new screen (under the username dropdown) to show system details as
 
 This screen is also available for regular users (those not in the Data Controller admin group), just without the additional buttons.
 
-
 ## Load More Values
 
 We've added the ability to 'load more' history on the history page, as well as the ability to [show more history by default](https://docs.datacontroller.io/dcc-options/#history_rows)
@@ -81,31 +77,14 @@ We've added the ability to 'load more' history on the history page, as well as t
 
 Some of the issues we've zapped:
 
-* Enable data-catalog refresh of a single library when invalid libraries are present
+- Enable data-catalog refresh of a single library when invalid libraries are present
-* Prevent error when attempting an UNLOCK of an already-unlocked table
+- Prevent error when attempting an UNLOCK of an already-unlocked table
-* Show Viya avatar when web app is served from a different domain
+- Show Viya avatar when web app is served from a different domain
-* Bug with delete-only uploads not appearing in the audit table
+- Bug with delete-only uploads not appearing in the audit table
-* Show special missing values on VIEW screen
+- Show special missing values on VIEW screen
 
 ## Roadmap
 
-Looking to the future, we are actively tidying up the codebase to publish it as 'source-available' (the source is already available to existing customers).  We are also investigating the HandsOnTable "Formula" feature to see if we can implement it on the EDIT grid.
+Looking to the future, we are actively tidying up the codebase to publish it as 'source-available' (the source is already available to existing customers). We are also investigating the HandsOnTable "Formula" feature to see if we can implement it on the EDIT grid.
 
 If you would like to see any new features in DC, or would like to kick the tyres and give it a whirl, do [get in touch](https://datacontroller.io/contact)!
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-

package.json

@@ -17,35 +17,35 @@
     "lint:fix": "npx prettier --write \"src/**/*.+(ts|tsx|js|jsx|json|css|scss)\" --ignore-path .gitignore"
   },
   "dependencies": {
-    "@browniebroke/gatsby-image-gallery": "^6.2.0",
+    "@browniebroke/gatsby-image-gallery": "^8.2.0",
-    "@mdx-js/mdx": "^1.6.22",
+    "@mdx-js/mdx": "^3.1.1",
-    "@mdx-js/react": "^1.6.22",
+    "@mdx-js/react": "^3.1.1",
-    "babel-plugin-styled-components": "^1.12.0",
+    "babel-plugin-styled-components": "^2.1.4",
-    "gatsby": "^3.5.1",
+    "gatsby": "^5.15.0",
-    "gatsby-plugin-google-analytics": "^3.3.0",
+    "gatsby-plugin-google-analytics": "^5.13.1",
     "gatsby-plugin-google-fonts": "^1.0.1",
-    "gatsby-plugin-image": "^1.3.1",
+    "gatsby-plugin-image": "^3.15.0",
     "gatsby-plugin-local-search": "^2.0.1",
-    "gatsby-plugin-manifest": "^3.3.0",
+    "gatsby-plugin-manifest": "^5.15.0",
-    "gatsby-plugin-matomo": "0.13.0",
+    "gatsby-plugin-matomo": "^0.17.0",
-    "gatsby-plugin-react-helmet": "^4.3.0",
+    "gatsby-plugin-react-helmet": "^6.15.0",
-    "gatsby-plugin-sharp": "^3.3.1",
+    "gatsby-plugin-sharp": "^5.15.0",
-    "gatsby-plugin-sitemap": "^3.3.0",
+    "gatsby-plugin-sitemap": "^6.15.0",
-    "gatsby-plugin-styled-components": "^4.3.0",
+    "gatsby-plugin-styled-components": "^6.15.0",
-    "gatsby-remark-embed-video": "^3.1.1",
+    "gatsby-remark-embed-video": "^3.2.1",
-    "gatsby-remark-images": "^4.2.0",
+    "gatsby-remark-images": "^7.15.0",
-    "gatsby-remark-responsive-iframe": "^4.2.1",
+    "gatsby-remark-responsive-iframe": "^6.15.0",
-    "gatsby-source-filesystem": "^3.3.0",
+    "gatsby-source-filesystem": "^5.15.0",
-    "gatsby-transformer-remark": "^4.0.0",
+    "gatsby-transformer-remark": "^6.15.0",
-    "gatsby-transformer-sharp": "^3.3.0",
+    "gatsby-transformer-sharp": "^5.15.0",
     "nostr-tools": "^2.7.0",
-    "react": "^17.0.1",
+    "react": "^18.3.1",
-    "react-dom": "^17.0.1",
+    "react-dom": "^18.3.1",
     "react-helmet": "^6.1.0",
-    "react-icons": "^4.2.0",
+    "react-icons": "^5.2.1",
-    "react-share": "^4.4.0",
+    "react-share": "^5.1.0",
     "react-use-flexsearch": "^0.1.1",
-    "styled-components": "^5.2.3",
+    "styled-components": "^6.1.11",
     "tseep": "^1.2.1"
   },
   "devDependencies": {

src/components/footer/index.tsx

@@ -1,9 +1,5 @@
 import React from 'react'
-import { Link } from 'gatsby'
+import { Section } from '../shared'
-
-// import dcLogo from "../../images/dclogo.png";
-
-import { Container, Section } from '../shared'
 import { SolidButton } from '../shared/styledComponents'
 import { StyledHeading, StyledDesc, InputStyled, StyledAnchor } from './style'
 
@@ -19,8 +15,14 @@ const Footer = () => (
         <StyledDesc>
           Data Controller is a product of 4GL Apps, a brand of Bowe IO Ltd,
           which is a UK company with a focus on SAS Software,{' '}
-          <StyledAnchor href="https://sasapps.io">Apps</StyledAnchor>, and
+          <StyledAnchor
-          Services.
+            href="https://sasapps.io"
+            target="_blank"
+            rel="noopener"
+          >
+            Apps
+          </StyledAnchor>
+          , and Services.
         </StyledDesc>
       </div>
       <div className="col-md-3">
@@ -64,8 +66,14 @@ const Footer = () => (
         <StyledHeading>Other Resources</StyledHeading>
         <StyledDesc>
           Visit our educational and fun SAS® software quiz{' '}
-          <StyledAnchor href="https://sasensei.com">Sasensei</StyledAnchor> and
+          <StyledAnchor
-          test your knowledge of SAS topics.
+            href="https://sasensei.com"
+            target="_blank"
+            rel="noopener"
+          >
+            Sasensei
+          </StyledAnchor>{' '}
+          and test your knowledge of SAS topics.
         </StyledDesc>
       </div>
     </div>

src/components/navibar/style.js

@@ -318,7 +318,7 @@ export const Li = styled.li`
 export const StyledLink = styled((props) => <Link {...props} />)`
   padding-right: 0.8rem !important;
   padding-left: 0.8rem !important;
-  color: white;
+  color: white !important;
 
   &:before {
     ${LinkUnderlineStyles}

src/components/shared/styledComponents.js

@@ -63,12 +63,14 @@ export const SolidButton = ({
   children,
   theme = 'light',
   type = 'submit',
+  disabled = false,
   onClick = undefined
 }) => (
   <StyledSolidButton
     type={type}
     className={`btn btn-${theme}`}
     onClick={onClick}
+    disabled={disabled}
   >
     {children}
   </StyledSolidButton>

src/markdown-pages/pricing.md

@@ -15,30 +15,29 @@ description: Data Controller for SAS® pricing starts from FREE, is fully TRA
 
 ## Packages
 
-Data Controller is available in four packages, with no limits on number of users (since [v6](https://datacontroller.io/v6-0-api-explorer/)).  Data Controller can run on ANY flavour of SAS - be that Viya, EBI, or [SASjs Server](https://server.sasjs.io).
+Data Controller is available in four packages, with no limits on number of users (since [v6](https://datacontroller.io/v6-0-api-explorer/)). Data Controller can run on ANY flavour of SAS - be that Viya, EBI, or [SASjs Server](https://server.sasjs.io).
 
-|                          | Community       | Bronze    | Silver    | Gold       |
+|                          | Community       | Bronze     | Silver     | Gold       |
-|--------------------------|-----------------|-----------|-----------|------------|
+| ------------------------ | --------------- | ---------- | ---------- | ---------- |
-| Number of Users          | Unlimited       | Unlimited | Unlimited | Unlimited  |
+| Number of Users          | Unlimited       | Unlimited  | Unlimited  | Unlimited  |
-| Environments             | Unlimited       | Unlimited | Unlimited | Unlimited  |
+| Environments             | Unlimited       | Unlimited  | Unlimited  | Unlimited  |
-| Libraries                | Unlimited       | Unlimited | Unlimited | Unlimited  |
+| Libraries                | Unlimited       | Unlimited  | Unlimited  | Unlimited  |
-| Tables (per library)     | 35              | 100       | Unlimited | Unlimited  |
+| Tables (per library)     | 35              | 100        | Unlimited  | Unlimited  |
-| Rows (VIEW)              | 15              | 100       | Unlimited | Unlimited  |
+| Rows (VIEW)              | 15              | 100        | Unlimited  | Unlimited  |
-| Rows (UPLOAD)            | 5               | 100       | Unlimited | Unlimited  |
+| Rows (UPLOAD)            | 5               | 100        | Unlimited  | Unlimited  |
-| Rows (HISTORY)           | 15              | 100       | Unlimited | Unlimited  |
+| Rows (HISTORY)           | 15              | 100        | Unlimited  | Unlimited  |
-| Lineage                  | 3 per day       | 100       | Unlimited | Unlimited  |
+| Lineage                  | 3 per day       | 100        | Unlimited  | Unlimited  |
-| Viewboxes                | 1               | 3         | 6 (max)   | 6 (max)    |
+| Viewboxes                | 1               | 3          | 6 (max)    | 6 (max)    |
-| Data Controller Support  | Best Endeavours | 8hr SLA   | 8hr SLA   | 4hr SLA    |
+| Data Controller Support  | Best Endeavours | 8hr SLA    | 8hr SLA    | 4hr SLA    |
-| SASjs Support            | ❌               | ✅         | ✅         | ✅          |
+| SASjs Support            | ❌              | ✅         | ✅         | ✅         |
-| In-house SAS App Support | ❌               | ❌         | ✅         | ✅          |
+| In-house SAS App Support | ❌              | ❌         | ✅         | ✅         |
-| Developer Days           | ❌               | ❌         | ❌         | ✅          |
+| Developer Days           | ❌              | ❌         | ❌         | ✅         |
-| Consulting Days          | ❌               | ❌         | ❌         | ✅          |
+| Consulting Days          | ❌              | ❌         | ❌         | ✅         |
-| Perpetual Licence Option | ❌               | ❌         | ❌         | ✅          |
+| Perpetual Licence Option | ❌              | ❌         | ❌         | ✅         |
-| Price                    | FREE            | 12k EUR pa | 25k EUR pa| Contact Us |
+| Price                    | FREE            | 12k EUR pa | 25k EUR pa | Contact Us |
 
 </br>
-<i>Data Controller Development Days (to build new product features etc) are available for all plans at a price of 600 EUR per day.  Consulting rates are published <a href="https://sasapps.io/pricing">here</a>.</i>
+<i>Data Controller Development Days (to build new product features etc) are available for all plans at a price of 600 EUR per day.  Consulting rates are published <a target="_blank" rel="noopener" href="https://sasapps.io/pricing">here</a>.</i>
-
 
 ## Support
 
@@ -48,10 +47,9 @@ Our support goal is to get you back up and running as soon as possible in the ca
 
 Tickets may be raised at either:
 
-1. Support Desk:  https://support.4gl.io
+1. Support Desk: https://support.4gl.io
 2. Support Mailbox: support@datacontroller.io
 
-
 The corresponding severity level should also be specified, eg as follows:
 
 1. Level 1 - App is non-functional and cannot be used
@@ -83,22 +81,21 @@ The following items are assumed:
 4. The client is able to provide relevant artefacts (logs, sample data, clarifications)
 
 ### Escalation
+
 If a ticket cannot be resolved within the agreed SLA conditions, escalation has to be started by the assigned support department. Escalation will be made via video call between managers at both the client and 4GL.
 
 Complaints on services should be directed to both the 4GL Engagement Manager and the client Application Owner.
 
-
 ### In Scope Support
 
 Support is unlimited, but restricted to items that actually relate to the tool (eg we cannot help with permissions, or data issues, beyond the guidance in the documentation).  All support is provided remotely, using video calls / screen sharing technology.no
 
-
 Examples of ‘in-scope’ (not an exhaustive list):
 
-* configuration & usage advice
+- configuration & usage advice
-* troubleshooting
+- troubleshooting
-* installation / deployment support
+- installation / deployment support
-* integration advice (actual integrations are a consulting effort)
+- integration advice (actual integrations are a consulting effort)
 
 ### Out of Scope Support
 
@@ -134,4 +131,3 @@ There may be situations where support cannot give immediate fixes, such as (but
 ### Partner Purchases
 
 Where Data Controller is purchased directly from one of our partners, the partner may provide first line support as per your agreement with them.
-

src/pages/about.tsx

@@ -132,7 +132,12 @@ const About: React.FC<PageProps<DataProps>> = ({ data, location }) => {
               <br />
               The company supports the development of the Data Controller web
               application and other{' '}
-              <a href="https://sasapps.io" title="SAS Apps">
+              <a
+                target="_blank"
+                rel="noopener"
+                href="https://sasapps.io"
+                title="SAS Apps"
+              >
                 SAS Apps
               </a>
               , and the Macro Cards range of websites listed below:

src/pages/contact.tsx

@@ -37,10 +37,28 @@ const Contact: React.FC<PageProps<DataProps>> = ({ data, location }) => {
 
   const [name, setName] = useState<string>()
   const [email, setEmail] = useState<string>()
+  const [emailError, setEmailError] = useState<boolean>(false)
   const [subject, setSubject] = useState<string>()
   const [message, setMessage] = useState<string>()
   const [notification, setNotification] = useState<string>()
 
+  const handleEmailChange = (e: React.ChangeEvent<HTMLInputElement>) => {
+    const value = e.target.value
+    setEmail(value)
+
+    if (
+      value &&
+      (value.toLowerCase().includes('@gmail.com') ||
+        value.toLowerCase().includes('@googlemail.com'))
+    ) {
+      setEmailError(true)
+      setNotification('Gmail addresses are not allowed.')
+    } else {
+      setEmailError(false)
+      setNotification(undefined)
+    }
+  }
+
   const getBorderStyle = (value: string | undefined) =>
     value === undefined ? {} : value ? {} : { border: '1px solid red' }
 
@@ -63,10 +81,14 @@ const Contact: React.FC<PageProps<DataProps>> = ({ data, location }) => {
             <StyledHeading>Contact Us</StyledHeading>
 
             <form
-              className="kwes-form"
               onSubmit={async (evt) => {
                 evt.preventDefault()
 
+                // Prevent sending if emailError is set
+                if (emailError) {
+                  return
+                }
+
                 if (name && email && subject && message) {
                   const res = await nostrController
                     .sendDM(
@@ -122,10 +144,7 @@ Message: ${message}`
                   name="email"
                   rules="required|email"
                   aria-describedby="emailHelp"
-                  onChange={(evt) => {
+                  onChange={handleEmailChange}
-                    setEmail(evt.target.value)
-                    setNotification(undefined)
-                  }}
                   style={getBorderStyle(email)}
                 />
                 <div id="emailHelp" className="form-text">
@@ -167,7 +186,9 @@ Message: ${message}`
                 ></textarea>
               </div>
               <div className="mb-3">
-                <SolidButton theme="dark">Submit</SolidButton>
+                <SolidButton theme="dark" disabled={emailError}>
+                  Submit
+                </SolidButton>
               </div>
             </form>
             {notification && <span>{notification}</span>}