Skip to content

Data Validation Framework

Data Validation Framework Overview

The validation process for all data destined for the UC Data Warehouse is validated via a two stage process:

  • Stage/Base Layer Validations - For the stage and base validation processes, test cases are designed and developed using key validation rules to examine the accuracy of the data. Reports are created and made available to applicable data stewards using IBM Cognos.
  • BI Audit Validations - The Business Intelligence (BI) validation process is part of IRAP's Data Audit Program. Test cases are designed to periodically examine the validity and accuracy of data within the reporting layer in an effort to consistently monitor the quality of data used for analyses and reporting.

UC Data Validation Process Overview

The validation process involves seven key steps:

  1. N0 - Input file(s) pre-submission notifications.
    1. N0a - Notification to campus that input file(s) is due. This is sent one week prior to submission deadline.
    2. N0b - Notification to campus and IRAP data stewards that input file(s) is due. This is sent on the day the file is due for submission.
    3. N0c - Notification to campus and IRAP data stewards that input file(s) is late. This is sent once a week after file submission deadline until file is received.
  2. N1 - Notification to IRAP data stewards that input file(s) has been submitted. Same notice goes to campus(es) acknowledging receipt of file(s).
  3. N2 - Notification to campus and IRAP staff about stage and base data load.
    1. N2a - Notification to campus and IRAP data stewards that input file(s) have been loaded successfully to the staging and base layers of UCDW.
    2. N2b - Notification to campus and IRAP data stewards that input file(s) have been rejected.
  4. N3 - Notification to campus and IRAP data stewards regarding availability of stage and base validation reports in IBM Cognos. Validation process begins at campus(es) and at UCOP.
  5. N4 - Notification to Data Services regarding campus and IRAP data stewards certification of stage and base data. This is an approval for data to be loaded into the Business Intelligence (BI) layer of UCDW.
  6. N5 - Notification to campus and IRAP data stewards regarding successful data load into the BI reporting layer.
  7. N6 - Notification to IRAP data stewards regarding availability of BI audit reports.

Training Materials