Skip to content
  1. Home > Support & Training > Data Validation Framework

The Enterprise Data Validation Framework

IRAP's Enterprise Data Validation Framework Overview

The validation process for all data destined for the UC Data Warehouse is a multi-step process:

  • Data Profiling of Incoming Files - This step checks the health of a file before processing them into UCOP systems. If the condition of the file is bad [based on the input files and additional profiling requirements provided by the content team], we can let campuses know almost immediately that a file has issues.
  • Stage and Base Validations -  This step validates the data based on a set of prebuilt rules post the data profiling stage but prior to loading data into the data warehouse reporting layer.
  • File to Stage to Base Validation & Balancing - This validation involves comparing counts by various predefined categories of data from file to stage and to base, to make sure records are not lost as data transitions from one stage of the process to another.
  • File to Stage to Base to BI Validation & Balancing - This validation function involves comparing counts by various predefined categories of data from file to stage, to base, and to BI to make sure records are not lost as data transitions from one stage of the process to another.
  • Business Intelligence Data layer Audit - This involves audit routines that run to compare data across years and report on variations and trends for review purposes.