SOP aou-ehr-file-check

  • Updated
Follow


SOP AoU-EHR-File-Check

All of Us Standard Operating Procedure

 

Purpose

To create a standardized method for sites to download and implement the latest version of the AoU-EHR-File-Check to use with the OMOP CDMv 5.3 during the 2021 Q3 submission cycle

Scope

The process applies to all program partner organizations that are responsible for sharing participant data (e.g., EHR) with the DRC. The recently updated file-check can be downloaded from a GCS bucket. 

Materials

Requirements: 

  • Python 3 (download from here and install)
  • pip (download get-pip.py and run python get-pip.py)

Background

While a submission from sites is not required this quarter, we are requiring that sites make a test submission with CDMv 5.3 using the new aou-ehr-file-check. This is required for all HPO sites during Q3 from 7/15 to 9/1. 

Procedure
  1. Cloning Repo (use the master branch) 
    1. Through GIT
    2. Or download Repo
  1. (First time) Install requirements by running
    pip install -r requirements.txt
  1. Run file validator
    1. Update _settings.py and rename it to settings.py
    2. Create a folder and place all the EHR submission files to be validated into it
    3. Set the "csv_dir" parameter in settings.py to the full path of the folder created above
    4. Ensure the resources folder is also downloaded and is located in the same place as omop_file_validator.py
    5. Execute the following at the command line:
       python omop_file_validator.py
  2. Validation logic
    1. File names must follow naming convention {table}.csv
      1. table an OMOP CDM table listed in resources/omop
    2. Files must be in CSV format (comma-delimited) as specified by rfc4180
    3. Column names and types must follow the conventions in resources/omop
    4. Date and timestamps formats must be compatible with bigquery
    5. There can be no blank lines in the csv file
      1. Unless all lines are blank
  1. File errors
    1. Errors will be stored in the files results.html and results.csv
    2. Errors need to be solved before uploading to the bucket
   

 

References:

https://github.com/all-of-us/aou-ehr-file-check#readme








Related articles

Powered by Zendesk