Contaminated Sediment Data ETL Tool Development

Objective

We seek to hire an IT professional to conduct data verification and data validation on a backlog of studies in the SLR AOC and to transform the data from these studies into a predefined formats so they may be added to Department enterprise Oracle database (SWIMS) and the SLR AOC database (extract, transform, and load processes).

Study Design

The data must be qc'd (verified and validated), and then transformed from the native formats in which they are received, to a format that allows them to be loaded into SWIMS as well as into a flat file format developed for the SLR AOC database. The process is basically an extract, transform, and load (ETL). The native formats may vary depending on the original source. Through the process of working on the backlog data, the IT professional will develop strategies and solutions for data flow processes and automated tools necessary for DNR staff to perform these steps on future data sets. This work will include developing the necessary referential tables, lookup tables, translation scripts or other automated tools, and documentation as needed, to build WDNR internal capacity. We have relied on incremental grant funding in the past to hire contractors to perform this work without the benefit of building our own capacity to perform these tasks internally. This approach is not sustainable since grant funding is becoming harder to obtain and we need to be able to use these data on a timely basis to conduct site assessments or to develop cleanup objectives for contaminated sites.

Related Reports

Run Project Summary Report
View Umbrella-Projects
View Related-Projects

Great Lakes Restoration Initiative
Toxics and Areas of Concern
SLRAOC_Database
2015
Active