4. Importing logger data¶
The Monitoring Module can be used to import continuous logger data. The timeseries data are first exported into a csv-file with one row per interval (timestep) and one or multiple columns for the recorded parameters.
Each csv-file represents a single monitoring location. Multiple csv-files can be imported at once.
Note
The Monitoring Module’s logger data importing tool has been tested to work with the following loggers:
- In-Situ® Troll 9000 Pro XP
Other loggers and csv input file formats may be supported by tweaking the tool’s configuration parameters.
4.1. Getting everything in place¶
In this example, the file to be imported 2015-01 Logger results.csv
is saved in a project file structure as follows:
C:\
Project\
Import data\
2015-01 Logger results.csv
Monitoring data.dss
logger_import.yml
4.2. The configuration file¶
The configuration file logger_import.yml
can be created as a simple text
file with the following content:
folder: Import data
files:
2015-01 Logger results.csv: Location A
site: Site name
version: RAW
date_format: "%d/%m/%Y"
interval_snap: yes
mapping:
Temperature: TEMP
Turbidity: TURB
pH: PH
Rugged DO Sat: DO%
Rugged DO: DO
Conductivity: EC
params:
TEMP:
unit: degC
TURB:
unit: FNU
PH:
unit: "-"
DO%:
unit: "%"
DO:
unit: mg/l
EC:
unit: µS/cm
The configuration file describes the files to be imported as well as information
about which csv-file parameter columns to be imported. The mapping
section
of the configuration file indicates which column headings map onto the
parameters to be saved into the database. Special symbols (non-ASCII characters)
should be omitted from the mapping
section.
The following tags can be used to specify the date format:
Day | Month | Year | |||
---|---|---|---|---|---|
Tag | Example | Tag | Example | Tag | Example |
%d | 01, 02, .., 31 | %b | Jan, Feb, .. | %y | 00, 01, .., 99 |
%B | January, February, .. | %Y | 1970, 2013, .. | ||
%m | 01, 02, .., 12 |
The time column is always assumed to be formatted as %H:%M:%S
.
Important
Unlike the other import tools, the files
section lists the file names
without a dash -
. Instead, the files are listed as a series of key:
value pairs like this:
files:
file name 1.csv: location name A
file name 2.csv: location name B
Parameter values in the import file starting with < are interpreted as being below the meter’s limit of detection (LOD). Such measurements are imported as 50% of the LOD to allow numeric evaluations and plotting in line with current best practice.
4.3. Running the import¶
When the configuration has been set up, the data can be imported as follows:
- Open the
Monitoring data.dss
file in HEC-DSSVue.- Select menu item
or alternatively click on on the toolbar.- Choose the configuration file
logger_import.yml
in the file s election window.
When successfully completed, a message is displayed how many records have been imported and the catalogue is refreshed.