DISKOS transformation

Overview

The DISKOS transformer takes data in the NPD-95 format and converts the individual blocks into tabular data (one sheet per block) then adds a combined sheet at the start (containing data merged from each block).

We use the information in the file definition block (at the start) to determine the delimiter, version, format and sender.

Merge

The sender is significant because the ID scheme used for some senders may require more intricate merge logic. For example APT files have a reference in the analysis blocks that is composed of 4 parts:

Sample Id (numeric digits)
Whole or fraction ('W' for whole or 'A', 'B', etc. for each fraction)
Analysis code e.g. "RE" for rock eval - not always 2 digits
Repeat ref ('A' for first run, 'B', 'C', etc to indicate repeats)

IGI treats a single sample/row as the combination of the above excluding the analysis code as the values for multiple analyses can be merged onto a single row.

For other senders we currently just merge on the ids given, without decomposing and removing elements (such as analysis ref).

Linking

The headers from the source file are used in the output file (with the exception of minor edits to add the indicator etc). These will not all automatically link if importing the data into p:IGI+ or Transform.

One point to be aware of is that some properties that are defined separately in the source headings use the same IGI analysis group e.g. whole oil and whole extract both go to WO-GC in the IGI property model. This means that if you have data for both in a single file, you will need to import separately (as the p:IGI import does not allow duplicate columns when defining the mapping for import).