Dataloader Reject Files

When updating from Metis v1.28 to Metis v2.0.0 the data from the current production environment will be migrated to the new environment.

As Metis v2.0.0 has improved data validation, any invalid data will be rejected as part of the migration process. This invalid data will be exported to reject files that can be corrected and upload to Metis v2.0.0 via Transform.

There are typically two reject files produced after the migration:

igi_extract_wells_transformed_rejected
igi_extract_samples_transformed_rejected

As the name suggests the first file contains the rejected well data. The second file contains the rejected sample data. Both well sample and non-well sample (outcrops etc) data are contained in the second file. In the unlikely event no data has been rejected then these files will not be produced.

It's important to understand, only individual invalid data values are rejected, all other valid data values will be present after the migration. For example, if a sample has one invalid data value the rest of the data for the sample will have been migrated successfully and be populated correctly in Metis v2.0.0. Only the invalid value will be rejected.

Before continuing it is best to examine these files by opening them up in excel or another application suitable for .csv files. Each file may contain hundreds, if not thousands, of rows of rejected data. However, if the files contain significantly more data than this, i.e 10,000+ rows, please contact us. As they may have been a configuration issue in the migration.

Although either file can be processed independently of the other, it is simpler to start with the well file as it is likely to contain fewer rows of rejected data. The first column of each reject file contains the reason that it has been rejected by the migration:

There are three types of reject reason:

Date-times:

"Cannot parse 'xyz' with header 'Date.Analyis' which has expected data type DateTime."

Date-time values are likely to be the most commonly rejected values during the migration from Metis v1.28 to Metis v2.0.0. As Metis v2.0.0 now fully supports querying on dates and times all values for properties relating to date-times must be a valid format.

Metis v1.28 did not enforce any validation on dates or times, as a result any value could be entered in a property such as "Sampling Date.Sam" and be accepted. Metis v2.0.0 is more strict and invalid date-time strings cannot be uploaded to Metis or imported via Transform. For more information about date-times read here:

link goes here

As a result, many strings may be rejected as they are an invalid format. It may be the case that the rejected value can simply be edited in excel into a valid date-time format. Once corrected, the file can be imported into Metis Transform, merged and uploaded to Staging.

However, it may also be the case that the many rejected value may not be close to a valid value at all. For example, the string "n/a" is not a date-time. These types of values cannot be corrected and the associated rows can be removed from the reject file and do not need to be processed by Metis Transform.

Dataloader Reject Files

Date-times:

Long strings

Wrong data-types