Metis Staging Upload

Overview

Well information and sample data that have been project merged in Transform, and thus given identity in the Metis system, can be uploaded to the central Staging database. The upload step uses a wizard that helps to ensure data is not duplicated on the Staging database and that newly added analyses are merged with the correct existing samples and analyses.

Version: 2.0+ (Oct 2021)

Usage: Metis Tools --> Merge and Upload to Staging...

How to use in practice

In the project merge we bring together one or more candidate rows to create project wells and project samples. Project wells are shown as well artefacts, and project samples are just samples (rows) in the spreadsheet. You can tell whether a sample is a candidate sample by inspecting the Sam status.Metis property - an empty value implies a candidate sample, but if the string "Metis sample" is shown you are looking at a project sample.

Only project wells and project samples will be considered for uploading to the Staging database. You must first select which Staging database to upload data to.

The Staging merge and upload proceeds somewhat differently to the project merge. In this case the aim is to find 'similar' wells and samples on the Staging database, when comparing each project well and sample in the Transform project. The process works on wells, and then on samples, similarly to the project merge.

Well merge and upload

Unlike Project Merge, the Staging merge does not allow you to select what properties are used to find potential matches, rather these are hard-coded in the merge algorithm.

The Staging works in the following manner:

For each project well in Transform we look for 'matching' wells on the Staging database. Here matching is based on wells which are 'similar' on certain properties - for example the well name, or the well location (lat, long), or other well identifiers. An exhaustive list of the matching properties can be found here.
For each matched well we calculate a matching score - this uses more properties to provide a numeric score in the range of 0 (no match) to 100 (identical). The scoring details can be found here.
The matched wells are ranked, based on their score and up to the top 10 matches are returned. Typically only a small number of wells will be matched - only matched wells are scored, so you might only expect a small number to be returned.

Once Transform has obtained the candidate matches, you are presented with a list of project wells. For each well you are shown whether it is a new well (green +), an existing well that merges without conflicts (green joining arrows) or merges with an existing well with conflicting values (red cross, shown above).

Clicking on each well provides a summary view, as illustrated above. This shows the values for the well properties in the project (row 2), the values for the well properties on Staging (row 3, and more if multiple matches are found, ranked by score, which is shown in the second column). Where multiple wells are returned from Staging the user can select which of the Staging wells to merge with. For wells with conflicting values in some of the properties the user must decide whether to retain the Staging value, or update this with the project value.

Once the conflicts have been resolved, the merge can proceed to the sample merge process.

Sample merge and upload

Well samples

When finding potential matches between samples associated with wells in the project and those on Staging we first ensure the samples are in the same well and then compare the sample name, depth, identity and type information. Details of the matching criteria for well samples can be found here.

The user will be presented with options depending on the outcome of the matching and scoring in a manner similar to wells. In this interface the data from each well is loaded from Staging on demand, so there can be some delay when selecting a well. Once a well is selected all project samples from within that well are shown. Selecting a sample reveal the analyses for that sample, in both the project and Staging, on Staging only and in the project only (new data). A red cross indicates conflicts, and red no-entry symbol indicates the system needs more information to proceed, a green merge arrow indicates the system is ready to upload the sample, and a green plus means the sample is entirely new to the system.

Users will always be informed of the implication of all actions, especially any merge conflicts when attempting to merge samples / wells with differences in properties other than the matching criteria.

Non-well samples

The merging of non-well samples proceeds similarly, but the criteria are slightly different.

The potential matches are based on criteria outlined here.

The scores are calculated based on properties described here.

Again the user is presented with information and options to select how to upload the samples to Staging.

Once you are happy with the merges and uploads proposed select Confirm and continue. This will summarise the number of well(bore)s and samples that will be uploaded. Note only those wells and samples for which the conflicts have been resolved will be uploaded. Selecting Upload and close will upload the data, and close the wizard. Note that uploading large numbers of wells and samples can take some time.

When a sample is uploaded, that sample will be removed from the project (this can be undone, although the uploaded data cannot be removed using undo). If all samples from a well are uploaded the well will also be removed from the project. This allows you to incrementally upload data in a project.