Who is this article for?

Administrators and Module Developers importing data to existing modules

Area reporting authorities, roles, and record access controls restrict visibility.

Data import supports header-level fields only. A common use case is loading new trend codes into a reference module. This feature is often used during implementation projects. The shortcut is available to users with the Data Import Administrator role.

For a user to gain access they must have roles in the appropriate reporting authority. The Data Import screen is not a back door for loading data a user wouldn't normally be able to create manually.

Classic UI data import limitations
Beginning with platform version 2.55.0 the data import tool is only supported in the Miramar UI. Data Import is no longer available in the Classic UI for any area running 2.55.0 or greater. Classic UI subscribers who need to use the data import tool, must use a 2.55.0 or lower version of the platform. If a subscriber is using 2.55.0 and greater, and does not have access to Miramar, contact a technical account manager to gain access.

The data import tool can load data from an Excel or a CSV file. On the initial load, the Data Import screen displays a list of all prior data import jobs.

Click the New Job button to create a new data import job, give the job a name, select the reporting authority, and the module to load the data into. Once complete, the system will email a summary report to the email address entered.

Once the file is selected, modify the file-specific settings such as starting row and date format. Click Upload file to display the data mapping region.

Batch importing hosting fees
Hosted subscribers who are using the batch-importing data feature may incur additional hosting fees. For information about current rates or limits, please contact your account representative.

Always follow security best practices to protect data 
If a spreadsheet needs to be saved to a computer in order to use the Data Import tool, make sure the computer is encrypted and password protected. Securely delete the file once the data and been imported. To send data files to a Ideagen employee, never use email -- use a secure FTP server, or another equally secure means of exchanging data. Ideagen employees are required to comply with our Information Security Management System for the protection of customer content and can provide guidance on appropriate means of exchanging data depending on its sensitivity.

Data mapping

In the Data Mapping region, a list of header-level fields, enabled for data import, will be shown. The columns list allows for mapping columns from the uploaded file to fields of the module. Selecting Map Fields by Name will attempt to match columns from the file to fields of the module by name where possible. Be sure to review all mappings before proceeding.

The default column can be used to specify a fixed value for all imported objects.

Key fields

Key fields allow the data import tool to know when to update an existing object or create a new object. The system will search for the combination of values from all fields marked as a key and update any existing objects that match. If no objects match, a new object will be created from the imported file.

The Key Field is automatically checked for the field which is defined as the identifying field in the module, this is typically the Identifier or another similar unique field, but this can be cleared, and other fields can used.

Use care when setting key fields
Use care when setting key fields, the system will update existing records that match on the combination of key fields and create new objects that don’t exist. Improperly setting the key fields can result in duplicate objects.

Running the import job

Click the Preview button to preview the data import job without importing any actual data. The Run button will perform the import and update of the database.

Job history, at the bottom of the screen, will show how many times the job has been done and if there were any issues. Viewing the details will provide load details and some performance metrics.

From the history, prior files used for import can be downloaded. When an error occurs, a retry file of only the data that failed to import can be downloaded.

Execution rules based on Required Fields will only be executed after the import begins.

Troubleshooting data imports
If the data import encounters a problem, review the data for complex symbols and remove extraneous (blank) rows at the end.