Managing your Collection

Owned by Luna
Last updated: Sept 04, 2012Version comment
19 min read

Managing your collection consists of the following tasks:

  • Import Data
  • Import Media
  • Manage User Access
  • Manage Collection Access
  • Modify Collection Properties
  • Modify Collection Theme
  • Modify Thumb/Sort Settings
  • Manage Enabled Standards

Choosing a Collection to Manage in Insight Studio

When you log into Insight Studio, you have the option to Create New Collection, Manage Catalog Templates, or to choose from a list of existing collections to manage.


To manage a specific collection, highlight it in the collection list and press select. Once you have chosen a collection to manage, Insight Studio will present you with a list of tasks to perform.

Importing Data

The Insight Studio Data Import Module enables users to import batches of content from simple delimited sources such as tab- or comma-separated files or via XML. tab- or comma-delimited files can be created with applications like FileMaker Pro and Excel.

When to Import Data

When importing data using Insight Studio, the importer MUST be the only tool updating the collection. If Insight Studio is importing data, it will preclude all other catalogers from making changes as it imports. It is therefore suggested that you perform data import in off-hours to ensure optimal productivity.

Using the Data Import Module with Collections Created in Versions Before Insight 5.x

Pre-5.x collections maintain a slightly different structure than 5.x collections. For a collection created prior to 5.x to use the Data Import Module, it must comply with the 5.x Catalog Template structure. Specifically, Catalog Templates add the following rules to Fields, Field Groups, and Records.

  1. All Field Names must be unique
  2. All Record Types must have a corresponding Field Group (even if empty)

NOTE:If you are using the Luna VRA v.1, distributed with Insight Standard 3.5–4.1 and ran the VRA upgrade script distributed with Insight 5.0, your collection complies with these requirements.

Using the Data Import Module with the Indexer

Many pre-5.x collections were built using external source tables. These tables were/are indexed to draw content into Insight. As content changes in the external database, Insight must be re-indexed to recognize those changes. In most cases, the Insight indexer only operates in one direction—that is importing content into Insight—but reflecting changes made via Inscribe or the data import module back into the source data. Although the data import module may be available for collections that utilize the indexer, it is strongly suggested that you choose only one mode for data import, either the data import module or the indexer.

Creating an Import File via CSV (comma- or tab-delimited)

Insight Studio, like all data import tools, maintains specific rules on the structure and content of an import file prior to its import. Without these rules, it would be impossible to maintain the structure of the data as well as ensure the integrity of the overall file.

General Rules on File Import

  • The file must be delimited using a comma, semicolon, tab, space, or other single character delimiter.
  • The first row of the data file must contain column names, or field headers.
  • Fields from the Object Record must be in the leftmost columns.
  • At least one field from the top level Field Group must have a value in each row.

For example: the field "Identifier" in the Dublin Core template must have a value for every record imported using the Insight Studio Client.

  • Each column name must be unique. Multiple columns may map to the same field but they require unique names.

For example: if you were importing two titles for the same record, the first could have the column "Title" the second column would need a different name like "Title2."

  • Field values containing the chosen delimiter must be quoted

For example if you were using a comma as the delimiter and your data contains commas, the sentence would need quotes. E.g. "How now, brown cow." Most applications like Excel do this work for you when you save a file in .csv text format.

Generating a Data Import Template

Insight Studio can create a data import template for your collection. This template can be populated within Excel, FileMaker, or other database applications.
To create a Data Import Template for your collection:

  1. Within Insight Studio, open the Import Data section.
  2. At the bottom is a button "Generate Descriptive Data Import Template." This will export a delimited file to the location that you specify.

Using the Data Import Template within Microsoft Excel

If you open the template within Excel, the first row should contain field names for the collection.

If you don't have existing data

If you don't have existing data, then you can simply start filling in the template. Please review the rules for file import below.

If you have existing data

If you have existing data, copy and paste the columns from your existing data into the template. If you have groups of fields that repeat, you can repeat the entire record group within the data field (simply change the column names).
NOTE:To determine the contents of an Insight Record Group, either look at the Record Forms available in Inscribe or view the Catalo Template in Insight Studio.

Specifying an Import File

Select the file you would like to import.
NOTE: It is good practice to maintain a copy of these import files for backup. They may be useful if you ever need to confirm what was imported or need a record of what happened and when it happened.

Specifying the Field Delimiter

The field delimiter is the ASCII character used to mark when one field ends and the next one begins. The four most commonly used delimiter characters (Comma, Semicolon, Tab, and Space) are provided as options, but "Other" allows for entry of a custom character to be used as a delimiter.

Handling Paragraphs or New-Line Characters in your Data

New-line characters, in the form of paragraphs or carriage-returns, will cause issues in data import, preventing Insight from importing complete data records. If the data being imported into Insight has new-line characters, it must be pre-processed to ensure the new-lines or paragraphs are properly imported.
To pre-process your new lines for Insight:

  1. Open a text editor, which allows you to replace new-line characters such as vi, UltraEdit, or TextPad.
  2. Find all new-line characters and replace them with CRLF.
  3. Save the file.

NOTE: If you want to be more specific when replacing new-line characters, replace "Carriage Returns" with CR and "Line Feeds" with LF.

Handling Unicode Characters or Non-Western European Characters in your Data

Insight has full support for Unicode or non-Western characters as long as your database is properly configured to support this content. If you are importing Unicode (UTF-8) files, choose the UTF-8 setting when specifying a file to import.
To verify that your database is properly configured to support Unicode values, please review the database configuration section of the Insight 5.1 or later Installation & Configuration Guide.
NOTE:The file import tool will default to the local character set, if you are importing data with special characters (i.e. accents or other diacritics) then make sure your data is saved in "Windows" or "Macintosh" format, depending on platform.

Mapping Columns or Fields in Your Data to Fields in Collection

The import file must be mapped to your Catalog Template. If the column names in your import file match field names in your data, then the importer will automatically match those for you. For every column in the import file that cannot be matched, it will place "MISSING MAPPING" in the Target Insight Field column.
To map an unmapped field, or remap an existing field:

  1. Locate the field name in the "Source Column Name".
  2. Click on the field name in the Target Insight Field column to open the mapping field pull-down menu.
  3. Select the correct field you want to map the data to.


NOTE: To automatically map repeating fields within your data file to the same field within Insight, simply append an incrementing number to the end of the field name. For example, if the field name within Insight is "Subject" and there were four Subject columns, simply name the columns "Subject1," "Subject2," "Subject3," and "Subject4."
NOTE: To ignore a field within your data file, choose "<SKIP>" from the pull-down menu.

NOTE: You can map multiple columns to the same field within Insight.
When you have completed mapping your fields, select Next.

Performing the Import

After reviewing your settings, to begin your import, select "Finish."

The data import process occurs in a series of phases. First, the importer will attempt to parse and validate each of the records. This will ensure a valid structure as well as attempt to prevent the tool from importing records that are missing fields. Second, it will package and send the data to the server. Finally, the server will process each of the records and import the data.
NOTE:Data import can be a complex operation, especially when there are many records involved. Please be patient while the import process is occurring, as it may take some time. A data file with 1,000 records may take an hour or more.

Creating an Import File via XML (new in 5.5)

To import data via XML, you will need the proper format for your Catalog Template. To generate an example, press the "Generate import template" button.

The following is an example of the output for the Simple Labels data template.
<?xml version="1.0" encoding="UTF-8"?>
<recordList xmlns="http://www.lunaimaging.com/xsd">
<record type="Simple Labels" >
<field type="Identifier" ></field>
<field type="Type" ></field>
<field type="Creator" ></field>
<field type="Date" ></field>
<field type="Title" ></field>
<field type="Description" ></field>
</record>
</recordList>
NOTE: All elements must have a value. If they are empty the import will fail. Remove any empty elements before import.
Below is an example of how the data should be populated in the simple labels XML record.
<?xml version="1.0" encoding="UTF-8"?>
<recordList xmlns="http://www.lunaimaging.com/xsd">
<record type="Simple Labels" >
<field type="Identifier" >A-4725</field>
<field type="Type" >engraving; laid paper</field>
<field type="Creator" >Albrecht Dürer</field>
<field type="Date" >1504</field>
<field type="Title" >Adam and Eve</field>
<field type="Description" >Inscribed in plate UL:ALBERT/DURER/NORICVS/FACIEBAT/1504</field>
</record>
</recordList>
Once you have created your data in an XML format, you can follow the same instructions for importing a CSV file.
More complex example using the Dublin core data model:
<?xml version="1.0" encoding="UTF-8"?>
<recordList xmlns="http://www.lunaimaging.com/xsd">
<record type="dublincore" >
<field type="Identifier" >AGO_.84/2</field>
<fieldGroup type="Title" >
<field type="Title" >Adam and Eve</field>
</fieldGroup>
<fieldGroup type="Creator" >
<field type="Creator" >Albrecht Dürer</field>
</fieldGroup>
<fieldGroup type="Description" >
<field type="Description" >Inscribed in plate UL:ALBERT/DURER/NORICVS/FACIEBAT/1504</field>
</fieldGroup>
<fieldGroup type="Description" >
<field type="Description" >Bull's Head Watermark</field>
</fieldGroup>
<fieldGroup type="Contributor" >
<field type="Contributor" >Art Gallery of Ontario</field>
</fieldGroup>
<fieldGroup type="Date" >
<field type="Date" >unknown</field>
</fieldGroup>
<fieldGroup type="Type" >
<field type="Type" >Prints</field>
</fieldGroup>
<fieldGroup type="Rights" >
<field type="Rights" ><a href="http://www.ago.net/2/repro.htm" target="_blank">http://www.ago.net/2/repro.htm</a></field>
</fieldGroup>
</record>
<record type="dublincore" >
<field type="Identifier" >MCAS.1984.2.1-7</field>
<fieldGroup type="Title" >
<field type="Title" >The First Day (Back In The World)</field>
</fieldGroup>
<fieldGroup type="Creator" >
<field type="Creator" >Terry Allen</field>
</fieldGroup>
<fieldGroup type="Contributor" >
<field type="Contributor" >Museum of Contemporary Art, San Diego</field>
</fieldGroup>
<fieldGroup type="Date" >
<field type="Date" >1983</field>
</fieldGroup>
<fieldGroup type="Type" >
<field type="Type" >Mixed Media</field>
</fieldGroup>
<fieldGroup type="Rights" >
<field type="Rights" ><a href="http://www.MCASanDiego.org/general/copy.html" target="_blank">http://www.MCASanDiego.org/general/copy.html</a></field>
</fieldGroup>
</record>
</recordList>

Data Duplication

The data import module will attempt to resolve duplication within source data. For example, if multiple source records reference identical Creator information, only one instance of that Creator will be imported. The data import module does not detect source data that duplicates content already in Insight, meaning that importing a Creator that already exists within Insight will create a duplicate entry. The next section describes a tool for finding and eliminating these duplicates.

Merging Duplicate Records

Duplicate records are a common issue within any database system. To address this issue, Inscribe includes a Record Merge Tool that enables users to combine duplicate values in controlled vocabularies and authorities (Independent Records).

The Merge Tool works by finding duplicate field values within a given record type. The tool then returns each set of duplicate values and allows the user to "re-map" the duplicate records to the "primary" record.
To load the Record Merge Tool:

  1. Launch Inscribe.


  1. Login to the collection you wish to work with.


  1. Go to the Edit Menu and select "Merge Records." (Note: this menu item will be disabled if you do not have permission to fully commit data updates or if there are no controlled vocabularies or authorities in the collection.)
  2. In the Merge Window, select the "Record Type" you wish to work with.


  1. Select the Field within the record that will be used to identify duplicate values.


  1. Click on the "Search" button.


  1. Each page returned will represent a set of records with duplicate values within that field.


NOTE: To filter the result sets (pages) by a specific value or set of values, you can enter that value in the "Filter By" area before you click search.
NOTE: The Merge Tool only finds exact matches in field values. If there are similar but not exact matches in values, use the "Find and Replace" tool to make values match exactly.

  1. Once a set of duplicate records has been found, identify the "Primary" record. The "Primary" is the record that will be kept once the merge is complete. Select the "Primary" checkbox for that record.


  1. Once you have identified the primary record, click the "Merge" checkbox for all of the rows that should be "merged" with the primary record.


NOTE: If you are working with a Controlled Vocabulary or Authority File, a count of the records that use or link to that term or record will appear in the "Ref Count" column. After the merge, all of the records that use or link to a row that was deleted will be re-linked to the entry marked as "Primary."

  1. Once you have identified all of the records to merge on that page, as well as the primary, click the "Merge Records" button. This will only affect the current page of results.



  1. Repeat for each of the pages of duplicate records.


  1. Once you have finished merging records, open that record type and search for one of the values you merged; you should now only see one entry.

NOTE: You can see the results of your merge most clearly in the "Table View".

Media Processing

Media processing is how images and multimedia are imported into Insight and linked with object (data) records. Media processing is broken into four separate stages:
Media Import & Upload: Creates derivative JPEGs and the Wavelet Encoded File (JPEG2000) from the original source media and uploads them to the Media Manager.
Media Linking: Associates media files with data records (objects).
Review: Allows you to view the newly-assigned links between data before publishing them.
Publishing: Publishes new media to a collection, making the images part of the collection.
NOTE:While processing media, you can link media to your data records, but only if the data has already been imported.
NOTE:Images must be linked and published to show within the Insight Java Client and Browser Insight. Images only need to be processed and uploaded to be available in Inscribe.

Introduction to Batches

Insight uses a concept of Batches to help administrators manage imported content. By default each batch represents one processing session in Insight Studio. Using batches allows administrators to review and modify older batches at any time. Common tasks may include re-linking or deleting items.
Batches are displayed on the left-hand side of the media processing window. Administrators can view, select, delete, and create batches from this panel.

!worddavdb8f70e51c1d9b138e65af7debd6f5b1.wmf|height=342,width=512!

Creating a New Batch

Batches are typically created at the beginning of an import session.
To create a batch:

  1. Open Insight Studio.


  1. Login to the collection to which you wish to add content.


  1. Select "Import Media" from the Main Menu.


  1. Click the "New" button in the Batch section of the Media Processing Tool.

NOTE: When you first open the Media Processing Tool, if you drag and drop content onto the processing window, a new batch will be automatically created.

Selecting an Existing Batch

An administrator can jump between batches at any time. This enables the administrator to compare or review all of the contents.
To select a different batch:

  1. Locate the batch in the batch section of the media processing tool.
  2. Click on the batch name.
  3. The Processing list, Linking, Review, and Publishing panels will now be updated to reflect the contents of the newly-selected batch.

Renaming an existing a Batch

Renaming batches is not currently allowed in Insight Studio. However, it can be accomplished within the Administrator Tools. For more Information on renaming batches, please see "Reviewing and Modifying Mapped Media" on page .

Deleting a Batch

Deleting a batch allows an administrator to remove the batch from Insight.

  • WARNING: Deleting a batch will delete the images/media from Insight. You will not be able to recover these images/media after deleting the batch they appear in.


To Delete a Batch:

  1. Locate the batch to delete in the list of batches on the left side of the media processing section.
  2. Click on the batch name.
  3. Click the "Delete" button at the bottom of the Batch area.


Selecting Media for Import

Insight supports a wide range of media formats (see Table 13: Image and Multimedia Formats Supported by Insight on page for a full list of supported media formats).

The Media Selection panel allows you to either drag-and-drop media files or entire folders onto the panel, or to select Browse and specify files or folders to import.
NOTE: When processing multimedia files (audio, video, QTVR, etc.), Insight Studio generates default icons for thumbnail derivative images (per media type). If you want custom thumbnail images for your multimedia, use Administrator Tools instead.
NOTE: If a batch is not selected within Insight Studio when a user drags images onto the window, a new batch will be created.

Managing Large Media Files

If you are importing large media files (larger than 40-50MB), you may have to adjust your application's memory settings to support these large files. If you encounter an out-of-memory error, please see Tuning Image Processing Application Memory for Insight Studio and Administrator Tools on page for more information.

Adjusting the JPEG2000 Compression Settings

If you would like to adjust the JPEG2000 compression settings (for example, to change the encoder to use "lossless compression"), please see "Changing the JPEG2000 Compression Settings for the Administrator Tools or Insight Studio" on page for more information.

Importing & Uploading Media

Insight Studio is designed to import, organize, and upload content to the Insight Media Manager. Once content is selected, click the Import button to begin importing media. Insight Studio will create derivative images and upload them to the server.

As Insight Studio creates the derivative images and uploads the media, it will update the status bar as well as the Status column for each image.
NOTE:(New in 6.3) Importing of Source Images. If you would like to store the source Image file on the server, select the "Import Source Media" check box. Source images can take a significant amount of storage and you should check with your Insight Administrator before making this selection.
NOTE:If you would like to use a custom image processing profile to create MrSID derivatives or to create wavelets at different sizes, use the Administrator Tools. (For more information on this please see Media Processing on page .)
Once Insight Studio has completed importing and uploading media, you can review the processed media in the Review Tab. If you are not planning to link the media to currently imported data records, it is suggested that you Publish the media files.

Mapping Objects to Media

Once media is processed, it can be bulk-linked to data records by using a mapping file. Mapping files are tab-separated files containing two columns: the first for the image file name and the other for a unique value in the data record. Insight uses the mapping file to associate object and image values.

Using a mapping file is the quickest method for creating associations between objects and images in Insight. To create a mapping file, simply open Excel or another database application and enter data in two columns. The first column should be the file name of the image that you would like to map. The second column should be a unique field value in the data that identifies the record to which the image maps.
NOTE:All field mappings must be to the same field. File names are case-sensitive.
NOTE:The * represents changed or new values.
NOTE:You can map more than one image to the same record by duplicating the field entry for both. You can map more than one record to the same image by duplicating the image name next to each record entry.
Figure 7: A Sample Mapping File


Once your mapping file has been loaded, you will see a preview of your mappings in the window at the bottom. Review the list to ensure that the file has been loaded properly.

Reviewing your Mappings

To apply your record-to-media mappings to the media you just processed, press Apply Links in the Linking panel. The Review panel will now be updated to display your mappings next to your processed media. Review the thumbnails for your processed media to ensure that they appear properly, and check your record-to-media mappings one last time. When you are sure everything is right, continue to the Publish tab.

Publishing Changes

After reviewing your processed media, you must publish it into your collection. Currently, your media is not associated with a collection, but instead with the Collection Manager. Publishing enables content to be seen within Insight.

To publish your processed media:

  1. Select specific media to publish in the left-hand Batch window to bring up thumbnails of the batch's contents. Press Select All Unpublished Media.


  1. Press ↓↓ to add the media to your collection.


  1. Press Apply Changes to publish your media files.


Updating Links between Object and Media Records after Publishing

Once images have been published to Insight, their links can be updated by returning to the batch and applying a new mapping file in the linking section. If you do not have a mapping file, you can also modify specific records using the record-to-media-mapping tool in either Inscribe or the Administrator Tools.

Modifying a Batch

Once the media has been published, it is still possible to modify a batch's contents, including adding new content and linking or re-linking existing content. Simply return to the Media Processing Tool and select the batch you wish to modify.


Deleting Media

To delete a media item, select the image in the Review panel or the Process List and press the delete button. You will be prompted for conformation then the Media will be removed from the system.
NOTE: Media Stored on the server will not be deleted in this process, only references within the Insight Collection Manager database will be deleted. You will not be able to use these images again in Insight.

Deleting Media in Inscribe

Inscribe has a new tool that allows you to delete media. You can find this tool under the edit menu in Manage Media.
The Manage Media Tool allows you to search for media by file name, MediaID or By keyword if it is linked to data.
To delete an item, select it by clicking on it. The item will turn blue. You can select a series by using the Shift key or various images by using the Control key.
Once the images are selected you can press Delete and the images will be marked in red for deletion. You can unmark the images by pressing UnDelete.
When you are ready to commit your selection for deletion, press the Save button.

Once a media item is deleted, you cannot recover it.

Modifying Collection Properties

Users may want to update basic Collection Properties:

  • Modify Collection Properties

To change the Brief Description, Website, or Copyright Statement

  • Modify Collection Theme

To replace the Collection Theme with another pre-existing theme, or to create a new theme

  • Modify Thumb/Sort Settings


NOTE: Once these settings have been changed, you will need to "Save Collection Settings."

Managing Standards for your Collection

Collection owners may choose to limit or control the data standards available for a given collection. The "Manage Field Standards" section enables administrators to affect two specific aspects of the cross-collection searching experience.

Enable/Disable Standards

This allows an administrator to enable/disable a data standard for use with cross-collection searching. It may be useful to disable a standard if only a few fields can properly be mapped between the local data standard and the selected standard or if you would like to increase the likelihood that a specific standard will be used in cross-collection searching.
NOTE: Reducing the enabled field standards to one cross-collection searching standard means that standard will be preferred for all cross-collection searching. If that standard is not available in other collections, users may not be able to cross-collection search between those collections.
To Enable/Disable a Field Standard:

  1. Login to Insight Studio with your administrative username.
  2. Select the collection to modify the standards settings for.
  3. Select Modify Metadata Standards from the main menu, or select Metadata Standards from the Collection Settings section of the task menu.
  4. Select the Standard you wish to modify.
  5. Click the checkbox to the left of the standard to enable or disable it.


NOTE:Remember to "Save Collection Settings" once you make changes.

Specify Standard Order Preference

When a user performs a cross-collection search in the Java Client, the client resolves the standards between the two or more collections. Any standard that is enabled across all selected collections could be used as the data standard returned to the client. Of the standards that are enabled, the one that has the highest preference order in ALL selected collections will be chosen.
To specify the preference order of a Field Standard:

  1. Login to Insight Studio with your administrative username.
  2. Select the collection to modify the standards settings for.
  3. Select Modify Metadata Standards from the main menu, or select Metadata Standards from the Collection Settings section of the task menu.
  4. Select the Standard you wish to modify.
  5. Double-click on the number to the right of the standard name and either raise or lower the number respectively to increase/decrease the preference order.


NOTE:Remember to "Save Collection Settings" once you make changes.

Managing Access Permissions

Collection administrators can manage user access permissions by creating User Groups and assigning specific Profiles to those User Groups. User Groups are then mirrored on the User Manager and associated with specific users.
To Access the Security Settings for your Collection in Insight Studio:

  1. Login to Insight Studio with your administrative username.
  2. Select the collection to manage access for.
  3. Either select Manage Collection Access from the main menu, or select Collection Access from the Access Management section of the task menu.


!worddavbf7ee78836ef05eacb554f282eca0733.wmf|height=339,width=396!

Viewing Existing User Groups

Before you create a new User Group, check the existing User Groups to see if any match the permissions you plan on using.
To view the properties of an existing User Group:

  1. Click on a User Group from the menu on the left.



  1. The name, access/code key, and profile will be displayed on the right.


NOTE: For a description of each of the default profiles, please see Table 16: Default Profiles for Collections and Associated Permissions on page .

Adding a New User Group

A User Group consists of a name, profile, and an access/code key.
NOTE: You can use the Administrator Tools to add IP Address restrictions to a User Group.
To add a new User Group:

  1. Once you are in the Collection Access section of Insight Studio, click the New button at the bottom left.


  1. Enter a User Group Name and Access/code key
  2. Choose a Profile from the pull-down menu.

NOTE: For a description of each of the default profiles, please see Table 16: Default Profiles for Collections and Associated Permissions and Table 17: Personal Insight Manager Default Profile Privleges on pages and .
NOTE:If none of the existing profiles match the security settings you need, you can create a custom profile in the Administrator Tools later. For more information on creating a custom profile, please see Adjusting Profile Settings for a User in a Collection on page .

  1. Press OK to save your new User Group.
  2. Once you've created your new User Group, you must create a matching User Group on your User Manager before you can associate it with a user. For more information on creating a User Group on the User Manager, please see Creating New User Groups on page .