Media Processing in the Administrator Tools

Owned by Luna
Last updated: Sept 04, 2012Version comment
20 min read

LPS, SPS and Image Files

Insight accommodates a wide range of storage structures through the use of a segmented URL path. Any time a piece of media is requested within Insight, these segments are joined together to form a whole URL path to the requested media.
The segments are referred to as SPS (Storage Path Segment), LPS (Logical Path Segment), and filename. The SPS refers to the machine on which the media resides. The LPS refers to the subdirectory structure between the SPS and the filename.
The following is an example of how this URL is formed:
+http://Insight.lunaimaging.com/size4/Disc0004/00044001.jpg+
In the example above, the SPS is http://Insight.lunaimaging.com/size4/ the LPS is Disc0004 and the filename is 00044001.jpg
To understand this even further, you will need to look at the underlying table structure. All media content contained in Insight must have references in the IRImageFiles table. This table is key to the retrieval of media within Insight. This information is produced by the Media Batches tool within the Collection Editing section of the Administrator Tools.
Information from the IRImageFiles table provides a map for the retrieval of media content within Insight. The following pieces of information are pulled from this table, based on the ImageID:

  • A size identifier (0-8) identifies which size will be retrieved. The size will determine which SPS is used.
  • LPSID determines which LPS to use.
  • Format help to specify the appropriate SPS.
  • File Name determines which file to load.

Create New LPS

The LPS (Logical Path Segment) refers to the subdirectory structure of images and refers to the intermediate directories between the SPS and individual media files. Any directory structure can be accommodated within Insight. Some Insight installations opt to create directories according to internal directory structure conventions. The LPS Data tool permits administrators to create and manage the LPS IDs and LPS names for a particular collection.

  1. Right click the LPS Data folder on the navigation tree to create a new LPS reference.
  2. Select New and a blank Edit LPS Data console will be available.
  3. LPS ID – Enter the logical path segment (LPS) ID. The LPS ID field will only accept an LPS ID not already in use.
  4. LPS – Enter the logical path segment (LPS).
  5. Once the Edit LPS Data console has been configured properly, click Save.


Edit LPS Reference

To edit an existing LPS reference, follow the steps below:

  1. Select the LPS Data folder on the navigation tree. Double-click the record set you wish to edit.
  2. The LPS value describes the directory structure or path to a set of media files. In order for Insight to locate and display an image, the URL to the image must be accurate. The Administrator Tools Media Batch tool automatically enters the LPS values when you process a batch of images or media files.
  3. A case where you may need to edit an LPS value is if you renamed the output directory. You would need to change the LPS to reflect the change to the file system directory.


Create New SPS Reference

The Storage Path Segment (SPS) refers to the machine on which the images accessed in Insight reside. This is a partial URL to the media files stored on the image server. To make a complete URL, the system combines the following:
SPS/LPS/File Name
SPS can be used to direct clients to different locations for image retrieval.

  1. Right click the SPS Data folder on the navigation tree to create a new SPS reference. Click New and a blank Edit SPS Data dialog appears.
      1. SPS Number - If you are creating a new SPS grouping, use a new SPS Number (numeric value identifying a SPS group) and make sure to create a SPS reference for each media type and size you need.
      2. Resolution Size - This numeric value (0-8) represents the Insight image size (see Table 14: Image Sizes in Insight on page 241 for exact sizes).
      3. Collection ID - Provide a numeric value for the collection.
      4. Media Type - Choose the appropriate Media type.
      5. Format - Select the appropriate Format type.
      6. URL - Provide the URL that points to source images for this size.
  2. Once you have completed your edits, click Save to apply the changes. You will need to repeat this process for each image size and for each multimedia format.

Review SPS Reference

To review an existing SPS reference:

  1. Select the SPS Data folder on the navigation tree, and double-click the record set you wish to edit.
  2. The selected SPS field values will appear in the Edit SPS Data window.
  3. After making the desired edits, click Save to apply changes. To close the window ignoring any edits, click Cancel to return to the main dialog panel.

Processing MrSID Files Using the Administrator Tools

The purpose of the Media Batch utility is to automate the process of generating derivative images and media files from batches of source images.

Processing a Media Batch

To process a new batch of media files for integration into Insight:

  1. Organize the data and media which you would like to process. Make sure that you have a unique identifier in the data which relates to the images you wish to process. Choose a data field that uniquely identifies a data record that relates to only one media record. (You can process more than one media record to one single data record, but this is rare.)

NOTE:It is recommended that you have descriptive data and related media identifiers (unique identifiers) loaded into your database before you process your media.
In the example below, the names of the media are in the left column and the corresponding unique identifiers are in the right column.

  1. The Media Batch tool can only match to indexed data. If not already indexed, index your data. (Please see Using Insight's Indexer on page for more information.)


  1. Use the Media Batch Tool to process the media.


    1. In the Administrator Tools, connect to your Collection Manager.
    2. In the Collection Manager, open the Collection Editor folder.
    3. Right click Media Batches, and select New.
    4. The Media Batch Window opens.


Figure 11 - Processing a Media Batch



All fields are initially empty in the form for creating a new media batch, except for the LPS Directory field. LPS Directory provides the logical path structure to the derivative images being generated. The LPS is used in conjunction with the storage path segment and the filename to generate the media file URL during an Insight client session. The LPS Directory is auto-generated when a new Media Batch is opened. This is to prevent inadvertent deletion of a former LPS or creation of a duplicate LPS ID.
You can change the LPS Directory by editing the Maximum IDs node in the Administrator Tools.
In the following example, we would like the next media batch to have an LPS Directory of D0030. Setting Max Batch ID to 29 will achieve this.
Figure 12 - Maximum IDs

NOTE:Change Max Batch ID instead of changing the LPS ID directly in Media Batch tool. Managing Batch ID's centrally prevents duplication of LPS ID's in your collection.
Once you have changed your Max Batch ID, open a new Media Batch. You do not need to rename your LPS Directory. You may want to avoid creating repeating LPS Directories for more than one Media Batch.
To view the LPS ID and LPS's already in use, go to the LPS Data node for the collection in Administrator Tools.

  1. Batch Name - The Batch Name serves as a unique identifier for the group of source media to be processed. This name should reflect the media that you are processing. For example, if you are processing maps, you may want to name the batch Map. Also, naming the batch the same name as your LPS Directory may help cut down on confusion.
  2. Profile - The profile defines the set of derivative images to be generated for each source media file and provides a set of processing rules, such as stored filenames and directory structures. You have the ability to create profiles for any situation, which allows for selection of an appropriate derivative scheme for any Media Batch.

NOTE:See Creating an Image Profile (on page ) for more information.

  1. Destination - The destination directory is the location on your local file system where the derivative images should be written. The destination may be the final destination on your server or it may be a temporary destination for the images. When processing is complete, you will want to move the processed media to its final location, which is entered in the SPS Data folder (see Review SPS Reference on page ).


  1. LPS Directory - Verify if this directory is correct. See step 3 for instructions to change the LPS directory. Make sure you do not assign an LPS that has already been used in this collection.


  1. Log File - Choose the location and name of the Log File. The log file is a tab-delimited text file that is generated from the media processing activity. Name the log file something that easily relates to the media you are processing, such as the Batch Name or LPS Directory name (use a .txt or .log extension). This will minimize the chance of confusion with other log files.

The field values generated are: Date/Time, Source File, Resolution Size Being Generated, Output Directory, Derivative File Name, Scale Ratio, Width, Height, Compression Level, and Quality Level. You may import the log files into a database table to track processed source files and the generated derivative images.By default, the Log File is saved in the installation directory. To override the default location, press Browse on the Media Batch console.

  1. (Optional) Starting Media ID (numeric value) - Generated media names will begin with the specified Starting Media ID (if it is not already in use). Not specifying a Starting Media ID results in the highest available ID being used.


Example: If the available ID's are 423, 645, and 800, the system will use ID 800.

  1. Mapping File - The mapping file is a two-column, tab-delimited text file. It lists the source media file (minus the file extensions) in the first column; and the object link value (such as accession) in the second column. This tab-delimited text file should be created by the administrator. The mapping file facilitates the linking of descriptive object records to media files. The mapping file lists the name of the media that you would like to process, minus the extension in the left column, and the unique identifier that is in the database in the right column. This mapping file is what creates the link between the data and the processed media in your database.

Figure 13 - Mapping File

To create the mapping text file, open a spreadsheet application and list the source media files to be processed in the first column. Next, in the second column, provide the object link value associated with the appropriate source media file. Save the file to your file system as a text (.txt) file, then select Browse from the Media Batch configuration window and navigate to it. Select the mapping file, then click load under the Object Link list.

  1. Field - The field function enables you to choose which data field the Object Links and Source Files will link to the media. Choose a field by clicking on the visible field name. This field is now available for linking. Only one field may be chosen from the drop down list.For example, a user would like to process an image, which has an object link named nu689, which will be linked to a source file called figure14.jpg (object links and source files are explained later). They know that the field Accession Number (which is available from the drop down box to the right of Field(smile) contains an object record named nu689 in the database. This link between the object link nu689 and object record in the field called Accession Number is confirmed once the user presses Match Objects & Media. This button will display all object links that have a corresponding link in the database based on the field they selected.If the particular object does not have a link, it will not be automatically moved to the Process list (database) once the Match Objects & Media button is selected. You will still see it in your Source File list.If you would like to select a field that is not in the Field list, you will need to add that field and data into your database. You will need to re-index before it will be available here (see Using Insight's Indexer on page for more information).


  1. Object Links - The object link values are listed in the selected mapping file. To load the mapping file, click Load. The mapping file destination must be entered in the mapping file column or the object links will not load.


  1. Source Files - The source files are records that are related to an image. They can be graphic (such as JPEG, TIF), sound (for example, Wav, MP3), or other multimedia files (such as Mov).Click Load, navigate to the directories containing the source files. Select the source files that are listed in the selected mapping file. The object link list should contain the object link values for the source files you select.

NOTE:You are able to select one to many media items at a time. To select more than one media item, select the first item and hold down the Shift key, select the last media item in the group that you would like to bring into the Media Batch. Now, there should be numerous media items selected. Click open, this will bring the selected media into the Source Files field.
The Media Batches utility currently creates JPEG, SID and JPEG2000 image file formats. Original images can be .tif or .jpeg files. Luna suggests using the JPEG file format for resolutions up to 1536 (lengthwise). By default, the Insight profile will generate the standard set of derivative image sizes and formats commonly used for Insight installations. The Insight profile consists of 5 JPEG resolutions (spanning from 96 pixels lengthwise to 1536 pixels lengthwise) and a single multi-resolution encapsulated wavelet image file (either SID or JPEG2000).

  1. Match Objects & Media - Compares the Object Link list from the selected mapping file with the selected source files and inserts the matching pairs into the Processing List. Only media that matches will be linked. If a source file and object do not link, that source file will remain in the Source Files column.

NOTE: You are permitted to link source files and data by hand. To do this, select the object link and source file to be linked and click the >> button. The object link and source file will be added to the Process List. This link, and any media files that have not yet been processed, will be labeled as PENDING.
To remove or unlink source files and data (object link), click on them from the Process List and press <<. This will put the source files and object links back in their appropriate columns. They will be available to link again later.

  1. Process List - The process list contains all media that will be and has been processed in a particular media batch.


After processing a media batch, a verification step is performed to check whether all media records were committed to the database correctly. If there is a discrepancy in the media record count, you will be alerted with a warning message.

Running Media Creation Process

Once the Process List is populated with linked Objects and Source Files, they will be categorized as PENDING in the Process list window. This means that the files have not been run but are ready to be. To run the media, click Start. A Media Creation window will come up and you can monitor the processing progress.
Figure 14 - Monitor Progress for Media Creation

When the media has been processed correctly, it will be labeled PROCESSED-SUCCESS in the Process List column.
If you browse to the destination location where processed media has been saved, you will find the Log File and processed media which has been formatted as defined in the Profile. A new LPS entry will be posted in the collection, which can be seen by selecting the LPS Data node in the Administrator Tools main screen.
For additional information on LPS Data, refer to LPS, SPS and Image Files on page in this manual.
Save the batch by pressing Save in the Media Batch window.
NOTE: If you stop the Media Batch, lose connection to your computer, or close the Administrator Tools, the Media Batch will stop where it is. If you open your saved Media Batch again and click start, it will pick up where it left off.
To verify that a Media Batch completed normally:

  1. To view all saved or processed Media Batches, select the Media Batches node under Collection Editor. Double click on a Media Batch to edit it. Right-click on the completed Media Batch and select Tools from the pop up context menu. Which will pop up another menu. Select Map objects to media, which will open the Manage Objects-to-Media Mappings Window.

Figure 15 - Working with Media Batches

  1. Verify links from the mapping file in the Manage Objects-to-Media Mappings window for the selected Media Batch. Only the Media Records processed in the selected Media Batch that have not yet been published will be available. You can link, unlink, and change links with this tool. (To learn more about the Map objects to media function, refer to Manage Objects-to-Media Mappings on page .)

Figure 16 - Objects to Data Expanded View

  1. Publish media – Use this option once the processed media group is correct and ready to be published into a collection. To publish the processed material in a New Collection Group, fill in the new group name and then select Publish To New. However, before you publish the media, physically move the processed media from the original destination location to the location defined in the SPS. This will ensure that the proper media is available when it is published.To publish the processed material into an existing group, select the desired group name and select Publish To Existing.


Figure 17 - Publishing Media Batches

  1. Finally, update the thumbnail caches. For instructions on updating thumbnail caches, please see Updating a Server's Thumbnail Caches on page .



Once you have updated the Thumbnail Cache, your media should be available in your Insight collection. Then, to review your log file, see Reviewing and Modifying Mapped Media on page .

Media Batch Tool Options

New media batch – If you select New media batch, a blank Media Batch window appears. You can design a media batch as described in the
Processing a Media Batch section on page .
Edit media batch – Enables the media batch that you select to be available with all of its fields pre-assigned. You can now edit or change this media batch.
Delete media batch – Deletes the media batch from Administrator Tools. You will receive a confirmation prompt to prevent you from accidentally deleting a Media Batch.
Tools – You will have the option to Process media batch, Map objects to media, and Publish media. For more information on Map objects to media or Publish media, see steps 2 and 3 above.
Process media batch – Enables you to process a Media Batch that is already created and saved. When this option is selected, the selected Media Batch window opens. Click start to process the media batch. You may choose to use the same set-up for every media batch, with only a unique LPS Directory, which will be automatically assigned. This way, you can process your media in the same fashion every time.

Reviewing and Modifying Mapped Media

After you have processed a Media Batch, Administrator Tools creates a log file listing field values for your image files. The media creation log file includes the media type codes for easier identification of your image files. An example of a type code would be 01052000.jpg. The media creation log file can be found in the directory that you originally assigned while you were creating a new Media Batch (for additional information, refer to
Processing a Media Batch on page ).
Figure 18 - Reviewing and Modifying Mapped Media

Creating an Image Profile

Image Profiles define the set of derivative sizes and file types that are generated for each source media file. Profiles may be reused on successive batches of source images.

  1. Press Configure (on the Media Batch dialog) to open the Image Creation Profile. The Image Creation Profile enables you to create, view, edit, copy, and delete profiles. The default Insight Profile cannot be altered. This profile contains specific settings needed to process images for Insight using Luna naming conventions (discussed later in this section).


  1. To create a new, blank profile by selecting New.


  1. To change the existing profile, select Copy. This will bring up a copy of the active profile. In this view, you are able to give the profile a new name, assign a URL Prefix, a SID URL Prefix and change other items from the original profile such as Image Type settings and select Preserve Filenames or Preserve Directory Structure.


  1. Check Preserve Filenames to preserve source file names (the tool will only apply an extension change).


If the user chooses Preserve Filenames, all derivatives (for all JPEG sizes and the Wavelet file) will have the same name.
If Preserve Filename is not selected, the media will be renamed by Luna naming conventions.
The Luna naming convention is a 12-character format
(e.g., 00012001.JPG ), made up of four parts:

    1. A Four-digit Batch ID# (ranging from 0000-9999). In the example 00012001.JPG, the batch is 0001.
    2. Single Digit Image Size Identifier. In the example 00012001.JPG, the Single Digit Image Size Identifier is 2. This is a size two image.
    3. A three-digit image sequence (ranging from 000-999). In the example 00012001.JPG, the three-digit image sequence number is 001. This is the first image in the batch 0001.
    4. A Three-Character File Format Extension. In the example 00012001.JPG, the three-character file format extension is .JPG. This is a JPEG image.

Figure 19 - Create Image Profile


Directory Structure Convention

  1. If the Preserve Directory Structure checkbox is selected for the active image creation profile, a new select button will appear to the right of the LPS Directory setting. The LPS Directory setting is the field below Destination on the Media Batch console.


Click Select (to the right of LPS Directory), to open the LPS Directory Structures dialog, which contains two path statements: Default Directory and End LPS At. For the Default Directory, click Browse and navigate to the level where everything below will be preserved and replicated for each of the derivative image sizes being generated. For example, if your source directory path was
E:\Dalton\TIFFS\Wwm9912\Cas009\007149.tif
and you wanted to maintain the bi-directory structure below TIFFS; you would select the TIFFS directory to be the default directory. As a result, Wwm9912/Cas009 will be the LPS and the Size0 directory path would be:
http://luna01.dalton.org/jpegs/Size0/Wwm9912/Cas009/007149.jpg
Figure 20 - LPS Directory Structure

If you choose not to select the Preserve Directory Structure, your batch process will process the images into the assigned LPS Directory listed in the Media Batch Console. Generally the LPS directory will follow a logical sequence after every batch processed, starting with D0001 and then going to D0002 and so forth. When you check the Preserve Directory structure option, this can be altered to your directory specifications.

  1. In the Image Types section of the Image Creation Profile, you are able to determine the type of images that will be created (JPEG, SID or JPEG2000 images), Insight sizes, as well as other specific settings by selecting the Settings button to the right of a particular image type.


These are the options available for the Image Type, JPEG:
Figure 21 - JPEG Image Type

These are the options available for the Image Type, SID:
Figure 22 - SID Image Type

There is not an options set for Image Type, JPEG2000.

  1. To add or remove Image Types, you will select either Add or Remove from the Image Creation Profile dialog to add or remove JPEG, SID and JPEG2000 images from Insight. The Image Creation Profile option enables you to customize your images (different sizes and resolutions) as well as to add or remove images from Insight. h2. Manage Objects-to-Media Mappings
    The Manage Objects-to-Media Mappings Tool provides you with an interface through which the relationship between published media files and descriptive records can be defined. The interface allows you to search through a collection's media files and descriptive object records to create, modify, and delete the relations between object records and media files.
  2. Expand the Collection Editor node, right-click the Map Objects-to-Media node and select Run Directly. This will open the Manage Objects-to-Media Mappings window.

Figure 23 - Map Objects to Media

The Manage Objects-to-Media Mappings window is organized in an easy-to-use fashion. The left panel displays the Object Data Records for the collection, the right panel displays the Media Records.

  1. When you click on either an item in either panel, the selected item will be highlighted in red. This item is now active. Any items linked to it will be highlighted in blue in the opposite panel. If no blue items appear in the opposite panel, the selected item is an Orphan. An Orphan is a Media Record or Object Data Record without a link, and will not be available in Insight.


To map an object data record to a particular media file, select the media file. The selected record should now have a red highlight indicating it as the active element. Double-click any media file to determine the number of object records that are mapped to that media file. The list of mapped object records will appear in blue highlight.
Figure 24 - Linking Object with Image

If the ID number for the selected media file is known (see
Processing a Media Batch on page ), it may be useful to search for the associated object data record using the Find function. If the known information about the media file is from another field, use the Change data fields function to update the display and search fields to aid you in your search.
Click on any Media Record to link it to your Object Data Record. You will be prompted with a Mapping Confirmation window which states that you are about to map the selected object to the selected medium.
Figure 25 - Mapping Confirmation

  1. Clicking a blue item will sever the mapping (a confirmation prompt will prevent you from doing this accidentally).


  1. Object Records can be mapped to Media Records. Media Records can be mapped to Object records. Mappings can be severed from either side.


  1. Press Undo to reverse the most recent mapping change. All mapping changes made during the session can be "Undone" until Save commits changes to the database. Once you have completed your mapping changes, press Save to apply the changes to the database.

Object Data Records

The Object Data Records pane, which is located on the left side of the Manage Objects-to-Media Mappings console, lists all the data for the selected collection.

  1. You can display up to two data fields for each record in this window. To view and change which two data fields are being displayed and in what order, click the Change data fields button, which is located in the bottom left corner of the dialog. This will allow you to choose which data fields you would like to display for the Object Data Records.
  2. To view all data for a particular data record, right-click the record and select View all data.







Figure 26 - Object Data Records Expanded View

  1. To search for a particular image, type in any data associated with the active data field in the white text box. The data entered does not have to exactly match the item for which you are searching. All similar records will be displayed in the Object Data Records window when the search is complete. Once the data for the search is entered, click Find. The results will display in the Object Data Records window. Change the searchable data field by clicking the Change data fields button and selecting the desired field.


  1. Roll your mouse over a particular data record, and hold it there for a second or so without clicking on it, the data identification number will appear. This can be useful for quickly identifying a particular data record without having to view all the data or performing a search.


If you would like to view all the Orphans (Data Records without a Media Record link), press Show Orphans. Only orphans will be available. To view all the data records, press Show All.

Media Records

The Media Records pane will display any available media in the collection. When a media record is selected it will become highlight in red.

  1. To see a larger view of a media record, right-click it. Select View Larger Image from the pop-up menu, and a larger image will open in its own viewing window. This function is useful when trying to view a detail of a particular image.

Figure 27 - Expanded Image View

  1. To search for a media record, you must know that record's exact ID number. Enter the number in the white box in the right corner of the dialog and click Find. The result will display in the Media Records window. You can also find a particular image ID number by rolling your mouse over an image and holding it there for a second or so without clicking on it. The ID number for that image will appear. This can be useful for quickly identifying a particular Media Record without having to perform a search.


To see all orphans (media records unmapped to any object records), press Show Orphans. Only orphans will now be available. To view all media records, press Show All.