LUNA 7 Backend Overview

Introduction

Here you'll find details related to the backend of LUNA 7.

The data in LUNA is stored and accessed in a several different ways. We use a database, files for the repository on the filesystem and Solr/Lucene. These different components allow us to provide quick access to content and relational database concepts where they make sense.

The LUNA database

Mostly used to store user generated information like permissions and data. Information stored includes:

Media Groups
Label names
Content references
Tiny URL's
Annotations
Users
Privileges

The Data Repository Filesystem

For each and every record, media item and authority we store an XML document that contains some type of descriptive metadata.

Below is an example of a data record. By storing the data in this XML format, the structure of data is maintained and accessible.

<?xml version="1.0" encoding="UTF-8"?>
<entityList>
  <entity name="object_type">
    <entity name="author" id="6398">
      <field name="author">
        <value>Lucas, Fielding Jr.</value>
      </field>
    </entity>
    <entity name="date" id="1528">
      <field name="date">
        <value>1823</value>
      </field>
    </entity>
    <entity name="publisher" id="6400">
      <field name="publisher_location">
        <value>Baltimore</value>
      </field>
      <field name="publisher">
        <value>Fielding Lucas Jr.</value>
      </field>
    </entity>
    <entity name="type" id="216">
      <field name="type">
        <value>Covers</value>
      </field>
    </entity>
    <entity name="world_area" id="102">
      <field name="world_area">
        <value>World</value>
      </field>
    </entity>
    <entity name="publication_author" id="6399">
      <field name="publication_author">
        <value>Lucas, Fielding Jr.</value>
      </field>
    </entity>
    <entity name="pub_type" id="3">
      <field name="pub_type">
        <value>World Atlas</value>
      </field>
    </entity>
    <field name="author_thumbnail_label">
      <value>Lucas, Fielding Jr.</value>
    </field>
    <field name="browse_title_sort">
      <value>Covers to) A General Atlas Containing Distinct Maps Of all the Known Countries in the World, Constructed from the Latest Authority. Written and Engraved by Jos. Perkins, Philadelphia. Baltimore: Published by Fielding Lucas, Jun. No. 138, Market Street. Baltimore. Entered ... by F. Lucas Jr. of the State of Maryland June 3, 1823.</value>
    </field>
    <field name="mrsidsoftwareurl">
      <value>&lt;a href=http://www.davidrumsey.com/mrsidviewer.zip target=_blank &gt;MrSID Image Viewer for Windows&lt;/a&gt;</value>
    </field>
    <field name="fullsidurl">
      <value>&lt;a href=http://www.davidrumsey.com/rumsey/download.pl?image=/D0001/4584A.sid target=_blank&gt;Full Image Download in MrSID Format&lt;/a&gt;</value>
    </field>
    <field name="publication_author_initialsort">
      <value>Lucas, Fielding Jr.</value>
    </field>
    <field name="ownership_statement">
      <value>Copyright 1998</value>
    </field>
    <field name="institution">
      <value>Rumsey Collection</value>
    </field>
    <field name="image_no">
      <value>4584A</value>
    </field>
    <field name="pub_width_cm">
      <value>32</value>
    </field>
    <field name="pub_height_cm">
      <value>39</value>
    </field>
    <field name="pub_maps">
      <value>100</value>
    </field>
    <field name="pub_list_no_initialsort">
      <value>4584.000_001</value>
    </field>
    <field name="pub_list_no">
      <value>4584.000</value>
    </field>
    <field name="pub_note">
      <value>The first edition, first state (see our first edition, second state for comparison) of Lucas' best general atlas and the finest general atlas produced in the U.S. at that time (In asserting this we omit the Tanner and Finley American Atlases of 1823 and 1826 respectively because they were not general atlases - cartographically they may have been superior to the Lucas, but not as broad in coverage). The quality of the engraving (most of the U.S. maps were engraved by B.T. Welch, others by Young &amp; Delker, J.V.N. Throop, Cone &amp; Freeman, and Kneass) is superb, the detail is very fine, and the coloring is delicate and elegant. While the same Lucas drawn base maps were used in this atlas and the 1822 American Atlas by Carey and Lea (and thereby created strained relations between him and Carey and Lea), the maps in this Lucas Atlas are far superior in quality - Welch reengraved many of the maps for Lucas that Young &amp; Delker had engraved for Carey and Lea. It is interesting to note that the Carey and Lea maps are usually earlier states of the Lucas maps. The Atlantic Islands and the West Indies Island Maps are mostly copied from Thomson's General Atlas of 1817 (see our copy) as is the Mountains and Rivers plate (reduced). Many maps are derived from the earlier Lucas Atlas of 1815-17 which copied the Oddy Atlas maps for the non U.S. state maps. Some copies have solid color. Maps of Mexico, Canada, and N. America (updated by Lucas in the arctic), are copied from Arrowsmith's General Atlas of 1823 (first ed. was 1817). The classical maps derive from Melish's Atlas of Ancient Geography of 1815. Half leather with marbled paper covered boards, thick outline color, title on spine - "Lucas's Cabinet Atlas."</value>
    </field>
    <field name="pub_reference">
      <value>P742.</value>
    </field>
    <field name="pub_title">
      <value>A General Atlas Containing Distinct Maps Of all the Known Countries in the World, Constructed from the Latest Authority. Written and Engraved by Jos. Perkins, Philadelphia. Baltimore: Published by Fielding Lucas, Jun. No. 138, Market Street. Baltimore. Entered ... by F. Lucas Jr. of the State of Maryland June 3, 1823.</value>
    </field>
    <field name="pub_date">
      <value>1823</value>
    </field>
    <field name="series_no">
      <value>1</value>
    </field>
    <field name="list_no">
      <value>4584A</value>
    </field>
    <field name="full_title">
      <value>(Covers to) A General Atlas Containing Distinct Maps Of all the Known Countries in the World, Constructed from the Latest Authority. Written and Engraved by Jos. Perkins, Philadelphia. Baltimore: Published by Fielding Lucas, Jun. No. 138, Market Street. Baltimore. Entered ... by F. Lucas Jr. of the State of Maryland June 3, 1823.</value>
    </field>
    <field name="obj_width_cm">
      <value>32</value>
    </field>
    <field name="obj_height_cm">
      <value>39</value>
    </field>
    <field name="short_title">
      <value>Cover: General atlas.</value>
    </field>
  </entity>
</entityList>

Solr/Lucene indexes

Solr/Lucene are used in a number of ways. Not as a database, but more like a cache and search index. We do this to create fast searches of large amounts of data. This eliminates complex queries that are required in database searches. It's also the current methodology/technology for this type of work. These Solr/Lucene indexes can be rebuilt from the XML repository if there is any corruption or inconsistencies.

Putting this all together

What generally happens on a typical record creation/edit is that a new set of XML documents are created and stored in the LUNA repository and the content is added to the Solr/lucene indexes.