Technology, Data Storage, and Backup

Maine Memory Network has pioneered the many-to-many model of information sharing for history-related websites. Input is "distributed," meaning content is created not just by the Maine Historical Society staff, but by collecting organizations across Maine, by students, teachers, historians, and individuals. Output is "dynamic" meaning end-users can create their own content and collections of historical items for collaboration, schoolwork, and information sharing.

When we built the prototype in 2000, rather than create a web application from the ground up (which would have cost an estimated $150,000), or adapt existing library or museum software (which was limited in its functionality), we licensed stock-photography software from a local photo agency in Portland, Maine. For a fraction of the cost, we adapted their existing software and had them host the site.

The entire system underwent an infrastructure overhaul and re-launch in late 2005. We migrated the software and database to our own servers and took over all aspects of technology production. W3C.org and ADA compliant XHTML 1.1 templates were designed and implemented, which improved usability, provided better access to the blind and visually impaired, and allows for future compatibility with compliant browsers. The ModPerl code was "refactored." Technical documentation was created, and a network of developers and designers was cultivated. Because resources are limited, we developed the second iteration of the site to allow for easy global changes: HTML and CSS are completely separate; mastheads, footers, and the navigation bars are single include files.

Browse Tool and authority files:

Our search abilities were greatly enhanced by the use of our browse tool. If you have ever used a search engine that uses a human-compiled index, the browse tool should look familiar. Information is indexed by category: there are 15 top level categories, approximately 85 second level categories, and approximately 300 third level categories. The categories were thoughtfully chosen – the product of extensive planning and endless focus groups, as no suitable state or national standard existed. We felt a library standard would be difficult for web users who, in their homes and schools, wouldn't have the aid of a librarian. Also, we know which topics and phrases are searched on most, so creating our own list was an opportunity to bring some of the more popular searches to the forefront,while providing related themes. You can download our list here (.pdf format).

Hosting:

From 2016 to present, all MHS websites (www.mainehistory.org, www.mainememory.net, www.vintagemaineimages.com, and www.hwlongfellow.org) are hosted on virtual private servers at Linode (https://www.linode.com/) and run the CentOS Linux operating system. The MHS websites also use the Amazon Simple Storage Service (Amazon S3) and Amazon CloudFront content delivery network (CDN) to host and serve the MMN image files. Amazon S3 is also used to provide one of the geographically separate backup locations of critical data.

Frontend:

The websites are served using the Apache webserver. Apache communicates with the website backends using mod_perl and FastCGI. The website frontends use HTML, CSS, and Javascript. Several Javascript libraries are used, including jQuery, MochiKit, Scriptaculous, and Prototype. Open Seadragon (previously known as Deep Zoom) is used to provide the image zooming capability.

Development:

Maine Memory Network was developed primarily in Perl, using "Modern Perl" development practices and packages, including the Catalyst web framework, DBIx::Class, and Moose. PHP is also used in a few select places as well. Sites use the MySQL database for data storage and retrieval, and use Lucene and Xapian to provide search capabilities. Many other software packages are used on the servers for their utility, and to maintain the servers' stability and security. These include the Pure-FTPd FTP server, the Postfix email server, fail2ban and rkhunter for security, supervisord for process management, and ImageMagick and GD for image manipulation.

MHS uses several tools for supporting the software development lifecycle. Primarily, these include Subversion for software version control, and the Trac software package (by Edgewall Software) for issue tracking, project management, and internal/technical documentation. Additionally, the MHS websites integrate a number of third-party services: Disqus provides the website comment system; SoundCloud, Vimeo, and YouTube are used to embed some content into the websites; Google Analytics is used to help understand the website usage, trends, and popular content. MHS is committed to Open Source software because of the advantages in cost, security, and stability, as well as its community of dedicated programmers.

Digital File Formats

Every items that is scanned on a flatbed scanner has two files. The preservation master and the accessible derivative. Preservation master: 41-43 MB TIF approx.. 3300 x 4200 pixels; Accessible image: 41-43 MB JPG (opened), 5-10 MB JPG (closed) (print quality of 300dpi @ 11"x14") 3300 x 4200 pixels, with minimal compression (approx. 10%); 24 bit depth; RGB mode. Originals are archived without retouching. Video: MOV or MP4; Audio: WAV or AIFF with 96 kHz sampling rate, 24 bit, if feasible, otherwise 44.1 kHz sampling rate. These are converted to MP3 for accessibility on the web.

We encourage contributors to Maine Memory Network to follow our guidelines.

We review our digital archiving media and file formats on an ongoing basis to ensure they remain accessible. We also have a photo studio to photograph oversized documents and three dimensional museum items. Learn more about our digitizing standards and procedures in the Contributing Partners manual.

Back to Top