YUL ITS Hydra News

The Avalon team at Indiana University and Northwestern University is pleased to announce the release of Avalon Media System 4.0.1. Version 4.0.1 adds the following capabilities over the previous version 3.3:

·         New Metadata Fields: Notes, More Identifiers, Table of Contents
·         Structural Metadata: Navigation by structure, create and edit structure
·         Wowza Support: Wowza Media Server integration with Avalon’s authorization
·         Hydra 8: Latest versions of Fedora 3 backed Hydra dependencies
·         Accessibility Improvements: Navigation of web page elements with screen reader and tab key; using player controls with keyboard
·         Active Encode: New transcoding API to support use of alternative transcoding engines
·         oEmbed Support: Avalon provides an oEmbed service with autodiscovery headers on item pages
·         LTI integration improvements: Avalon provides LTI URLs for direct access to item or sections when used with a Learning Management System such as Canvas
·         Offset URLs: Avalon users can now create URLs that jump to specified points within video or audio files.

Current users of Avalon 3.3 can take advantage of these new features by upgrading Avalon 3.3 to 4.0.1.

For a more comprehensive list of changes, see the 4.0 release notes. For more details on each of these new features, visit the What’s New in Avalon 4.0 wiki page.

The quarterly report for Fedora for January – March 2015 is now available:

http://duraspace.org/node/2625

The report details much of what happened in the first three months of this year but also covers a number of general topics from January to June.

Highlights include:

• Fedora Development
• Fundraising
• Fedora Committers
• Upcoming training and events

Big news! On Tuesday [June 16], eleven world-leading insttitutions agreed to form the IIIF Consortium, a member organization dedicated to sustaining and advancing the International Image Interoperability Framework (IIIF). The consortium will support the work of IIIF by pooling and allocating funding from members for exposing content via IIIF; doing outreach, training, and advocacy to grow the community; maintaining and elaborating on the IIIF technical specifications; providing catalytic support for IIIF-compatible software development; helping coordinate the IIIF communit, and more.

The Consortium will reinforce and amplify the work currently done within IIIF community by providing ongoing support for the IIIF working group, editorial efforts, community calls and meetings. A core principle of the Consortium is that it derives its strengths from the active participation of a distributed community, and therefore values the diversity, inclusiveness and transparency that have characterized IIIF since its inception.

The consortium will extend invitations to additonal Founding Members later this year. The first eleven institutions in the IIIF Consortium have each played prominent roles in helping develop IIIF to its current state, and have committed to helping grow it over the longterm. These institutions are:

• Artstor
• Bavarian State Library
• British Library
• Cornell Univeristy
• National Library of France
• National Library of Norway
• Oxford Univeristy
• Princeton University
• Stanford Univeristy
• Yale University
• Wellcome Trust

For next steps, we will continue to work to expand the number of IIIF-compatible repositories and software packages in the world, and do ongoing development of the technical specifications on authentication, search within objects, and discovery of IIIF resources for use.

This is a watershed development as it signals major, international commitment at an institutional level for the Framework, and the beginning of organized support to help sustain the benefits of interoperability over the longterm.

To read more about IIIF visit: http://iiif.io/

Indiana University and WGBH recently presented their plans for the grant funded HydraDAM 2 project. Some interesting bullets from their presentation:

• HydraDAM 1 came from a need for WGBH to migrate off the vendor product Artesia which was heading in a new direction
• Indian University’s use case is to ingest 10 Terabytes per day for 4 years for a total of 6.6 Petabytes of master and use copy video files along with associated files for preservation into HydraDAM 2
• HydraDam 1 is too slow for ingest so ingest is handled externally
• HydraDam2 will use two different storage system models with Fedora 4 managing both online/nearline and offline tape copies
• Out of region copies are out of scope for the size of the collection going in, however, IU is a DPN member and plans to use that for high risk items. Currently they are in the process of setting policies and preservation levels associated with the content.
• Preservation services to be offered in HydraDam2 include:
  • Storage and retrieval of files
  • Scheduled fixity checks and file characterization on demand
  • Auditing based on Fedora 4
  • Reporting
  • Media migration (from one storage solution to another storage solution)
  • Format migration for risk of obsolescence
• There is a working version of Avalon using Fedora 4

This was a preliminary presentation. IU and WGBH will be giving a detailed presentation at the upcoming Open Repositories conference in June.

Excerpt:

The University of Michigan Press, which is part of the University of Michigan Library, will collaborate with the university presses at Indiana, Minnesota, Northwestern and Penn State universities (each institution will receive funds from Mellon) to build workflows and infrastructure using Hydra/Fedora, a robust and flexible technical framework and repository system.

Read more about this 3 year, $899,000 grant:
http://ns.umich.edu/new/releases/22771-mellon-funds-u-m-press-collaboration-to-create-new-ecosystem-for-digital-scholarship

FOR IMMEDIATE RELEASE

April 15, 2015

Read it online: http://bit.ly/1OwQQU7

Contact: Dan Cohen (dan@dp.la), Tom Cramer (tcramer@stanford.edu) or Debra Hanken Kurtz (dkurtz@duraspace.org)

Far-reaching “Hydra-in-a-Box” Joint Initiative Funded by IMLS

A tripartite DPLA, Stanford University, and DuraSpace partnership will produce a turnkey, Hydra-based solution that can be widely and easily adopted by institutions nationwide.

Boston, MA  The Digital Public Library of America (DPLA), Stanford University, and the DuraSpace organization are pleased to announce that their joint initiative has been awarded a $2M National Leadership Grant from the Institute of Museum and Library Services (IMLS). Nicknamed Hydra-in-a-Box, the project aims foster a new, national, library network through a community-based repository system, enabling discovery, interoperability and reuse of digital resources by people from this country and around the world.

This transformative network is based on advanced repositories that not only empower local institutions with new asset management capabilities, but also interconnect their data and collections through a shared platform.

“At the core of the Digital Public Library of America is our national network of hubs, and they need the systems envisioned by this project,” said Dan Cohen, DPLA’s executive director. “By combining contemporary technologies for aggregating, storing, enhancing, and serving cultural heritage content, we expect this new stack will be a huge boon to DPLA and to the broader digital library community. In addition, I’m thrilled that the project brings together the expertise of DuraSpace, Stanford, and DPLA.”

Each of the partners will fulfill specific roles in the joint initiative. Stanford will use its existing leadership in the Hydra Project to develop core components, in concert with the broader Hydra community. DPLA will focus on the connective tissue between hubs, mapping, and crosswalks to DPLA’s metadata application profile, and infrastructure to support metadata enhancement and remediation. DuraSpace will use its expertise in building and serving repositories, and doing so at scale, to construct the back-end systems for Hydra hosting.

“DuraSpace is excited to provide the infrastructure for this project,” said Debra Hanken Kurtz, DuraSpace CEO. “It aligns perfectly with our mission to steward the scholarly and cultural heritage records and make them accessible for current and future generations. We look forward to working with DPLA and Stanford to support their work and that of the community to ensure a robust and sustainable future for ‘Hydra-in-a-Box.’”

Over the project’s 30-month time frame, the partners will engage with libraries, archives, and museums nationwide, especially current and prospective DPLA hubs and the Hydra community, to systematically capture the needs for a next-generation, open source, digital repository. They will collaboratively extend the existing Hydra project codebase to build, bundle, and promote a feature-complete, robust digital repository that is easy to install, configure, and maintain—in short, a next-generation digital repository that will work for institutions large and small, and is capable of running as a hosted service. Finally, starting with DPLA’s own metadata aggregation services, the partners will work to ensure that these repositories have the necessary affordances to support networked aggregation, discovery, management and access to these resources, producing a shared, sustainable, nationwide platform.

“The Hydra Project has already demonstrated enormous traction and value as a best-in-class digital repository for institutions like Stanford,” said Tom Cramer, Chief Technology Strategist at the Stanford University Libraries. “And yet there is so much more to do. This grant will provide the means to rapidly accelerate Hydra’s rate of development and adoption–expanding its community, features and value all at once.”

To find out more about the Hydra-in-a-Box initiative contact Dan Cohen (dan@dp.la), Tom Cramer (tcramer@stanford.edu) or Debra Hanken Kurtz (dkurtz@duraspace.org). An information page is available here: https://wiki.duraspace.org/display/hydra/Hydra+in+a+Box.

About DPLA

The Digital Public Library of America (http://dp.la) strives to contain the full breadth of human expression, from the written word, to works of art and culture, to records of America’s heritage, to the efforts and data of science. Since launching in April 2013, it has aggregated over 8.5 million items from over 1,700 institutions. The DPLA is a registered 501(c)(3) non-profit.

About DuraSpace

DuraSpace (http://duraspace.org), an independent 501(c)(3) not-for-profit organization providing leadership and innovation for open technologies that promote durable, persistent access to digital data. We collaborate with academic, scientific, cultural, and technology communities by supporting projects (DSpace, Fedora, VIVO) and creating services (DuraCloud, DSpaceDirect, ArchivesDirect) to help ensure that current and future generations have access to our collective digital heritage. Our values are expressed in our organizational byline, “Committed to our digital future.”

About Stanford University Libraries

The Stanford University Libraries (http://library.stanford.edu) is internationally recognized as a leader among research libraries, and in leveraging digital technology to support scholarship in the age of information. It is a founder of both the Hydra Project and the Fedora 4 repository effort, and a leading institution in the International Image Interoperability Framework (IIIF) (http://iiif.io).

About the Hydra Project

The Hydra Project (http://projecthydra.org) is both an open source community and a suite of software that provides a flexible and robust framework for managing, preserving, and providing access to digital assets. The project motto, “One body, many heads,” speaks to the flexibility provided by Hydra’s modern, modular architecture, and the power of combining a robust repository backend (the “body”) with flexible, tailored, user interfaces (“heads”). Co-designed and developed in concert with Fedora 4, the extensible, durable, and widely used repository software, the Hydra/Fedora stack is centerpiece of a thriving and rapidly expanding open source community poised to easy-to-implement solution.

From the Hydra Community:

The PSU team successfully deployed ScholarSphere on Fedora 4 into production this past Saturday April 12th.

It took approximately 20 hours to migrate our code and our data from Fedora 3 (Sufia 5) to Fedora 4 (Sufia 6) on our production servers.

The transition went smoothly and our upgraded system has been running well for the past two days.

Thank you to my team (Adam Wead, Hector Correra, and Michael Tribone), Penn State ITS Services an Solutions, Penn State Libraries, and the Hydra community for all the hard work and funding that made this transition possible!

The following was recently announced to the IIIF community. While we have not yet adopted IIIF or Mirador in the Yale Library, these technologies are in use on campus. We anticipate adopting them as we plan to update our current Hydra platform.

 

We are excited to (officially) announce the release of Mirador version 2.0.  Please visit our new project website at http://projectmirador.org. Here you will find a live demo, a four minute screencast demonstrating Mirador 2.0’s features, and links to the code repository and documentation (https://github.com/IIIF/mirador/).

The 2.0 release of Mirador builds and improves upon the first release with major user interface improvements and a rich feature set. These include:

• Deep zoom and pan using OpenSeadragon
• Multiple viewing modes, including single image, two-page, horizontal scroll and thumbnail gallery
• Synchronized navigation of multi-image objects by filmstrip or table of contents (when available)
• Metadata view
• Comparison of multiple images in a fully configurable workspace
• State saving and bookmarking for sharing a workspace
• Embeddable in blogs and third-party web apps
• Annotation 

Notably, Mirador now supports viewing and creation of annotations on regions of images. The annotation functionality is fully compatible with the OpenAnnotation specification (http://www.openannotation.org/), and of course Mirador 2.0 is fully compliant with the IIIF Image and Presentation API’s (http://iiif.io). 

A variety of features are in the backlog for the next version, and you can view the updated roadmap at https://github.com/IIIF/mirador/wiki/Mirador-2.1-Roadmap. 

Mirador 2.0 is the result of a gratifying global collaboration. Many thanks and congratulations to the lead development team, which consists of Drew Winget from Stanford University and Rashmi Singhal from Harvard University. Mirador 2.0 would not have been possible without contributions of code, advice, testing and support by many others at Harvard, Stanford and the IIIF community. See a full list of acknowledgements on the project website. 

As we look forward to subsequent releases, improvements and extensions to Mirador, we invite contributions of issues, bug fixes, and new features by others. If you are interested, please sign up for the mirador-tech@googlegroups.com list, and head to Github to read the contributor guidelines and get started.  

We are delighted to announce that the Digital Public Library of America (DPLA) has become the latest formal Hydra Partner.  In their Letter of Intent Mark Matienzo, DPLA’s Director of Technology, writes of their “upcoming major Hydra project, generously funded by the IMLS, and in partnership with Stanford University and Duraspace, [which] focuses on developing an improved set of tools for content management, publishing, and aggregation for the network of DPLA Hubs. This, and other projects, will allow us to make contributions to other core components of the Hydra stack, including but not limited to Blacklight, ActiveTriples, and support for protocols like IIIF and ResourceSync. We are also interested in continuing to contribute our metadata expertise to the Hydra community to ensure interoperability across our communities.”

Northeast Fedora User Group Meeting Location: Sterling Memorial Library, Lecture Hall, Yale University Dates: Monday May 11 and Tuesday May 12 Times: 9am – 4pm Fee: Free to attend Monday’s agenda is less structured with unconference style discussion with lightning rounds in the afternoon. Tuesday’s agenda is structured to provide training on Fedora 4 and will be led by David Wilcox and Andrew Woods Additional info can be found on the wiki: https://wiki.duraspace.org/display/Events/Northeast+Fedora+User+Group+Meeting%3A+11-12+May+2015 Please register for the event here: https://docs.google.com/forms/d/1b4ntNkhRuJvtNEfi0vXSjk7C9uuR3bwa2A8e3U-6w08/viewform

The Royal Library (National Library of Denmark and Copenhagen University Library) recently released a new Hydra application named Chronos.

Details of the project can be found here: http://www.kb.dk/en/nb/afdelinger/db/index.html

The project spanned a little over two years during which the bulk of the work  was in establishing policies for long term digital preservation and then setting a strategic plan based on the policies.

The work then segued into the cost models to support the newly developed policies and strategic plan. Using a shared set of principle and guidelines from Collaboration to Clarify the Cost of Curation (http://www.4cproject.eu/), they developed a sustainable cost model for long term preservation of their digital assets.

Once they had policies, strategies and costs established, they moved their work into a more detailed level and focused on the metadata requirements for preservation. The focus was on event data to be stored in PREMIS and structure data to be stored in METS. This led to much more detailed discussions related to the discovery of the digital assets for public discovery as well as metadata required for creating internal reports for performing tasks related to digital preservation.

Once this work was complete, they moved into the process of specifications for the system. They selected Hydra as the best approach for digitally preserving millions of documents. The planning process started in June 2014 and continued through the end of October 2014. This past December work begun on the new system and the week of March 16, 2015 they will release videos and additional information to the Hydra community.

Hydra community:

I’m pleased to be able to announce that the Indiana University Libraries, in partnership with Northwestern University Library, have received a $750,000 grant from the Andrew W. Mellon Foundation to support work on the Avalon Media System project through January 2017.

This funding will help to support the following activities: 1) developing additional features and functionality for Avalon to better meet needs of collection managers and users; 2) conducting studies of use of audio and video collections by researchers in humanities disciplines to help ensure future support for scholarly use; 3) integrating the Spotlight exhibit tool with Avalon to allow librarians, archivists, and scholars to showcase and provide additional context for media items and collections; 4) developing and implementing a community-funded business and governance model to sustain ongoing support and development for Avalon; and 5) deploying Avalon in a hosted software-as-a-service model for use by institutions that need the functionality of Avalon but would prefer to utilize a cloud-based software-as-a-service option rather than support a locally hosted instance.

I’d like to offer thanks to the Hydra community for building and maintaining a solid technical foundation that enables systems such as Avalon to be built and to members of the Hydra community who have assisted with Avalon’s development by providing feedback on requirements and implementation experiences.

More information is available in a press release from Indiana University at http://news.indiana.edu/releases/iu/2015/03/mellon-grants-digital-preservation.shtml

Best,
Jon

—

Jon Dunn

Interim Assistant Dean for Library Technologies

Indiana University Bloomington Libraries

Recent Announcement from Princeton:

We’ve finally soft launched our first public application that is serving content from Hydra: http://lae.princeton.edu/. There is still some tweaking to do to Solr and the CSS, but we’re getting close. Staff have been adding data to this application at a rate of 150-300 items per month since for about 6 months now, and we expect to be at a consistent rate of 300/month or more by the summer.

This public interface has some features that may not be obvious to the average end-user:
* All of the images are served via IIIIF (e.g. http://libimages.princeton.edu/loris2/puls%2F0%2Fj%2F0%2Fg%2F1.jp2/full/75,/0/default.jpg)
* The individual catalog pages are displaying data drawn from IIIF Presentation manifests: http://lae.princeton.edu/catalog/0d7hm.jsonld (and, as you can see, also available as IIIF Manifests)
* All of the data is also available as RDF, e.g.: http://lae.princeton.edu/catalog/0d7hm.ttl

Spotlight is a Hydra head we are currently investigating as part of an Academic Repository project with Central ITS. Here’s the one sentence pitch that defines Spotlight:

“Enable librarians, curators, and others who are responsible for digital collections to create attractive, feature-rich websites that highlight these collections.” – taken from GitHub

 

Recent Communication to the Hydra Community:

While we are long overdue for a community update on Spotlight, the team at Stanford (Jessie Keck, Chris Beer and Gary Geisler) has been heads down working hard for the past several months. As we near an end of our current development cycle we wanted to report on current status and goals, and share a view visuals.

The current round of development is focussed on three broad goals: 1. Building out an end-to-end, self-service workflow for creating a new Spotlight exhibit using items and collections in the Stanford Digital Repository. Because we have built Spotlight to be repository agnostic, the technical work to accomplish this goal is somewhat specific to Stanford’s digital library architecture. The code developed for this does not ship as part of Spotlight.  However, we hope that this can serve as an example and model for others to implement a repository-based self-service workflow for creating new exhibits.  We intend to document the workflow we’ve implemented for reference. A demo of this workflow is now available on YouTube at:  http://youtu.be/ZyJ2wzzzunc 2. Enable the addition of items not stored in a formal repository system to a new or existing exhibit.   We refer to this feature set as “support for non-repository items”, although we likely want to re-label it.  This set of features is intended to make Spotlight useful for those institutions that don’t have a fully baked repository backend with which to integrate Spotlight, or for many good reasons may want to build exhibits from contents not stored in a repository.  It also includes the ability to augment any exhibit with non-repsitory items, for example a faculty member or curators local collection of images.  We have implemented two approaches:

• Single image upload: Using a form an exhibit creator can upload a single image file from their local system and add a few simple metadata fields.  If exhibit-specific fields have been created for the exhibit these fields will also be available in the form. Upon submission, the single image and associated metadata is added to items available for building feature pages, and is indexed and available in search results and browse categories.
• Bulk-add via CSV: A CSV template is provided to the exhibit creator to populate with a list of image URLs and associated metadata.  Upon submission of the CSV, the images are fetched over the web and copied, and indexed records are created for all items.  The bulk feature is pulling images in via the web, so exhibit creators can upload images to popular cloud services (box, dropbox, google drive, google images, flickr, etc.) or add any URI to a publicly available image.

This feature is nearly complete, and we’ll send a video demo out in the next few days. See: https://github.com/sul-dlss/spotlight/issues/787 and https://github.com/sul-dlss/spotlight/pull/842 and https://github.com/sul-dlss/spotlight/pull/895 3. Enhance the visual design and user experience to better support image-heavy exhibits. The goal here has been to enhance the visual design to provide a more “museum-like” or visually oriented look and feel.  Our design team has developed a proposal for a variety of new elements and widgets to produce a more visual, immersive and interactive experience. The developers are just starting to implement these now, and certainly your feedback is welcome.  The initial design proposal can be seen here: https://github.com/sul-dlss/spotlight/wiki/Enhanced-Visual-Design Of course there are a variety of tickets and features that we have added and will be adding that fall somewhat outside the scope of these high-level goals.  For example we have just recently added simple analytics for exhibits using the Google Analytics API – https://github.com/sul-dlss/spotlight/pull/942 . We anticipate 2-3 weeks more of development on Spotlight and the next release will also include improved documentation and a project site (at something like spotlight.github.io – not claimed or built yet). We’ll be back in touch soon with more frequent updates as we wind down this phase of development. -Stu Snydman ****************************************
Stuart Snydman
Associate Director for Digital Strategy
Stanford University Libraries

The National Endowment for the Humanities recently awarded the Indiana University Libraries and WGBH Boston a grant to support the development of HydraDAM2. This preservation-oriented digital asset management system for time-based media will improve upon WGBH’s existing HydraDAM system and work seamlessly with the Avalon Media System for user access, among other features.

Both HydraDAM and the Avalon Media System grew from the Hydra community. Hydra is an open source technology framework that supports the creation of preservation and access applications for digital assets based on the Fedora repository system. A community of institutions known as the Hydra Partners works together to maintain the framework and create applications for local or shared use by libraries, archives, and cultural institutions. Both Indiana University and WGBH Boston are among the 25 Hydra Partner institutions. Indiana University is collaborating with Northwestern University on the development of the Avalon Media System and WGBH developed the original HydraDAM system with help from the Data Curation Experts group.

[complete article]

HydraDam is based on the popular Hydra application Sufia. You can view some interesting examples of institutions using Sufia for digital preservation here:

Penn State: ScholarSphere

Notre Dame: CurateND

Case Western: Digital Case

 

 

ProjectHydra.org
Avalon Media Systems