More Related Content
Similar to The Open Archives Initiative and the Sheet Music Consortium (15)
The Open Archives Initiative and the Sheet Music Consortium
- 1. The Open Archives Initiative and the Sheet Music Consortium Jon Dunn, Jenn Riley IU Digital Library Program October 10, 2003
- 2. October 10, 2003 DL Brown Bag: OAI/Sheet Music 2 Presentation outline Jon: OAI introduction Sheet Music Consortium background Jenn: Data mapping issues Sheet music harvester demonstration Next steps
- 3. October 10, 2003 DL Brown Bag: OAI/Sheet Music 3 OAI: Open Archives Initiative Original problem: searching across e-print archives Distributed searching hard e.g. Z39.50 Varying search semantics, capabilities Network, server problems Solution: metadata harvesting OAI-PMH: OAI Protocol for Metadata Harvesting
- 4. October 10, 2003 DL Brown Bag: OAI/Sheet Music 4 Metadata Harvesting Extract metadata from various sources Build services on local copies of metadata user . . . search for “Indiana” local copy of metadata metadata harvested offline metadata harvested offline metadata harvested offline metadata harvested offline all searching, browsing, etc. performed on the metadata hereIndividual repositories can still support direct user interaction Data providers Service provider
- 5. October 10, 2003 DL Brown Bag: OAI/Sheet Music 5 OAI-PMH roles Data Providers Repositories of digital content and metadata Support harvesting of metadata via the OAI protocol Service Providers Harvest metadata from data providers using the OAI protocol Implement user interface to data Usually for searching, but other services also possible Can be selective
- 6. October 10, 2003 DL Brown Bag: OAI/Sheet Music 6 OAI Protocol for Metadata Harvesting Originally developed in 1999 (Santa Fe Convention) Original focus on E-prints Has grown into general metadata harvesting protocol Version 1.0: January 2001 Version 1.1: June 2001 Conform to XML Schema 1.0 Version 2.0: June 2002 Transition period through December 2002 Currently 120 registered OAI data providers (up from 53 in March 2003)
- 7. October 10, 2003 DL Brown Bag: OAI/Sheet Music 7 OAI-PMH tech details Carried over HTTP Requests: HTTP GET or POST Responses encoded in XML Format defined via XML schema Metadata in unqualified Dublin Core (and potentially other formats)
- 8. October 10, 2003 DL Brown Bag: OAI/Sheet Music 8 Dublin Core elements Coverage Description Type Relation Source Subject Title Contributor Creator Publisher Rights Date Format Identifier Language
- 9. October 10, 2003 DL Brown Bag: OAI/Sheet Music 9 OAI-PMH verbs Verb Function Identify description of archive ListMetadataFormats metadata formats supported by archive ListSets sets defined by archive ListIdentifiers OAI unique ids contained in archive ListRecords listing of N records GetRecord listing of a single record
- 10. October 10, 2003 DL Brown Bag: OAI/Sheet Music 10 OAI resources Web site, mailing lists Repository explorer Data/service provider software www.openarchives.org
- 11. October 10, 2003 DL Brown Bag: OAI/Sheet Music 11 OAI data providers at IU OAI data provider for DLP collections Lilly: Hohenberger Photograph Collection, DeVincent Sheet Music Collection IUN: U.S. Steel Photograph Collection eventually all Eprints: Digital Library of the Commons AISRI ReciprocalNet
- 12. October 10, 2003 DL Brown Bag: OAI/Sheet Music 12 OAI data provider for DLP PHP OAI Data Provider Developed by University of Oldenburg PHP, mySQL database Perl scripts used to map USMARC, other formats to DC MARC.pm Perl module
- 13. October 10, 2003 DL Brown Bag: OAI/Sheet Music 13 Examples of OAI service providers UIUC Digital Gateway to Cultural Heritage Materials http://oai.grainger.uiuc.edu/ UMich OAIster http://www.oaister.org/ RLG Cultural Materials (licensed) http://www.rlg.org/culturalres/ OLAC: Open Language Archives Community http://www.language-archives.org/
- 14. October 10, 2003 DL Brown Bag: OAI/Sheet Music 14 Sheet Music Consortium Partners UCLA Johns Hopkins IU Goal: Integrate access to sheet music collections Online and print collections
- 15. October 10, 2003 DL Brown Bag: OAI/Sheet Music 15 Sheet music Definition Based on physical format: generally loose sheets or folio, 1-10 pages Much is “popular music,” but not all Variety of research uses Currently hard to access Variety of metadata Much uncataloged Many valuable collections MLA list At IU: Lilly, Archives of Traditional Music
- 20. October 10, 2003 DL Brown Bag: OAI/Sheet Music 20 Sheet Music Consortium Harvester: Timeline March 2002: Initial planning meeting at IU Fall 2002: Initial system prototype Winter 2002/2003: Usability evaluation, interface redesign Focus groups and usability testing at several sites Fall 2003 – Version 1 of system released
- 21. October 10, 2003 DL Brown Bag: OAI/Sheet Music 21 Why did we have to map data? OAI requires unqualified Dublin Core Sheet Music Harvester version 1 only collected Dublin Core Contributed data only needed to support resource discovery Dublin Core field definitions need interpretation For efficient searching, data from different institutions must be consistent
- 22. October 10, 2003 DL Brown Bag: OAI/Sheet Music 22 Some mapping issues Field formatting important, not just contents Choices heavily influenced by LC practice Can’t force institutions to comply with guidelines Sheet music has many alternative titles Creator vs. contributor Plate numbers: they’re important, where to put and how to label? Uncertain dates and date ranges
- 23. October 10, 2003 DL Brown Bag: OAI/Sheet Music 23 Mapping guidelines Examples: Creator: Invert name. Use the authorized form of name where possible. If needed (e.g. for an alias) repeat the field for the alternative form. Date: Date of publication. The most recent date to appear on the music, or, the actual date of publication if not present but known. Include other dates (e.g. date of composition) if known. Codes “c” for copyright and “ca.” for circa in front of the date is allowed for now. Use repeated DC fields for each date if needed.
- 24. October 10, 2003 DL Brown Bag: OAI/Sheet Music 24 Existing metadata formats MARC Encoded Archival Description (EAD) Dublin Core (DC) Local custom formats
- 25. October 10, 2003 DL Brown Bag: OAI/Sheet Music 25 MARC (1) Library of Congress – mostly from Music for the Nation: American Sheet Mus almost 50,000 records available via OAI already had data mapped “based on” MARC to Dublin Core crosswalk not able to alter their mapping for participation in sheet music project
- 26. October 10, 2003 DL Brown Bag: OAI/Sheet Music 26 MARC (2) IU – Starr collection little authority control determined LC MARC2DC mapping inadequate mapping in progress using MARC.pm Duke – Weinmann collection rare materials emphasis also customized own mapping mapping in progress
- 27. October 10, 2003 DL Brown Bag: OAI/Sheet Music 27 EAD Duke – Historic American Sheet Music Item level finding aid very robust and specific conversion was relatively simple because data was converted to EAD from collection- specific database included virtually all information in EAD documents to DC records
- 28. October 10, 2003 DL Brown Bag: OAI/Sheet Music 28 Dublin Core UCLA – Archive of Popular American Music 4 types of DC records songs sheet music covers et al recordings mapping only required inheritance of songs and sheet music data elements down to the covers level recordings data ignored for OAI data provider purposes
- 29. October 10, 2003 DL Brown Bag: OAI/Sheet Music 29 Local custom formats (1) publication (location, publisher, date) subject call num (box, item) title composer/lyricist/ arranger form of composition instrumentation first line first line of chorus performer dedicatee engraver/lithographer/ artist advertisement plate num duplication Johns Hopkins – Levy collection Simple SGML DTD
- 30. October 10, 2003 DL Brown Bag: OAI/Sheet Music 30 Local custom formats (2) title composer lyricist place of publication publisher copyright first line first line of chorus subject form of composition performance medium copies call # IU – DeVincent collection Simple MS Access database Conversion done with Perl
- 31. October 10, 2003 DL Brown Bag: OAI/Sheet Music 31 Harvester demonstration <http://digital.library.ucla.edu/sheetmusic>
- 32. October 10, 2003 DL Brown Bag: OAI/Sheet Music 32 Data inconsistencies Different depths of description Different levels of authority control No common subject vocabulary between collections Despite mapping guidelines, differences in DC interpretation
- 33. October 10, 2003 DL Brown Bag: OAI/Sheet Music 33 Next steps? Authority control for names Date formats Data clean-up: what can be done at harvester end and what must we ask data providers to do? What will more robust data format look like? How do we make it easier for more institutions to participate?
- 34. October 10, 2003 DL Brown Bag: OAI/Sheet Music 34 More information Presentation on DLP web site, with links: www.dlib.indiana.edu/workshops/bbfall2003.htm Email: Jon Dunn: [email protected] Jenn Riley: [email protected]
Editor's Notes
- #22: Unqualified DC required, but more robust formats also allowed. More on this later. Since the purpose of the harvester is discovery of resources, and a user is taken out of the harvester to view items at individual institutions, it was not necessary to force all of the information from each institution’s records into DC. We needed to define what was required for discovery only, not figure out how to squeeze every marc field into dc. The name “Dublin Core” reveals something about its purpose. It was designed to be a core set of metadata elements applicable to all types of resources. Thus it’s meant to be flexible, with a low entry barrier. This means the definitions of fields are open to wide interpretations. We needed to develop a single interpretation that all contributors followed to make searching and browsing more effective.
- #26: Duke (rare materials emphasis) and IU (little authority control) had records in MARC too, but these weren’t contributed in phase 1 of the project.
- #27: Duke (rare materials emphasis) and IU (little authority control) had records in MARC too, but these weren’t contributed in phase 1 of the project.
- #28: Explain what EAD is and what a finding aid is Very specific info, for example: subject type (LCSH, AAT, TGM), dedicatee, recordings available
- #30: Very specific for sheet music, not good even for other types of printed music.
- #31: No authority control
- #33: MARC has more fields than custom DBs, but custom DBs have more applicable fields. Many MARC records don’t have relator codes, so we don’t know who’s a composer and who’s a lyricist. Even if each individual collection was under name authority, they still might not interoperate. But the problem was much worse – there wasn’t even agreement on name order! Local subject vocabs in use, so even a complex mapping between LCSH, AAT, TGM wouldn’t solve the problem.
- #34: but we don’t know how to solve these things yet