(on using META-SHARE records in CLARIN)

Two large language resource metadata systems-to-be in Europe:

• CLARIN centers provide metadata records for harvesting by search indexes, using Component Metadata Initiative (CMDI) for form and ISOCat links for meaning
• META-SHARE integrates metadata creation and presentation in its synchronized nodes; its editor supports a defined resourceInfo schema that has a documented meaning

http://www.clarin.eu/external/ (go to VLO)

http://www.meta-net.eu/ http://www.meta-share.org/ http://metashare.csc.fi/

• We are CLARIN
• We are META-SHARE
• At least some META-SHARE records are ours
• We don't want to maintain the same metadata twice
• So we desire a bridge
• That is, we wish CLARIN to harvest our META-SHARE records, too

CLARIN has the component metadata model.

• component types for stating particular facts
• profiles to organize component types into record types
• links to ISOCat definitions give fields their meaning (and linguistic expression)
• types can be defined and shared in public registries

http://www.clarin.eu/cmdi http://catalog.clarin.eu/ds/ComponentRegistry/# http://www.isocat.org/

META-SHARE has (will have) a defined content model for metadata

• resourceInfo (after the document element name)
• rich record types for different kinds of language resources
  • corpora (text, audio, video, …)
  • lexical resources (dictionaries, wordnets)
  • tools and services
  • couple of others, different media types
• documented meaning http://metashare.ilsp.gr/portal/knowledgebase/resourceInfo

META-SHARE nodes provide an editor for resoureInfo records.

Records can be published directly from the editor.

UHEL node: http://metashare.csc.fi

CLARIN uses OAI-PMH to provide records for harvesting

• request records from a provider with http GET or POST
• the actual metadata records are enveloped in some more XML
• different record types (with DC mandatory) form an item about a resource

META-SHARE nodes will synchronize with each other automatically

• unsure if there is programmable way for public access
• node administrators have full access to the database
• synchronization is not yet but should start in this summer

http:architecture.svg

http://www.openarchives.org/OAI/openarchivesprotocol.html

I see three issues in sending META-SHARE records to CLARIN:

1. Format shift from resourceInfo to CMDI/ISOCat and OAI-PMH
   • should be a natural match (except mandatory DC)
   • may still need much work
2. Tracking the identity of the reused records
   • important
   • not trivial
3. Authorization or identification of reusable records
   • whose records are they?
   • is this an issue?

Shifting from resourceInfo to CMDI/ISOCat should be straightforward:

• resourceInfo is built of components already: personInfo, sizeInfo, licenseInfo, usageInfo, …
• make precise counterparts in the CMDI component registry
• profile or profiles will be needed (can be lax if one-way)
• resourceInfo has documented ISOCat links already (not in records)
• more may be needed or useful
• then create and publish more in ISOCat (may be a process)

Most likely implemented as XSL Transforms (XSLT). Identify the META-SHARE records as the original.

http://metashare.ilsp.gr/portal/knowledgebase/resourceInfo

http://catalog.clarin.eu/ds/ComponentRegistry/# http://www.isocat.org/

Records in META-SHARE get edited, sometimes removed.

CLARIN OAI-PMH providers must track the changes to an existing record accurately.

• duplicates would be bad
• systematic duplication would be worse
• a duplication cycle between CLARIN and META-SHARE would be self-destruction (if two-way reuse)

(Different records on the same resource are not a major issue. Also, they may be detected by persistent identifiers, but hardly resolved automatically.)

META-SHARE plans to have an identifier for each record.

• It was there in v1, not used now in v2, presumably needed in v3.
• If it is the right kind, use it to identify a record.
• Use last-modification date to identify changes.

It may not be the right kind, though.

• This identifier looked like a content hash in v1.
• Editing seemed to change it, so it may have been a content hash.

So there may be a problem. Not sure. Wait for v3, this summer.

When a META-SHARE record is considered by an OAI-PMH provider for CLARIN:

If already there:

• replace an older record
• leave the same or newer record untouched

If not there, add it as a new item with a new OAI-PMH identifier

When a META-SHARE record is already in an OAI-PMH provider:

• if changed, replace
• if removed, replace with a deletion record

Might notice absence of old record, presence of new, but don't really want a trail of deletions when there have really been only changes.

How do we notice when a new META-SHARE record is available for CLARIN? Scenarios:

1. Someone personally makes their own records available.
2. CLARIN harvests META-SHARE directly somehow.
   • is there a mechanism for such access?
   • a node admins could leak the contents
   • not sure of legal issues at all
3. Other.

At least some of the records there are ours.

META-SHARE has a defined content model with a supporting editor for describing language resources. META-SHARE synchronizes itself.

CLARIN has a compatible but flexible format and semantics. CLARIN harvests its own providers.

We are in both. We desire a bridge.

1. Authorize inclusion of (some) META-SHARE records in CLARIN somehow.
2. Track the identity of an included record through changes and deletions somehow.
3. Need CMDI profiles, OAI-PMH envelopes, mandatory DC, transforms (likely XSL). Adding to ISOCat might be useful for CLARIN. (Profile can be loose if the bridge is one-way but should be precise if two-way.)