Evaluation and Testing Applied to ENRICH WP5 NEW (2)

 

The evaluation focuses results achieved during aggregation of documents from each individual content provider (both regular Content Partners and Associated partners).

As there are different ways of cooperation there are different evaluation questionnaires:

  • The questionnaire located in this page is dedicated to those who create theirs documents metadata using Manuscriptorium on-line tools.
  • Another questionnaire is dedicated to those who provide existing metadata exports for import to Manuscriptorium.

The questions below focus the interoperability of produced documents, usability of tools and processes and as well their adaptability as experienced during preparation of routine cooperation with each individual content provider.

From each content provider at least one evaluation should be provided. It is suggested to fill the questionnaire separately for each different collection of documents aggregated in Manuscriptorium as the collections have different properties and there are different connectors (a part of Manuscriptorium import interface) created for each different collection so the results achieved through each the connector may vary.

Each partner is kindly asked to perform the evaluation as soon as receives necessary mappings documentation and XSL transformation. This will be sent to the partners during first days of evaluation. In parallel the documentation and XSL will be published in the project website and within dedicated section of new Manuscriptorium beta website.

 

Please tick the appropriate box indicating that your opinions are based on the experience of:

 

Content provider, information manager
Technical personnel, supporting staff
Scholar, researcher in historical documents
End-user having general interests

 

General scale from 0 to 4 has to be used for any approach of evaluation applied (your personal opinion or according the check list added separately)

   
The metrics used for evaluation means:

0 - Failed or feature does not exist

1 - Has poor support and/or it can be done but with significant effort

2 - Fair support but needs modification to reach the desired level of support

3 - Good support and needs a minimal amount of effort

4 - Excellent support and meets the criteria, minimal effort.
 

Task 5.1: On-line tools for structuring of existing metadata and data related to manuscripts (m3-m16). Task leader: AIP.

5.1.1. Evaluate the tools used for structuring existing metadata / newly created digitized documents and their implementation into Manuscriptorium structures.

Set of tools provided in the frame of Manuscriptorium was:

  • The individual Connector prepared according to the metadata properties in large- scale digitization projects;
  • M-Tool (off-line or on-line version): an application which enables to create the descriptive and structural metadata;
  • M-Can: on-line application which enables upload of the metadata into the Manuscriptorium environment, check of correctness and subsequently transfer for import

Evaluate the tool(s) and process you have used in respect of Criteria /categories indicated:

The Content/Associated Partners are kindly asked to answer the questions below by ticking "Yes" or "No". The resulting score of the corresponding criteria, assigned by you to that item - the sum of yours "Yes", will be shown automatically in the next line after those questions.

 

Interoperability/Objects

Yes

No
1) Are all produced documents metadata well formed?

2) Are all produced documents metadata valid against the TEI P5 ENRICH schema?

3) Suits the produced information structure to your usual cataloguing practice? (tick Yes if the structure of information allows you to create at least the same amount of information you create using your usual cataloguing practice)

4) Was it possible to attach necessary identifiers that would bind the new metadata to objects in your repository?

   
 

Failed

      

Excellent
Resulting evaluation of the INTEROPERABILITY/Objects


0


1


2


3


4
 

 

Adaptability/Tools

Yes

No
1) Are the tools of MNS well adapted to the properties of your data repository? (the more otherwise unnecessary changes you had to implement on your side in order to make the data (images) usable within ENRICH, the more likely you will tick No; the less changes were necessary, the more likely you will answer Yes)

2) Are the tools adapted to create and/or process metadata for all types of data you intent to provide?

3) Is the documentation of the M-Can well designed and helpful?

4) Is the single (English) language of the M-Can interface sufficient for your needs?

   
 

Failed

      

Excellent
Resulting evaluation of the ADAPTABILITY / Tools:


0


1


2


3


4

 

 

Usability/Processes

Yes

No
1) Is the process of passing your metadata via M-Can application well usable?

2) Is the documentation of the M-Can application sufficient and well usable?

3) Is the response of Manuscriptorium editors and administrators adequate your expectations when processing documents you offer for import?

4) Do you find the one month database update period to be sufficient? (if you think that the accepted documents should be available in MNS sooner than within 30 days since accetptation, then tick No)

   
 

Failed

      

Excellent
Resulting evaluation of the USABILITY / Process:


0


1


2


3


4

 

 

Usability/Tools

Yes

No
1) Have you encountered no fatal errors causing loosing work in process?

2) Have you encountered no browser error alerts or display difficulties?

3) Javascript dependency: it was no dependency on Javascript, affecting negatively your work with the M-Can?

4) Is the interface intuitive / user friendly enough?

   
 

Failed

      

Excellent
Resulting evaluation of the USABILITY / Tools:


0


1


2


3


4

 

 

Adaptability/Object

Yes

No
1) Is the documentation describing structure of the metadata well prepared?

2) Are all documents prepared by M-Tool well retrievable within Manusriptorium?

3) Are the documents prepared by M-Tool well usable when browsing their digital copies?

4) Is the way of updating/changing records in MNS well implemented?

   
 

Failed

      

Excellent
Resulting evaluation of the ADAPTABILITY/Object:


0


1


2


3


4

 

 

USABILITY/Object

Yes

No
1) Are the documents metadata easily editable outside the MNS environment?

2) Is it possible for you to use the digital document with TEI metadata in your local presentation systems?

3) Does Manuscriptorium lead the end-users to your local systems by appropriate links?

4) Did you manage to find your documents in other sources (e.g. TEL)?

   
 

Failed

      

Excellent
Resulting evaluation of the USABILITY/Object:


0


1


2


3


4

 

Are you satisfied with your evaluation? The rating is from 0 = <failled/feature don't exist> to 4 = <Excellent>:

 

 

Failed

      

Excellent
INETROPERABILITY / Object


0


1


2


3


4
ADAPTABILITY / Tools


0


1


2


3


4
USABILITY / Processes


0


1


2


3


4
USABILITY / Tools


0


1


2


3


4
ADAPTABILITY / Object


0


1


2


3


4
USABILITY / Object


0


1


2


3


4

 

If the results seems acceptable, please click on <Submit>, if not, return back to the questions.

You were invited to evaluate results by: *


Your free comments / suggestions:


See statistics...

 

Evaluation and Testing Applied to ENRICH WP 3, WP 4, WP 5, and WP 6

 

The all project partners have to participate in evaluation process. Thank you for your efforts in evaluating the first results derived in WPs until December 2008.

 

Home