Evaluation and Testing Applied to ENRICH WP5 NEW (1)

 

The evaluation focuses results achieved during aggregation of documents from each individual content provider (both regular Content Partners and Associated partners).

As there are different ways of cooperation there are different evaluation questionnaires:

  • The questionnaire located in this page is dedicated to those who provide existing metadata exports for import to Manuscriptorium.
  • Another questionnaire is dedicated to those who create theirs documents metadata using Manuscriptorium on-line tools.

The questions below focus the interoperability of produced documents, usability of tools and processes and as well their adaptability as experienced during preparation of routine cooperation with each individual content provider.

From each content provider at least one evaluation should be provided. It is suggested to fill the questionnaire separately for each different collection of documents aggregated in Manuscriptorium as the collections have different properties and there are different connectors (a part of Manuscriptorium import interface) created for each different collection so the results achieved through each the connector may vary.

Each partner is kindly asked to perform the evaluation as soon as receives necessary mappings documentation and XSL transformation. This will be sent to the partners during first days of evaluation. In parallel the documentation and XSL will be published in the project website and within dedicated section of new Manuscriptorium beta website.

 

Please tick the appropriate box indicating that your opinions are based on the experience of:

 

Content provider, information manager
Technical personnel, supporting staff
Scholar, researcher in historical documents
End-user having general interests

 

General scale from 0 to 4 has to be used for any approach of evaluation applied (your personal opinion or according the check list added separately)

   
The metrics used for evaluation means:

0 - Failed or feature does not exist

1 - Has poor support and/or it can be done but with significant effort

2 - Fair support but needs modification to reach the desired level of support

3 - Good support and needs a minimal amount of effort

4 - Excellent support and meets the criteria, minimal effort.
 

Task 5.1: On-line tools for structuring of existing metadata and data related to manuscripts (m3-m16). Task leader: AIP.

5.1.1. Evaluate the tools used for structuring existing metadata / newly created digitized documents and their implementation into Manuscriptorium structures.

Set of tools provided in the frame of Manuscriptorium was:

  • The individual Connector prepared according to the metadata properties in large- scale digitization projects;
  • M-Tool (off-line or on-line version): an application which enables to create the descriptive and structural metadata;
  • M-Can: on-line application which enables upload of the metadata into the Manuscriptorium environment, check of correctness and subsequently transfer for import

Evaluate the tool(s) and process you have used in respect of Criteria /categories indicated:

The Content/Associated Partners are kindly asked to answer the questions below by ticking "Yes" or "No". The resulting score of the corresponding criteria, assigned by you to that item - the sum of yours "Yes", will be shown automatically in the next line after those questions.

 

Interoperability/Objects

Yes

No
1) Are all TEI P5 documents which originated as conversion results from your original metadata valid against the TEI P5 ENRICH schema?

2) Does the structure of produced TEI based descriptions suit your original records properties/cataloguing practice (all descriptive information well structured and correctly tagged/labeled)?

3) Does the structure of produced TEI based structural metadata suit your original structural metadata (do the produced records contain all structural information from the original metadata)?

4) Were all the objects identifiers and identifiers of other related objects kept in the converted records?

   
 

Failed

      

Excellent
Resulting evaluation of the INTEROPERABILITY/Objects


0


1


2


3


4
 

 

Adaptability/Tools

Yes

No
1) Are the tools of MNS well adapted to the properties of your data repository? (the more otherwise unnecessary changes you had to implement on your side in order to make the data (images) usable within ENRICH, the more likely you will tick No; the less changes were necessary, the more likely you will answer Yes)

2) Are the tools adapted to process metadata for all types of data you intent to provide?

3) Are the tools well adapted to the information structure of your metadata? (choose No if there were changes required by MNS platform, that you had to perform above your metadata structure; choose Yes if no changes were necessary)

4) Are the tools well adapted to accept format and form which your metadata repository provide? (In case you provide your data in a native form/format, choose Yes, if you had to implement new forms/formats, which are otherwise unnecessary, choose No)

   
 

Failed

      

Excellent
Resulting evaluation of the ADAPTABILITY / Tools:


0


1


2


3


4

 

 

Usability/Processes

Yes

No
1) Is the process of passing your metadata that is/will be used during routine cooperation well usable? (the answer is No if you had/will have to implement a new way of metadata passing in your system)?

2) Do you find the one month update period to be sufficient?

3) Does the conversion process produce consistent results for all documents within your collections aggregated within ENRICH?

4) Is the documentation to the process of metadata mappings sufficient?

   
 

Failed

      

Excellent
Resulting evaluation of the USABILITY / Process:


0


1


2


3


4

 

 

Usability/Tools

Yes

No
1) Do you find the making of XSL transformation available via MNS website for further use as a sufficient increase of usability of the produced XSL?

2) Is the documentation to using of the XSL transformations sufficient?

3) Does the version (1.1) of XSL transformation suit your needs?

4) Did you encounter no errors when using the XSL on your own?

   
 

Failed

      

Excellent
Resulting evaluation of the USABILITY / Tools:


0


1


2


3


4

 

 

Adaptability/Object

Yes

No
1) Is the documentation describing mappings well prepared?

2) Are your digital documents in MNS interface well adapted for presentation to end-users?

3) Are all documents adapted to be used with the complete suite of MNS end-users features (personal collections, virtual documents)

4) Is the way of updating/changing records in MNS well implemented?

   
 

Failed

      

Excellent
Resulting evaluation of the ADAPTABILITY/Object:


0


1


2


3


4

 

 

USABILITY/Object

Yes

No
1) Is the object well retrievable in the Manuscriptorium system?

2) Is it possible to use the original digital object both in MNS interface and your local presentation systems without any changes at your side?

3) Does Manuscriptorium lead the end-users to your local systems by appropriate links?

4) Did you manage to find your documents in other sources (e.g. TEL)?

   
 

Failed

      

Excellent
Resulting evaluation of the USABILITY/Object:


0


1


2


3


4

 

Are you satisfied with your evaluation? The rating is from 0 = <failled/feature don't exist> to 4 = <Excellent>:

 

 

Failed

      

Excellent
INETROPERABILITY / Object


0


1


2


3


4
ADAPTABILITY / Tools


0


1


2


3


4
USABILITY / Processes


0


1


2


3


4
USABILITY / Tools


0


1


2


3


4
ADAPTABILITY / Object


0


1


2


3


4
USABILITY / Object


0


1


2


3


4

 

If the results seems acceptable, please click on <Submit>, if not, return back to the questions.

You were invited to evaluate results by: *


Your free comments / suggestions:


See statistics...

 

Evaluation and Testing Applied to ENRICH WP 3, WP 4, WP 5, and WP 6

 

The all project partners have to participate in evaluation process. Thank you for your efforts in evaluating the first results derived in WPs until December 2008.

 

Home