The quality of an ontology : the development and demonstration of an instrument for ontology quality assessment

Abstract

In the Information Sciences ontology specifies the conceptual structure of a real-world domain through its vocabulary and its meaning, or semantics. The conceptual structure comprises definitions of concepts and the rules for the relationships between those concepts. Examples of traditional ontologies include database schemas, taxonomies and library classification schemes. Modern ontologies are designed for the computer and, as well as specifying a common understanding of a domain for its different human users, allow automated information processes such as interpretation, aggregation and manipulation. As an ontology is fundamental to systems in an organisation its quality is critical for the sound operation of those systems. The problem is that there seems no systematic basis or method for identifying the criteria for undertaking an assessment of ontology quality. An abundance of literature proposes ideas about what constitutes a 'good' ontology, but it is diverse, scattered and incoherent. This means that, in practice, it is difficult to conduct a holistic quality evaluation. The aim of this research is to review the literature and produce and demonstrate an instrument for assessing the quality of an ontology. The production of the instrument places this research in the design-science paradigm, that is, it contributes a new and innovative artefact for practitioner use in addition to scholarly contribution. The Ontology Quality Assessment Instrument was demonstrated using two case studies. The first study concerned assessing whether the Australian Government's Standard Business Reporting (SBR) ontology has the quality required to replace the Standard Chart of Accounts used by the Queensland University of Technology when reporting its grants funding to government. The instrument demonstrated some of the difficulties associated with assessing quality, particularly the difficulty of grounding the criteria in the actuality of the case because the criteria evolved from the literature and this required further explanation or a glossary. It also demonstrated the difficulty interpreting the results of an assessment as criteria are met or not met. Operationalizing the criteria became necessary and this was useful in making sense of the findings in the case studies. The second case study applied the instrument to a proposed taxonomy for defining the term 'information' in the Information Systems literature. The use of the instrument identified risk areas in the taxonomy. In future, the research and practitioner communities could refine the criteria and add to the examples within the instrument for use by ontology. The contribution of this research is in the assembly and organisation of quality attributes into a qualitative assessment instrument that provides the criteria for assessing ‘The Quality of an Ontology’.

Date of Award	2012
Original language	English
Awarding Institution	University of Canberra
Supervisor	Craig Mcdonald (Supervisor) & John Campbell (Supervisor)