VerbOWL
Verbalisation of Afrikaans OWL 2 DL Ontologies

Theoretical Analysis - How it's done


This sections aims to provide an overview of the proof of how the templates together with the program can provide a verbalisation for any construct in an ontology written in OWL 2. It defines how constructs of the ontology map to different elements of the templates. Further discussion of each OWL 2 construct is available in the full report.

  • Definition 1: Let O be an Ontology with signature Φ = {VC, VOCE, VOP, VDP, VI, VA, VDR} such that:
  • VC = Set of OWL Classes, C
  • VOCE = Set of OWL Class Expressions, OCE
  • VOP = Set of OWL Object Property, OP
  • VDP = Set of OWL Data Property, DP
  • VI = Set of Non-negative Integers, I
  • VA = Set of Axioms, A; VA ⊇ {VC, VOCE, VOP, VDP, VI, VDR
  • VDR = Set of Data Ranges, DR; VDR ⊆ VOCE
  • And semantics as specified in the standard.
  • Definition 2: Let t be a template with Γ = {NO, NR, NCE, NC} such that:
  • NO = Object Node
  • NR = Role Node
  • NCE = Class Expression Node
  • NC = Cardinality Node
  • There is a set of templates, T such that T → VA, and some template, t ∈ T, such that t → A and t is a tuple consisting of:
  • At least 1 text element;
  • At least 1 n ∈ (NO ∪ NCE ∪ NC);
  • Zero or more v ∈ NR;
  • Zero or more loop elements, which allow for sections of the template to be iterated multiple times.
  • Definition 3: Using Definitions 1 and 2, the corespondence between them are as follows:
  • n ∈ NO, n → C
  • v ∈ NR, v → (OP ∪ DP)
  • if v → DP: n ∈ NC, n → I
  • else: n ∈ NCE, n → OCE
  • Definition 4: There is a set of text elements from the set of templates, T. Each such text element maps to some symbol in the axiom.
  • ∀ → net 'only'
  • ∃ → ten minste een 'at least one'
  • ⊆ → is 'n 'is a'
  • ∩ → en 'and'
  • ∪ → of 'or'
  • ≡ → is 'is'
  • ¬ → nie 'not'

Definition 3 shows the mapping from the templates defined in Definition 2 to the constructs in OWL 2. Each element in the template directly relates to a construct in the ontology or to words in the structure of the sentence. Object nodes, Role nodes and Cardinality nodes relate to simple constructs in the ontology. When the Class Expression node, NCE, is encountered, another template will be used to structure the corresponding expression into a partial sentence, which can be inserted into the original template's Class Expression node. This is illustrated in Figure 1.

Example: Let template t1 be "Elke <n1> <v1> <n2>" and template t2 be: "<v2> <n3>".
n1, n3 ∈ NO, n1 → C
v1, v1 ∈ NR, v1, v2 → OP
n2 ∈ NCE, n2 → OCE → t2
Figure 1: Example mapping for nested class expressions

Templates can also have other nodes which do not map directly to OWL constructs. Text elements allow for specified Afrikaans words to be inserted into the sentence so the constraint can be properly verbalised (Definition 4). Loop nodes initiate a while-loop in the program to iterate through multiple variables for Object, Role or ClassExpression nodes.

The World Wide Web Consortium has compiled a quick reference for OWL 2 constructs and axioms which was used to ensure that the template accounts for full coverage of OWL 2 DL.


Results and Discussion


Results

The full test output of the Template program is included as a supplementary file. The following section presents a summary of the results.

The program was tested using six ontologies. These ontologies are written in OWL 2 DL and two of them have been translated to Afrikaans. With regard to the translation of ontologies, a user of this program may not necessarily translate the ontology. Some of the test ontologies were not manually translated to give an indication of what the output would be when the input is not in the specified format.

Two approaches to ontology verbalisation were compared. The two prototype verbalisers shall be referred to as the Template program and the GF program.

Both programs calculate the total number of axioms in the ontology; the number of axioms processed; the number of sentences written; and the time take to run the program.

The programs output the verbalised sentences into a text file so that structure of sentences and appearance of grammatical errors per axiom type can be compared.

Verbalisation Coverage

Table 1: Percentage ontology verbalisation for Template program
Ontologies Total Axioms Axioms verbalised Percentage verbalised Sentences written Time taken (s) Time per sentence (ms)
Pizza 712 707 99.30 707 1.72 2.433
African Wildlife 56 55 98.21 57 1.32 23.158
Computer Science 52 48 92.31 48 1.11 23.125
Wine 657 635 96.65 635 5.59 8.803
University 95 91 95.79 91 1.09 11.978
Stuff 136 134 98.53 175 1.21 6.914
Average 96.80 285.5 2.01 12.735
Table 2: Percentage ontology verbalisation for GF program
Ontologies Total Axioms Axioms verbalised Percentage verbalised Sentences written Time taken (s) Time per sentence (ms)
Pizza 712 711 99.86 711 2.11 2.968
African Wildlife 56 56 100.00 56 1.73 30.893
Computer Science 52 44 84.62 44 1.69 38.409
Wine 657 628 95.59 628 5.46 8.694
University 95 94 98.95 94 1.43 15.213
Stuff 136 110 80.88 110 1.51 13.727
Average 93.32 273.83 2.32 18.317

Tables 1 and 2 show what percentage of the axioms in the ontology were verbalised for each of the programs. The total axioms refer to the total number of axioms in the ontology. Although the Template program doesn't have 100% coverage for any of the ontologies, the only missing axioms are those that are explicitly ignored.

The last three columns of Tables 1 and 2 show that there is a difference in number of sentences generated for all the ontologies. For the pizza ontology, the GF program ignores a single equivalent class axiom (InterestingPizza). Regarding the African Wildlife ontology, the Template program generates nine sentences from the seven disjoint axioms. In the Computer Science ontology, the GF program doesn't verbalise a data property domain; data property range; three equivalent classes containing data ranges; and two sub classes containing data ranges. With the wine ontology, the GF program ignores 29 axioms which contain cardinality restrictions. For the university ontology verbalisation the GF program ignores a subclass that has a data range. In the stuff ontology verbalisation, the Template program creates 41 disjoint sentences from the sixteen disjoint axioms, while the GF program doesn't verbalise the sixteen equivalent classes; three disjoint union classes and seven subclasses involving nested class expressions.

Grammar

The full list of axiom types and whether their verbalisations contain grammatical errors is included in supplementary materials.

BarChart of Percentage Axiom types Verbalised
Figure 2: Bar Chart of Percentage Axiom Types Verbalised
BarChart of Grammar Errors in Text Axioms
Figure 3: Bar Chart of Number of Grammar Errors in Test Axioms

Out of the 33 axiom types in OWL 2, 21 were found in the test ontologies. One of each type of axiom was evaluated, except for the subclass and equivalent class axioms. Due to the variability of the nested class expressions, two subclass axioms and three equivalent class axioms were tested. This resulted in 24 test axioms being evaluated. Figure 2 shows how the Template program verbalised all 24 axioms, while the GF program verbalised 18 of the axioms.

The generated sentences were evaluated on a scale from 0-2 (0 - No error; 1 - Minor error; 2 - Multiple errors). Figure 3 shows how each program scored on this scale.

Of the 18 axioms that were verbalised by both programs, three were found to differ in meaning. The sentences were compared with the explanation of the axioms given in the OWL 2 syntax guide and in all three cases, the GF program's verbalisation was found to be incorrect. Table 3 shows how the comparison was conducted by showing one of the axiom types as an example (all three comparisons are provided in the supplementary material). Each sentence was roughly translated to English before being compared with the standard explanation.

Table 3: Comparison of Verbalised Axioms that have Different Meanings
Description Sentence
Originals Template program As iets het wyn descriptor, dit het Body.
GF Program as X [HetBody] Y, dan Y [HetWynDescriptor] X.
OWL 2 explanation if an individual x is connected by OPE1 to an individual y, then x is also connected by OPE2 to y.
English Comparisons Template program (English) If something has wine descriptor, it has Body.
GF Program (English) If X has Body Y, then Y has wine descriptor X.
OWL 2 explanation with classes/properties from example inserted If an individual x is connected by "has wine descriptor" to an individual y, then x is also connected by "has Body" to y.

Discussion

Although the Template program verbalises about 96.8% of the axioms in the ontology, it explicitly ignores all the axioms that it does not verbalise. The GF program verbalises on average only 93.32% of the axioms and doesn't explicitly state any ignored axioms. The difference in percentage coverage is also highlighted in the grammar evaluation. Figure 14 shows how the Template program verbalises 100% of the axiom types found in the test ontologies, while the GF program only verbalises 75%. This shows that the Template program provides a more detailed description of the ontology than the GF program.

The Template program produces on average more sentences than the GF program. This could be because the Template program verbalises a larger percentage of the ontology. However, the number of sentences written is always greater or equal to the number of axioms verbalised. This could imply that the Template program is more verbose and expresses a single axiom over multiple sentences instead of in one sentence. This is possible, as it separates disjoint classes into pairwise classes and each pairwise group is verbalised separately. This is not the best option, however it does mean that all the axioms are still properly covered by the program.

The Template program appears to be faster than the GF program. Both the average time taken and the average time per sentence is less than for the GF program, despite the fact that the Template program has more sentences. However, the result of a t-test is 0.41, which is above 0.05. Therefore, there is not a statistically significant difference.

Regarding grammar, the Template program performed better than the GF program. It had fewer errors in its verbalisations and it did not have any axiom types with multiple grammatical errors. It also followed the standard axiom explanation more closely than the GF program where it was found that the verbalisations differed between the two programs.

Some referring expressions were used in the program where nested class expressions required the insertion of iets 'something' in order to make the sentence flow (Table 3). However, these were not always very effective and were occasionally used in the wrong place or omitted when they should be included. Simple aggregation was used when encountering intersections in axioms in the ontology. However, future work could look at performing document structuring on the output so that descriptions where the named class is the same are placed together. Aggregation could then be used to join these sentences if they are not already too long and referring expressions could be used to refer to repeated names.

There are a few limitations to the Template approach. The first is that it requires a manual translation of the ontology for the tool to be most effective, whereas the GF program automatically attempts to translate the ontology into the required language. There are also limitations with regard to grammar. The template can only provide a limited choice of grammatical structures. Further rules would need to be added to the code in order to achieve perfect grammar, which cannot be achieved by the template alone.

Despite the limitations, the template approach does appear to be more feasible for future development than the GF program. The Template program would require the addition of a user interface and extensive coding to handle grammar rules that can't be dealt with in the template, yet the Template program already shows better results than the GF program. The GF program requires a developer to learn grammatical framework, and the framework is aimed more at producing text in multiple languages than just in one language.

This is the first Afrikaans template created to verbalise ontologies. Although it does not have a user interface like some other verbalising tools, it is the first verbaliser of Afrikaans ontologies and shows promise for future development.