Experiments

Experiments for FOIKS 2012

We conducted the experiments on an openSUSE 11.1 (x86 64) server having a Intel Xeon CPU E5450 3.00GHz and 15.7 GB of RAM.

First-Order (FO) Rewritability

The FO rewritability experiments were evaluated on top of a PostgreSQL 8.4 database with increased shared buffers and work mem parameters. For the DL rewritting a modified version of Owlgres was used. For creating the DL KB we applied Owlgres' TBox and ABox loading facilites on the OWL files dbpedia_tbox.rdf, lubm_tbox_lite.owl, dbpedia_10k.rdf, university_10k.owl, and so on.

Download Test data and MOR

Usage of MOR for query FO3_Lubm.dlv on the database lubm_10 (which has to be created with the Owlgres loading facilites):

 $ ./mor.sh --pgm FO3_Lubm.dlv --db lubm_10 --user test --passwd test --str CLA --silent

Random

This experiment was conducted on randomly generated data with a high selectivity among the join attributes and without a DL KB. We compared MOR with DLV and DLV^DB regarding binary joins with negation. One finding was that MOR and DLV^DB scale similar and have a constant factor differing them.

Instance	MOR	DLV^DB	DLV
R 10k	<1	<1	1
R 100k	1	1	105
R 250k	3	4	977
R 500k	5	9	2,795
R 1M	11	19	11,446

DBpedia

For the DBpedia experiment, we evaluate the dl-program based on a simple DL KB with books, periodicals, and publications extraced from DBpedia. The dl-program selects a range of individuals from the DL KB upon extension with books from an external source. The results let observe a linear runtime behavior of MOR and DLV^DB. The entries with "-" (out of memory) for dlvhex are due to RacerPro’s usage of a Graph library, which limited the instance size. (The optimized RacerPro 2.0 release is not available with dlvhex at present.)

Instance	MOR	DLV^DB	dlvhex[DL]
D 10k	1	<1	7
D 100k	4	6	-
D 250k	9	25	-
D 500k	18	50	-
D 1M	42	145	-

LUBM

The instances for the LUBM benchmarks are created according to the well-known Lehigh University Benchmark (LUBM) with a simplified TBox. We altered roughly 10% of the TBox axioms like transitive roles and equality axioms. Again, the results let observe a linear runtime behavior of MOR and DLV^DB.

Instance	MOR	DLV^DB	dlvhex[DL]
R 10k	1	<1	36
R 100k	4	6	117
R 250k	11	19	-
R 500k	20	39	-
R 1M	44	87	-

LUBM Recursive

This experiment was conducted on the LUBM KB with limited recursion (negation is stratified and no predicate cycles through dl-atoms) in the rules part. As DL-LiteR misses transitive roles, we calculate the transitive closure of the sparse, tree-like structure of the subOrganization role (thus the organizational hierarchy of the LUBM university) in the rules part and update the DL KB. We observed in the results an indication for quadratic runtime.

Instance	MOR	dlvhex[DL]
R 10k	1	35
R 100k	1	108
R 250k	2	-
R 500k	4	-
R 1M	11	-

Datalog Rewriting

Download Test data

el-galen

The experiment shows the efficiency of the EL datalog rewriting for a large TBox. As test ontology we used an EL variant of Galen, a large biomedical ontology. In the test, we created four ontology instances G1 to G4 that have a fixed Galen TBox and increasing ABoxes with 10i assertions, each using roughly ten concepts and roles.

We have original OWL files EL-GALEN_i.owl, and the datalog rewritng results. To run a query, you can use DLV or clingo.

Usage: For example, if you want to query q-1 with abox-2, you can use dlv or clingo by:

$ dlv el.dl el-galen.dl abox-2.dl q-1.dlv

$ clingo el.dl el-galen.dl abox-2.dl q-1.clingo

Ontology	Query	DReW [DLV]	DReW [clingo]	HermiT
G1	1	2.0	1.3	8.1
	2	2.0	1.3	8.2
	3	2.0	1.3	8.
	4	2.0	1.4	8.1
G2	1	2.0	1.3	8.9
	2	2.0	1.4	8.9
	3	2.1	1.4	8.7
	4	2.0	1.3	9.0
G3	1	2.0	1.3	9.5
	2	2.1	1.4	9.5
	3	2.0	1.4	9.7
	4	2.1	1.4	9.5
G4	1	2.1	1.4	10.3
	2	2.1	1.4	10.2
	3	2.1	1.4	10.2
	4	2.1	1.4	10.2

df-policy

This experiment was conducted on terminological default KBs with an EL ontology. In the test, we created ontology instances L_i, i = 1, 5, 10, 25, that have a fixed TBox and increasing Aboxes with i∗1000 instances of user requests. The query imposed was then whether a set of particular individuals, designated by concepts Q_k , k = 5, 50, 100,

Usage: for example, if you want to query 5 request with 1000 users, you can use dlv or clingo by:

$ dlv -filter=in users-1000.dl requests-5.dl policy.dl el.dl el-i.dl el-i-n.dl  df-policy-query.dl

$ clingo policy.dl users-1000.dl requests-5.dl el.dl el-i.dl el-i-n.dl df-policy-query.dl df-filter.clingo

KB	Typing	DReW[DLV]	DReW[clingo]
Δ ₁	5	1.1	0.8
	50	2.4	1.3
	100	6.0	3.0
Δ ₅	5	6.6	4.4
	50	8.3	5.0
	100	12.2	7.4
Δ ₁₀	5	13.9	9.4
	50	15.7	10.1
	100	20.5	13.3
Δ ₂₅	5	35.8	26.0
	50	40.0	26.4
	100	43.7	32.7

MLP Rewriting

The algorithm for solving input-call stratified MLPs has been implemented in the TD-MLP solver, which is based on the dlvhex system. We were testing with dl-programs where U₁ and U₁₅ are EL versions of the LUBM ontology programs P₁ to P₉ encode variants of the LUBM queries; all dl-programs where acyclic. We used two Datalog engines to compute the models of the native and the modular encodings: clingo 3.0.3 and DLV 2010-10-14.