dlvhex

Experiments

This page describes experiments conducted mainly with dlvhex version 2.x. It contains information how to obtain and build dlvhex 2.x, as well as measurements for

comparing dlvhex 2.x with the previous version dlvhex 1.7.3,
comparing different parameter settings of dlvhex 2.x, and
comparing dlv and clingo backends of dlvhex 2.x.

dlvhex 2.x

dlvhex 2.x is not yet released, but you can get it from sourceforge using subversion:

you find the branch online here: dlvhex/branches/dlvhex-refactoring,

checking out and building the sources works as follows:

$ svn co https://dlvhex.svn.sf.net/svnroot/dlvhex/dlvhex/branches/dlvhex-refactoring
$ cd dlvhex-refactoring
$ ./bootstrap.sh
<ignore all warnings>
$ ./configure <insert your parameters here>
$ make
$ make check
<optional>
$ make install
<optional>
$ sudo ldconfig
<required on some machines>

Test Instances and Detailed Benchmark Results:

DMCS (Distributed multi-context systems)
Reviewer Selection 1
Reviewer Selection 2

DMCS Instances

These test instances were created using the DMCS benchmark instance generator and converted to the input format of MCS-IE using generate_master_file.py. Please refer to the original DMCS benchmark instance generator page for details of topologies and instance parameters.

Download

All DMCS instances are contained in the following archive: dmcs_instances.tar.bz2.

Complete and Detailed Results

				dlvhex 1.x			dlvhex 2.x
				(dlv backend)					clingo backend		dlv backend
topology	parameters	instance	#models	exit status	memory used (MB)	time used (s)	#units	#unit- dependencies	memory used (MB)	time used (s)	memory used (MB)	time used (s)
diamond	7-7-3-3	a	4	OK	11.1	1.91	17	35	5.8	0.16	5.5	0.22
		b	16	ERR	3000	600	17	34	9.8	0.28	9.9	0.38
		c	4	OK	70.6	4.89	16	32	7.6	0.24	6.1	0.31
		d	24	OK	21.5	1.88	17	34	5.9	0.14	4.5	0.17
		e	12	OK	81	4.73	17	33	7.5	0.22	7.7	0.27
		f	24	OK	118.3	5.48	16	32	6.1	0.17	5.4	0.22
		g	28	OK	83.9	5.3	17	35	5.9	0.19	5.9	0.22
		h	8	OK	912.9	23.15	16	32	7.2	0.25	9.1	0.35
		i	32	OK	566.6	22.99	17	36	7.6	0.26	8	0.32
		j	28	OK	1257.3	276.8	16	32	8.3	0.3	8.1	0.41
	7-7-4-4	a	36	ERR	3000	600	17	34	12.3	0.56	11.9	0.87
		b	12	OK	2017.2	80.2	17	34	8	0.25	6.6	0.37
		c	48	ERR	1012.6	600	17	36	21	1.23	26.5	1.71
		d	14	ERR	3000	600	17	34	11.3	0.43	9.7	0.67
		e	16	ERR	547.8	600	16	32	10	0.48	12.8	0.63
		f	40	ERR	466	600	17	35	14.4	0.68	14.4	1.23
		g	16	ERR	3000	600	17	35	10.8	0.47	13	0.56
		h	768	ERR	882.8	600	16	33	34.5	1.23	32	2.21
		i	122	OK	1666.6	348.14	17	36	10.2	0.39	18	0.61
		j	32	OK	198.9	6.96	16	32	7.4	0.22	6	0.26
	7-7-5-5	a	48	ERR	3000	600	17	34	54.9	4.01	90.1	11.08
		b	16	ERR	3000	600	17	35	36.7	3.08	70.4	5.45
		c	336	ERR	449.6	600	16	32	18.7	0.77	22.8	1.14
		d	108	ERR	3000	600	17	34	36.9	2.13	64.2	4.89
		e	32	ERR	800.2	600	17	35	15.3	0.9	15.7	1.12
		f	32	ERR	752.3	600	16	32	15.9	0.8	33	1.11
		g	112	ERR	2267.9	600	16	32	45.9	1.44	23.3	2.37
		h	64	ERR	3000	600	17	36	24.2	1.32	66.1	2.24
		i	24	ERR	457.5	600	17	35	14.9	0.75	15	1.02
		j	12	ERR	413.4	600	16	31	18	0.56	11.3	0.89
house	9-9-3-3	a	812	ERR	3000	600	19	38	23.3	0.86	59.6	0.97
		b	68	ERR	956.5	600	20	41	15	0.51	16	0.71
		c	384	ERR	491.7	600	20	40	12.9	0.47	11.6	0.55
		d	48	ERR	398.5	600	20	41	8	0.24	7	0.38
		e	84	ERR	482.8	600	19	37	10.2	0.34	9.2	0.39
		f	200	ERR	1588.2	600	19	38	12.7	0.46	17.6	0.68
		g	384	ERR	812.9	600	20	42	13.4	0.46	15	0.61
		h	384	ERR	3000	600	20	41	39.3	1.33	39.7	1.99
		i	1344	ERR	3000	600	19	39	54.1	1.21	32.9	1.32
		j	84	ERR	3000	600	20	42	12.8	0.67	13.5	1.07
	9-9-4-4	a	512	ERR	3000	600	19	39	44.5	2.91	77.5	4.78
		b	1920	ERR	3000	600	20	43	87.9	3.77	259.3	13.78
		c	20	ERR	3000	600	20	42	67.7	3.01	109.5	6.89
		d	40	ERR	3000	600	20	42	11.8	0.58	11.6	0.87
		e	846	ERR	3000	600	20	41	24.6	0.72	23.1	0.89
		f	60	ERR	3000	600	20	41	35.1	2.03	71.2	3.88
		g	144	ERR	3000	600	20	41	30.3	0.99	19.8	1.5
		h	368	ERR	3000	600	20	41	31.3	1.56	50.3	2.41
		i	200	ERR	3000	600	20	41	87.9	3.37	250.2	10.13
		j	19152	ERR	3000	600	20	42	514.9	142.21	332.4	132.02
ring	7-7-4-4	a	288	ERR	854.5	600	15	28	16.4	0.55	10.8	0.6
		b	12	ERR	933.8	600	15	28	13	0.43	8.2	0.46
		c	288	ERR	3000	600	15	28	16	0.62	18.4	0.99
		d	456	ERR	3000	600	15	28	28.7	0.77	22.3	1.12
		e	80	ERR	3000	600	15	28	12.7	0.76	15.4	0.8
		f	324	ERR	3000	600	15	28	22.5	1.2	30.2	1.23
		g	24	ERR	905.7	600	15	28	8.5	0.41	7.5	0.39
		h	376	ERR	893.8	600	15	28	17.7	0.49	11.8	0.59
		i	288	ERR	3000	600	15	28	25	1.26	42.6	2.37
		j	128	ERR	494	600	15	28	9	0.38	7.1	0.41
	7-7-5-5	a	24	ERR	3000	600	15	28	21.6	0.93	16.6	0.88
		b	16	ERR	3000	600	15	28	16.5	0.98	27.5	1.33
		c	172	ERR	3000	600	15	28	42.8	1.7	20.3	1.89
		d	40	ERR	3000	600	15	28	25.2	1.61	55	2.22
		e	156	ERR	3000	600	15	28	18.9	1.49	48.6	1.53
		f	14	ERR	3000	600	15	28	20.7	1.43	48.1	2.27
		g	102	ERR	3000	600	15	28	32.5	1.4	68.8	1.91
		h	224	ERR	3000	600	15	28	67.4	2.07	47.5	2.57
		i	840	ERR	3000	600	15	28	34.8	1.78	74.9	2.65
		j	3	ERR	3000	600	15	28	17	0.5	10.2	0.55
	7-8-5-5	a	256	ERR	3000	600	15	28	34.3	1.1	24.3	1.05
		b	40	ERR	3000	600	15	28	57	1.43	24.5	1.73
		c	264	ERR	3000	600	15	28	24.1	0.73	14.1	0.8
		d	184	ERR	3000	600	15	28	33.9	1.33	39.2	2.09
		e	24	OK	2947.3	295.19	15	28	10.7	0.38	6.9	0.32
		f	864	ERR	3000	600	15	28	53.1	1.6	31.1	2.13
		g	120	ERR	1910.9	600	15	28	16.2	0.72	11.8	0.98
		h	1344	ERR	3000	600	15	28	38.8	1.95	62.9	3.53
		i	3276	ERR	3000	600	15	28	139.7	2.95	120.8	4.36
		j	168	ERR	3000	600	15	28	15.9	1.33	14.2	0.97
	7-9-5-5	a	54	ERR	3000	600	15	28	45.5	3.31	39.2	4.87
		b	116	ERR	3000	600	15	28	38.7	1.49	18.4	1.93
		c	400	ERR	3000	600	15	28	61.6	1.4	67	1.88
		d	608	ERR	3000	600	15	28	23.7	0.93	23.2	1.24
		e	32	ERR	3000	600	15	28	28.2	2.1	46.2	3.92
		f	40	ERR	3000	600	15	28	12.3	0.8	15.7	1.03
		g	576	ERR	3000	600	15	28	25.7	1.65	74.2	2.19
		h	336	ERR	3000	600	15	28	32.5	1.62	24.9	1.93
		i	234	ERR	3000	600	15	28	60.5	2.73	42.5	4.12
		j	272	ERR	3000	600	15	28	20.8	1.42	29.1	1.57
	8-7-5-5	a	54	ERR	2219.2	600	17	32	8.7	0.33	7.2	0.5
		b	138	ERR	3000	600	17	32	53.1	3.67	85.1	4.65
		c	36	ERR	3000	600	17	32	11.4	0.5	10.1	0.65
		d	672	ERR	3000	600	17	32	108.7	4.16	58.5	6.88
		e	428	ERR	3000	600	17	32	39.1	2.29	46.6	3.39
		f	1440	ERR	3000	600	17	32	50.7	1.86	37.7	2.35
		g	168	ERR	3000	600	17	32	61.3	2.21	79.4	2.97
		h	168	ERR	3000	600	17	32	18	1.39	25.3	2.16
		i	240	ERR	3000	600	17	32	41.8	1.39	26.6	1.74
		j	64	ERR	3000	600	17	32	10.7	0.56	13.1	0.75
	8-8-5-5	a	760	ERR	3000	600	17	32	28.7	1.64	56	1.81
		b	312	ERR	3000	600	17	32	25	1.55	25.6	2.05
		c	72	ERR	3000	600	17	32	22.6	1.3	23.7	1.96
		d	520	ERR	3000	600	17	32	39.1	3.02	55.1	4.72
		e	364	ERR	3000	600	17	32	19.1	1.18	42.7	1.39
		f	2400	ERR	3000	600	17	32	116.2	2.36	80.7	3.87
		g	528	ERR	3000	600	17	32	91.2	2.96	44.1	4.09
		h	1812	ERR	3000	600	17	32	75.2	2.56	67.3	2.52
		i	192	ERR	3000	600	17	32	15	0.93	16	1.33
		j	180	ERR	3000	600	17	32	14.4	0.65	14.3	1.12
zigzag	7-7-3-3	a	8	OK	61	11.63	16	34	9.9	0.35	14.1	0.42
		b	28	ERR	3000	600	17	36	13.8	0.44	9.6	0.77
		c	128	OK	258.3	6.61	17	36	9.2	0.21	6.9	0.24
		d	184	ERR	3000	600	17	34	17.1	0.55	14.2	1.13
		e	64	OK	1255.2	362.68	16	35	9.5	0.38	10	0.56
		f	42	OK	351.8	73.76	17	35	13.1	0.4	9.2	0.64
		g	6	OK	246.3	12.09	17	36	7.5	0.27	6.8	0.38
		h	10	OK	465.1	74.19	17	35	9.1	0.31	10.1	0.45
		i	8	OK	42.9	4.13	17	33	5.8	0.18	9.3	0.24
		j	14	OK	574.9	19.7	16	33	10.9	0.34	9.8	0.38
	7-7-4-4	a	80	ERR	3000	600	16	34	69	2.38	260.5	6.81
		b	36	ERR	1640.2	600	17	38	15.6	0.75	14.3	1.08
		c	24	ERR	855.5	600	16	33	19.4	1.07	54.8	1.99
		d	56	OK	674.2	265.04	17	35	11.5	0.43	16.5	0.53
		e	36	ERR	894.7	600	17	38	21.6	1.26	32.8	1.83
		f	192	ERR	3000	600	16	33	52.9	1.84	89.8	3.1
		g	134	ERR	377	600	16	35	23.8	0.71	16.8	0.89
		h	22	ERR	251.3	600	15	31	12.1	0.55	12.8	0.84
		i	18	OK	451	69.89	17	37	9	0.37	8.5	0.48
		j	4	ERR	473.6	600	16	34	17.9	0.66	20.5	1.07
	7-7-5-5	a	112	ERR	2813.4	600	16	33	29.5	1.59	51.1	3.1
		b	32	ERR	3000	600	16	33	101.4	9.77	291.8	44.59
		c	56	ERR	1763.9	600	16	35	64.8	1.83	117.1	4.36
		d	64	ERR	495.3	600	16	33	18.7	1	29.4	1.76
		e	80	ERR	3000	600	16	34	27.9	1.89	46.5	3.1
		f	42	ERR	3000	600	17	36	45.1	2.74	75.9	6.34
		g	32	ERR	3000	600	17	38	29.5	1.77	38.8	4.18
		h	8	ERR	872	600	17	38	18.9	1.2	52.7	2.35
		i	32	ERR	1824.4	600	16	34	43.6	2.21	55.1	5.78
		j	12	ERR	3000	600	17	39	35	2.84	42.4	4.48
minimum			3	-	11.1	1.88	15	28	5.8	0.14	4.5	0.17
maximum			19152	-	3000	600	20	43	514.9	142.21	332.4	132.02
average			387.84	-	2118.48	515.55	16.63	33.09	32.1	2.29	40.18	3.24
standard deviation			1656.14	-	1152.95	196.75	1.52	4.25	47.51	11.93	52.77	11.69

Reviewer Selection 1 - Instances

These test instances were created using revsel2.cpp with varied parameter -t (the number of conference track per instance).

Download

All instances are contained in the following archive: revsel1_instances.tar.bz2.

Complete and Detailed Results

		dlvhex 1.x (old heuristics H1 + dlv backend)			dlvhex 2.x + former heuristics H1 + dlv backend			dlvhex 2.x + simple heuristics H2 + dlv backend
# tracks	# papers	exit status	memory used (MB)	time used (s)	exit status	memory used (MB)	time used (s)	exit status	memory used (MB)	time used (s)
1	20	OK	43.9	1.49	OK	31.7	1.45	OK	31.5	1.36
2	20	OK	73.1	3.31	OK	72.7	2.65	OK	56.2	2.11
3	20	OK	120.8	5.89	OK	112.1	4.17	OK	82	3.15
4	20	OK	152.8	11.64	OK	142.4	5.82	OK	106.3	4.02
5	20	OK	197.5	28.47	OK	180.2	8.21	OK	131.3	5.25
6	20	OK	239.7	71.64	OK	239.6	11.92	OK	165.2	6.17
7	20	OK	325.8	109.14	OK	258.5	16.31	OK	164.5	7.17
8	20	OK	777.3	477.51	OK	309.4	28.08	OK	213.7	7.92
9	20	ERR	1716.3	600	OK	422.4	51.74	OK	250.1	9.58
10	20	ERR	2533.7	600	OK	588.3	100.26	OK	242.2	10.87
11	20	ERR	3000	600	OK	1413.7	206.24	OK	262.4	12.01
12	20	ERR	3000	600	OK	3000	458.02	OK	294.3	12.65
13	20	ERR	3000	600	OK	3000	600	OK	337.9	13.95
14	20	ERR	3000	600	ERR	3000	600	OK	295.7	15.03
15	20	ERR	3000	600	ERR	3000	600	OK	397.8	16.73
16	20	ERR	3000	600	ERR	3000	600	OK	380.5	17.32
17	20	ERR	3000	600	ERR	3000	600	OK	461.4	18.85
18	20	ERR	3000	600	ERR	3000	600	OK	481.2	19.9
19	20	ERR	3000	600	ERR	3000	600	OK	520.3	21
20	20	ERR	3000	600	ERR	3000	600	OK	444.4	21.94
21	20	ERR	3000	600	ERR	3000	600	OK	561.5	23.11
22	20	ERR	3000	600	ERR	3000	600	OK	605.1	24.48
23	20	ERR	3000	600	ERR	3000	600	OK	628.5	25.54
24	20	ERR	3000	600	ERR	3000	600	OK	643.3	27.07
25	20	ERR	3000	600	ERR	3000	600	OK	683.8	28.29
26	20	ERR	3000	600	ERR	3000	600	OK	694.6	29.74
27	20	ERR	3000	600	ERR	3000	600	OK	726	31.09
28	20	ERR	3000	600	ERR	3000	600	OK	756.9	31.94
29	20	ERR	3000	600	ERR	3000	600	OK	745	33.45
30	20	ERR	3000	600	ERR	3000	600	OK	761.2	34.26
31	20	ERR	3000	600	ERR	3000	600	OK	848.4	36.66
32	20	ERR	3000	600	ERR	3000	600	OK	894.2	37.29
33	20	ERR	3000	600	ERR	3000	600	OK	841.8	39.43
34	20	ERR	3000	600	ERR	3000	600	OK	914.3	39.57
35	20	ERR	3000	600	ERR	3000	600	OK	889.9	42.45
36	20	ERR	3000	600	ERR	3000	600	OK	946.1	43.3
37	20	ERR	3000	600	ERR	3000	600	OK	1019	44.7
38	20	ERR	3000	600	ERR	3000	600	OK	998.6	45.95
39	20	ERR	3000	600	ERR	3000	600	OK	1015	46.23
40	20	ERR	3000	600	ERR	3000	600	OK	1118.2	47.89
41	20	ERR	3000	600	ERR	3000	600	OK	1051.6	49.73
42	20	ERR	3000	600	ERR	3000	600	OK	1113.6	52.66
43	20	ERR	3000	600	ERR	3000	600	OK	1213.4	53.47
44	20	ERR	3000	600	ERR	3000	600	OK	1291	54.54
45	20	ERR	3000	600	ERR	3000	600	OK	1358.2	56.84
46	20	ERR	3000	600	ERR	3000	600	OK	1287.4	58.74
47	20	ERR	3000	600	ERR	3000	600	OK	1347.6	60.08
48	20	ERR	3000	600	ERR	3000	600	OK	1286.4	61.41
49	20	ERR	3000	600	ERR	3000	600	OK	1403.1	62.61
50	20	ERR	3000	600	ERR	3000	600	OK	1408.2	65

Reviewer Selection 2 - Instances

These test instances were created using revsel1.cpp with varied parameter -p (the number of papers per conference track).

Download

All instances are contained in the following archive: revsel2_instances.tar.bz2.

Complete and Detailed Results

			dlv			clingo (clasp+gringo)			dlvhex 2.x + simple heuristics H2 + dlv backend			dlvhex 2.x + simple heuristics H2 + clingo backend
#tracks	#papers	#models	exit status	memory used (MB)	time used (s)	exit status	memory used (MB)	time used (s)	exit status	memory used (MB)	time used (s)	exit status	memory used (MB)	time used (s)
5	1	0	OK	0	0	OK	0	0	OK	0	0.01	OK	0	0.01
5	2	9	OK	0	0	OK	0	0	OK	0	0.04	OK	0	0.03
5	3	9	OK	0	0	OK	0	0.01	OK	0	0.05	OK	0	0.03
5	4	9	OK	0	0.01	OK	0	0.01	OK	0	0.06	OK	0	0.06
5	5	9	OK	0	0.03	OK	0	0.02	OK	0	0.07	OK	0	0.06
5	6	9	OK	0	0.06	OK	0	0.04	OK	4.2	0.13	OK	4.5	0.12
5	7	9	OK	0	0.11	OK	0	0.04	OK	5.1	0.16	OK	4.7	0.13
5	8	9	OK	4.4	0.18	OK	5.4	0.1	OK	6.1	0.22	OK	4.9	0.17
5	9	9	OK	7.2	0.29	OK	6.4	0.19	OK	8.1	0.33	OK	5	0.26
5	10	9	OK	9.8	0.45	OK	6.4	0.2	OK	8.8	0.47	OK	5.3	0.28
5	11	9	OK	14.6	0.66	OK	9.6	0.3	OK	14.9	0.73	OK	5.8	0.35
5	12	9	OK	27.5	1.02	OK	14.3	0.42	OK	22.7	0.88	OK	6.5	0.63
5	13	9	OK	33.5	1.29	OK	18.3	0.61	OK	26.6	1.1	OK	7.4	0.63
5	14	9	OK	42.8	1.82	OK	22.6	0.81	OK	29.2	1.45	OK	8.1	0.67
5	15	9	OK	57.8	2.28	OK	28.8	1.1	OK	38.8	1.78	OK	9.3	1.79
5	16	9	OK	69.3	2.97	OK	36.4	1.47	OK	51.2	2.44	OK	10.8	1.62
5	17	9	OK	116.8	3.88	OK	44.2	1.91	OK	58.6	2.87	OK	12.6	1.54
5	18	9	OK	125.1	5.14	OK	54.5	2.49	OK	79.7	3.47	OK	14.2	1.77
5	19	9	OK	151.1	6.93	OK	66.2	3.21	OK	97.3	4.16	OK	16.7	2.39
5	20	9	OK	230.8	8.57	OK	80.6	3.92	OK	112.8	5.48	OK	18.7	2.99
5	21	9	OK	231.7	10.18	OK	95.1	5.04	OK	155.6	6.41	OK	22.2	3.76
5	22	9	OK	256.3	10.59	OK	114.4	6.02	OK	180.1	7.63	OK	25.1	4.73
5	23	9	OK	330.8	13.45	OK	133.6	7.43	OK	201.9	9.99	OK	30.2	6.99
5	24	9	OK	458.6	15.71	OK	162.1	8.96	OK	220	10.9	OK	33.2	6.43
5	25	10	OK	459.8	17.77	OK	186.5	10.52	OK	243.1	12.8	OK	38.8	10.09
5	26	11	OK	520.6	21.97	OK	214.2	13.1	OK	341.5	14.71	OK	44.5	8.89
5	27	12	OK	600.1	25.41	OK	251.1	15.06	OK	385.5	17.68	OK	49.1	12.53
5	28	13	OK	911.8	28.1	OK	290.7	17.63	OK	527.2	19.97	OK	55.9	13.59
5	29	14	OK	913.3	35.46	OK	331.7	20.5	OK	432.7	23.43	OK	64.5	13.22
5	30	15	OK	914.9	36.62	OK	380.8	23.93	OK	398	26.79	OK	70.6	14.61
5	31	16	OK	1013.5	44.53	OK	427.4	27.78	OK	519.4	28.94	OK	80.1	20.39
5	32	17	OK	1227.8	48.05	OK	492.9	32.12	OK	295.6	33.01	OK	92.7	23.68
5	33	18	OK	1816.3	60.42	OK	549.8	37.41	OK	947.6	39.03	OK	101.4	24.63
5	34	19	OK	1818.4	63.68	OK	623.1	44.07	OK	821.5	43.31	OK	117.1	32.27
5	35	20	OK	1820.5	72.94	OK	691.1	48.91	OK	822	49.35	OK	132.3	31.37
5	36	21	OK	1958.9	78.54	OK	779.3	55.41	OK	683.3	52.84	OK	141.8	36.52
5	37	22	OK	2107.6	92.86	OK	865.3	63.04	OK	685.3	61.2	OK	157.6	39.54
5	38	23	OK	2329.9	99.25	OK	951.3	75.02	OK	939.4	68.49	OK	172.6	45.87
5	39	24	OK	2794.8	117.66	OK	1045.2	80.86	OK	1211.5	76.46	OK	194.9	49.82
5	40	25	ERR	3000	600	OK	1134.8	90.33	OK	855.4	83.5	OK	210.1	65.25
5	41	26	ERR	3000	600	OK	1296.2	105.07	OK	488.3	93.92	OK	233.4	67.82
5	42	27	ERR	3000	600	OK	1383.3	113.2	OK	788.1	97.51	OK	251	73.18
5	43	28	ERR	3000	600	OK	1520.5	127.61	OK	1234.9	116.37	OK	280.6	81.97
5	44	29	ERR	3000	600	OK	1644.1	143.91	OK	1723.9	121.46	OK	308.9	89.94
5	45	30	ERR	3000	600	OK	1783.1	159.53	OK	1372.8	133.25	OK	328.6	100.56
5	46	31	ERR	3000	600	OK	1952.1	172.73	OK	1809.8	149.21	OK	354.1	118.54
5	47	32	ERR	3000	600	OK	2094.7	202.25	OK	2091.4	167.52	OK	386.1	121.87
5	48	33	ERR	3000	600	OK	2261.4	215.39	OK	1431.8	175.74	OK	423.5	142.17
5	49	34	ERR	3000	600	OK	2503.8	242.77	OK	1991.7	194.42	OK	460.9	155.78
5	50	35	ERR	3000	600	OK	2890.7	262.91	OK	2061.6	206.17	OK	497.9	193.36
5	51	36	ERR	3000	600	OK	2943.4	300.58	OK	1598.3	225.69	OK	531.6	203.94
5	52	37	ERR	3000	600	ERR	3000	600	OK	2211.3	236.35	OK	583.2	207.76
5	53	38	ERR	3000	600	ERR	3000	600	OK	2340.5	261.56	OK	641.7	214.53
5	54	39	ERR	3000	600	ERR	3000	600	OK	2384.7	278.85	OK	684.4	265.44
5	55	40	ERR	3000	600	ERR	3000	600	OK	2313.8	309.97	OK	720.5	280.53
5	56	41	ERR	3000	600	ERR	3000	600	OK	2598.7	324.43	OK	764.2	288.02
5	57	42	ERR	3000	600	ERR	3000	600	OK	2447.8	350.54	OK	814.7	370.84
5	58	43	ERR	3000	600	ERR	3000	600	OK	1972.2	392.71	OK	868.2	340.77
5	59	44	ERR	3000	600	ERR	3000	600	OK	2197.3	425.45	OK	951.9	425.85
5	60	45	ERR	3000	600	ERR	3000	600	OK	2074.8	437.47	OK	1002.4	411.15

External Behavior Learning

This section compares dlvhex 2.x with and without external behavior learning (EBL).

General Information

All benchmarks were carried out on a machine with 24 Dual-Core AMD Opteron Magny-Cours 6176 SE CPUs and 128 GB RAM, running Linux and using Clasp 2.0.5 as solver backend.

Set Partitioning

In our first (synthetic) example program we consider a set S of elements which is partitioned into two subsets S₁ and S₂, such that S₁ contains at most two elements. This is realized with HEX-programs using external atoms of kind &setminus[p,q](X), which computes the set difference of the extensions of p and q.

We use the program template partitioning.hex, specifying two cyclic rules which realize the partitioning, and give the solver a hint on monotonicity in the first input predicate. For benchmarking we use a benchmark script which automatically iterates through instance sizes and instantiates the program template accordingly. The script can be downloaded here (and is also contained in the dlvhex repository): benchmark.sh. It is called like this:

benchmark.sh [Program] [Min Domain Size] [Max Domain Size] [Timeout/s] [Configuration Strings (optional)],

where

Program is the Name of a HEX-program
Min Domain Size and Max Domain Size specify the range of instance sizes
Timeout specifies the maximum runtime per instance
Configuration String

The benchmark script evaluates Program for each integer i between Min Domain Size and Max Domain Size, and adds all facts domain(1), ..., domain(i) to the program prior to evaluation. dlvhex is then called once per configuration specified in the last argument.

An example call is:

./benchmark.sh partitioning.hex 1 20 300 "--solver=genuinegc;--solver=genuinegc --extlearn"

It evaluates partitioning.hex for each instance size from 1 to 20, once with arguments --solver=genuinegc and once with arguments --solver=genuinegc --extlearn, i.e., it compares the genuine solver (with gringo+clasp backend) with and without external learning.

We summarized the runtimes for the configurations

--solver=genuinegc (no external learning, all models)
--solver=genuinegc --extlearn (external learning, all models)
--solver=genuinegc -n=1 (no external learning, one model)
--solver=genuinegc --extlearn -n=1 (external learning, one model)

in benchmarks.ods.

Bird-Penguin

We use DL-programs for realizing default reasoning over a description logic knowledge base, realized as a plugin to dlvhex. Program birdpenguin.hex queries all birds from ontology (template) bird.owl and encodes that bird typically fly (if the contrary cannot be derived). For varying the input size, we use an adopted version of the benchmark script above which produces ontology instances with A-boxes of different sizes. The script can be found in the repository of the DL-plugin

The runtime results of 5 runs are summarized in birdpenguin.ods.

Wine Ontology

The findings from the bird-penguin example carry over to the larger wine ontology http://www.w3.org/TR/owl-guide/wine.rdf (and scaled variants available at http://kaon2.semanticweb.org/download/test_ontologies.zip). We made three experiments which implement three different default rules as follows:

1. "A wine is dry, unless the contrary can be derived"

For this experiment we used three scripts drywine_pi.sh, drywine_omega.sh and drywine_y.sh to produce HEX programs for classifying different categories of wines, based on different transformations [dek2009].

2. "A wine is dry, unless it is known that it is sweet"

For this experiment we used the Omega-transformation, which turned out to be the most efficient one. We used the script sweetwine.sh to produce the HEX programs for classifying different categories of wines.

3. "A wine is white, unless it is known to be red"

For this experiment we used the Omega-transformation, which turned out to be the most efficient one. We used the script redwine.sh to produce the HEX programs for classifying different categories of wines.

The runtime results of 3 runs are summarized in benchmarks.ods.

Multi-context Systems

For our multi-context benchmarks we used the instances from mcs.zip and the MCS plugin for dlvhex.

Runtime results are summarized in benchmarks.ods.

Literature

Minh Dao-Tran, Thomas Eiter, and Thomas Krennwallner.
Realizing Default Logic over Description Logic Knowledge Bases.
In Claudio Sossai and Gaetano Chemello, editors, Proceedings of the 10th European Conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty (ECSQARU 2009), Verona, Italy, July 1-3, 2009 , volume 5590 of LNAI, pages 602-613. Springer, July 2009.
[ DOI ]

Search this Website

General
dlvhex source code @ github.com
Description-Of-A-Project

Documentation
User Guide
README
doxygen
Writing Plugins in C++
Writing Plugins in Python