SWI-Prolog External Database
Jan Wielemaker
SWI,
University of Amsterdam
The Netherlands
E-mail: jan@swi.psy.uva.nl
This package realised external storage of Prolog terms based on the Berkeley
DB library from Sleepycat
Software. The DB library implements modular support for the bottom
layers of a database. The database itself maps unconstrained keys onto
values. Both key and value are binary blobs.
The SWI-Prolog interface for DB allows for fast storage of general Prolog terms in the database. |
The native Prolog database is not very well suited for either very large data-sets or dynamically changing large data-sets that need to be communicated between Prolog instances or need to be safely guarded against system failure. These cases ask for an external database that can be attached quickly and provides protection against system failure.
The Berkeley DB package by SleepyCat software is a GPL'ed library realising the bottom-layers of a database. It is a modular system, which in it's simplest deals with resource management on a mapped file and in its most complex form deals with network transparency, transaction management, locking, recovery, life-backup, etc.
The DB library maps keys to values. Optionally multiple values can be associated with a key. Both key and value are arbitrary-length binary objects.
This package stores arbitrary Prolog terms, using PL_record_external() introduced in SWI-Prolog 3.3.7, in the database. It provides an interface similar to the recorded-database (recorda/3). In the future we plan to link this interface transparently to a predicate.
This manual is by no means complete. The Berkeley DB documentation should be consulted directly to resolve details on security, resource usage, formats, configuration options etc. This interface passed default values for most DB API calls. Supported options hint to the corresponding DB API calls, which should be consulted for details.
Berkeley DB is an embedded database. This implies the library provides access to a file containing one or more database tables. The Berkeley DB database tables are always binary, mapping a key to a value.
Accessing a database consists of four steps:
read
,
providing read-only access or update
, providing read/write
access. Options is a list of options. Currently supported
options are:
key
for details.
The predicates in this section are used to read and write the
database. These predicate use a Key and a Value.
These should satisfy the key and value-types specified with db_open/4.
If a value is declared using the type term
(default),
arbitrary Prolog terms may be put in the database.
If a non-ground term is used as Key, it is matched using
structural equivalence. See =@=/2 in the SWI-Prolog
reference manual for details. For short, if a term a(A,B)
is used as key, it will only be found using a key of the same format: a
term with functor a
and two unbound arguments that are not
shared.
Using the DB transaction protocol, security against system failure, atomicy of multiple changes and accessing a database from multiple writers is provided.
Accessing a DB under transactions from Prolog is very simple. First
of all, the option transactions(true)
needs to be provided
to
db_init/1 to
initialise the transaction subsystem. Next, the predicate
db_transaction/1
may be used to execute multiple updates inside a transaction.
Of special interest is the exception
error(package(db, deadlock)
, _)
This exception indicates a deadlock was raised by one of the DB predicates. Deadlocks may arise if multiple processes or threads access the same keys in a different order. The DB infra-structure causes one of the processes involved in the deadlock to abort its transaction. This process may choose to restart the transaction.
For example, a DB application may define to realise transactions and restart these automatically is a deadlock is raised:
{}(Goal) :- catch(db_transaction(Goal), E, true), ( var(E) -> true ; E = error(package(db, deadlock), _) -> { Goal } ; throw(E) ). |
The Berkeley DB routines are not signal-safe. Without precaution, this implies it is not possible to interrupt Prolog programs using the DB routines in a safe manner. To improve convinience, interrupt signals are blocked during the execution of the DB calls. As db_transaction/1 handles aborts gracefully, PrologDB applications can be interrupted and aborted safely.
Signals other than SIGINT
caught during the execution of
one of the DB interaction predicates may leave the DB in an inconsistent
state. Fatal signals thrown by other Prolog or foreign language
facilities are handled gracefully.
Optionally, the DB environment may be initialised explicitely. Without initialisation, the DB predicates may be used to access a database file without transaction support and using default caching. This is generally satisfactory for not-too-large databases that have no strong security demands and are accessed by at most one application in update mode.
Options is a list of options. The currently supported are listed below. For details, please refer to the DB manual.
Name(Value)
.
true
, create any underlying file as required. By
default, no new files are created. This option should be set for
prograns that create new databases.
logging
and locking
. See
section 2.4.
This package was developed for DB version 3.1. The interface of DB 3.x is fundamentally different from previous releases and db4pl relies on functionality provided by DB 3.x. Unfortunately many distributions of DB are still based on DB 2.x. Please make sure to install DB 3.1 or later before building db4pl.
Installation on Unix system uses the commonly found configure,
make and make install sequence. SWI-Prolog should be
installed before building this package. If SWI-Prolog is not installed
as pl, the environment variable PL
must be set to
the name of the SWI-Prolog executable. Installation is now accomplished
using:
% ./configure % make % make install |
This installs the foreign libraries in $PLBASE/lib/$PLARCH
and the Prolog library files in $PLBASE/library
, where
$PLBASE
refers to the SWI-Prolog `home-directory'.
Configure recognises the following options in addition to the default GNU configure options.
./configure --with-db=/usr/local/BerkeleyDB.3.1 |