How can I prevent XMLDB corruption?

The eXist XMLDB can get corrupted after an unclean database shutdown. An unclean shutdown may be caused by power failures, OS reboots, or hanging processes that are subsequently killed.

A good improvement was seen after adding the recovery parameter, which configures the journaling and recovery of the database. With recovery enabled, it is much easier to recover an unclean database. For this to work correctly, all database operations must be logged to a journal file.

To add the recovery parameter, one must follow these steps:

a) make a backup for the ${MUSE_HOME}/xmldb/webapps/exist/WEB-INF/conf.xml file;

b) edit the ${MUSE_HOME}/xmldb/webapps/exist/WEB-INF/conf.xml file and just before the ““(no quotes) add the lines from below:

<!–
This element configures the journaling and recovery of the database.
With recovery enabled, the database is able to recover from an unclean database shutdown due to,
for example, power failures, OS reboots, and hanging processes. For this to work correctly,
all database operations must be logged to a journal file. The location, size and other parameters
for this file can be set using the element.
recovery Attributes:
– enabled:
If this attribute is set to yes, automatic recovery is enabled.
– size:
This attributes sets the maximum allowed size of the journal file.
Once the journal reaches this limit, a checkpoint will be triggered and the journal will be cleaned.
However, the database waits for running transactions to return before processing this checkpoint.
In the event one of these transactions writes a lot of data to the journal file, the file will grow
until the transaction has completed. Hence, the size limit is not enforced in all cases.
– journal-dir:
This attribute sets the directory where journal files are to be written.
If no directory is specified, the default path is to the data directory.
– sync-on-commit:
This attribute determines whether or not to protect the journal during operating system failures.
That is, it determines whether the database forces a file-sync on the journal after every commit.
If this attribute is set to “yes”, the journal is protected against operating system failures.
However, this will slow performance – especially on Windows systems. If set to “no”, eXist will rely
on the operating system to flush out the journal contents to disk. In the worst case scenario,
in which there is a complete system failure, some committed transactions might not have yet been written to the journal, and so will be rolled back.
– group-commit:
If set to “yes”, eXist will not sync the journal file immediately after every transaction commit.
Instead, it will wait until the current file buffer (32kb) is really full. This can speed up eXist on some systems
where a file sync is an expensive operation (mainly windows XP; not necessary on Linux).
However, group-commit=”yes” will increase the chance that an already committed operation is rolled back after a database crash.
– force-restart:
Try to restart the db even if crash recovery failed. This is dangerous because there might be corruptions inside the data files.
The transaction log will be cleared, all locks removed and the db reindexed. Set this option to “yes” if you need to make sure that the db is online, even after a fatal crash. Errors encountered during recovery are written to the log files.
Scan the log files to see if any problems occurred.
-consistency-check:
If set to “yes”, a consistency check will be run on the database if an error was detected during crash recovery.
This option requires force-restart to be set to “yes”, otherwise it has no effect.
The consistency check outputs a report to the directory {files}/sanity and if inconsistencies are found in the db, it writes an emergency backup to the same directory.

–>
<recovery enabled="yes" size="100M" sync-on-commit="no" group-commit="yes" force-restart="no" consistency-check="no" />

c) Restart Apache Embedded Tomcat server.