[Mulgara-dev] CRITICAL: Bug fix to Backup operation

Fri Mar 28 19:40:51 UTC 2008

Andrae,

You are completely correct with this bug described in this email and I
was wrong in my previous email.  In my previous email I stated:

But if a backup has node numbers in the triples section and not in the
string pool and we restore that backup and then query it the query will
return "_node######".  If you then backup that Mulgara and look in the
string pool you'll then find: ## "_node######".  

My second statement is incorrect, I am unable to reproduce this behavior
using Andrae's method below.  Following the steps below with Andrae's
smaller backup, a smaller backup created from our data that exhibits
blank nodes (which I have attached), and using a full database of our
data I cannot re-create this scenario in the manner I initially
described.  I had assumed that after a period of time if we were not
watching our backups the "_node####" that appeared when the database was
queried was due to restoring a database with blank nodes.  This appears
to be an incorrect assumption on my part.

We have however ended up with the '## "_node######"' in our backup
files...I am back to square one on how these actually get into the
backup file.  In reference to these "_node###" that do eventually show
up...we are not inserting them, they used to reference real data until
somewhere something 'happens' and the backups start spitting out
"_node####" in the backup file.  I'm not sure if this is a time issue, a
transaction issue, or both.  The size of our server1 directory starts at
5.5 GB, we often start having problems when it grows and always perform
a restore before we hit 20 GB.  Our longest stretch is two weeks before
we have either hit the 20 GB mark or we are starting to see blank nodes
in the backup file, shorter if we are doing a lot of transactions.  

The issues we are still tracking:
1. how do "_node###" ever get into the backup file
2. Duplicate string pool entries
3. why are we getting blank nodes in our backup file to begin with.  We
can still query the running Mulgara to find the data, but we can't back
it up.
4. In my attached backup file I actually can't load it in the present
form, when I do I get the following Java error on win2k3 server dual
cpu, quad cores with 8 Gb ram...its an interesting error that appears to
be because of the very large node numbers listed in the backup,
thoughts? (I can get it to load if I make the node numbers small):

C:\mulgara_rev710>java -Xms4000m -Xmx8000m -XX:GCTimeRatio=19 -jar
mulgara-1.1.1
.jar -k 192.168.100.70 -o 192.168.100.70
Mulgara Semantic Store Version 1.1.1 (Build 1.1.1.local)
 INFO [main] (EmbeddedMulgaraServer.java:739) - RMI Registry started
automatical
ly on port 1099
0 [main] INFO org.mulgara.server.EmbeddedMulgaraServer  - RMI Registry
started a
utomatically on port 1099
 INFO [main] (EmbeddedMulgaraServer.java:788) - java.security.policy set
to jar:
file:/C:/mulgara_rev710/mulgara-1.1.1.jar!/conf/mulgara-rmi.policy
0 [main] INFO org.mulgara.server.EmbeddedMulgaraServer  -
java.security.policy s
et to
jar:file:/C:/mulgara_rev710/mulgara-1.1.1.jar!/conf/mulgara-rmi.policy
2008-03-28 15:14:26,407 INFO  Database - Host name aliases for this
server are:
[localhost, 127.0.0.1, 192.168.100.70, todd]
2008-03-28 15:14:29,657 ERROR ? - Delete existing temp dir
C:\DOCUME~1\benh\LOCA
LS~1\Temp\1\Jetty_192_168_100_70_8080__webui for
WebApplicationContext[/webui,ja
r:file:/C:/mulgara_rev710/mulgara-1.1.1.jar!/webapps/webui.war]
java.lang.Error: Cleaner terminated abnormally
        at sun.misc.Cleaner$1.run(Cleaner.java:130)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.misc.Cleaner.clean(Cleaner.java:127)
        at
java.lang.ref.Reference$ReferenceHandler.run(Reference.java:124)
Caused by: java.io.IOException: The process cannot access the file
because anoth
er process has locked a portion of the file
        at sun.nio.ch.FileChannelImpl.unmap0(Native Method)
        at
sun.nio.ch.FileChannelImpl.access$000(FileChannelImpl.java:26)
        at
sun.nio.ch.FileChannelImpl$Unmapper.run(FileChannelImpl.java:680)
        at sun.misc.Cleaner.clean(Cleaner.java:125)
        ... 1 more

C:\mulgara_rev710>

-ben

-----Original Message-----
From: mulgara-dev-bounces at mulgara.org
[mailto:mulgara-dev-bounces at mulgara.org] On Behalf Of Andrae Muys
Sent: Thursday, March 27, 2008 10:09 PM
To: Mulgara Developers
Subject: Re: [Mulgara-dev] CRITICAL: Bug fix to Backup operation

On 28/03/2008, at 5:05 AM, Ben Hysell wrote:
> We restore to a brand new Mulgara instance that has an empty server1 
> directory.

Ok, in this case I am unable to reproduce the 2nd bug discussed below.

Just confirming my understanding of what's going on:

1. You are using the latest mulgara from svn trunk/ 2. You backup the
mulgara instance - and the resulting backup contains blank-nodes 3. You
startup a new mulgara instance (pointing at a new directory) 4. You
restore the backup produced in 2.
5. You backup the new instance - and the resulting backup has replaced
the blank-nodes with "_node#####" literals.

I have attempted to reproduce this on my system and failed - I have
included the two backup files (decompressed) at the end of this email.

If I haven't misunderstood what you are seeing, then I am going to need
a fuller description of your environment - I hate these sort of bugs.

Andrae

FIRST BACKUP (backup1)

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: backup large node numbers.txt
URL: <http://lists.mulgara.org/pipermail/mulgara-dev/attachments/20080328/8b590f32/attachment.txt>