[Mulgara-dev] CRITICAL: Bug fix to Backup operation

Ben Hysell BenH at viewpointusa.com
Tue Mar 25 21:43:08 UTC 2008


We check for inconsistencies using the following methods with the
decompressed backup file:

1. Read in the string pool and ensure there are not duplicate entries in
the string pool, i.e. node numbers listed twice

2. Read through the string pool looking for any entries that start with
_node followed by a number

3. Look up each node number in the TRIPLES section to ensure there is a
corresponding string pool entry for the node number in question. 

At one point someone had sent out on the list how to search your
restored running Mulgara instance to check and see if any _node entries
existed, however I've lost the email, and every time we ran the query we
would crash Mulgara.

As for our testing:

I took the server1 folder from production that was causing problems,
copied over the new jar files, ran a backup and examined it using the
steps from above.

1. I still have duplicate entries in the string pool, however this time
they are grouped together, i.e. the one instance in this back up is node
6290, which is listed twice:

6288
<http://www.viewpoint.com/viewpointcfca/rado/elements/1.0/CCDfile#04C99C
68-4CCC-4D00-8C49FC8B698DFF3C>
6289
<http://www.viewpoint.com/viewpointcfca/rado/elements/1.0/CCDfile#04C99F
E5-2191-CE4A-A77A-0A0858C4D78D>
6290
<http://www.viewpoint.com/viewpointcfca/rado/elements/1.0/CCDfile#04C9DA
35-5C74-480D-A08457DF764FE61A>
6290
<http://www.viewpoint.com/viewpointcfca/rado/elements/1.0/CCDfile#04C9DA
35-5C74-480D-A08457DF764FE61A>
6291
<http://www.viewpoint.com/viewpointcfca/rado/elements/1.0/CCDfile#04C9FB
02-1D3F-8A4C-9461-3C5320315477>
6292
<http://www.viewpoint.com/viewpointcfca/rado/elements/1.0/CCDfile#04CA29
44-8B93-462A-887EAB5B15960508>

2. There are no listings of _node in the backup file, this tells me the
string pool is 'clean', where if I query the database I'll never return
an entry that has _node followed by a large number.

3. There is roughly 17k triples that contain node numbers represented in
the TRIPLES section that do not have corresponding string pool entries.
If I restored this backup I would introduce 17k triples that would have
one of the triples represented with _node.

4. I did a backup of the same server1 with rev 570 with the following
results:

-there is roughly 17k node numbers in the TRIPLES section that do not
have corresponding string pool entries
-there are no _node string pool entries
-there is a duplicate in the string pool, but as I put out in my
original email the duplicate string pool entry is near the bottom of the
backup file, included with the node IDs of 6427720.


I'm still concerned we have a duplicate in the string pool entries and
not all of the strings in the string pool are making it out to the
backup file.

-ben

-----Original Message-----
From: mulgara-dev-bounces at mulgara.org
[mailto:mulgara-dev-bounces at mulgara.org] On Behalf Of Russell Uman
Sent: Tuesday, March 25, 2008 4:21 PM
To: Mulgara Developers
Subject: Re: [Mulgara-dev] CRITICAL: Bug fix to Backup operation

we've done backup and then restore on production databases - is there
anything we need to do to repair our data and/or to look for
inconsistencies? 

thanks!

> -----Original Message-----
> From: mulgara-dev-bounces at mulgara.org 
> [mailto:mulgara-dev-bounces at mulgara.org] On Behalf Of Andrae Muys
> Sent: Tuesday, March 25, 2008 2:51 AM
> To: Mulgara Developers
> Cc: Ben Hysell
> Subject: [Mulgara-dev] CRITICAL: Bug fix to Backup operation
> 
> 
> Revision 710 in trunk/ is a corrects a critical bug in 
> mulgara's backup-operation.
> 
> Could anyone who has been experiencing instability in 
> performing backups please try out 710 to see if this fix 
> helps.  Given the critical nature of this bug I strongly 
> encourage anyone using old versions of mulgara to please 
> consider backporting this fix to whichever version you are 
> using - I am more than willing to provide help in doing this 
> if help is required.
> 
> This bug is consistent with both the NPE being experienced by 
> PLoS/ Topaz; and the appearance of Blank-nodes in the 
> backup-files being observed by Viewpoint.  Absent a second 
> bug also causing these symptoms, this fix should resolve both 
> these issues.
> 
> Andrae
> 
> --
> Andrae Muys
> andrae at netymon.com
> Senior RDF/SemanticWeb Consultant
> Netymon Pty Ltd
> 
> 
> _______________________________________________
> Mulgara-dev mailing list
> Mulgara-dev at mulgara.org
> http://mulgara.org/mailman/listinfo/mulgara-dev
> 

------------------------------------------------------------------------
------------------------------------------------------------------------
------------------------------------------------------------------------
--
This email is confidential to the intended recipient. If you have
received it in error, please notify the sender and delete it from your
system. Any unauthorized use, disclosure or copying is not permitted.
The views or opinions presented are solely those of the sender and do
not necessarily represent those of Public Library of Science unless
otherwise specifically stated. Please note that neither Public Library
of Science nor any of its agents accept any responsibility for any
viruses that may be contained in this e-mail or its attachments and it
is your responsibility to scan the e-mail and attachments (if any).

_______________________________________________
Mulgara-dev mailing list
Mulgara-dev at mulgara.org
http://mulgara.org/mailman/listinfo/mulgara-dev



More information about the Mulgara-dev mailing list