From LotzM at niehs.nih.gov  Wed Nov  2 13:23:45 2011
From: LotzM at niehs.nih.gov (Lotz, Michael (NIH/NIEHS) [C])
Date: Wed, 2 Nov 2011 09:23:45 -0400
Subject: [Mulgara-dev] Mulgara high CPU
In-Reply-To: <BANLkTikGL-T2rG4DwG-SQuav13vs9AJNUw@mail.gmail.com>
Message-ID: <CAD6BB9B.CB09%lotzm@niehs.nih.gov>

Paul -

This issue has reared in ugly head again - but I don't see any strange log
errors.

Can you look at the backup and see if you can tell what's going on -
<http://ehp01.niehs.nih.gov/mulgara/backup_1101am.gz>

I'm sorry to bug - you but a backup/restore and moving mulgara to another
server did not solve the problem.

Thanks.
Mike


---------------------------------------------------------------------------
-
Mike Lotz 
Technology Manager
EHP/NIEHS (Brogan & Partners ? Contractor)
Environmental Health Perspectives / NIEHS
919-541-7661
www.ehponline.org
---------------------------------------------------------------------------
-





On 4/27/11 12:08 PM, "Paul Gearon" <gearon at ieee.org> wrote:

>Hi Michael,
>
>Thanks, though if a backup/restore hasn't worked, then it must be
>something else. I'll get to this when I can, but it probably won't be
>today.
>
>Paul
>
>On Wed, Apr 27, 2011 at 11:54 AM, Lotz, Michael (NIH/NIEHS) [C]
><LotzM at niehs.nih.gov> wrote:
>> http://ehp01.niehs.nih.gov/mulgara/topazproject/
>>
>> Here is the actual mulgara data folders.
>>
>>
>> 
>>-------------------------------------------------------------------------
>>--
>> -
>> Mike Lotz
>> Technology Manager
>> EHP/NIEHS (Brogan & Partners) (Contractor)
>> Environmental Health Perspectives
>> National Institute of Environmental Health Sciences
>> [voc] 919-541-7661                [fax] 919-541-0273
>> 
>>-------------------------------------------------------------------------
>>--
>> -
>>
>>
>>
>>
>>
>> On 4/26/11 5:12 PM, "Lotz, Michael (NIH/NIEHS) [C]"
>><LotzM at niehs.nih.gov>
>> wrote:
>>
>>>Paul -
>>>
>>>Here is the error from Ambra.log - might help:
>>>
>>>2011-04-26 11:00:07,496 ERROR UnhandledException(AmbraJournal)>  Too
>>>many
>>>values for 'public void
>>>org.topazproject.ambra.models.DublinCore.setCreated(java.util.Date)' :
>>>[2009-10-20, 2009-11-19] [TP-Processor138
>>>org.topazproject.ambra.UnhandledException]
>>>> org.topazproject.otm.OtmException: Too many values for 'public void
>>>>org.topazproject.ambra.models.DublinCore.setCreated(java.util.Date)' :
>>>>[2009-10-20, 2009-11-19]
>>>
>>>Mike
>>>
>>>________________________________________
>>>From: Lotz, Michael (NIH/NIEHS) [C]
>>>Sent: Tuesday, April 26, 2011 12:06 PM
>>>To: Mulgara Developers; rcave at plos.org
>>>Subject: RE: [Mulgara-dev] Mulgara process 100% CPU
>>>
>>>http://ehp01.niehs.nih.gov/mulgara/
>>>
>>>I think it's Mulgara 2.0.1 - Rich Cave can verify.
>>>
>>>Mike
>>>
>>>________________________________________
>>>From: Paul Gearon [gearon at ieee.org]
>>>Sent: Tuesday, April 26, 2011 11:18 AM
>>>To: Mulgara Developers
>>>Subject: Re: [Mulgara-dev] Mulgara process 100% CPU
>>>
>>>On Tue, Apr 26, 2011 at 10:32 AM, Lotz, Michael (NIH/NIEHS) [C]
>>><LotzM at niehs.nih.gov> wrote:
>>>> Here is the error:
>>>>
>>>> 2011-04-26 10:30:23,848 INFO  jotm> stop JOTM [Standard shutdown hook
>>>>org.objectweb.jotm]
>>>> 2011-04-26 10:30:23,865 WARN  TuplesOperations> RuntimeException
>>>>thrown
>>>>in join [RMI TCP Connection(52)-157.98.93.161
>>>>org.mulgara.store.tuples.TuplesOperations]
>>>> java.lang.NullPointerException
>>>>        at
>>>>org.mulgara.store.xa.MappedBlockFile.readBlock(MappedBlockFile.java:268
>>>>)
>>>>        at
>>>>org.mulgara.store.xa.ManagedBlockFile$Phase.readBlock(ManagedBlockFile.
>>>>ja
>>>>va:388)
>>>>        at org.mulgara.store.xa.AVLNode.init(AVLNode.java:1144)
>>>>        at org.mulgara.store.xa.AVLNode.<init>(AVLNode.java:142)
>>>>        at org.mulgara.store.xa.AVLNode.newInstance(AVLNode.java:180)
>>>>        at
>>>>org.mulgara.store.xa.AVLFile$Phase.getRootNode(AVLFile.java:304)
>>>>        at
>>>>org.mulgara.store.statement.xa.TripleAVLFile$Phase$AbstractStoreTuples.
>>>><i
>>>>nit>(TripleAVLFile.java:1800)
>>>>        at
>>>>org.mulgara.store.statement.xa.TripleAVLFile$Phase$TuplesImpl.<init>(Tr
>>>>ip
>>>>leAVLFile.java:2269)
>>>>        at
>>>>org.mulgara.store.statement.xa.TripleAVLFile$Phase.allTuples(TripleAVLF
>>>>il
>>>>e.java:1226)
>>>>        at
>>>>org.mulgara.store.statement.xa.XAStatementStoreImpl$Phase.findTuples(XA
>>>>St
>>>>atementStoreImpl.java:1853)
>>>>        at
>>>>org.mulgara.store.statement.xa.XAStatementStoreImpl$ReadOnlyGraph.findT
>>>>up
>>>>les(XAStatementStoreImpl.java:1342)
>>>
>>>This has been solid code for a long time (~10 years), so it's concerning
>>>for me.
>>>
>>>I can try running the queries in the log against the data and see what
>>>is failing. It looks like it's somehow asking a query against a closed
>>>phase, but I can't really be certain, since the current state isn't
>>>visible. (I mostly do functional programming now, and this kind of
>>>thing is exactly the reason why functional is so much nicer!)
>>>
>>>I may need to duplicate the state to follow what's happening. Is there
>>>any way to look at that backup file please? What size is it? Also,
>>>which version of Mulgara is Ambra using?
>>>
>>>Regards,
>>>Paul
>>>_______________________________________________
>>>Mulgara-dev mailing list
>>>Mulgara-dev at mulgara.org
>>>http://lists.mulgara.org/mailman/listinfo/mulgara-dev
>>
>> _______________________________________________
>> Mulgara-dev mailing list
>> Mulgara-dev at mulgara.org
>> http://lists.mulgara.org/mailman/listinfo/mulgara-dev
>>
>_______________________________________________
>Mulgara-dev mailing list
>Mulgara-dev at mulgara.org
>http://lists.mulgara.org/mailman/listinfo/mulgara-dev


From gearon at ieee.org  Wed Nov  2 13:49:19 2011
From: gearon at ieee.org (Paul Gearon)
Date: Wed, 2 Nov 2011 09:49:19 -0400
Subject: [Mulgara-dev] Mulgara high CPU
In-Reply-To: <CAD6BB9B.CB09%lotzm@niehs.nih.gov>
References: <BANLkTikGL-T2rG4DwG-SQuav13vs9AJNUw@mail.gmail.com>
 <CAD6BB9B.CB09%lotzm@niehs.nih.gov>
Message-ID: <CAGZNPFn4R3N1YLQcTEcMvD21A5Ph=GcSpmHS0GARqo_DWARvKw@mail.gmail.com>

Hi Mike,

I have limited time today, but I'll see what I can do.

Does the data load up properly, or is that where the error is?

Paul

On Wed, Nov 2, 2011 at 9:23 AM, Lotz, Michael (NIH/NIEHS) [C]
<LotzM at niehs.nih.gov> wrote:
> Paul -
>
> This issue has reared in ugly head again - but I don't see any strange log
> errors.
>
> Can you look at the backup and see if you can tell what's going on -
> <http://ehp01.niehs.nih.gov/mulgara/backup_1101am.gz>
>
> I'm sorry to bug - you but a backup/restore and moving mulgara to another
> server did not solve the problem.
>
> Thanks.
> Mike
>
>
> ---------------------------------------------------------------------------
> -
> Mike Lotz
> Technology Manager
> EHP/NIEHS (Brogan & Partners ? Contractor)
> Environmental Health Perspectives / NIEHS
> 919-541-7661
> www.ehponline.org
> ---------------------------------------------------------------------------
> -
>
>
>
>
>
> On 4/27/11 12:08 PM, "Paul Gearon" <gearon at ieee.org> wrote:
>
>>Hi Michael,
>>
>>Thanks, though if a backup/restore hasn't worked, then it must be
>>something else. I'll get to this when I can, but it probably won't be
>>today.
>>
>>Paul
>>
>>On Wed, Apr 27, 2011 at 11:54 AM, Lotz, Michael (NIH/NIEHS) [C]
>><LotzM at niehs.nih.gov> wrote:
>>> http://ehp01.niehs.nih.gov/mulgara/topazproject/
>>>
>>> Here is the actual mulgara data folders.
>>>
>>>
>>>
>>>-------------------------------------------------------------------------
>>>--
>>> -
>>> Mike Lotz
>>> Technology Manager
>>> EHP/NIEHS (Brogan & Partners) (Contractor)
>>> Environmental Health Perspectives
>>> National Institute of Environmental Health Sciences
>>> [voc] 919-541-7661 ? ? ? ? ? ? ? ?[fax] 919-541-0273
>>>
>>>-------------------------------------------------------------------------
>>>--
>>> -
>>>
>>>
>>>
>>>
>>>
>>> On 4/26/11 5:12 PM, "Lotz, Michael (NIH/NIEHS) [C]"
>>><LotzM at niehs.nih.gov>
>>> wrote:
>>>
>>>>Paul -
>>>>
>>>>Here is the error from Ambra.log - might help:
>>>>
>>>>2011-04-26 11:00:07,496 ERROR UnhandledException(AmbraJournal)> ?Too
>>>>many
>>>>values for 'public void
>>>>org.topazproject.ambra.models.DublinCore.setCreated(java.util.Date)' :
>>>>[2009-10-20, 2009-11-19] [TP-Processor138
>>>>org.topazproject.ambra.UnhandledException]
>>>>> org.topazproject.otm.OtmException: Too many values for 'public void
>>>>>org.topazproject.ambra.models.DublinCore.setCreated(java.util.Date)' :
>>>>>[2009-10-20, 2009-11-19]
>>>>
>>>>Mike
>>>>
>>>>________________________________________
>>>>From: Lotz, Michael (NIH/NIEHS) [C]
>>>>Sent: Tuesday, April 26, 2011 12:06 PM
>>>>To: Mulgara Developers; rcave at plos.org
>>>>Subject: RE: [Mulgara-dev] Mulgara process 100% CPU
>>>>
>>>>http://ehp01.niehs.nih.gov/mulgara/
>>>>
>>>>I think it's Mulgara 2.0.1 - Rich Cave can verify.
>>>>
>>>>Mike
>>>>
>>>>________________________________________
>>>>From: Paul Gearon [gearon at ieee.org]
>>>>Sent: Tuesday, April 26, 2011 11:18 AM
>>>>To: Mulgara Developers
>>>>Subject: Re: [Mulgara-dev] Mulgara process 100% CPU
>>>>
>>>>On Tue, Apr 26, 2011 at 10:32 AM, Lotz, Michael (NIH/NIEHS) [C]
>>>><LotzM at niehs.nih.gov> wrote:
>>>>> Here is the error:
>>>>>
>>>>> 2011-04-26 10:30:23,848 INFO ?jotm> stop JOTM [Standard shutdown hook
>>>>>org.objectweb.jotm]
>>>>> 2011-04-26 10:30:23,865 WARN ?TuplesOperations> RuntimeException
>>>>>thrown
>>>>>in join [RMI TCP Connection(52)-157.98.93.161
>>>>>org.mulgara.store.tuples.TuplesOperations]
>>>>> java.lang.NullPointerException
>>>>> ? ? ? ?at
>>>>>org.mulgara.store.xa.MappedBlockFile.readBlock(MappedBlockFile.java:268
>>>>>)
>>>>> ? ? ? ?at
>>>>>org.mulgara.store.xa.ManagedBlockFile$Phase.readBlock(ManagedBlockFile.
>>>>>ja
>>>>>va:388)
>>>>> ? ? ? ?at org.mulgara.store.xa.AVLNode.init(AVLNode.java:1144)
>>>>> ? ? ? ?at org.mulgara.store.xa.AVLNode.<init>(AVLNode.java:142)
>>>>> ? ? ? ?at org.mulgara.store.xa.AVLNode.newInstance(AVLNode.java:180)
>>>>> ? ? ? ?at
>>>>>org.mulgara.store.xa.AVLFile$Phase.getRootNode(AVLFile.java:304)
>>>>> ? ? ? ?at
>>>>>org.mulgara.store.statement.xa.TripleAVLFile$Phase$AbstractStoreTuples.
>>>>><i
>>>>>nit>(TripleAVLFile.java:1800)
>>>>> ? ? ? ?at
>>>>>org.mulgara.store.statement.xa.TripleAVLFile$Phase$TuplesImpl.<init>(Tr
>>>>>ip
>>>>>leAVLFile.java:2269)
>>>>> ? ? ? ?at
>>>>>org.mulgara.store.statement.xa.TripleAVLFile$Phase.allTuples(TripleAVLF
>>>>>il
>>>>>e.java:1226)
>>>>> ? ? ? ?at
>>>>>org.mulgara.store.statement.xa.XAStatementStoreImpl$Phase.findTuples(XA
>>>>>St
>>>>>atementStoreImpl.java:1853)
>>>>> ? ? ? ?at
>>>>>org.mulgara.store.statement.xa.XAStatementStoreImpl$ReadOnlyGraph.findT
>>>>>up
>>>>>les(XAStatementStoreImpl.java:1342)
>>>>
>>>>This has been solid code for a long time (~10 years), so it's concerning
>>>>for me.
>>>>
>>>>I can try running the queries in the log against the data and see what
>>>>is failing. It looks like it's somehow asking a query against a closed
>>>>phase, but I can't really be certain, since the current state isn't
>>>>visible. (I mostly do functional programming now, and this kind of
>>>>thing is exactly the reason why functional is so much nicer!)
>>>>
>>>>I may need to duplicate the state to follow what's happening. Is there
>>>>any way to look at that backup file please? What size is it? Also,
>>>>which version of Mulgara is Ambra using?
>>>>
>>>>Regards,
>>>>Paul
>>>>_______________________________________________
>>>>Mulgara-dev mailing list
>>>>Mulgara-dev at mulgara.org
>>>>http://lists.mulgara.org/mailman/listinfo/mulgara-dev
>>>
>>> _______________________________________________
>>> Mulgara-dev mailing list
>>> Mulgara-dev at mulgara.org
>>> http://lists.mulgara.org/mailman/listinfo/mulgara-dev
>>>
>>_______________________________________________
>>Mulgara-dev mailing list
>>Mulgara-dev at mulgara.org
>>http://lists.mulgara.org/mailman/listinfo/mulgara-dev
>
>

From LotzM at niehs.nih.gov  Wed Nov  2 13:55:08 2011
From: LotzM at niehs.nih.gov (Lotz, Michael (NIH/NIEHS) [C])
Date: Wed, 2 Nov 2011 09:55:08 -0400
Subject: [Mulgara-dev] Mulgara high CPU
In-Reply-To: <CAGZNPFn4R3N1YLQcTEcMvD21A5Ph=GcSpmHS0GARqo_DWARvKw@mail.gmail.com>
Message-ID: <CAD6C3EA.CB1D%lotzm@niehs.nih.gov>

It loads but very slowly.

Mike

---------------------------------------------------------------------------
-
Mike Lotz 
Technology Manager
EHP/NIEHS (Brogan & Partners ? Contractor)
Environmental Health Perspectives / NIEHS
919-541-7661
www.ehponline.org
---------------------------------------------------------------------------
-





On 11/2/11 9:49 AM, "Paul Gearon" <gearon at ieee.org> wrote:

>Hi Mike,
>
>I have limited time today, but I'll see what I can do.
>
>Does the data load up properly, or is that where the error is?
>
>Paul
>
>On Wed, Nov 2, 2011 at 9:23 AM, Lotz, Michael (NIH/NIEHS) [C]
><LotzM at niehs.nih.gov> wrote:
>> Paul -
>>
>> This issue has reared in ugly head again - but I don't see any strange
>>log
>> errors.
>>
>> Can you look at the backup and see if you can tell what's going on -
>> <http://ehp01.niehs.nih.gov/mulgara/backup_1101am.gz>
>>
>> I'm sorry to bug - you but a backup/restore and moving mulgara to
>>another
>> server did not solve the problem.
>>
>> Thanks.
>> Mike
>>
>>
>> 
>>-------------------------------------------------------------------------
>>--
>> -
>> Mike Lotz
>> Technology Manager
>> EHP/NIEHS (Brogan & Partners ? Contractor)
>> Environmental Health Perspectives / NIEHS
>> 919-541-7661
>> www.ehponline.org
>> 
>>-------------------------------------------------------------------------
>>--
>> -
>>
>>
>>
>>
>>
>> On 4/27/11 12:08 PM, "Paul Gearon" <gearon at ieee.org> wrote:
>>
>>>Hi Michael,
>>>
>>>Thanks, though if a backup/restore hasn't worked, then it must be
>>>something else. I'll get to this when I can, but it probably won't be
>>>today.
>>>
>>>Paul
>>>
>>>On Wed, Apr 27, 2011 at 11:54 AM, Lotz, Michael (NIH/NIEHS) [C]
>>><LotzM at niehs.nih.gov> wrote:
>>>> http://ehp01.niehs.nih.gov/mulgara/topazproject/
>>>>
>>>> Here is the actual mulgara data folders.
>>>>
>>>>
>>>>
>>>>-----------------------------------------------------------------------
>>>>--
>>>>--
>>>> -
>>>> Mike Lotz
>>>> Technology Manager
>>>> EHP/NIEHS (Brogan & Partners) (Contractor)
>>>> Environmental Health Perspectives
>>>> National Institute of Environmental Health Sciences
>>>> [voc] 919-541-7661                [fax] 919-541-0273
>>>>
>>>>-----------------------------------------------------------------------
>>>>--
>>>>--
>>>> -
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On 4/26/11 5:12 PM, "Lotz, Michael (NIH/NIEHS) [C]"
>>>><LotzM at niehs.nih.gov>
>>>> wrote:
>>>>
>>>>>Paul -
>>>>>
>>>>>Here is the error from Ambra.log - might help:
>>>>>
>>>>>2011-04-26 11:00:07,496 ERROR UnhandledException(AmbraJournal)>  Too
>>>>>many
>>>>>values for 'public void
>>>>>org.topazproject.ambra.models.DublinCore.setCreated(java.util.Date)' :
>>>>>[2009-10-20, 2009-11-19] [TP-Processor138
>>>>>org.topazproject.ambra.UnhandledException]
>>>>>> org.topazproject.otm.OtmException: Too many values for 'public void
>>>>>>org.topazproject.ambra.models.DublinCore.setCreated(java.util.Date)'
>>>>>>:
>>>>>>[2009-10-20, 2009-11-19]
>>>>>
>>>>>Mike
>>>>>
>>>>>________________________________________
>>>>>From: Lotz, Michael (NIH/NIEHS) [C]
>>>>>Sent: Tuesday, April 26, 2011 12:06 PM
>>>>>To: Mulgara Developers; rcave at plos.org
>>>>>Subject: RE: [Mulgara-dev] Mulgara process 100% CPU
>>>>>
>>>>>http://ehp01.niehs.nih.gov/mulgara/
>>>>>
>>>>>I think it's Mulgara 2.0.1 - Rich Cave can verify.
>>>>>
>>>>>Mike
>>>>>
>>>>>________________________________________
>>>>>From: Paul Gearon [gearon at ieee.org]
>>>>>Sent: Tuesday, April 26, 2011 11:18 AM
>>>>>To: Mulgara Developers
>>>>>Subject: Re: [Mulgara-dev] Mulgara process 100% CPU
>>>>>
>>>>>On Tue, Apr 26, 2011 at 10:32 AM, Lotz, Michael (NIH/NIEHS) [C]
>>>>><LotzM at niehs.nih.gov> wrote:
>>>>>> Here is the error:
>>>>>>
>>>>>> 2011-04-26 10:30:23,848 INFO  jotm> stop JOTM [Standard shutdown
>>>>>>hook
>>>>>>org.objectweb.jotm]
>>>>>> 2011-04-26 10:30:23,865 WARN  TuplesOperations> RuntimeException
>>>>>>thrown
>>>>>>in join [RMI TCP Connection(52)-157.98.93.161
>>>>>>org.mulgara.store.tuples.TuplesOperations]
>>>>>> java.lang.NullPointerException
>>>>>>        at
>>>>>>org.mulgara.store.xa.MappedBlockFile.readBlock(MappedBlockFile.java:2
>>>>>>68
>>>>>>)
>>>>>>        at
>>>>>>org.mulgara.store.xa.ManagedBlockFile$Phase.readBlock(ManagedBlockFil
>>>>>>e.
>>>>>>ja
>>>>>>va:388)
>>>>>>        at org.mulgara.store.xa.AVLNode.init(AVLNode.java:1144)
>>>>>>        at org.mulgara.store.xa.AVLNode.<init>(AVLNode.java:142)
>>>>>>        at org.mulgara.store.xa.AVLNode.newInstance(AVLNode.java:180)
>>>>>>        at
>>>>>>org.mulgara.store.xa.AVLFile$Phase.getRootNode(AVLFile.java:304)
>>>>>>        at
>>>>>>org.mulgara.store.statement.xa.TripleAVLFile$Phase$AbstractStoreTuple
>>>>>>s.
>>>>>><i
>>>>>>nit>(TripleAVLFile.java:1800)
>>>>>>        at
>>>>>>org.mulgara.store.statement.xa.TripleAVLFile$Phase$TuplesImpl.<init>(
>>>>>>Tr
>>>>>>ip
>>>>>>leAVLFile.java:2269)
>>>>>>        at
>>>>>>org.mulgara.store.statement.xa.TripleAVLFile$Phase.allTuples(TripleAV
>>>>>>LF
>>>>>>il
>>>>>>e.java:1226)
>>>>>>        at
>>>>>>org.mulgara.store.statement.xa.XAStatementStoreImpl$Phase.findTuples(
>>>>>>XA
>>>>>>St
>>>>>>atementStoreImpl.java:1853)
>>>>>>        at
>>>>>>org.mulgara.store.statement.xa.XAStatementStoreImpl$ReadOnlyGraph.fin
>>>>>>dT
>>>>>>up
>>>>>>les(XAStatementStoreImpl.java:1342)
>>>>>
>>>>>This has been solid code for a long time (~10 years), so it's
>>>>>concerning
>>>>>for me.
>>>>>
>>>>>I can try running the queries in the log against the data and see what
>>>>>is failing. It looks like it's somehow asking a query against a closed
>>>>>phase, but I can't really be certain, since the current state isn't
>>>>>visible. (I mostly do functional programming now, and this kind of
>>>>>thing is exactly the reason why functional is so much nicer!)
>>>>>
>>>>>I may need to duplicate the state to follow what's happening. Is there
>>>>>any way to look at that backup file please? What size is it? Also,
>>>>>which version of Mulgara is Ambra using?
>>>>>
>>>>>Regards,
>>>>>Paul
>>>>>_______________________________________________
>>>>>Mulgara-dev mailing list
>>>>>Mulgara-dev at mulgara.org
>>>>>http://lists.mulgara.org/mailman/listinfo/mulgara-dev
>>>>
>>>> _______________________________________________
>>>> Mulgara-dev mailing list
>>>> Mulgara-dev at mulgara.org
>>>> http://lists.mulgara.org/mailman/listinfo/mulgara-dev
>>>>
>>>_______________________________________________
>>>Mulgara-dev mailing list
>>>Mulgara-dev at mulgara.org
>>>http://lists.mulgara.org/mailman/listinfo/mulgara-dev
>>
>>


From LotzM at niehs.nih.gov  Wed Nov  2 18:40:48 2011
From: LotzM at niehs.nih.gov (Lotz, Michael (NIH/NIEHS) [C])
Date: Wed, 2 Nov 2011 14:40:48 -0400
Subject: [Mulgara-dev] Mulgara high CPU
In-Reply-To: <CAGZNPFn4R3N1YLQcTEcMvD21A5Ph=GcSpmHS0GARqo_DWARvKw@mail.gmail.com>
Message-ID: <CAD706D8.CBA6%lotzm@niehs.nih.gov>

The logs show:

Wed Nov  2 14:27:22 EDT 2011 Stopping ambra (14061)
WARN: The method class
org.apache.commons.logging.impl.SLF4JLogFactory#release() was invoked.
WARN: Please see http://www.slf4j.org/codes.html#release for an
explanation.
WARN: The method class
org.apache.commons.logging.impl.SLF4JLogFactory#release() was invoked.
WARN: Please see http://www.slf4j.org/codes.html#release for an
explanation.
WARN: The method class
org.apache.commons.logging.impl.SLF4JLogFactory#release() was invoked.
WARN: Please see http://www.slf4j.org/codes.html#release for an
explanation.
WARN: The method class
org.apache.commons.logging.impl.SLF4JLogFactory#release() was invoked.
WARN: Please see http://www.slf4j.org/codes.html#release for an
explanation.
Error closing connection
'org.mulgara.connection.SessionConnection at c186073':
java.lang.IllegalStateException: Attempt to access a closed connection
Error closing connection
'org.mulgara.connection.SessionConnection at 14642439':
java.lang.IllegalStateException: Attempt to access a closed connection
Error closing connection
'org.mulgara.connection.SessionConnection at 3a8624a4':
java.lang.IllegalStateException: Attempt to access a closed connection
Error closing connection
'org.mulgara.connection.SessionConnection at 3ccfa900':
java.lang.IllegalStateException: Attempt to access a closed connection
Error closing connection
'org.mulgara.connection.SessionConnection at 65605e30':
java.lang.IllegalStateException: Attempt to access a closed connection
Error closing connection
'org.mulgara.connection.SessionConnection at e8542c2':
java.lang.IllegalStateException: Attempt to access a closed connection
Error closing connection
'org.mulgara.connection.SessionConnection at 4ee1d600':
java.lang.IllegalStateException: Attempt to access a closed connection
Error closing connection
'org.mulgara.connection.SessionConnection at 4ca8c275':
java.lang.IllegalStateException: Attempt to access a closed connection
Error closing connection
'org.mulgara.connection.SessionConnection at 5b76b6a4':
java.lang.IllegalStateException: Attempt to access a closed connection
Error closing connection
'org.mulgara.connection.SessionConnection at 4ed0f7e5':
java.lang.IllegalStateException: Attempt to access a closed connection
Error closing connection
'org.mulgara.connection.SessionConnection at 794303cf':
java.lang.IllegalStateException: Attempt to access a closed connection
Error closing connection
'org.mulgara.connection.SessionConnection at 1eaf493b':
java.lang.IllegalStateException: Attempt to access a closed connection
Error closing connection
'org.mulgara.connection.SessionConnection at 5f0d03a':
java.lang.IllegalStateException: Attempt to access a closed connection
Error closing connection
'org.mulgara.connection.SessionConnection at 4f692e2':
java.lang.IllegalStateException: Attempt to access a closed connection
Error closing connection
'org.mulgara.connection.SessionConnection at 582535f9':
java.lang.IllegalStateException: Attempt to access a closed connection
Error closing connection
'org.mulgara.connection.SessionConnection at 72fa9ba4':
org.mulgara.query.QueryException: Java RMI failure
Error closing connection
'org.mulgara.connection.SessionConnection at 230066f5':
org.mulgara.query.QueryException: Java RMI failure
Wed Nov  2 14:28:35 EDT 2011 Starting ambra in /usr/local/topaz/ambra via
java







On 11/2/11 9:49 AM, "Paul Gearon" <gearon at ieee.org> wrote:

>Hi Mike,
>
>I have limited time today, but I'll see what I can do.
>
>Does the data load up properly, or is that where the error is?
>
>Paul
>
>On Wed, Nov 2, 2011 at 9:23 AM, Lotz, Michael (NIH/NIEHS) [C]
><LotzM at niehs.nih.gov> wrote:
>> Paul -
>>
>> This issue has reared in ugly head again - but I don't see any strange
>>log
>> errors.
>>
>> Can you look at the backup and see if you can tell what's going on -
>> <http://ehp01.niehs.nih.gov/mulgara/backup_1101am.gz>
>>
>> I'm sorry to bug - you but a backup/restore and moving mulgara to
>>another
>> server did not solve the problem.
>>
>> Thanks.
>> Mike
>>
>>
>> 
>>-------------------------------------------------------------------------
>>--
>> -
>> Mike Lotz
>> Technology Manager
>> EHP/NIEHS (Brogan & Partners ? Contractor)
>> Environmental Health Perspectives / NIEHS
>> 919-541-7661
>> www.ehponline.org
>> 
>>-------------------------------------------------------------------------
>>--
>> -
>>
>>
>>
>>
>>
>> On 4/27/11 12:08 PM, "Paul Gearon" <gearon at ieee.org> wrote:
>>
>>>Hi Michael,
>>>
>>>Thanks, though if a backup/restore hasn't worked, then it must be
>>>something else. I'll get to this when I can, but it probably won't be
>>>today.
>>>
>>>Paul
>>>
>>>On Wed, Apr 27, 2011 at 11:54 AM, Lotz, Michael (NIH/NIEHS) [C]
>>><LotzM at niehs.nih.gov> wrote:
>>>> http://ehp01.niehs.nih.gov/mulgara/topazproject/
>>>>
>>>> Here is the actual mulgara data folders.
>>>>
>>>>
>>>>
>>>>-----------------------------------------------------------------------
>>>>--
>>>>--
>>>> -
>>>> Mike Lotz
>>>> Technology Manager
>>>> EHP/NIEHS (Brogan & Partners) (Contractor)
>>>> Environmental Health Perspectives
>>>> National Institute of Environmental Health Sciences
>>>> [voc] 919-541-7661                [fax] 919-541-0273
>>>>
>>>>-----------------------------------------------------------------------
>>>>--
>>>>--
>>>> -
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On 4/26/11 5:12 PM, "Lotz, Michael (NIH/NIEHS) [C]"
>>>><LotzM at niehs.nih.gov>
>>>> wrote:
>>>>
>>>>>Paul -
>>>>>
>>>>>Here is the error from Ambra.log - might help:
>>>>>
>>>>>2011-04-26 11:00:07,496 ERROR UnhandledException(AmbraJournal)>  Too
>>>>>many
>>>>>values for 'public void
>>>>>org.topazproject.ambra.models.DublinCore.setCreated(java.util.Date)' :
>>>>>[2009-10-20, 2009-11-19] [TP-Processor138
>>>>>org.topazproject.ambra.UnhandledException]
>>>>>> org.topazproject.otm.OtmException: Too many values for 'public void
>>>>>>org.topazproject.ambra.models.DublinCore.setCreated(java.util.Date)'
>>>>>>:
>>>>>>[2009-10-20, 2009-11-19]
>>>>>
>>>>>Mike
>>>>>
>>>>>________________________________________
>>>>>From: Lotz, Michael (NIH/NIEHS) [C]
>>>>>Sent: Tuesday, April 26, 2011 12:06 PM
>>>>>To: Mulgara Developers; rcave at plos.org
>>>>>Subject: RE: [Mulgara-dev] Mulgara process 100% CPU
>>>>>
>>>>>http://ehp01.niehs.nih.gov/mulgara/
>>>>>
>>>>>I think it's Mulgara 2.0.1 - Rich Cave can verify.
>>>>>
>>>>>Mike
>>>>>
>>>>>________________________________________
>>>>>From: Paul Gearon [gearon at ieee.org]
>>>>>Sent: Tuesday, April 26, 2011 11:18 AM
>>>>>To: Mulgara Developers
>>>>>Subject: Re: [Mulgara-dev] Mulgara process 100% CPU
>>>>>
>>>>>On Tue, Apr 26, 2011 at 10:32 AM, Lotz, Michael (NIH/NIEHS) [C]
>>>>><LotzM at niehs.nih.gov> wrote:
>>>>>> Here is the error:
>>>>>>
>>>>>> 2011-04-26 10:30:23,848 INFO  jotm> stop JOTM [Standard shutdown
>>>>>>hook
>>>>>>org.objectweb.jotm]
>>>>>> 2011-04-26 10:30:23,865 WARN  TuplesOperations> RuntimeException
>>>>>>thrown
>>>>>>in join [RMI TCP Connection(52)-157.98.93.161
>>>>>>org.mulgara.store.tuples.TuplesOperations]
>>>>>> java.lang.NullPointerException
>>>>>>        at
>>>>>>org.mulgara.store.xa.MappedBlockFile.readBlock(MappedBlockFile.java:2
>>>>>>68
>>>>>>)
>>>>>>        at
>>>>>>org.mulgara.store.xa.ManagedBlockFile$Phase.readBlock(ManagedBlockFil
>>>>>>e.
>>>>>>ja
>>>>>>va:388)
>>>>>>        at org.mulgara.store.xa.AVLNode.init(AVLNode.java:1144)
>>>>>>        at org.mulgara.store.xa.AVLNode.<init>(AVLNode.java:142)
>>>>>>        at org.mulgara.store.xa.AVLNode.newInstance(AVLNode.java:180)
>>>>>>        at
>>>>>>org.mulgara.store.xa.AVLFile$Phase.getRootNode(AVLFile.java:304)
>>>>>>        at
>>>>>>org.mulgara.store.statement.xa.TripleAVLFile$Phase$AbstractStoreTuple
>>>>>>s.
>>>>>><i
>>>>>>nit>(TripleAVLFile.java:1800)
>>>>>>        at
>>>>>>org.mulgara.store.statement.xa.TripleAVLFile$Phase$TuplesImpl.<init>(
>>>>>>Tr
>>>>>>ip
>>>>>>leAVLFile.java:2269)
>>>>>>        at
>>>>>>org.mulgara.store.statement.xa.TripleAVLFile$Phase.allTuples(TripleAV
>>>>>>LF
>>>>>>il
>>>>>>e.java:1226)
>>>>>>        at
>>>>>>org.mulgara.store.statement.xa.XAStatementStoreImpl$Phase.findTuples(
>>>>>>XA
>>>>>>St
>>>>>>atementStoreImpl.java:1853)
>>>>>>        at
>>>>>>org.mulgara.store.statement.xa.XAStatementStoreImpl$ReadOnlyGraph.fin
>>>>>>dT
>>>>>>up
>>>>>>les(XAStatementStoreImpl.java:1342)
>>>>>
>>>>>This has been solid code for a long time (~10 years), so it's
>>>>>concerning
>>>>>for me.
>>>>>
>>>>>I can try running the queries in the log against the data and see what
>>>>>is failing. It looks like it's somehow asking a query against a closed
>>>>>phase, but I can't really be certain, since the current state isn't
>>>>>visible. (I mostly do functional programming now, and this kind of
>>>>>thing is exactly the reason why functional is so much nicer!)
>>>>>
>>>>>I may need to duplicate the state to follow what's happening. Is there
>>>>>any way to look at that backup file please? What size is it? Also,
>>>>>which version of Mulgara is Ambra using?
>>>>>
>>>>>Regards,
>>>>>Paul
>>>>>_______________________________________________
>>>>>Mulgara-dev mailing list
>>>>>Mulgara-dev at mulgara.org
>>>>>http://lists.mulgara.org/mailman/listinfo/mulgara-dev
>>>>
>>>> _______________________________________________
>>>> Mulgara-dev mailing list
>>>> Mulgara-dev at mulgara.org
>>>> http://lists.mulgara.org/mailman/listinfo/mulgara-dev
>>>>
>>>_______________________________________________
>>>Mulgara-dev mailing list
>>>Mulgara-dev at mulgara.org
>>>http://lists.mulgara.org/mailman/listinfo/mulgara-dev
>>
>>