[Mulgara-dev] Lost phase token

[Alex Hall]

Paul Gearon wrote:
> On Wed, Sep 23, 2009 at 1:15 PM, James Leigh <james-nospam at leighnet.ca> wrote:
>> On Wed, 2009-09-23 at 12:29 -0400, Paul Gearon wrote:
>>> On Wed, Sep 23, 2009 at 12:22 PM, Paul Gearon <gearon at ieee.org> wrote:
>> My understanding is the the lost phase token happens before the
>> OutOfMemoryException, which is before the rollback. I am using the
>> mulgara-core.jar and am happy to try this out with a snapshot version.
> 
> Because of various exception frames, it's possible that these things
> are being logged out of order.
> 
> I can guarantee that if you run out of heap while in a write
> transaction then you'll definitely get a lost phase token. I should
> have remembered this earlier, but I've seen this often enough myself.
> 
> We can run out of heap, particularly when using Lucene indexes. The
> only solution I have for the general case is to simply run the JVM
> with -Xmx512m. (Or more, but the more heap you reserve the slower it
> gets)

I'm thinking that orphaned phases are at the root of the underlying
out-of-memory problem. The reason I say that is the heap profile in the
original email is dominated by 47340 live references to character arrays
allocated in the StackTrace constructor, which is generally kept around
for debugging transaction-related issues. However, those are created all
over the place so the stack trace in the lost phase token message might
not be the problem one.

James, can you look in the original java.hprof.txt file and get the
actual stack traces identified in the heap snapshot (305231 and 305221)?
Those will give a clue as to what resources are being created but not
properly released.

-Alex