[Mulgara-general] Mulgara size limits

Life is hard, and then you die ronald at innovation.ch
Fri Jul 11 00:15:40 UTC 2008


On Wed, Jul 09, 2008 at 01:58:58PM -0700, Chuck Borromeo wrote:
> I pulled the latest version of Mulgara from svn and built it.  I ran
> it using Java 1.6 (as per Ronald's suggestion).  Now the files load.
> It seems to take a long time to load these files.  It takes about 40
> minutes to load a 97 MB RDF/XML file.  I'll try to put together a
> more formal measurement of the loading times.

Interesting! Which version exactly were you running before? It sounds
like the only major change was Paul's object-pool removal - if that's
the case, this would be a great confirmation of that work.


  Cheers,

  Ronald


> --- On Tue, 7/8/08, Alex Hall <alexhall at revelytix.com> wrote:
> 
> > From: Alex Hall <alexhall at revelytix.com>
> > Subject: Re: [Mulgara-general] Mulgara size limits
> > To: cborromeo3 at yahoo.com, "Mulgara General" <mulgara-general at mulgara.org>
> > Date: Tuesday, July 8, 2008, 4:36 PM
> > Chuck Borromeo wrote:
> > > Hi Everyone,
> > >   I just wanted to check to see if anyone has figured
> > out the issue I was encountering when I tried to load
> > RDF/XML files larger than 98 MB.
> > 
> > I've been looking at this off and on since last week,
> > but with little 
> > results to show.  I can tell the offending class that is
> > filling the 
> > heap, and I can tell where it is being allocated, but I
> > haven't been 
> > able to track down where the references to the class are
> > not being 
> > released.  To add a degree of difficulty, I'm
> > developing on a 32-bit 
> > Windows laptop so I can't duplicate the conditions
> > causing the 
> > out-of-memory error.
> > 
> > However, I do have a couple of suggestions for you to try:
> > 
> > 1.  Paul recently committed some changes that affect how
> > objects are 
> > allocated.  These were meant to simplify our code and make
> > it easier to 
> > track object allocation, but there's an outside chance
> > they might 
> > address your memory problem.  Try building and running the
> > latest 
> > revision from SVN (directions at the Mulgara Download page)
> > and see if 
> > you still get out-of-memory errors.
> > 
> > 2.  I would also try running Mulgara with assertions
> > enabled.  Do this 
> > by adding the "-ea" flag to the Java VM
> > arguments.  Scan the log file or 
> > console for AssertionError stack traces, and let us know if
> > you find 
> > anything.
> > 
> > In both cases, I would also gather the heap profile, just
> > to be sure 
> > that the AVLNode class is still the culprit.
> > 
> > Beyond these two suggestions, I can't tell how the
> > AVLNode references 
> > are not being released.  The new Mulgara XA2 storage layer,
> > planned for 
> > next year, will eliminate the use of the AVLNode class
> > altogether, so 
> > this should not be an issue under the next-generation
> > architecture. 
> > Given my limited time and resources, I just can't
> > justify putting much 
> > more effort into trying to track this down when I'm
> > unable to reproduce 
> > the problem myself.  If either of the previous suggestions
> > produces new 
> > results, I'll be happy to take a look at them, but
> > absent any new 
> > developments I'm afraid there's not much more I can
> > do at the moment.
> > 
> > Regards,
> > Alex
> 



More information about the Mulgara-general mailing list