[Mulgara-general] Memory error during rdf load

Bill OConnor wtoconnor at gmail.com
Thu Aug 14 18:35:52 UTC 2008


Hello,

Thanks for getting back to me.

There is no big hurry. DMOZ only has an outdated RDF formatted data set so
N3 is not an easy option.

Maybe I'm showing my ignorance but as far as 64 bit is concerned it isn't
clear to me how that would help.
It would expand the addressable memory space and increase the data bandwidth
but I still only have swap
and real memory available. These will not change and are addressable by  32
bits. Longs in Java are 64 bit
regardless of JVM.

How does 64 bit help?

It seems more likely that there is a data structure being built in Mulgara
that is proportional to the size
of the file being loaded. (?)

Would it help to break the file down into more manageable chunks? (say 500
meg?)
What are the limitations are far as the number of triples are concerned?
(100k, millions etc)

Thanks for the help,
-bill



On Thu, Aug 14, 2008 at 10:10 AM, Paul Gearon <gearon at ieee.org> wrote:

> Hi Bill,
>
> Well, the obvious answer is to go to 64 bit, but if you're really
> against that then you can try a couple of things.
>
> First off, does DMOZ have an N3 format? That has different memory
> requirements while loading, so it may do better.
>
> Another thing to try would be to set the heap size for Mulgara. That's
> done with the -Xms (initial size) and -Xmx (maximum size) options on
> the Java command line.
>
> Finally, I'm working on some tweaks to indexing that will make loading
> a little more efficient. However, I won't be done with this for a few
> days yet, and it won't be in a formal release for nearly 2 weeks. But
> if you can hang on, then this should help 32 bit installations.
>
> Regards,
> Paul Gearon
>
>
> On Wed, Aug 13, 2008 at 12:59 PM, Bill OConnor <wtoconnor at gmail.com>
> wrote:
> > Hello,
> >
> > Any more ideas about how I might fix some of these problems?
> >
> > On Mon, Aug 11, 2008 at 11:00 AM, Bill OConnor <wtoconnor at gmail.com>
> wrote:
> >>
> >> Hello,
> >>
> >> Thanks for responding.
> >>
> >> I am running a 32 bit version of Ubuntu on a AMD 64 bit processor. I
> would
> >> assume the JVM
> >> that is running is 32 bit.  The 64 bit operating systems have given me
> >> compatibility problems
> >> in the past so I generally try to avoid it. The system has 1 GB physical
> >> and 3 GB swap.
> _______________________________________________
> Mulgara-general mailing list
> Mulgara-general at mulgara.org
> http://mulgara.org/mailman/listinfo/mulgara-general
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mulgara.org/pipermail/mulgara-general/attachments/20080814/7456f582/attachment.htm>


More information about the Mulgara-general mailing list