[Mulgara-dev] Mulgara Apps

Paul Gearon gearon at ieee.org
Fri Dec 3 20:10:12 UTC 2010


On Fri, Dec 3, 2010 at 2:07 PM, Gregg Reynolds <dev at mobileink.com> wrote:
> On Fri, Dec 3, 2010 at 12:20 PM, Paul Gearon <gearon at ieee.org> wrote:
>> I think I need a clearer picture of what you are defining to be an "App".
>>
>>
>> So what's is an app for you, and where do you want to see a new
>> "clean" delineation?
>>
> I hope to get a better picture myself this weekend.  I'm thinking of Mulgara
> as basically a library that developers use as a component in their
> applications, rather than being itself an application.

Some projects definitely approach Mulgara that way. Sort of like
BerkeleyDB. On the other hand, others want to use it like MySQL, in
that the database runs in its own space, and is accessed remotely.
This is much more popular, though the embedded approach is still being
used by several projects. It's because we want to support both that we
have implemented all of the required interfaces through RMI. There's a
couple of layers of wrapping to hide whether or not RMI is in use
(since it's not needed in the embedded scenario).

One argument for putting Mulgara in its own JVM is access to the heap.
Mulgara is a big and complex system and it does a lot of tricks with
memory to try and get the most out of the system. Sharing address
spaces gets in the way. It would nice to say that 64 bit systems avoid
all of this, but Java still has some 32 bit roots, even on a 64 bit
system.

> I know the binary
> distro ships as a server app,

Well, the mulgara-x.y.z.jar file is a server app, but the others
aren't. driver and querylang are specifically for clients.
mulgara-lite is specifically for embedding (I don't think it can be
run as a standalone app, but I haven't checked).

> but it looks to me like the real goal of the
> project is to ship a good infrastructure component for developers, and the
> app is secondary.  Which doesn't necessarily mean anybody on the dev team
> looks at it that way, but it's what it looks like from here.  And it looks
> good that way from here.
> From this perspective mulgara-x.y.z.jar looks a lot like a sample
> application.  That's no judgement as to its quality, just saying it looks an
> assembly of non-mulgara functionality (Jetty, servlet) wrapped around a
> Mulgara "kernel".

That's true, at least when it was first created. Initially, there was
a wrapper that started up the database and provided an RMI service.
Then the Jetty server was introduced to support the Descriptors, and
was also used to serve up the documentation. The web UI came later
(incidentally, that WebUI was completely discarded and replaced). Last
of all came the SPARQL endpoint.

However, now that most developers want to talk to RDF datastores via
SPARQL, the need for an HTTP service is more important. Also, the size
and scope of the system makes it more desirable to keep it as a
service (regardless of the protocol for talking to it). So the
perception of the system has evolved, even if the architecture didn't.

> I guess the language you want to use depends on how you look at the overall
> thrust of the Mulgara project.  I'm using "app", "kernel", "backend", etc.
> mainly because I'm thinking about how to represent the whole/parts in
> documentation in way that is simple and transparent and that harmonizes with
> "standard" language about architectures.  I expect my usage will change as I
> become more familiar with the system.
> "Clean delination" meaning both clearly defined (sounds like it is already)
> and clearly documented.
>
>>
>> > Add apps/mulgara-sparql, which is a plain sparql frontend with embedded
>> > Jetty and memory/file (store?) resolvers
>> > Add apps/mulgara-sparqlet, which is a servlet that can run in any
>> > servlet
>> > container
>>
>> The SPARQL frontend is already a servlet that can run in any servlet
>> container. Well, that's a broad claim. I can vouch that it runs in
>> Jetty and Tomcat.  :-)  It *should* run in anything.
>
> Here I'm thinking of the various ways one can implement such a front-end.
>  There's the standard, container-agnostic servlet way, but then with e.g.
> Jetty one could also implement a plain "handler", which is not a servlet.

Yes, but to do SPARQL that way you'd end up having to duplicate most
of what is already done for you with servlets.

>  Another possibility is to make Mulgara/SPARQL service available to non-Java
> webapps by slapping an FCGI wrapper around a standalone server.  Then
> Mulgara could run directly behind e.g. nginx, without an embedded http
> server (but with some means of launching the indpendent Mulgara process).

This seems like a little much in order to avoid the servlet architecture.

Of course, Mulgara doesn't need an *embedded* HTTP server to do
SPARQL. For instance, it works just fine in Tomcat. In that case, you
can either run a Mulgara server (providing only RMI and no HTTP), or
you can start it as a servlet that doesn't have a frontend
(ServletMulgaraServer) by configuring it in the web.xml in the WAR
file.

It would be possible to implement an HTTP service without going
through servlets, but servlets really seem to be the correct approach
here.

> I guess this is all part of the documentation I have in mind, which I'll try
> to write up this weekend.  There are only so many basic architectural
> patterns one can use to implement e.g. a SPARQL service using Mulgara, with
> this or that resolver; I'd like to provide at least an overview, and maybe
> and example, of each.
>
>>
>> > Ship some demos in a demo dir; these would have more specific
>> > functionality
>> > than apps, e.g. a TODO list app with an embedded RDF triplestore, using
>> > only
>> > the Mulgara API.  Only included in the source distrib.  The purpose is
>> > to
>> > show developers how to do it, rather than provide real apps.
>>
>> That's what I started in the tools/ directory (it was a demo that grew
>> into some useful tools for my testing). However, that's just
>> client-side code. A useful demo would be creating an embedded database
>> (a common request).
>
> Can you be more specific as to what people are looking for?  E.g., somebody
> wants an app the runs a Mulgara DB, but only within the app process space,
> and accessed by a Java API?

That's a good description of what some projects want. For instance
Fedora does that, as does the Sesame-Mulgara bridge.

Right now, instantiating a Database object requires a complex
configuration. That's fine when it's being set up in the standalone
system, but too complex for developers. A simpler constructor that
uses sane defaults would be a welcome addition. Until that time, at
the very least we could document what's needed (a lot of it comes from
XML configuration files).

>  What sorts of applications?  Or maybe a better
> question would be: would do you envision as the ideal embedded DB demo?
>  Might be a good project for me.

Well, just something that I can instantiate, load data into, and query
it (not necessarily with SPARQL - possibly just via an API). For
instance, Firefox uses RDF for some of its internal configuration.

In all honesty, if I need something embedded like that, I wouldn't be
using Mulgara. I'd be using MRG (which is why I created MRG).

Regards,
Paul


More information about the Mulgara-dev mailing list