[Mulgara-dev] Re: Blobs and MulgaraTransactionManager

Andrae Muys andrae at netymon.com
Tue Apr 3 09:38:19 UTC 2007


On 03/04/2007, at 4:56 PM, Eric Brown wrote:

> Hi Andrae, (Your name is on the MulgaraTransactionManager)
>
> Topaz would like to modify mulgara slightly so that we can write a  
> BLOB management service next to mulgara but within the same JOTM  
> session. As you know, we use SOAP to talk to mulgara. Our plan is  
> to use REST to talk to our BLOB service.  Essentially what we're  
> shooting for is this:
>
> begin
> POST /blobservlet -> identifier
> insert s p identifier (via soap)
> end

> I'm new to JTA and mulgara internals so am feeling around a bit in  
> the dark. But my thought is that I'd like to try and make a call to  
> the ItqlInterpreter between _begin_ and _POST_ and get a handle (a  
> string) to the mulgara transaction (or mulgara session which can  
> then map to the transaction) and send that handle with the POST so  
> that I can get the specific BLOB resource to enlist its XAResource.  
> Do you see a better way or a problem here? Any suggestions on how  
> to do this mapping and expose relevant interfaces?

The problem is that while moving from our own custom transaction  
interfaces to JTA is the first step towards supporting participation  
in externally mediated distributed transactions, it is only the first  
step.  Mulgara does use JTA internally to mediate its transactions,  
however this integration is not currently sufficient to support what  
you are trying to achieve.

The biggest problem is that it was only in the process of getting  
this first step working that Simon Raboczi and I came to understand  
that our initial mapping between JTA and Mulgara concepts was  
flawed.  We currently map XAResources to Resolvers; this works as  
long as JTA remains internal to Mulgara.  For this to work externally  
the mapping will need to be changed to between XAResources and  
ResolverFactorys, (Resolvers map to XIDs).  Once this is done, we  
would then need to decide on an interface to expose the JTA support  
to external containers.

> It looks like Mulgara is acting like both a container (application  
> server) and the application with the resolvers being the resource  
> managers? That's my read anyway. Do you know any other way to hook in?

As JTA is currently constrained to use internally it does indeed act  
like both a container and an application.  Yes resolvers are acting  
as resource managers, but as mentioned above this is in error -  
resolver-factories should be in that role instead.  Ultimately until  
the development resources are found to correct this misunderstanding,  
there really is _no_ way to hook in except as a ResolverFactory/ 
Resolver.

> BTW, our fall-back idea is to write a resolver that also has a  
> servlet service() method and would maintain some resource-id to  
> XAResource mapping such that we'd have the following (simplified)  
> semantics:
>
> begin
> select $urlWithEmbededResourceId from <blobModel>
>   where s <blob:isMutable> $urlWithEmbededResourceId;
> POST urlWithEmbededResourceId
> end
>
> The returned url of course would be specific to the current  
> transaction and invalid outside the transaction. The url could be a  
> literal or a uri, doesn't matter as far as I can tell.

I don't think you gain much here - but I may be mistaken.  For this  
to work you will already have to use setAutoCommit() to demarcate the  
transaction manually.  If your blob service is capable of  
participating in a distributed transaction, it already supports a 3- 
phase commit protocol.  In that case you can treat mulgara as a dumb  
2-phase participant - distributed transaction protocols support a  
single 2-phase participant.

The appropriate incantation would be:

blobserv.begin
session.setAutoCommit(false)

blobserv.POST BLOB -> URI
session.insert s p URI

blobserv.prepare
try:
   session.setAutoCommit(true)
fail:
   blobserv.rollback
succeed:
   blobserv.commit


> I believe this can be done without any modifications to mulgara.  
> The advantages of this approach are that we can add additional  
> predicates to our resolver to get meta-data about the blobs. The  
> disadvantages though are this ugly transient url and the fact that  
> mulgara now needs to manage more things - knowledge of blobs and  
> this resource-id to XA mapping. It is hard to say how important  
> querying meta-data inside mulgara is (instead of through the BLOB  
> REST service). Of course we can still write a custom resolver later  
> just to get the meta-data.

My current thinking in this is to implement BLOBs via a typed-literal  
containing a URL, the type being defined as the invariant content  
obtained by dereferencing the URL at some specific point in time ie.  
now().  It is perfectly acceptable for a resolver to dereference a  
URL or other pointer to a BLOB in some way - we already have  
resolvers that do that.  The reason I keep thinking of typed-literals  
is that this would eventually permit the StringPool to do the  
dereferencing instead of a resolver, meaning that BLOBs would just be  
another data-type instead of objects in a distinguished model.

To fit with the current mulgara-store, a distinguished model is  
likely to be easier to work with.  It also allows you to use an  
XAResource to let you notify the blobserv when to execute any deletes  
performed on the URL between query and fetch.

So in summary we have two possible solutions:

1. Fix the JTA/Mulgara implementation so Mulgara can participate in  
distributed JTA transactions.
2. Use a blob-model, which works very similarly to the lucene  
resolver in treating all _object_'s as either BLOBs or references to  
BLOBs that should be dereferenced and stored.

There is of course also the 'default' option

3. Wait for XA2, and implement BLOBs as first-class datatypes in the  
string-pool.

Andrae

-- 
Andrae Muys
andrae at netymon.com
Principal Mulgara Consultant
Netymon Pty Ltd





More information about the Mulgara-dev mailing list