[Mulgara-dev] Monitoring resource use

[Paul Gearon]

On Fri, Nov 7, 2008 at 8:40 PM, Life is hard, and then you die
<ronald at innovation.ch> wrote:
> On Fri, Nov 07, 2008 at 05:37:49PM -0500, Alex Hall wrote:
>> Does anybody have any insight into the topic of monitoring resource
>> usage of a Mulgara server?  The idea is, once we put up a publicly
>> available, Mulgara-backed website that incorporates the ability to
>> execute SPARQL queries, we expose ourselves to denial-of-service
>> attacks.  The sort of thing that I would be interested in doing is
>> some sort of throttling or rate control; assigning different
>> priorities to queries; and monitoring queries to either kill or
>> lower the priority of ones that are running too long.  One possible
>> approach would be to extend MBean capability to the Session level
>> (right now there is an MBean interface for the server object) but
>> I'm sure there are others.  I'd love to hear back from anybody with
>> experience or thoughts on this topic.

Yes, the current MBean isn't all that informative, and I don't think
it lets you do any control at all (does it?).

> Right we just monitor by looking at os stats such as cpu and memory
> used; more JMX support would be really great, though (there is the
> ServerMBean, but it's not actually exported as an MBean).

What do you mean by this? Standard MBeans don't require anything
except to meet the naming conventions. Are you talking about something
the server needed to do?

> Regarding throttling etc, I'm not sure how you want to achieve that
> within mulgara, unless you start getting really intrusive and add
> things like explicit sleep()'s and checks all over.

Agreed. Though this may be worth considering.

> You can't rely on
> thread priorities in Java, btw, (e.g. they're wholly ignored on Linux)
> and even if they're implemented I don't think they'll do much as soon
> as you're disk IO bound.

Definitely. And because processes get scheduled when they do IO, then
this means a busy Mulgara is more than happy to let other processes
use the CPU. They just won't have any fun with the hard drive.

Paul