[Mulgara-dev] iitql shell without X?
Life is hard, and then you die
ronald at innovation.ch
Wed Mar 19 08:25:36 UTC 2008
On Mon, Mar 17, 2008 at 06:25:45PM -0700, William Mills wrote:
> I'm working from home and don't want to run X back to here. Is
> there an itql shell that's really just a shell?
We have a simple shell that we'd be happy to donate. It's written in
groovy, and it uses the jline (http://jline.sourceforge.net/) library
to get basic editing and history, and seems to work on all major
platforms. It shouldn't be hard to turn into straight java, though.
I'm attaching the two groovy files for this, RunItql.groovy and
Answer.groovy . In addition to groovy, you'll need the following jars:
commons-lang: http://mirrors.ibiblio.org/pub/mirrors/maven2/commons-lang/commons-lang/2.3/commons-lang-2.3.jar
mulgara-client: http://maven.topazproject.org/maven2/org/topazproject/mulgara-client/0.8.3-SNAPSHOT/mulgara-client-0.8.3-20080318.212714-56.jar
driver: http://maven.topazproject.org/maven2/org/mulgara/driver/1.2-SNAPSHOT/driver-1.2-20080318.094904-5.jar
(this is mulgara trunk rev 691, so you can also build it yourself)
Run with
groovy -cp mulgara-client-0.8.3-20080318.212714-56.jar:driver-1.2-20080318.094904-5.jar:commons-lang-2.3.jar:. RunItql.groovy rmi://...
This code is not supported, use at your own risk, yada yada... If you
have fixes, we'll happily accept them though :-)
Cheers,
Ronald
-------------- next part --------------
/*
* Copyright (c) 2008 by Topaz, Inc.
* http://www.topazproject.org/
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not
* use this file except in compliance with the License. You may obtain a copy of
* the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations under
* the License.
*/
import jline.ConsoleReader;
import jline.History;
import org.apache.commons.lang.text.StrMatcher;
import org.apache.commons.lang.text.StrTokenizer;
import org.topazproject.mulgara.itql.DefaultItqlClientFactory;
import org.topazproject.mulgara.itql.ItqlClient;
import org.topazproject.mulgara.itql.ItqlClientFactory;
// Constants
csv = "csv" // In case somebody runs %mode = csv instead of %mode = "csv"
table = "table reduce quote" // Allows %mode = table
// Parse command line
def cli = new CliBuilder(usage: 'runitql [-f script] [-itpvN] <mulgara-uri>')
cli.h(longOpt:'help', 'usage information')
cli.v(longOpt:'verbose', 'turn on verbose mode')
cli.e(longOpt:'echo', 'echo script file when running')
cli.f(args:1, 'script file')
cli.p(longOpt:'prompt', 'show the prompt even for a script file')
cli.N(longOpt:'noprettyprint', 'Do not pretty-print results')
cli.i(longOpt:'runinit', 'Run ~/.runitql even if running a script')
cli.m(args:1, 'mode')
cli.t(args:1, 'number of characters to truncate literals to')
if (args.size() > 0 && args[0] == null) args = [ ]
if (args != null && args.length == 1)
args = new StrTokenizer(args[0], StrMatcher.trimMatcher(), StrMatcher.quoteMatcher()).tokenArray
def opt = cli.parse(args)
if (!opt) return
if (opt.h || opt.arguments().size != 1) { cli.usage(); return }
def file = (opt.f) ? new File(opt.f).newInputStream() : System.in
bPrompt = (opt.p || !opt.f)
bInit = (opt.i || !opt.f)
pp = !opt.N
mode = opt.m ?: table
echo = opt.v || opt.e || !opt.f
trunc = opt.t
running = true
def writer = echo ? new OutputStreamWriter(System.out) : new StringWriter()
verbose = opt.v
def mulgaraUri = opt.arguments()[0]
if (verbose) {
println "Mulgara URI: $mulgaraUri"
}
factory = new DefaultItqlClientFactory();
client = factory.createClient(mulgaraUri.toURI());
help = new HashMap()
help[null] = '''All commands are sent to mulgara once a semicolon (;) is found except for
lines starting with #, . or %. Additional help is available via .help <topic>.
These are interpreted as follows:
# - Comment lines
% - Remainder of line executed as groovy. See ".help variables"
. - Runs special commands. See ".help cmds"
Available topics:
variables cmds .alias .quit init'''
help["cmds"] = '''The following commands are supported: .alias, .quit
Run ".help .<cmd>" for help with a specific command'''
help[".alias"] = """.alias [list|set alias uri]
list - lists currently active aliases (but doesn't load them)
set alias uri - adds an alias to the interpreter (but doesn't save it)"""
help[".quit"] = """.quit - Exit the interpreter"""
help["variables"] = '''Variables can be used for a number of things:
- Controlling features of the interpreter (see list of variables below)
- As place holders in itql.
e.g. itql> %modelX = "<rmi://localhost/topazproject#foo>"
itql> select $s $p $o from ${modelX} where $s $p $o;
If a %-character is the first character of the line, the rest of the line is
sent to the groovy interpreter. Thus, %x=3 sets x to 3. %foo=bar is an error
because bar is not in quotes. '%println mode' will printout the value of the
variable mode.
Special variables:
mode (str) - Sets display output. General options are: csv, tsv, table
You can also append sub-modes: quote reduce
These quote literals and uris appropriately and/or reduce uris
via aliases for easier viewing. eg. %mode="table quote reduce"
trunc (int)- If set to an integer, literals are truncated to this number of
characters
verbose - If set to true, will output more information'''
help["init"] = "On startup, ~/.runitql is loaded."
// Various functions
def showHelp(args) {
def desc = help[args == null ? null : args[0]]
println (desc != null ? desc : "Invalid topic: ${args[0]}")
}
def alias(args) {
switch(args[0]) {
case 'list': showAliases(); break
case 'set' :
def aliases = client.getAliases()
aliases.put(args[1], URI.create(args[2]).toString())
client.setAliases(aliases);
break
case 'help': println ".alias [list|set alias uri]"; break
}
}
def showAliases() {
def aliases = client.getAliases()
def len = new ArrayList(aliases.keySet())*.size().max()
aliases.keySet().sort().each() { printf "%${len}s: %s\n", it, aliases[it] }
}
def reduceUri(uri) {
for (alias in client.getAliases()) {
if (uri == alias.value) return uri
def val = uri.replace(alias.value, alias.key + ":")
if (val != uri) return val
}
return uri
}
def reduce(s) {
for (alias in client.getAliases())
s = s.replaceAll(alias.value, alias.key + ":")
return s
}
def expand(s) {
for (alias in client.getAliases())
s = s.replaceAll(alias.key + ":", alias.value)
return s
}
def showResults(result) {
if (result.message) {
if (echo) println result.message
} else {
switch (mode) { case ~/.*csv.*/: showCsv(result); break
case ~/.*tsv.*/: showTsv(result); break
case ~/.*tab.*/: showTable(result); break
default: showTable(result)
}
}
}
def showCsv(res) {
def ans = new Answer(res)
ans.flatten()
def ops = [ ]
if (mode =~ "red") ops.add(ans.createReduceClosure(client.getAliases()))
if (mode =~ "quote") ops.add(ans.csvQuoteClosure)
ans.quote(ops)
ans.each() { println it.toString()[1..-2] }
}
def showTsv(res) {
def ans = new Answer(res)
ans.flatten()
def ops = [ ]
if (mode =~ "red") ops.add(ans.createReduceClosure(client.getAliases()))
if (mode =~ "quote") ops.add(ans.rdfQuoteClosure)
ans.quote(ops)
ans.each() { row ->
int cols = row.size()
int col = 0
row.each() {
print it
if (++col < cols) print "\t"
}
println()
}
}
def showTable(res) {
def ans = new Answer(res)
def cnt = ans.data.size()
println "cnt = ${cnt}"
ans.flatten()
def ops = [ ]
if (trunc instanceof Integer && trunc > 3) ops.add(ans.createTruncateClosure(trunc))
if (mode =~ "red") ops.add(ans.createReduceClosure(client.getAliases()))
if (mode =~ "quote") ops.add(ans.rdfQuoteClosure)
ans.quote(ops)
def lengths = ans.getLengths()
def seps = [ ]
lengths.each() { seps += "-"*it }
([ ans.getHeaders(), seps ] + ans.data).each() { row ->
def col = 0
def line = ""
row.each() { val ->
def st = val.toString()
line += st + " "*(lengths[col++] - st.size() + 1)
}
println line.trim()
}
def rowCnt = ans.data.size()
if (rowCnt == cnt)
println "${cnt} rows"
else
println "${rowCnt} total rows (${cnt} rows from main query)"
}
/**
* Expand ${} variables in a query string.
*
* This allows things like:
* <pre>
* @model = "<local:///topazproject#mymodel>"
* select $s $p $o from ${model} where $s $p $o;
* </pre>
*/
def expandVars(query) {
def result = query
(query =~ /\$\{([^}]*)}/).each() { st, var ->
result = result.replace(st, evaluate(var))
}
return result
}
def execute(query) {
try {
query = expandVars(query)
if (verbose)
println query
def result = doQuery(query)
showResults result
} catch (Throwable e) {
println "Error running query '${query}':"
Throwable c = e;
while (c.cause)
c = c.cause
println c
if (verbose)
e.printStackTrace()
}
}
def doQuery(query) {
if (!query.trim().endsWith(';'))
query <<= ';'
return client.doQuery(query.toString()).get(0)
}
/**
* Ask groovy to evaluate some string in our context
*
* Be very careful on refactoring that global variables (really instance variables)
* are available to evaluate() below or things may get very messy
*/
def eval(s) {
try {
this.expand = { expand(it) }
this.reduce = { reduce(it) }
this.expandVars = { expandVars(it) }
evaluate(s)
} catch (Throwable e) {
println "Error evaluating groovy: %$s"
println e
if (verbose)
e.printStackTrace()
}
}
// Handle special commands that start with .
def handleCmd(s) {
try {
def args = s.split(/ +/)
def cmd = args[0]
args = (args.size() > 1 ? args[1..-1] : null)
// Look for a matching command (allow abbreviations)
if ("alias".startsWith(cmd)) { alias(args) }
else if ("help".startsWith(cmd)) { showHelp(args) }
else if ("quit".startsWith(cmd)) { running = false }
else { println "Unknown command: .$s" }
} catch (Throwable e) {
println "Error running command: .$s"
if (verbose)
e.printStackTrace()
}
}
// Queries can exist on multiple lines, so need to stash previous partial queries
query = ""
def processLine(line, console) {
if (line != "" && line[0] == '#')
line = '' // strip comments
else if (line != "" && (line[0] == '%' || line[0] == '@')) { // @ is for backward compatibility
eval(line.substring(1))
console?.getHistory()?.addToHistory(line)
line = '' // strip expression
} else if (line != "" && line[0] == '.') {
handleCmd(line.substring(1))
console?.getHistory()?.addToHistory(line)
line = '' // strip expression
}
while (line.indexOf(';') != -1) {
pos = line.indexOf(';')
query += " " + line[0..pos-1].trim()
if (query.trim() != "") {
execute query.trim()
console?.getHistory()?.addToHistory(query.trim() + ";")
}
query = ""
line = line.substring(pos+1)
}
if (line.trim() != "")
query += " " + line.trim()
}
// Read init file if it exists
if (bInit) {
def initfile = new File(new File(System.getProperty("user.home")), ".runitql")
if (initfile.exists())
initfile.eachLine { line -> processLine(line, null) }
}
// Show the initial prompt
if (bPrompt)
println 'Itql Interpreter. Run ".help" for more information.'
// Use jline for some attempt at readline functionality
def cr = new ConsoleReader(file, writer)
if (bPrompt)
cr.setDefaultPrompt("itql> ")
try {
histfile = new File(System.getProperty("user.home"), ".runitql_history")
cr.setHistory(new History(histfile))
} catch (IOException e) {
println "Error loading history: $e"
}
// process the input
while (running && (line = cr.readLine()) != null) {
processLine(line, cr)
}
// clean up
try { client.rollbackTxn("cleanup") } catch (Throwable t) { }
println()
System.exit(0)
-------------- next part --------------
/*
* Copyright (c) 2008 by Topaz, Inc.
* http://www.topazproject.org/
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not
* use this file except in compliance with the License. You may obtain a copy of
* the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations under
* the License.
*/
import org.topazproject.mulgara.itql.Answer;
/** Base class for itql/rdf types below */
class Value {
/** The raw value from mulgara */
String value
/** The quoted value if quote() was called */
String quotedValue
String toString() { return quotedValue ? quotedValue : value }
int size() { return toString().size() }
/**
* Use the suplied closure(s) to quote our value
*
* @param f a closer or list of closures that are passed ourself and must return
* the quoted value as a string.
*/
void quote(f) {
if (f instanceof Closure) {
quotedValue = f(this)
} else {
quotedValue = null
f.each() { quotedValue = it(this) }
}
}
}
class Literal extends Value { Literal(value) { this.value = value } }
class RdfDate extends Value { RdfDate(value) { this.value = value } } // TODO: extend Literal
class RdfInt extends Value { RdfInt(value) { this.value = value } } // TODO: extend Literal
class Resource extends Value { Resource(value) { this.value = value } }
class Blank extends Value { Blank(value) { this.value = value } }
class Empty extends Value { Empty() { this.value = "" } }
/**
* Represents one row of an itql answer. Some columns may also contain rows and
* so on if there were subqueries. The flatten() and quote() methods modify
* instance data.
*/
class Row {
def vars
def hdrs = [ ]
def vals = [ ]
Row(res, vars) {
// TODO: Handle now vars (like count())
this.vars = vars
vars.each() { hdrs.add(it) }
vars.each() { var ->
def val = res.getString(var)
switch (res) {
case {it.isURI(it.indexOf(var))}: val = new Resource(val); break
case {it.isBlankNode(it.indexOf(var))}: val = new Blank(val); break
case {it.isLiteral(it.indexOf(var))}:
switch (res.getLiteralDataType(res.indexOf(var))) {
case "http://www.w3.org/2001/XMLSchema#int": val = new RdfInt(val); break
case "http://www.w3.org/2001/XMLSchema#date": val = new RdfDate(val); break
default: val = new Literal(val)
}
break
case {it.isSubQueryResults(it.indexOf(var))}:
// TODO: Handle a subquery that returns multiple subrows per row
Answer sqr = res.getSubQueryResults(var)
if (sqr.next())
val = new Row(sqr, sqr.variables)
else
val = new Empty()
sqr.close()
break
default: val = new Value(val);
}
vals.add(val)
}
}
/** Flatten any subqueries into a simple row */
void flatten() {
// Assume all rows have the same type
int col = 0
def newvals = [ ]
def newhdrs = [ ]
vals.each() { val ->
if (val instanceof Row) {
val.flatten()
newvals += val.vals
newhdrs += val.hdrs
} else {
newvals.add(val)
newhdrs.add(hdrs[col])
}
col++
}
vals = newvals
hdrs = newhdrs
}
// Duck typing: Make Row function as if it was an array of columns
void quote(f) { vals.each() { it.quote(f) } }
def getLengths() { return vals*.size() }
String toString() { return vals.toString() }
def getAt(int pos) { return vals[pos] }
def iterator() { return vals.iterator() }
def size() { return vals.size() }
}
/**
* Represent an answer from itql
*/
class Answer {
def vars
def data = [ ]
/**
* Construct an Answer
*
* @param res should be Result from Session
*/
Answer(res) {
vars = res.variables
println "variables: ${vars}"
while (res.next())
data.add(new Row(res, vars))
res.close();
}
/** flatten any subquery results into main query */
void flatten() {
data.each() { row ->
row.flatten()
}
}
def getHeaders() {
if (data)
return data[0].hdrs
}
/**
* quote data with suplied closure(s)
*
* @param f a closer or list of closures that are passed ourself and must return
* the quoted value as a string.
*/
def quote(f) {
data.each() { it.quote(f) }
}
/** Return the maximum lengths of all columns */
def getLengths() {
def lengths = getHeaders()*.size()
data.each() { row ->
def col = 0
row.getLengths().each() { length ->
lengths[col] = [ lengths[col], length ].max()
col++
}
}
return lengths
}
// Duck typing helpers ... make us look like our data
def getAt(int pos) { return data[pos] }
def iterator() { return data.iterator() }
// Closure helpers for quote()
static def csvQuoteClosure = { v ->
println "Quoting: ${v.getClass().getName()}: $v"
switch (v) {
case Literal: return '"' + v.toString().replace('"', '""') + '"'
case Resource: return "<$v>"
default: return v.toString()
}
}
static def rdfQuoteClosure = { v ->
switch (v) {
case RdfDate: return v.toString(); break
case RdfInt: return v.toString(); break
case Literal: return "'" + v.toString().replace("'", "\\'") + "'"; break
case Resource: return "<$v>"; break
default: return v.toString()
}
}
static def createReduceClosure(aliases) {
return { v ->
if (!(v instanceof Resource)) return v.toString()
for (alias in aliases) {
if (v.value == alias.value) return v.toString()
def val = v.toString().replace(alias.value, alias.key + ":")
if (val != v.toString()) return val
}
return v.toString()
}
}
static def createTruncateClosure(int length) {
return { v ->
if (!(v instanceof Literal)) return v.toString()
if (v.toString().size() > length) {
return v.toString()[0..(length-3)] + "..."
} else {
return v.toString()
}
}
}
}
More information about the Mulgara-dev
mailing list