Gustavo Lacerda

One of the annoyances in R is dealing with functions that don't evaluate one or more arguments that you pass, or who otherwise use the name of the variable passed. The problem appears when you try to write abstractly.

e.g. with(data, ZQ/Total.Z) will compute data$ZQ / data$Total.Z . What 'with' is doing is parsing that expression, figuring out which variable tokens are already present in the current environment, and putting "data$" in the front of the rest. Yesterday, in my naivety, I implemented just that (28 easy-to-read lines of R).

However, it's hard to do something more abstract, e.g. with(data, property) will try to get a property named "property". To circumvent this, one can make a call to eval:

withExpr <- jPaste("with(x,",property,")")
eval(parse(text=withExpr))

I am not happy with this, but there is NO OTHER WAY. I say this confidently because 'with' appears to completely discard the value of the variable passed, while only using its name, i.e. something like:

property <- deparse(substitute(property))

Having to call eval is the price we pay for the convenience of not using quotes.

And, guess what, I take the deal! Yesterday, I wrote 'violinPlot', which is like a 'boxplot' but with kernel density estimates instead of quantiles. The two basic arguments to violinPlot are 'datasets' and 'property': for each dataset, it extracts the property and plots a violin.

l <- list(mon, tue, wed, thu, fri, sat, sun)
violinPlot(l, ZQ/Total.Z, col=c(rep("#AAAAFF",5), rep("orange", 2)), horizontal=FALSE)

My code starts with:

violinPlot <- function(datasets, property,
                       labels=c("M", "T", "W", "R", "F", "Sa", "Su"),
                       horizontal=TRUE, colors=NA){
  property <- deparse(substitute(property))
  colors <- rep(colors, length(datasets)/length(colors)+1)
  densities <- lapply(datasets, function(x) density(with2(x,property)))
  ...
}

with2 <- function(data, expr, ...)
  with(data, eval(parse(text=expr)), ...)

You can see above that I also wanted to pass 'property' without quotes. Having essentially reimplemented 'with', I am in a position to modify it so that the syntax becomes with(data, "ZQ/Total.Z"), and spare myself the eval next time... but I don't wanna.

But here's what I might do: instead of with(data,expr), make it with(data, exprLiteral=NULL, exprToEvaluate=NULL), and you would only pass one of these expr arguments. The difference is that 'exprToEvaluate' gets evaluated into a string (so it better be a string!); whereas 'exprLiteral' gets turned into a string directly, and corresponds to the current syntax of 'with'... and since 'exprLiteral' comes first (in the second position of the argument list), current calls to 'with' would continue working. Yay, backward-compatibility!


More pretty graphics:( Read more... )

A lot of people in the field of machine learning like to trash R. But as someone who comes from machine learning and who has programmed all his life, in many languages and paradigms, I have to say that R can be pretty pleasant to work with. It's not very fast (supposedly much slower than Matlab on matrix computations, and a lot slower than C++); its commands are a bit quirky at first and many defaults are annoying (e.g. whitespace is the default separator); and there are plenty of imperfections and missing features (e.g. hashes). And, there is no serious type system.

However, I find that R readily accommodates my desire to reinvent the language, which makes me very happy. Functions are first-class objects. apply and Reduce often spare me from writing looping code. We have eval! Although there is no defmacro, a lot can be accomplished with deparse and substitute (to be honest, I have yet to do any serious macro-ing). In function calls, "all remaining arguments" bind to '...'. The source code is within easy reach, in case you ever wonder how e.g. plot implements its default axes labels. do-while has a substitute in the form of repeat; if T then break (a.k.a. where repeat is the same as while(TRUE)).

---

Anyway, I have produced a substantial library for myself, and almost everything I do nowadays depends on it. Since I think this code could be useful for a lot of people (my debugging function, in particular), I should release a package of general-purpose R goodies someday.

Today I'm addressing the annoyance of having to remember parameter values, and pass them again and again to the different distribution-specific functions (e.g., in the case of the normal distribution, the set pnorm,qnorm,rnorm,dnorm). This code bundles together the 4 distributions functions for any given distribution:
( Read more... )

One thing that drives me crazy about R is that problems sometimes go away spontaneously, and I don't feel like I learned anything. Part of this is due to working in a REPL, part mystery.

doPlots <- TRUE; doPdflatex <- TRUE ##default settings
args <- commandArgs(TRUE)
if (length(args)>0) {
  eval(parse(text=args[1]))
}

The idea is that, since we using R's reflection mechanism, we can change the variables from the command-line.

Rscript fix-multiple.R "doPlots=FALSE; doPdflatex=FALSE"

(You could also theoretically write code that actually does stuff, but that would be strange)

In R, you loop over your list l as follows

for (i in 1:length(l))

But if length(l) is zero, you still visit the loop twice... I do not know an elegant solution to this.

Another one has to do with operator precedence, 1:k+3 gets parsed as (1:k)+3 rather than 1:(k+3).

Image you are to write a certain function 'f'. The purpose of calling it is to do X for every element of a list (and optionally Y; this is controlled by a flag)

Which of the following is better?

(a)

function f()
  for 1:1000
    do X
    if(flag)
      do Y

(b)

function f()
  for 1:1000
    do X
  if(flag)
    for 1:1000
      do Y

In all languages I know, (a) is more elegant but (b) is more efficient.

In Lisp, another alternative exists: you can write a macro to do a partial evaluation given the value of 'flag'. This is as efficient as (b), but whether it's elegant is debatable. Personally I've never seen an IDE that makes code-rewriting macros nice to work with.

I'm pleased about the Tetrad code being in Java 5, which really is a slightly different language. Java 5 has two nice features:

* nicer 'for' loops (++), such as:

for (int sampleSize : new int[]{1000, 10000}){ ... }

* Parametric types, which avoids all that unnecessary casting that Java is notorious for (+):

List nodes = dag.getNodes();
Node node0 = nodes.get(0);

Combined, we can now do:

for (Node node : dag.getNodes()) { ... }

instead of

for (int i=0; i < dag.getnodes().size(); i++) {
    Node node = dag.getnodes().get(i);
    ...
}

(although this translation would probably break if you remove an element from the list inside this loop without decrementing i)

-----------------

Sometimes, after your program has finished execution, you notice that some object (e.g. a graph) had a really interesting property... or, if the program is non-deterministic, and you notice a bug, you may want to run the same example again for debugging purposes (or the entire state of the program, if that were possible).

Unfortunately, the object is now gone, and all you have is its footprints. By the time you think of putting Object.serialize() in the code, it's too late: you have to pray that you'll run into that situation again.

What do you do?

I don't like most programming books, tutorials and reference manuals. They tend to be too low-level, walking you through too many obvious steps, and they not helping you find the RightThing for your sort of problem.

I think I much prefer books that goal-oriented, like:

* How to Solve Problems with X
* Patterns in X

They are better for someone who needs to write code, rather than someone who needs to read it.

With these books, it should be much easier to find the answers my questions, partly because they tend to *have* the answers, partly because they have less boring, distracting junk.

Right now, I want to find a good book about How to Solve Problems with Java.

Is it possible to write a function that does something only after its caller is finished?

I am writing a function that creates a temporary file, writes some stuff to it, and returns the filename. The expression where this is called gets the returned value, uses the file, but nobody does after that. The file should therefore be deleted, but by whom?

If I delete it inside the function that creates it, then the file can't be used by the expression that called it.

I can just delete the file on the next S-expression, right after the expression that calls the function, but that seems like a violation of modularity, since the creation and deletion are not in the same place (not to mention that this requires me to save the returned value, introducing a let).

It would be good if my function could stay in the background after returning, and somehow know when to delete the file. Of course, I would need to tell it when: however, something like "two levels up the call stack" sounds like a non-robust solution. Maybe you could have a scope for temporary files (i.e. a let): as soon as you fall out of that scope, all temporary files created inside get deleted.

Maybe there is a way of creating temporary files so that they get cleaned by a garbage collector as soon as the program ends.

--

This reminds me of an old debate, about a solution that seemed much easier by using GOTO. It was probably a reaction to "GOTO Considered Harmful".