Gustavo Lacerda

R: semantics and pragmatics / names vs values

Sat, 21 Apr 2012 16:57:43 GMT

One of the annoyances in R is dealing with functions that don't evaluate one or more arguments that you pass, or who otherwise use the name of the variable passed. The problem appears when you try to write abstractly.

e.g. with(data, ZQ/Total.Z) will compute data$ZQ / data$Total.Z . What 'with' is doing is parsing that expression, figuring out which variable tokens are already present in the current environment, and putting "data$" in the front of the rest. Yesterday, in my naivety, I implemented just that (28 easy-to-read lines of R).

However, it's hard to do something more abstract, e.g. with(data, property) will try to get a property named "property". To circumvent this, one can make a call to eval:

withExpr <- jPaste("with(x,",property,")")
eval(parse(text=withExpr))

I am not happy with this, but there is NO OTHER WAY. I say this confidently because 'with' appears to completely discard the value of the variable passed, while only using its name, i.e. something like:

property <- deparse(substitute(property))

Having to call eval is the price we pay for the convenience of not using quotes.

And, guess what, I take the deal! Yesterday, I wrote 'violinPlot', which is like a 'boxplot' but with kernel density estimates instead of quantiles. The two basic arguments to violinPlot are 'datasets' and 'property': for each dataset, it extracts the property and plots a violin.

l <- list(mon, tue, wed, thu, fri, sat, sun)
violinPlot(l, ZQ/Total.Z, col=c(rep("#AAAAFF",5), rep("orange", 2)), horizontal=FALSE)

My code starts with:

violinPlot <- function(datasets, property,
                       labels=c("M", "T", "W", "R", "F", "Sa", "Su"),
                       horizontal=TRUE, colors=NA){
  property <- deparse(substitute(property))
  colors <- rep(colors, length(datasets)/length(colors)+1)
  densities <- lapply(datasets, function(x) density(with2(x,property)))
  ...
}

with2 <- function(data, expr, ...)
  with(data, eval(parse(text=expr)), ...)

You can see above that I also wanted to pass 'property' without quotes. Having essentially reimplemented 'with', I am in a position to modify it so that the syntax becomes with(data, "ZQ/Total.Z"), and spare myself the eval next time... but I don't wanna.

But here's what I might do: instead of with(data,expr), make it with(data, exprLiteral=NULL, exprToEvaluate=NULL), and you would only pass one of these expr arguments. The difference is that 'exprToEvaluate' gets evaluated into a string (so it better be a string!); whereas 'exprLiteral' gets turned into a string directly, and corresponds to the current syntax of 'with'... and since 'exprLiteral' comes first (in the second position of the argument list), current calls to 'with' would continue working. Yay, backward-compatibility!

More pretty graphics:( Read more... )

comments

what I love about R

Thu, 05 Apr 2012 03:45:40 GMT

One thing I really love about R is how I can write improperly-scoped code, and everything still works.

gSmooth <- function(x,y, kernelSd=1, kernel=function(z) dnorm(z,mean=x[i],sd=kernelSd)){
  v <- c()
  for (i in seq_len(length(x))){
    weights <- sapply(x, kernel)
    v[i] <- sum(weights*y)/sum(weights)
  }
  list(x=x,y=v)
}

plot(data$ZQ, type="l", ylim=c(0,130))
ss <- gSmooth(1:n,data$ZQ)
pplot(ss$x, ss$y, type="l", col="red")
ss <- gSmooth(1:n,data$ZQ, kernelSd=3)
pplot(ss$x, ss$y, type="l", col="blue")

----

This is much cleaner:

gSmooth <- function(x,y, kernel=gaussKernel){
  v <- c()
  for (i in seq_len(length(x))){
    center <- x[i]
    weights <- sapply(x, function(z) kernel(z, center))
    v[i] <- sum(weights*y)/sum(weights)
  }
  list(x=x,y=v)
}

gaussKernel <- function(z, center) dnorm(z, mean=center, sd=kernelSd)
emaKernel <- function(z, center) if(z<=center) return(exp((z-center)/kernelSd))
                                   else return(0) ## Exponential Moving Average

plot(data$dayNumber, data$ZQ, type="p")
pplot(data$dayNumber, data$ZQ, type="l")


kernelSd <- 3
ss <- gSmooth(data$dayNumber,data$ZQ)
pplot(ss$x,ss$y, type="l", col="red")

kernelSd <- 3
ss <- gSmooth(data$dayNumber,data$ZQ, kernel=emaKernel)
pplot(ss$x,ss$y, type="l", col="blue")

Note how the Exponential Moving Average (in blue) is backward-looking, and less smooth than the Gaussian one (in red), even though these kernels, when viewed as distributions, have the same standard deviation of 3.

I think that this is in part due to the Exponential kernel not being as smooth as the Gaussian one, but I also suspect that it weights the points less evenly.

comments

in defense of R

Thu, 13 Jan 2011 19:08:33 GMT

A lot of people in the field of machine learning like to trash R. But as someone who comes from machine learning and who has programmed all his life, in many languages and paradigms, I have to say that R can be pretty pleasant to work with. It's not very fast (supposedly much slower than Matlab on matrix computations, and a lot slower than C++); its commands are a bit quirky at first and many defaults are annoying (e.g. whitespace is the default separator); and there are plenty of imperfections and missing features (e.g. hashes). And, there is no serious type system.

However, I find that R readily accommodates my desire to reinvent the language, which makes me very happy. Functions are first-class objects. apply and Reduce often spare me from writing looping code. We have eval! Although there is no defmacro, a lot can be accomplished with deparse and substitute (to be honest, I have yet to do any serious macro-ing). In function calls, "all remaining arguments" bind to '...'. The source code is within easy reach, in case you ever wonder how e.g. plot implements its default axes labels. do-while has a substitute in the form of repeat; if T then break (a.k.a. where repeat is the same as while(TRUE)).

---

Anyway, I have produced a substantial library for myself, and almost everything I do nowadays depends on it. Since I think this code could be useful for a lot of people (my debugging function, in particular), I should release a package of general-purpose R goodies someday.

Today I'm addressing the annoyance of having to remember parameter values, and pass them again and again to the different distribution-specific functions (e.g., in the case of the normal distribution, the set pnorm,qnorm,rnorm,dnorm). This code bundles together the 4 distributions functions for any given distribution:
( Read more... )

comments