Friday, September 13, 2013

How to expand color palette with ggplot and RColorBrewer

Histograms are almost always a part of data analysis presentation. If it is made with R ggplot package then it may look like this:

ggplot(mtcars) +
  geom_histogram(aes(factor(cyl), fill=factor(cyl)))

The elegance of ggplot functions is in simple yet compact expression of visualization formula while hiding many options assumed by default. Hiding doesn't mean lacking as most options are just a step away. For example, color selection can change with one of scale functions such as scale_fill_brewer:

ggplot(mtcars) +
  geom_histogram(aes(factor(cyl), fill=factor(cyl))) +

In turn, scale_fill_brewer palette can be changed too:

ggplot(mtcars) +
  geom_histogram(aes(factor(cyl), fill=factor(cyl))) +

Palettes used live in the package RColorBrewer - to see all available choices simply attach the package with library(RColorBrewer) and run display.brewer.all() to show this:
There are 3 types of palettes, sequential, diverging, and qualitative; each containing from 8 to 12 colors (see data frame and help ?RColorBrewer for details).

Curious reader may notice that if a histogram contains 13 or more bars (bins in case of continuous data) we may get in trouble with colors:

ggplot(mtcars) + 
  geom_histogram(aes(factor(hp), fill=factor(hp))) +

Indeed length(unique(mtcars$hp)) finds 22 unique values for horse power in mtcars, while specified palette Set2 has 8 colors to choose from. Lack of colors in the palette triggers ggplot warnings like this (and invalidates plot as seen above):
1: In brewer.pal(n, pal) :
  n too large, allowed maximum for palette Set2 is 8
Returning the palette you asked for with that many colors
RColorBrewer gives us a way to produce larger palettes by interpolating existing ones with constructor function colorRampPalette. It generates functions that do actual job: they build palettes with arbitrary number of colors by interpolating existing palette. To interpolate palette Set1 to 22 colors (number of colors is stored in colourCount variable for examples to follow):

colourCount = length(unique(mtcars$hp))
getPalette = colorRampPalette(brewer.pal(9, "Set1"))
ggplot(mtcars) + 
  geom_histogram(aes(factor(hp)), fill=getPalette(colourCount)) + 

While we addressed color deficit other interesting things happened: even though all bars are back and are distinctly colored we lost color legend. I intentionally added theme(legend.position=...) to showcase this fact: despite explicit position request the legend is no more part of the plot.

The difference: fill parameter was moved outside of histogram aes function - this effectively removed fill information from aesthetics data set for ggplot. Hence, there is nothing to apply legend to.

To fix it place fill back into aes and use scale_fill_manual to define custom palette:

ggplot(mtcars) + 
  geom_histogram(aes(factor(hp), fill=factor(hp))) + 
  scale_fill_manual(values = getPalette(colourCount))

Another likely problem with large number of bars in histogram plots is placing of the legend. Adjust legend position and layout using theme and guides functions as follows :

ggplot(mtcars) + 
  geom_histogram(aes(factor(hp), fill=factor(hp))) + 
  scale_fill_manual(values = getPalette(colourCount)) +
  theme(legend.position="bottom") +

Finally, the same example using  in place palette constructor with different choice of library palette:

ggplot(mtcars) + 
  geom_histogram(aes(factor(hp), fill=factor(hp))) + 
  scale_fill_manual(values = colorRampPalette(brewer.pal(12, "Accent"))(colourCount)) +
  theme(legend.position="bottom") +
Created by Pretty R at

There are quite a few more scale functions to choose from depending on aesthetics type (colour, fill), color types (gradient, hue, etc.), data values (discrete or continuous).

Wednesday, July 31, 2013

Quick R tip: ggplot in functions needs some extra care

When building visualizations with ggplot2 in R I decided to create specialized functions that encapsulate plotting logic for some of my creations. In this case instead of commonly used aes function I had to use its alternative -  aes_string - for aesthetic mapping from a string.

And now goes this handy tip:
while original aesthetic mapping function aes accepts x and y parameters by position:

p = ggplot(data, aes(x, y)) + ...

aes_string even though silently accepts them won't work like this:

my_plot_fun = function(data, xname, yname) {
  p = ggplot(data, aes_string(xname, yname)) + ...

It will run to compile plot object without problems but when plot p (returned from the function my_plot_fun) executed this rather cryptic error appears:

Error in as.environment(where) : 'where' is missing

What it means is that ggplot never got aesthetics defined right. This is due to aes_string function lacking the same position parameters as in its aes counterpart above. Instead, define both x and y parameters (and others if necessary) by name:

p = ggplot(data, aes_string(x=xname, y=yname)) + ...

Created by Pretty R at


One more vote for using aes_string in place of aes comes from CRAN submission policy, i.e.:
In principle, packages must pass R CMD check without warnings or significant notes to be admitted to the main CRAN package area. If there are warnings or notes you cannot eliminate (for example because you believe them to be spurious) send an explanatory note as part of your covering email, or as a comment on the submission form. 
 (source: CRAN Repository Policy)
 What happens is that  R CMD check  reports notes like this for every aes call:

no visible binding for global variable [variable name]
It turns out that the most sensible solution is using aes_string instead.