R is a case sensitive programming language, which sometimes creates unusual and yet annoying problems for users. A common mistake that I often make when using R is to press the shift button too early when I use the head() function – which results in heaD(). However, this returns an error message (not surprisingly) rather than printing the first six rows of the data.
At the beginning, this didn’t bother me since I could easily fix this typo but over time it has become a bit annoying. The faster I typed in R, the more I repeated this mistake. Then I thought creating a heaD() function that does the same job as head() would be the ultimate solution to my problem. Because I would need this “new” function every time I open R, I decided to add it into my .Rprofile file. The .Rprofile file contains R code to be run when R starts up. Typically .Rprofile is located in the user’s home directory. In my computer, I place it under: C:\Users\okanb\OneDrive\Documents
To create a new .Rprofile file, you can simply create a text file using a text editor, add the content to be used when R starts up, and save it as .Rprofile (i.e., no file name and the file extension is .Rprofile). If you already have this file in your computer, you can simply run usethis::edit_r_profile() to open and edit the file in R. In the file, I added the following line:
# Create a heaD function
heaD <- function(x) head(x) # or just: heaD <- head
After editing and saving .Rprofile, you need to restart R. From this point on, R will recognize both head() and heaD().
In my daily work I often have to transform a long table to a wide matrix so accommodate some function. At some stage in my life I came across the reshape2 package, and I have been with that philosophy ever since – I find it makes data wrangling easy and straight forward. I particularly like the tidyverse philosophy where data should be in a long table, where one row is an observation, and a column a parameter. It just makes sense.
However, I quite often have to transform the data into another format, a wide matrix especially for functions of the vegan package, and one day I wondering how to do that in the fastest way.
The code to create the test sets and benchmark the functions is in section ‘Settings and script’ at the end of this document.
I created several data sets that mimic the data I usually work with in terms of size and values. The data sets have 2 to 10 groups, where each group can have up to 50000, 100000, 150000, or 200000 samples. The methods xtabs() from base R, dcast() from data.table, dMcast() from Matrix.utils, and spread() from tidyr were benchmarked using microbenchmark() from the package microbenchmark. Each method was evaluated 10 times on the same data set, which was repeated for 10 randomly generated data sets.
After the 10 x 10 repetitions of casting from long to wide it is clear the spread() is the worst. This is clear when we focus on the size (figure 1). Figure 1. Runtime for 100 repetitions of data sets of different size and complexity.
And the complexity (figure 2). Figure 2. Runtime for 100 repetitions of data sets of different complexity and size.
Close up on the top three methods
Casting from a long table to a wide matrix is clearly slowest with spread(), where as the remaining look somewhat similar. A direct comparison of the methods show a similarity in their performance, with dMcast() from the package Matrix.utils being better — especially with the large and more complex tables (figure 3). Figure 3. Direct comparison of set size.
I am aware, that it might be to much to assume linearity, between the computation times at different set sizes, but I do believe it captures the point – dMcast() and dcast() are similar, with advantage to dMcast() for large data sets with large number of groups. It does, however, look like dcast() scales better with the complexity (figure 4). Figure 4. Direct comparison of number groups.
