Data Manipulation in R with dplyr

  • The R language is widely used among data scientists, statisticians, researchers and students.

    It is simply the leading tool for statistics, data analysis and machine learning. It is platform-independent, open-source, and has a large, vibrant community of users.

    The Comprehensive R Archive Network is the one-stop-shop for all R packages.

    This really brings us to the package to be discussed on this blog – dplyr. The CRAN documentation for dplyr can be found here.

    For this blog, I would be demonstrating the 5 operations of the package. The first thing we would need is to install the package and load the library.

    > install.packages(“dplR”)

    > library(dplR)

    We then need to find a dataset on which we could run these operations. CRAN makes the download logs of their packages publicly available here – CRAN package download logs. Let us download the file for July 8, 2014 (we could really pick a log from any date) onto RStudio’s working directory.

    Once the file has been copied onto the working directory of R, execute the below line (where the variable path2csv stores the location of the csv)

    > mydf <- read.csv(path2csv, stringsAsFactors = FALSE)

     

    we then save the data frame onto a variable called cran by converting it to a tbl_df to improve readability. Calling the variable cran prints out the contents.

    > cran <- tbl_df(mydf)
    > cran
    

    Capture_Dplyr1.PNG
    

    The dplyr philosophy is to have small functions that do one thing well. There are basically 5 commands that cover most of the fundamental data manipulation tasks.
    

      

    • select()
      • 

    

    

    

    

    Usually in the entire data set that we use for analyis, we would really be interested in a few columns. This function is used to select / fetch the columns which are required. If I only need the columns ip_id, package and country. I execute the following statement –
    

    

    

    > select(cran, ip_id, package, country)
    

    CaptureDplyr2.PNG
    

    

    It is important to note that the columns are returned in the order in which we specified, irrespective of how it was in the original dataframe.
    

    We could also use the ‘-‘ sign to ommit the columns we do not need.
    

    

    

    > select(cran, -time)
    

    CaptureDplyr3.PNG

    

    

    

    

    

      

    • filter()
      • 

    

    

    

    

    Now that we know how to select columns, the next logical thing would be to be able to select rows. That is where the filter() function comes in.
    

    This is like the ‘where’ clause in SQL. Let us understand this by an example –
    

    

    

    > filter(cran, package == "swirl")
    

    CaptureDplyr4.PNG
    

    

    If you look at the column ‘package’, we now see that the resulting dataframe has only rows which have the package as ‘swirl’.
    

    

    Multiple conditions can be passed to filter() one after the other. For example, if I want to fetch all swirl packages downloaded on the OS – linux in India:
    

    

    > filter(cran, package == "swirl", r_os == "linux-gnu", country == "IN")
    

    CaptureDplyr5.PNG
    

    

    

    

    

    

      

    • arrange()
      • 

    

    

    

    

    This is used to order the rows of a dataset according to the values of a particular variable. Suppose we want to order all rows of a dataset in ascending / descending order of a column. Notice the ip_id column listed in descending order.
    

    

    > arrange(cran2, desc(ip_id))
    

    CaptureDplyr6.PNG
    

    

    

    

    

      

    • mutate()
      • 

    

    

    

    

    This function is used to edit or add additional columns to the dataframe. Suppose I want to convert the size column which is in bytes to megabytes and store the values in a column called size_mb.
    

    

    

    

    

    > mutate(cran3, size_mb = size / 2^20)
    

    

    

    

    

    CaptureDplyr7.PNG
    

    

    

    

      

    • sumarize()
      • 

    

    

    

    

    This function is used to collapse the dataset into a single row, the go-to function to calculate the mean in a sanitized dataframe.
    

    For example – I want to know the average download size from the size column.
    

    > summarize(cran, avg_bytes = mean(size))
    

    CaptureDplyr8.PNG
    

    

    

    

    sumarize() can also be used to fetch records in groups using the FOR EACH construct.
    

    

    

    Disclosure: The above example is from the dplyR lesson on the swirl package.
    

    

    

    

    

    •