Tidyverse Functions (Part 2)

Introduction

The tidyverse metapackage offers even more useful functions than the ones we have covered to date. In this tutorial, we’ll be learning some new functions that will help us tansform our data, compute new variables, and visualize things more effectively.

First, here is a summary of the data analysis functions covered in this tutorial and a quick description of when you’ll want to use them:

• Use distinct() whenever you want to see the list of unique values within a variable

• Use top_n() whenever you want to get the top X rows (based on the values from some variable)

• Use mutate() whenever you want to add a new variable (column) to the existing data frame based on an operation

• Use pivot_longer() and pivot_wider() whenever you want to convert data from wide format to long format, and vice versa

Second, here is a summary of some helpful functions for creating data visualizations using ggplot2:

• Use the group, color, shape, and size aesthetic mappings to group information

• Use fct_reorder() to sort the labels on your axes

• Use expand_limits() to automatically extend an axis in a certain direction (e.g., to show the value at zero)

• Use scale_{x,y}_continuous() to add more tick marks to an axis

• Use labs() to provide easier-to-read labels for each axis

• Use facet_wrap() to create a grid of similar charts based on a variable

• Use theme() to move (or hide) the chart legend

Our dataset for this tutorial

We will be illustrating each function by applying them to a dataset combining the gross salaries for Town of Amherst employees in 2020 with the salaries from 2018. These data were extracted from the PDF files using Tabula, cleaned up to standardize department names, and saved to this CSV file.

I can load this dataset by using the read_csv() function from the readr package (part of tidyverse), supplying the URL for the dataset as my lone argument, and specifying that I want to disable scientific notation for the rest of my working session.

options(scipen=999)
library(tidyverse)

## ── Attaching packages ─────────────────────────────────────── tidyverse 1.3.1 ──

## ✓ ggplot2 3.3.5     ✓ purrr   0.3.4
## ✓ tibble  3.1.3     ✓ dplyr   1.0.7
## ✓ tidyr   1.1.3     ✓ stringr 1.4.0
## ✓ readr   2.0.1     ✓ forcats 0.5.1

## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## x dplyr::filter() masks stats::filter()
## x dplyr::lag()    masks stats::lag()

salaries <- read_csv("https://dds.rodrigozamith.com/files/amherst_gross_wages_2018_2020.csv")

## Rows: 1559 Columns: 5

## ── Column specification ────────────────────────────────────────────────────────
## Delimiter: ","
## chr (3): last_name, first_name, department
## dbl (2): year, gross_wages

##
## ℹ Use `spec()` to retrieve the full column specification for this data.
## ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

Let’s review the first few rows in the dataset to see what we’re working with.

head(salaries)

Next, let’s double-check the structure of the dataset:

glimpse(salaries)

## Rows: 1,559
## Columns: 5
## $ year        <dbl> 2020, 2018, 2020, 2018, 2020, 2020, 2018, 2018, 2020, 2018...
## $ last_name   <chr> "AARONSON", "ABDEL-MAKSOUD", "ABDEL-MAKSOUD", "ABELLI", "A...
## $ first_name  <chr> "JEREMY", "ALI", "AMMAR", "NICOLE", "NICOLE", "ANDREW", "D...
## $ gross_wages <dbl> 252.71, 9878.00, 1433.96, 45931.00, 57203.87, 81.96, 102.0...
## $ department  <chr> "Fire", "Amherst Recreation", "Amherst Recreation", "Amher...

As we can see, there are five variables in this dataset, and a total of 1,559 observations. The names are correctly stored as chr values and the numbers (year and gross_wages) correctly present themselves as numerical (dbl) values. Finally, it still seems like the data mostly exist for just two years: 2018 and 2020. (We can easily double-check that using the distinct() function, which is described below.)

distinct()

We use the distinct() to get a quick listing of all the unique values within a given variable.

This is helpful when a variable (like department) has a lot of values that repeat, yet you want to quickly see which different values (different departments) are present in the dataset. For example, here’s how we can get a basic data frame consisting of just the unique values for department:

salaries %>%
  distinct(department)

We can double-check that we only have data for 2018 and 2020 by using the following code:

salaries %>%
  distinct(year)

That is indeed the case. We only have data for those two years.

top_n()

The top_n() function is useful whenever we want to look at the top X rows of the dataset based on some variable. This is a faster alternative to the arrange() and head() pairings we’ve previously covered.

This function takes two arguments: (1) the number of rows you want and (2) the variable to base the selection on.

For example, here’s how we can produce a data frame with the three people that have the highest salaries (gross_wages):

salaries %>%
  top_n(3, gross_wages)

mutate()

The mutate() function is used whenever we want to add (compute) a new variable to the data frame based on an operation. (Put differently, when you want to add a new data point for each observation (row) within the data frame.)

We supply the mutate() function with an argument structured with (1) the name of the variable we want to create (e.g., new_salary), (2) followed by the equal sign (=) to denote what we want that new variable to be equal to, which is followed by (3) the operation we want to perform to calculate the values for the new variable (e.g., old_salary + 10).

This syntax is similar to what we previously covered with the summarize() function. However, unlike summarize(), the mutate() function does not strip information from the dataset (i.e., it does not aggregate data). Because we’re just adding a new column, all of our original variables (like first_name) are preserved.

Here’s how we might compute what a 2% raise would look like for each person in this dataset, storing that information in a variable called increased_wages:

salaries %>%
  mutate(increased_wages=gross_wages*1.02) %>%
  select(year, first_name, last_name, gross_wages, increased_wages)