Expanded Data Visualization Options

Several examples and graphical techniques in this lab are adapted from Robert Kabacoff’s Modern Data Visualization with R, particularly Chapters 4 and 5. The lab also incorporates additional examples, datasets, and exercises developed specifically for this course.

Today we will focus on choosing visualizations for distributions. You have already learned the basic grammar of ggplot2; the goal today is not to memorize a long list of new commands. Instead, we will expand the kinds of graphs you recognize and think about what different displays make easy—or difficult—to see.

Some graphs in this lab are supplied for you. Read the code and experiment with it when useful, but you are not expected to memorize every new function. When you are asked to construct a graph yourself, the question will generally describe what you want the graph to communicate rather than tell you which geom to use.

A note on data manipulation: Throughout this lab, you will encounter code that prepares and modifies datasets before we visualize them. Pay attention to what these steps accomplish, but don’t worry about memorizing the code or understanding every command just yet. Next week, we will begin learning dplyr, a package that makes working with and manipulating data much easier. By the end of next week, many of the data preparation steps you encounter in this lab should seem much less mysterious.

Data Stuff

# install.packages(c("ggplot2", "mosaicData", "nycflights13",
#                    "treemapify", "ggridges", "patchwork"))

library(ggplot2)
library(mosaicData)
library(nycflights13)

data("Marriage", package = "mosaicData")

# Select five airlines and exclude missing arrival delays.
flight_pool = subset(
  nycflights13::flights,
  carrier %in% c("AA", "B6", "DL", "UA", "WN") &
    !is.na(arr_delay)
)

# Draw a reproducible simple random sample.
set.seed(230)
flights_sample = flight_pool[
  sample(nrow(flight_pool), size = 6000),
]

# Replace airline codes with readable labels.
airline_names = c(
  AA = "American",
  B6 = "JetBlue",
  DL = "Delta",
  UA = "United",
  WN = "Southwest"
)

flights_sample$airline = factor(
  airline_names[flights_sample$carrier],
  levels = airline_names
)

# Give departure airports an explicit order.
flights_sample$origin = factor(
  flights_sample$origin,
  levels = c("EWR", "JFK", "LGA")
)

# Keep only the variables needed for the lab.
flights_sample = flights_sample[
  , c("airline", "origin", "arr_delay")
]

Marriage Data

The Marriage dataset comes from marriage records maintained by the Probate Court of Mobile County, Alabama. These records were originally collected as part of the ordinary administrative process of applying for and recording marriages, rather than for a particular research study. The records contain information from marriage applications and ceremonies, including dates, ages, previous marriages, education, and the title of the official who performed the ceremony. The version included in mosaicData contains 98 observations drawn from these public records.

The flights Data

The flights data contain records for all 336,776 flights that departed from New York City’s three major airports (JFK, LaGuardia, and Newark) during 2013. The flight records originate from the U.S. Bureau of Transportation Statistics and include scheduled and actual departure and arrival times, delays, airlines, flight numbers, origins, destinations, and other information about each flight. The data were therefore collected as transportation records rather than as part of a designed experiment or survey. The nycflights13 package assembled these records, together with related weather, airport, airline, and aircraft information, to make it possible to investigate questions such as when and why flights are delayed. (We’ll need more intricate stuff for that later)

Lab

Graphs for One Categorical Variable

A categorical distribution describes how observations are divided among categories. The most familiar display is a bar chart. Bar length encodes the number of observations in each category. Because the bars share a common scale and baseline, differences in frequency can usually be compared directly.

ggplot(Marriage, aes(x = officialTitle)) +
  geom_bar(fill = "steelblue") +
  labs(
    x = "Official title",
    y = "Count",
    title = "Marriage records by official title"
  ) +
  theme_minimal()

When category labels are long, a horizontal display can be easier to read. We can also reorder categories by frequency. Ordering changes the organization of the graph rather than the underlying data: the same counts are present, but large and small categories become easier to locate. This can be useful when magnitude is the important feature, although some categorical variables have a meaningful ordering that we would not want to replace.

ggplot(
  Marriage,
  aes(
    x = reorder(
      officialTitle,
      officialTitle,
      FUN = function(x) -length(x)
    )
  )
) +
  geom_bar(fill = "steelblue") +
  coord_flip() +
  labs(
    x = "Official title",
    y = "Count",
    title = "Marriage records ordered by frequency"
  ) +
  theme_minimal()

Question #1

Compare the two bar charts.

A. What becomes easier to see after the categories are ordered by frequency?

B. Describe a situation in which ordering categories from largest to smallest would not be a good choice.

C. What do you think the x = reorder(...) code is doing explicitly?

\(~\)

Bar charts emphasize comparisons among categories, but sometimes we want the display to emphasize that the categories together make up a whole. A pie chart represents category frequencies using angles and areas rather than aligned lengths. This changes what the viewer attends to: the entire circle has an immediate part-to-whole interpretation, while individual category sizes are encoded differently than they are in a bar chart.

officiant_counts = as.data.frame(table(Marriage$officialTitle))
names(officiant_counts) = c("officialTitle", "count")
officiant_counts = officiant_counts[officiant_counts$count > 0, ]

ggplot(
  officiant_counts,
  aes(x = "", y = count, fill = officialTitle)
) +
  geom_col(width = 1) +
  coord_polar(theta = "y") +
  labs(
    x = NULL,
    y = NULL,
    fill = "Official title",
    title = "Marriage officiants as parts of the observed whole"
  ) +
  theme_void() #Why void? =)

Question #2

Think about what would happen if the same categorical distribution were shown as a bar chart instead.

A. Which display would make it easier to distinguish two categories that are fairly close in frequency? Explain what you are using visually to make that comparison.

B. Which display more immediately emphasizes that the categories are pieces of one whole?

C. Recreate the pie chart by changing the geom_col layer to geom_col(width = 1, color="black"). Comment on any increased readability.

\(~\)

There are other ways to emphasize part-to-whole relationships. A treemap uses rectangular area. Treemaps can fit many categories into a compact space and can also represent hierarchical categories, although the viewer must compare areas rather than positions along a common axis.

library(treemapify)

ggplot(
  officiant_counts,
  aes(
    area = count,
    fill = officialTitle,
    label = officialTitle
  )
) +
  geom_treemap() +
  geom_treemap_text(
    place = "centre",
    reflow = TRUE
  ) +
  labs(
    fill = "Official title",
    title = "Marriage officiants represented with a treemap"
  ) +
  theme(legend.position = "none")

A waffle chart divides a whole into many equal-sized units. In this example, the observed proportions are converted into 100 squares, so each square represents approximately one percent of the observations. This can make approximate proportions intuitive, but it also requires us to transform the original counts into a different visual representation.

officiant_prop =
  officiant_counts$count / sum(officiant_counts$count)

officiant_squares =
  round(100 * officiant_prop)

# Adjust rounding so that the total is exactly 100 squares.
officiant_squares[which.max(officiant_squares)] =
  officiant_squares[which.max(officiant_squares)] +
  (100 - sum(officiant_squares))

waffle_data = data.frame(
  officialTitle = rep(
    officiant_counts$officialTitle,
    officiant_squares
  ),
  square = 1:100
)

waffle_data$row =
  ceiling(waffle_data$square / 10)

waffle_data$column =
  ((waffle_data$square - 1) %% 10) + 1

ggplot(
  waffle_data,
  aes(
    x = column,
    y = row,
    fill = officialTitle
  )
) +
  geom_tile(color = "white") +
  coord_equal() +
  scale_y_reverse() +
  labs(
    x = NULL,
    y = NULL,
    fill = "Official title",
    title = "Marriage officiants represented as 100 equal units"
  ) +
  theme_void()

Question #3

Pie charts, treemaps, and waffle charts all represent categories as parts of a whole, but they do so in different ways.

A. Which of the three makes approximate proportions easiest for you to judge? What feature of the graph helps?

B. Which would become difficult to use if the variable had a large number of categories? Explain.

\(~\)

You have now seen categorical frequencies represented using aligned bar lengths, angles, rectangular areas, and equal-sized units. None of these representations changes the underlying counts, but the graphical encoding changes which comparisons are visually prominent.

Question #4

A. Suppose your primary goal is to identify the largest category and compare categories that are fairly close in frequency. Which categorical display would you prefer? Which would you be least likely to use? Explain.

B. Suppose instead that your goal is to emphasize how categories divide up an entire collection of observations. Does your preferred display change? Explain.

C. The variable airline in flights_sample identifies the airline associated with each sampled flight. Construct one graph that you think clearly communicates which airlines make up larger and smaller portions of this sample. Choose the representation yourself and briefly explain why it is appropriate for that goal.

# Your code here

Graphs for One Quantitative Variable

For a quantitative variable, we are usually interested in its distribution: where observations tend to concentrate, how much variability is present, whether the distribution is symmetric or skewed, and whether unusual values occur. Different graphs preserve different pieces of this information.

In probability, the distribution of a continuous random variable can be described using a probability density function. A density function represents the relative concentration of probability across possible values. The height of the density at a particular value is not itself a probability. Instead, probabilities correspond to areas under the density curve, and the total area under a density function is one.

The following illustration shades the probability accumulated up to a particular value, \(x\).

The accumulated probability is described by the cumulative distribution function (CDF). For a continuous random variable \(X\) with density function \(f(x)\),

\[ F(x) = P(X \leq x) = \int_{-\infty}^{x} f(t)\,dt. \]

When we work with observed data, however, we generally do not know the underlying probability distribution. Later in this section we will use an empirical cumulative distribution function (ECDF), which builds a cumulative distribution directly from the observed sample.

Histograms

A histogram divides the range of a quantitative variable into intervals called bins and counts how many observations fall into each interval. Histograms maintain a fairly direct connection to observed frequencies, while still grouping nearby values together so that the overall shape of the distribution can emerge.

The grouping matters. The observations themselves do not change when we change the bin width, but the apparent shape of the histogram can.

ggplot(Marriage, aes(x = age)) +
  geom_histogram(
    binwidth = 5,
    boundary = 0,
    fill = "steelblue",
    color = "white"
  ) +
  labs(
    x = "Age",
    y = "Count",
    title = "Distribution of age"
  ) +
  theme_minimal()

Question #5

A. Describe the overall shape of the age distribution. Where do ages appear to be most concentrated? Are there relatively few or many observations at older ages?

B. Change binwidth = 5 to two substantially different values. What features of the histogram remain fairly stable? What features change?

C. Experiment with removing the boundary=0 argument. What happens? Why did I specify this for a histogram of this particular data?

\(~\)

Density Plots

A density plot replaces the bins of a histogram with a smooth estimate of the distribution. This can make broad patterns in shape easier to see and can be convenient when several distributions need to be compared. The curve is an estimate, however, rather than a direct display of individual observations.

The amount of smoothing is controlled by the bandwidth. A smaller bandwidth follows local changes in the data more closely, while a larger bandwidth smooths across more of those local changes. The two graphs below use the same observations.

library(patchwork)

density_small = ggplot(Marriage, aes(x = age)) +
  geom_density(
    bw = 2,
    fill = "steelblue",
    alpha = 0.4
  ) +
  labs(
    title = "Smaller bandwidth",
    x = "Age",
    y = "Density"
  ) +
  theme_minimal()

density_large = ggplot(Marriage, aes(x = age)) +
  geom_density(
    bw = 8,
    fill = "steelblue",
    alpha = 0.4
  ) +
  labs(
    title = "Larger bandwidth",
    x = "Age",
    y = "Density"
  ) +
  theme_minimal()

density_small + density_large

Question #6

Compare the two density plots.

A. How does increasing the bandwidth change the appearance of the distribution?

B. Identify a feature that is more visible with the smaller bandwidth and a feature that is clearer with the larger bandwidth.

C. What does this comparison suggest about interpreting every small bump or peak in a density curve as an important feature of the distribution?

\(~\)

Dotplots

A dotplot keeps the observations themselves visually connected to the display. Repeated or nearby observations stack, allowing us to see where the sample is concentrated without replacing it with a smooth curve. This directness is especially useful for smaller datasets. As the number of observations grows, however, individual dots can become crowded and the display can become difficult to read.

ggplot(Marriage, aes(x = age)) +
  geom_dotplot(
    binwidth = 2,
    dotsize = 0.7,
    fill = "steelblue"
  ) +
  labs(
    x = "Age",
    y = NULL,
    title = "Dotplot of age"
  ) +
  theme_minimal()

Question #7

Compare the dotplot with the histogram and density plots.

A. What can you see about the observed sample in the dotplot that is less direct in the density plot?

B. Imagine making this graph with 100,000 observations instead of the current dataset. What problem would you expect to encounter?

\(~\)

Empirical Cumulative Distribution Functions

An ECDF accumulates the observed values rather than displaying where they pile up. At a particular value \(x\), the height of the ECDF is the proportion of observations that are less than or equal to \(x\). The graph therefore rises from 0 toward 1 as we move through the observed values.

This makes threshold and percentile questions directly accessible from the graph. In exchange, familiar features of distributional shape such as peaks and modes are much less visually prominent.

ggplot(Marriage, aes(x = age)) +
  stat_ecdf(
    geom = "step",
    color = "steelblue",
    linewidth = 1
  ) +
  labs(
    x = "Age",
    y = "Cumulative proportion",
    title = "Empirical cumulative distribution of age"
  ) +
  theme_minimal()

Question #8

A. Approximately what proportion of the observations have age at or below 30? Explain how you used the ECDF to determine this.

B. Suppose you wanted to find an age such that approximately 75% of the observations were at or below that age. Describe how you would use the ECDF to find it.

C. Why would the ECDF be less useful than a histogram or density plot if your main interest were identifying peaks in the distribution?

\(~\)

A histogram, density plot, dotplot, and ECDF can all be constructed from the same quantitative variable, but they do not make the same features equally visible. Histograms group observations into bins, density plots smooth across observations, dotplots retain a close connection to individual observations, and ECDFs accumulate observations from left to right.

Question #9

A. For each goal below, identify the display or displays you would consider using. More than one answer may be reasonable. Explain your reasoning.

1. Describing the overall shape of a distribution.
2. Showing the actual observed values in a relatively small dataset.
3. Estimating the proportion of observations at or below a particular value.
4. Communicating where values tend to be concentrated to a general audience.

B. Arrival delay, arr_delay, is much less well-behaved than age. A negative value indicates an early arrival, while a positive value indicates a late arrival. Suppose you want a reader to understand where arrival delays are concentrated and how common very long delays are.

Construct one visualization of arr_delay that you think is useful for this purpose. Choose the representation yourself rather than simply reproducing one of the examples. Briefly explain what your graph reveals and why you chose it.

# Your code here

Graphs for Comparing Quantitative Distributions Across Groups

Often we want to compare a quantitative variable across categories. There is no single graph that preserves every feature of each distribution equally well. Some displays summarize aggressively so that groups are easy to compare, while others retain more information about shape or individual observations.

Boxplots

A boxplot compresses each distribution into a small set of summaries. The line inside the box marks the median, the box spans the middle 50% of the observations, and the whiskers and plotted points help identify observations farther from the central portion of the distribution.

Because every group is summarized in the same way, boxplots make centers and spreads convenient to compare across several groups. The compression is also their main limitation: distributions with noticeably different shapes can sometimes produce similar-looking boxes.

ggplot(flights_sample, aes(x = airline, y = arr_delay)) +
  geom_boxplot() +
  labs(
    x = "Airline",
    y = "Arrival delay (minutes)",
    title = "Arrival delays by airline"
  ) +
  theme_minimal()

Question #10

Use the boxplots to compare the airlines.

A. Which airlines appear to have higher or lower typical arrival delays?

B. What can you learn about variability from these boxplots?

C. What can you not determine very well about the shapes of the five distributions from the boxplots alone?

\(~\)

Violin Plots

A violin plot uses a mirrored density estimate for each group. Wider portions of the violin correspond to values where observations are more concentrated. Unlike a boxplot, the violin therefore preserves information about distributional shape. It does not, by default, display summary values such as the median as directly as a boxplot does.

ggplot(flights_sample, aes(x = airline, y = arr_delay)) +
  geom_violin(
    fill = "steelblue",
    alpha = 0.5
  ) +
  labs(
    x = "Airline",
    y = "Arrival delay (minutes)",
    title = "Arrival-delay distributions by airline"
  ) +
  theme_minimal()

Question #11

Compare the violin plots with the boxplots.

A. Identify something about the airline distributions that is easier to see in the violin plots.

B. Identify something that the boxplots communicate more directly.

C. If two groups had similar medians and quartiles but noticeably different distributional shapes, which of these displays would be more likely to reveal the difference? Explain.

\(~\)

Showing More of the Distribution

Boxplots and violin plots both transform the observed data in some way. A boxplot summarizes the observations using a small collection of statistics, while a violin plot replaces the observations with a smooth estimate of their distribution. Sometimes we may want to retain more of the original data or combine several kinds of information in the same display.

Jittered Observations

One possibility is simply to plot the observations themselves. Because many flights have the same or similar delays, ordinary points would overlap heavily. Jittering moves points slightly sideways so that more observations become visible.

This preserves information that a boxplot or density estimate discards, including the presence of individual observations and extreme values. With thousands of observations, however, the points can still overlap substantially and the amount of visual information can become difficult to process.

ggplot(flights_sample, aes(x = airline, y = arr_delay)) +
  geom_jitter(
    width = 0.18,
    alpha = 0.15,
    size = 0.8
  ) +
  labs(
    x = "Airline",
    y = "Arrival delay (minutes)",
    title = "Individual sampled flights by airline"
  ) +
  theme_minimal()

Comparing Density Curves

Instead of displaying individual observations, we can place the density estimates for several groups on a common coordinate system. Because all of the curves use the same quantitative axis, we can directly compare where the distributions concentrate and how their shapes differ.

This works well when there are only a few groups whose curves remain visually distinguishable. As the number of groups increases, however, overlapping curves and colors can become difficult to trace.

ggplot(flights_sample, aes(x = arr_delay, color = airline)) +
  geom_density(linewidth = 1) +
  labs(
    x = "Arrival delay (minutes)",
    y = "Density",
    color = "Airline",
    title = "Arrival-delay density estimates"
  ) +
  theme_minimal()

Combining Layers

Because ggplot2 builds graphs in layers, we do not always have to choose between a summary and the observations themselves. For example, a boxplot can be combined with jittered observations.

ggplot(flights_sample, aes(x = airline, y = arr_delay)) +
  geom_jitter(
    width = 0.18,
    alpha = 0.10,
    size = 0.7
  ) +
  geom_boxplot(
    width = 0.45,
    outlier.shape = NA,
    alpha = 0.5
  ) +
  labs(
    x = "Airline",
    y = "Arrival delay (minutes)",
    title = "Arrival delays: observations and summaries"
  ) +
  theme_minimal()

The combined display preserves several kinds of information at once: the boxplot provides a compact summary while the points retain a connection to the individual observations. That additional information also increases the visual complexity of the graph. Adding more information to a visualization does not necessarily make the information easier to understand.

Question #12

Compare the three displays above with the boxplots and violin plots you saw earlier.

A. Which display gives you the clearest connection to the individual sampled flights? What information does it retain that the other displays do not?

B. Which display gives you the clearest view of the overall shapes of the five distributions? Why?

C. The combined boxplot and jitter display contains more kinds of information than the ordinary boxplot. Describe one advantage and one disadvantage of combining the two layers.

D. Suppose the dataset contained 100,000 flights instead of 6,000. Which of these displays do you think would become substantially less useful? Explain why.

\(~\)

The graphs in this section illustrate a recurring tradeoff. Boxplots provide compact summaries of center and spread. Violin and density plots preserve more information about distributional shape. Plotting individual observations retains still different information, while layered displays can combine several representations at the cost of additional visual complexity.

Choosing among them therefore depends on what we want the reader to be able to compare.

Question #13

A. Suppose your primary goal were each of the following. Which representation from this section would you consider using, and why?

1. Comparing typical arrival delays among several airlines.
2. Comparing the shapes of the airline delay distributions.
3. Drawing attention to individual unusually delayed flights.
4. Showing both a compact summary and the observed data.

More than one choice may be reasonable for some of these goals.

B. The variable origin identifies whether a sampled flight departed from EWR, JFK, or LGA. Suppose you want to investigate whether the typical arrival delay and the variability in arrival delay differ among these three airports.

Construct one graph that you think communicates this comparison effectively. Choose the representation yourself and briefly justify your choice.

# Your code here

When the Data Fight the Graph

Arrival delays have a long right tail: most flights are relatively close to their scheduled arrival time, while a small number are delayed by several hours. This creates a genuine visualization problem. If the graph includes every extreme value, the central mass of the data can be compressed into a small part of the plotting area. If we zoom in, the typical observations become easier to distinguish, but the most extreme delays are no longer visible.

Neither choice automatically makes the graph correct or incorrect. What matters is understanding what the display allows the reader to see and what it leaves out.

ggplot(flights_sample, aes(x = airline, y = arr_delay)) +
  geom_boxplot() +
  labs(
    x = "Airline",
    y = "Arrival delay (minutes)",
    title = "Full range of sampled arrival delays"
  ) +
  theme_minimal()

The following graph uses the same data but changes the visible range. coord_cartesian() zooms the display without deleting the observations before the boxplots are calculated.

ggplot(flights_sample, aes(x = airline, y = arr_delay)) +
  geom_boxplot() +
  coord_cartesian(
    ylim = c(-60, 180)
  ) +
  labs(
    x = "Airline",
    y = "Arrival delay (minutes)",
    title = "A closer view of the central arrival-delay distributions"
  ) +
  theme_minimal()

Question #14

Compare the full-range and zoomed boxplots.

A. What becomes easier to see after zooming?

B. What information becomes less apparent?

C. If you presented the zoomed graph by itself, what would a reader need to know about the choice you made?

\(~\)

Faceting gives us another way to separate groups. With fixed scales, every panel uses the same axes. This maintains a common visual reference across panels. Allowing a scale to vary gives each panel more freedom to display its own data, which can reveal within-group structure that would otherwise occupy little of the plotting region. The tradeoff is that the panels no longer provide the same visual reference for every comparison.

ggplot(flights_sample, aes(x = arr_delay)) +
  geom_histogram(
    binwidth = 15,
    fill = "steelblue",
    color = "white"
  ) +
  facet_wrap(~airline) +
  labs(
    x = "Arrival delay (minutes)",
    y = "Count",
    title = "Faceted arrival-delay histograms: fixed scales"
  ) +
  theme_minimal()

ggplot(flights_sample, aes(x = arr_delay)) +
  geom_histogram(
    binwidth = 15,
    fill = "steelblue",
    color = "white"
  ) +
  facet_wrap(
    ~airline,
    scales = "free_y"
  ) +
  labs(
    x = "Arrival delay (minutes)",
    y = "Count",
    title = "Faceted arrival-delay histograms: free vertical scales"
  ) +
  theme_minimal()

Question #15

A. What comparisons are easier when the facets use the same vertical scale?

B. What becomes easier to see when each panel is allowed to use its own vertical scale?

C. Why would you want to know whether a faceted graph used fixed or free scales before comparing the heights across panels?

\(~\)

Finally, a ridgeline plot places several density estimates in parallel. It separates the groups vertically while preserving a common quantitative axis, avoiding some of the line-crossing and color-tracing problems that can occur when many density curves are drawn directly on top of one another. Like other density displays, it still represents smoothed estimates rather than individual observations.

library(ggridges)

ggplot(
  flights_sample,
  aes(x = arr_delay, y = airline)
) +
  geom_density_ridges(
    scale = 1.1,
    alpha = 0.6
  ) +
  labs(
    x = "Arrival delay (minutes)",
    y = "Airline",
    title = "Arrival-delay distributions as ridgelines"
  ) +
  theme_minimal()

Question #16

Compare the ridgeline plot with the overlaid density curves from earlier.

A. What becomes easier to follow when the density estimates are separated into ridges?

B. What comparison might be easier when the curves are instead drawn directly on the same coordinate system?

\(~\)

Looking Back

Today you encountered many more graphical forms than you were asked to construct. That is intentional. Knowing how to visualize data does not mean memorizing a separate command for every possible graph. It means recognizing the kinds of information different displays preserve, understanding the compromises they make, and being able to choose a representation that fits the question you are trying to answer.

Question #17

Imagine that you are preparing one figure for a short report comparing arrival-delay distributions among the five airlines in flights_sample. You may choose any representation you encountered today.

A. Which representation would you choose, and what would you want the reader to notice?

B. What feature of the data would be less visible, summarized away, or otherwise harder to judge in your chosen graph?

C. If a classmate chose a different representation, what would make their choice reasonable even if you prefer your own?

You do not need to construct another graph for this question.

On the next day of the lab, we will move from distributions of individual variables to relationships between variables. As the lab progresses, you will receive less guidance about which graphical form to use and will increasingly make that choice yourself.