Class Introduction

Syllabus Link

Important notes:

Attendance is important. Probably the 2nd most important thing in this class. Continual repeated exposure to programming is one of the best ways of learning, and we will have a fair amount of programming ahead of us.

Class Format

Most classes will have the following format:

5-15 minutes: Class announcements, preamble to lab (together)
Work on Lab (solo or with partner)
(occasional) 0-5 minutes: Class wrap up, reminder of goals/what you should have learned

Notes:

This semester I am experimenting with breaking up and reorganizing content from previous semesters. Some labs will span multiple class periods. I will try to give you an idea of how far you should be at the end of class.
We will (usually) only go over the preamble on the first day of a lab.
If you don’t finish a lab in class, you will need to finish it outside of class. You can finish the lab on your own or work with other students in the course (each student still needs to turn in their own lab)

Preamble

Today’s preamble will be longer than usual in order to review important parts of R as a class.

Goals

Introduce the class and class format
Refresh your memory of R and Rstudio
Remind you of the differences between an R script and an R markdown file
Introduce additional miscellaneous topics that will make working in R better

The Layout of R Studio

After you open RStudio, the first thing you’ll want to do is open a file to work in. You can do this by navigating: File -> New File -> RScript, which will open a new window in the top left of the RStudio interface for you to work in. At this point you should see four panels:

Your R Script (top left)
The Console (bottom left)
Your Environment (top right)
The Files/Plots/Help viewer (bottom right)

An R Script is like a text-file that stores your code while you work on it. At any point you can send some or all of the code in your R Script to the Console to execute. You can also type commands directly into the Console. The Console will echo any code you run, and it will display any textual/numeric output generated by your code.

The Environment shows you the names of data sets, variables, and user-created functions that have been loaded into your work space and can be accessed by your code. The Files/Plots/Help Viewer will display graphics generated by your code and a few other useful entities (like help documentation and file trees).

Packages

To facilitate more complex tasks in R, many people have developed their own sets of functions known as packages. If you plan on working with a new package for the first time, it must be installed:

install.packages("ggplot2")

Once a package is installed, it still needs to be loaded into your R session using the library() function (or require()) before its contents can be used.

You’ll need to re-load a package every time you open R Studio, but you’ll only need to install it on your computer once.

my_data <- read.csv("https://remiller1450.github.io/data/HappyPlanet.csv")
library(ggplot2)
qplot(my_data$Region)

## `stat_bin()` using `bins = 30`. Pick better value `binwidth`.

R scripts vs R markdown

R Scripts are built to contain only executable R code and comments.

R Studio supports several other types of files, some of which use the “Markdown” authoring framework. An “R Markdown” file allows you to both:

Write and execute R code
Generate a high quality, reproducible report

To use R Markdown, you’ll need the rmarkdown package. In order to knit to pdf instead of html, you will likely need the tinytex package:

install.packages("rmarkdown")
library("rmarkdown")
install.packages("tinytex")
tinytex::install_tinytex()

Once you have the package installed and loaded, you can create a new R Markdown file by selecting: File -> New File -> R Markdown.

At the top of the document is the header:

This section initiated by three ‘-’ characters and closed by another three ‘-’ characters
It contains the title, author, etc. that appears at the top of the document created by your code
You can use it to add elements like a table of contents, page numbers, etc.

The second thing you’ll see is a code chunk:

Code chunks are initiated by ```{r} and closed by ```
The ``` wrappers tell R Markdown that what appears inside is code that should be executed. The first code chunk, initiated by ```{r setup} sets up options that will be used in executing your R code when your report is built. For now, you should keep this chunk as it appears and place your actual code inside of other code chunks.
You can execute the R code in a chunk by clicking the small green arrow in the upper right corner. You can also highlight individual code pieces and execute them using Ctrl-Enter.

Next you’ll see section headers:

Sections are created using strings of the # character.
The number of # characters used determines the level (size) of the header.

Finally, R Markdown allows you to type ordinary text outside of code chunks. Thus, you can easily integrate written text into the same document as your code and its output.

The primary purpose of R Markdown is to create documents that blend R code, output, and text into a polished report. To generate this document you must compile your R Markdown file using the “Knit” button (a blue yarn ball icon) located towards the upper left part of your screen.

The R Markdown cheat sheet can be found here: https://raw.githubusercontent.com/rstudio/cheatsheets/main/rmarkdown.pdf

Question 0: Create a new R Markdown file and delete all of the template code that appears beneath the “r setup” code block. Change the title to “Lab #1” and the author to your name(s). Next, create section labels for each question in the lab using three # characters followed by “Question X” (where X is the number of the question).

Using R

R is an interpreted programming language, which allows you to have the computer execute any piece of code contained your R Script at any time without a lengthy compiling process.

To run a single piece of code, simply highlight it and either hit Ctrl-Enter or click on the “Run” button near the top right corner of your R Script. You should see an echo of the code you ran in the Console, along with any response generated by that code.

4 + 6 - (24/6)

## [1] 6

5 ^ 2 + 2 * 2

## [1] 29

The examples shown above demonstrate how R can be used as a calculator. However, most of code we will write will rely upon functions, or pre-built units of code that translate one or more inputs into one or more outputs.

log(x = 4, base = 2)

## [1] 2

The example above demonstrates the log() function. The input named “x” is set to be 4, and the input named “base” is set to 2. The labels given to these inputs, “x” and “base”, are the function’s arguments. The function returns the output “2”, which is \(\log_2{(4)}\). Note that log(4, 2) will also produce the output “2” as any unlabeled inputs are mapped to arguments in the order defined by the creator of the function.

Help Documentation

You’ll eventually end up memorizing the arguments of common R functions; however, while you’re learning I strongly encourage you to read the help documentation for any R function used in your code. You can access a function’s documentation by typing a ? in front of the function name and submitting to the console.

?log

In addition, if you think there should be a function but you don’t know what it is called, you can use two ‘??’:

??mean

Adding Comments

When coding, it is good practice to include comments that describe what your code is doing. In R the character “#” is used to start a comment. Everything appearing on the same line to the right of the “#” will not be executed when that line is submitted to the console.

# This entire line is a comment and will do nothing if run

1:6 # The command "1:6" appears before this comment

## [1] 1 2 3 4 5 6

In your R Script, comments appear in green. You also should remember that the “#” starts a comment only for a single line of your R Script, so long comments requiring multiple lines should each begin with their own “#”.

Lab

The remainder of the lab is to be completed by you . You should work at a comfortable pace that ensures you thoroughly understand the lab’s contents and examples.

Loading Data

An important part of data science is reproducibility, or the ability for two people to independently replicate the results of a project.

To help make an analysis reproducible, raw data should be imported and manipulated using documented code rather than relying on manual changes or the point-and-click “Import Dataset” interface.

Below are two different examples:

## Loading a CSV file from a web URL (storing it as "my_data")
my_data <- read.csv("https://some_webpage/some_data.csv")

## Loading a CSV file with a local file path
my_data <- read.csv("H:/path_to_my_data/my_data.csv")

A few things to note.

Both <- or = can be used to assign something to a named object. We can use the two interchangeably, it’s mostly a stylistic choice.
File paths must use / or \\. A single \ is used by R to start an instance of a special text character. For example, \n creates a new line in a string of text.

Question 1A: Add code to your script that uses the read.csv() function to create an object named my_data that contains the “Happy Planet” data stored at: https://remiller1450.github.io/data/HappyPlanet.csv

After running your Question 1A code, an entry named “my_data” should appear in the Environment panel (top right).

You can click on the small arrow icon to reveal the data’s structure, or you can click on the object’s name to view the data in spreadsheet format.

Question 1B: Inspect the structure of my_data and view the data set in spreadsheet format. In an R comment, briefly describe how this data set is structured (ie: what does each row and column represent, what are some of the columns, etc.)

Objects and Assignments

R stores data in containers called objects. Data is assigned into an object using <- or =. After assignment, data can be referenced using the object’s name. The simplest objects are scalars, or a single element:

x = 5 # This assigns the integer value '5' to an object called 'x'
x^2    # We can now reference 'x'

## [1] 25

R stores ordered sequences of elements of the same data type in an object called a vector:

x <- 1:3 # The sequence {1, 2, 3} is assigned to the vector called 'x'
print(x)

## [1] 1 2 3

y <- c(1,2,3) # The function 'c' concatenates arguments (separated by commas) into a vector
print(y)

## [1] 1 2 3

z <- c("A","B","C") # Vectors can contain many types of values
print(z)

## [1] "A" "B" "C"

Note: scalars are actually vectors of length 1 =)

The three most important types of vectors are:

numeric vectors - for example: x = c(1,2,3)
character vectors - for example: x = c("A","B","C")
logical vectors - for example: x = c(TRUE, FALSE, TRUE)

You should always consider a vector’s type before using it. Many functions expect specific input types and will produce an error if the wrong type is used. You can check the type of an object using the typeof() function:

chars <- c("1","2","3") # Create a character vector
typeof(chars)

## [1] "character"

nums <- c(1,2,3) # Create a numeric vector
typeof(nums)

## [1] "double"

mean(chars) # This produces an error, mean() only works for numeric vectors

## Warning in mean.default(chars): argument is not numeric or logical: returning
## NA

## [1] NA

mean(nums) # This works as intended

## [1] 2

Errors are a normal part of working in R. When an error occurs, read the error message before changing your code. R will often tell you what operation failed and give you a clue about why it failed.

For example, look at the error produced by mean(chars). What does the error message tell you about why R cannot calculate the mean of this object?

Certain R functions are vectorized, meaning they can accept a scalar input, or they can accept a vector input. The benefit of vectorized functions is that we can quickly apply the function to many values at the same time.

For example, sqrt() is vectorized, meaning I can take the square root of a whole bunch of numbers at once, as long as I store them all in the same vector.

sqrt(1) # works just fine on a scalar value

## [1] 1

nums <- c(1,2,3,4)
sqrt(nums) # also works on all values in my vector

## [1] 1.000000 1.414214 1.732051 2.000000

Data is usually stored in objects called data.frames, which are composed of several vectors of the same length:

DF <- data.frame(A = x, B = y, C = z) # Creates a data.frame object 'DF'
print(DF)

##   A B C
## 1 1 1 A
## 2 2 2 B
## 3 3 3 C

Functions like read.csv() will automatically store their output as a data frame:

my_data <- read.csv("https://remiller1450.github.io/data/HappyPlanet.csv")
typeof(my_data)

## [1] "list"

However, notice typeof() describes my_data as a list object. Lists are a flexible class of objects whose elements can be of any type. A data frame is actually a special type of a list, where each component is a vector represent a column (variable) of data.

Shown below is an example list containing three components, two different data frames and the character string “ABC”:

my_list <- list(my_data, DF, "ABC")

Question 2: Create a data frame named my_DF containing two vectors, J and K, where J is created using the seq function to be a sequence from 0 to 100 counting by 10 and K is created using the rep function to replicate the character string “XYZ” the proper number of times. Hint: read the help documentation for each function (seq and rep) to determine the necessary arguments.

Indexing

Suppose we have a vector “x” and would like to extract the element in its second position and assign it to a new object called “b”:

x <- 5:10
b <- x[2]
b

## [1] 6

The square brackets, [ and ], are used to access a certain position (or multiple positions) within an object. In this example we access the second position of the object “x”. Note that R is a 1-indexed language not a 0-indexed language (e.g. C). The first index is 1.

Some objects, such as data frames, have multiple dimensions, requiring indices in each dimension (separated by commas) to describe a single element. A few examples are shown below:

DF <- data.frame(x = x, y = y, z = z) 
DF[2,3] # The element in row 2, column 3

## [1] "B"

DF[2,] # Everything in row 2

##   x y z
## 2 6 2 B

For list objects, double square brackets [[ ]] can be used to extract an individual component from the list.:

my_list[[2]] ## The 2nd component of the list

##   A B C
## 1 1 1 A
## 2 2 2 B
## 3 3 3 C

Question 3: Use indices to print the Happiness score (column #3) of Hong Kong (row #57) in the object my_data (the Happy Planet data from Question 1). Be sure your code does not print any other information about this observation.

Working with Data Frames

Suppose we want to access a single variable from a data set, there are a few different ways we can do so:

# The $ accesses the component named 'Country' within 'my_data'
countries <- my_data$Country  

# Position indexing to access the variable 'Country' (since its the first column)
countries2 <- my_data[,1] 

# Use the name of the variable in place of an index position
countries3 <- my_data[,'Country']

Suppose we want to access a single observation (data point) in our dataset:

Albania <- my_data[1,] # This stores the entire first row

Suppose we want a range of observations:

FirstFive <- my_data[1:5,] # This stores the first five rows
head(FirstFive)

##     Country Region Happiness LifeExpectancy Footprint  HLY   HPI HPIRank
## 1   Albania      7       5.5           76.2       2.2 41.7 47.91      54
## 2   Algeria      3       5.6           71.7       1.7 40.1 51.23      40
## 3    Angola      4       4.3           41.7       0.9 17.8 26.78     130
## 4 Argentina      1       7.1           74.8       2.5 53.4 58.95      15
## 5   Armenia      7       5.0           71.7       1.4 36.1 48.28      48
##   GDPperCapita   HDI Population
## 1         5316 0.801       3.15
## 2         7062 0.733      32.85
## 3         2335 0.446      16.10
## 4        14280 0.869      38.75
## 5         4945 0.775       3.02

The head() function prints the first few rows of a data frame (default = 6 rows). The tail() functions prints the last few rows (also default = 6 rows).

Here are a few other functions you might use when working with a new data set:

dim(my_data) # prints the dimensions of 'my.data'

## [1] 143  11

nrow(my_data) # prints the number of rows of 'my.data'

## [1] 143

ncol(my_data) # prints the number of columns of 'my.data'

## [1] 11

colnames(my_data) # prints the names of the variables (columns) of 'mydata'

##  [1] "Country"        "Region"         "Happiness"      "LifeExpectancy"
##  [5] "Footprint"      "HLY"            "HPI"            "HPIRank"       
##  [9] "GDPperCapita"   "HDI"            "Population"

Another useful function is str(), which displays the structure of an object. For a data frame, str() displays its dimensions along with the name, type, and some of the values of each variable. It is useful as a time saver for investigating all of these quickly and recreates the data information you would see in the Environment tab.

str(my_data)

## 'data.frame':    143 obs. of  11 variables:
##  $ Country       : chr  "Albania" "Algeria" "Angola" "Argentina" ...
##  $ Region        : int  7 3 4 1 7 2 2 7 5 7 ...
##  $ Happiness     : num  5.5 5.6 4.3 7.1 5 7.9 7.8 5.3 5.3 5.8 ...
##  $ LifeExpectancy: num  76.2 71.7 41.7 74.8 71.7 80.9 79.4 67.1 63.1 68.7 ...
##  $ Footprint     : num  2.2 1.7 0.9 2.5 1.4 7.8 5 2.2 0.6 3.9 ...
##  $ HLY           : num  41.7 40.1 17.8 53.4 36.1 63.7 61.9 35.4 33.1 40.1 ...
##  $ HPI           : num  47.9 51.2 26.8 59 48.3 ...
##  $ HPIRank       : int  54 40 130 15 48 102 57 85 31 104 ...
##  $ GDPperCapita  : int  5316 7062 2335 14280 4945 31794 33700 5016 2053 7918 ...
##  $ HDI           : num  0.801 0.733 0.446 0.869 0.775 0.962 0.948 0.746 0.547 0.804 ...
##  $ Population    : num  3.15 32.85 16.1 38.75 3.02 ...

Question 4A: Write code that prints the last ten observations (countries) that appear in the Happy Planet data.

Question 4B: Write code that finds the values of the “LifeExpectancy” variable for the last 10 observations in the Happy Planet data.

Question 4C: Write code that gives me both the country names and values of the “LifeExpectancy” variable for the last 10 observations in the Happy Planet data.

Data Summaries

Descriptive summaries are an essential component of any data analysis. A few functions used to calculate several basic numerical summaries are shown below:

mean(my_data$LifeExpectancy) # mean

## [1] 67.83846

sd(my_data$LifeExpectancy) # standard deviation

## [1] 11.04193

min(my_data$LifeExpectancy) # minimum

## [1] 40.5

max(my_data$LifeExpectancy ) # maximum

## [1] 82.3

quantile(my_data$LifeExpectancy, probs = .35) # the 35th percentile

##   35% 
## 66.18

Each of these functions operates on a single variable. For a broader set of summary statistics, you can input an entire data frame into the summary() function:

summary(my_data)

##       Country        Region        Happiness     LifeExpectancy 
##  Length   :143   Min.   :1.000   Min.   :2.400   Min.   :40.50  
##  N.unique :143   1st Qu.:2.000   1st Qu.:5.000   1st Qu.:61.90  
##  N.blank  :  0   Median :4.000   Median :5.900   Median :71.50  
##  Min.nchar:  4   Mean   :3.832   Mean   :5.919   Mean   :67.84  
##  Max.nchar: 24   3rd Qu.:6.000   3rd Qu.:7.000   3rd Qu.:76.05  
##                  Max.   :7.000   Max.   :8.500   Max.   :82.30  
##                                                                 
##    Footprint           HLY             HPI           HPIRank     
##  Min.   : 0.500   Min.   :11.60   Min.   :16.59   Min.   :  1.0  
##  1st Qu.: 1.300   1st Qu.:31.10   1st Qu.:34.47   1st Qu.: 36.5  
##  Median : 2.200   Median :41.80   Median :43.60   Median : 72.0  
##  Mean   : 2.877   Mean   :41.38   Mean   :43.38   Mean   : 72.0  
##  3rd Qu.: 3.850   3rd Qu.:53.20   3rd Qu.:52.20   3rd Qu.:107.5  
##  Max.   :10.200   Max.   :66.70   Max.   :76.12   Max.   :143.0  
##                                                                  
##   GDPperCapita        HDI           Population      
##  Min.   :  667   Min.   :0.3360   Min.   :   0.290  
##  1st Qu.: 2107   1st Qu.:0.5790   1st Qu.:   4.455  
##  Median : 6632   Median :0.7720   Median :  10.480  
##  Mean   :11275   Mean   :0.7291   Mean   :  44.145  
##  3rd Qu.:15711   3rd Qu.:0.8680   3rd Qu.:  31.225  
##  Max.   :60228   Max.   :0.9680   Max.   :1304.500  
##  NAs    :2       NAs    :2

For categorical variables frequency tables are often much more informative than this summary() output.

A one-way frequency table shows the frequencies of categories in a single categorical variable, while a two-way frequency table shows the relationship between two categorical variables. Both are created by the table() function:

# one-way table for Region frequencies
table(my_data$Region)

## 
##  1  2  3  4  5  6  7 
## 24 24 16 33  7 12 27

# two-way table for the year and mode of residence for a sample of students
year <- c("First", "First", "First", "Second", "Second",
          "Second", "Third", "Third", "Third", "Third")

residence <- c("Campus", "Campus", "Off Campus",
               "Campus", "Off Campus", "Off Campus",
               "Campus", "Campus", "Off Campus", "Off Campus")

table(year, residence)

##         residence
## year     Campus Off Campus
##   First       2          1
##   Second      1          2
##   Third       2          2

Tables are their own type of object, and they can be used as an input to functions like barplot():

my_table <- table(my_data$Region) # Tables can be stored as objects
barplot(my_table) # Creates a bar plot from a table

Frequency tables can also be converted to proportions using the prop.table() function. For a one-way table, calling prop.table() without any additional arguments gives the proportion of observations belonging to each category:

prop.table(my_table)

## 
##          1          2          3          4          5          6          7 
## 0.16783217 0.16783217 0.11188811 0.23076923 0.04895105 0.08391608 0.18881119

For a two-way table, we may instead want to calculate proportions within each row or column. The margin argument determines which totals are used to calculate these proportions. Setting margin = 1 calculates proportions within each row, while margin = 2 calculates proportions within each column:

# recreate data table of students: years, residence
student_table = table(year, residence)
# Proportion of students within each year
prop.table(student_table, margin = 1)

##         residence
## year        Campus Off Campus
##   First  0.6666667  0.3333333
##   Second 0.3333333  0.6666667
##   Third  0.5000000  0.5000000

# Proportion of students within each residence type
prop.table(student_table, margin = 2)

##         residence
## year     Campus Off Campus
##   First     0.4        0.2
##   Second    0.2        0.4
##   Third     0.4        0.4

Notice that the proportions across each row sum to 1 when margin = 1, while the proportions down each column sum to 1 when margin = 2.

Question 5: Find the mean, median, and range (maximum - minimum) of the variable LifeExpectancy in the Happy Planet data. Without graphing the distribution, Briefly comment on whether the distribution of this variable seems to be symmetric or skewed using plain text beneath your answer’s code chunk.

Coercion

Earlier we introduced three important types of vectors:

numeric vectors - for example: x = c(1,2,3)
character vectors - for example: x = c("A","B","C")
logical vectors - for example: x = c(TRUE, FALSE, TRUE)

Many functions require their inputs be of a certain type. Fortunately, data can be coerced into another type using the as. family of functions:

## A character vector where the text strings are numbers
x <- c("1","12","123")
typeof(x)

## [1] "character"

## Coerce 'x' to a numeric vector
x <- as.numeric(x)
x

## [1]   1  12 123

typeof(x)

## [1] "double"

Question 6: Coerce the variable “Region” into a character variable. Use the typeof function to verify the change. Hint: you should overwrite the “Region” vector within “my_data” as part of this question.

Missing Data

Real data sometimes contain missing values, which R stores as the special element NA. Missing values may be present in your raw data, but they can also be introduced by coercion or other operations/functions:

## The second element is a blank space
x <- c("1"," ","123")
typeof(x)

## [1] "character"

## Coerce to a numeric vector (stored as 'y'), notice the NA
y <- as.numeric(x)
y

## [1]   1  NA 123

Missing values can cause problems for many functions, but some functions have arguments that control how missing values are handled. The example below shows how to remove any missing values when calculating the mean of y:

mean(y) ## Doesn't handle the missing value

## [1] NA

mean(y, na.rm = TRUE) ## calculates mean while ignoring missing values

## [1] 62

Another useful function is na.omit(), which will subset a data frame to remove any rows that contain missing data in any variable. This function is demonstrated on the Happy Planet data below:

## Store the subset without missing data
my_data_without_na <- na.omit(my_data)

## Compare dimensions
dim(my_data)

## [1] 143  11

dim(my_data_without_na)

## [1] 141  11

Question 7: Find the median value of the variable “GDPperCapita” in the Happy Planet data, removing any missing values in this variable if necessary. Report the country names corresponding to any missing values that you removed (if applicable).

After completing Question 7 you are finished witht he lab. Instructions for submission will be provided to you at some point by Mr. Friedrichsen

R Basics – Refresher with a bit of new stuff

Nathan Friedrichsen