---
title: "<br/>Hands-on data exploration using R <br/>"
#subtitle: "A tidyverse approach"
author: "Sebastian Sauer"
date: "Last update: `r Sys.Date()`"
output:
  xaringan::moon_reader:
    css: ["ses-xa.css","default", "default-fonts", "hygge"]
    #css: [default, metropolis, metropolis-fonts]
    lib_dir: libs
    nature:
      highlightStyle: github
      highlightLines: true
      ratio: '16:9'
      countIncrementalSlides: false
      titleSlideClass: [center, right, inverse]
    seal: false
editor_options: 
  chunk_output_type: console
---



```{r setup-knitr, include=FALSE}
options(htmltools.dir.version = FALSE)


knitr::opts_chunk$set(
  comment = "#>",
  collapse = TRUE,
  message = FALSE,
  warning = FALSE,
  cache = TRUE,
  echo = TRUE,
  out.width = "70%",
  fig.align = 'center',
  fig.width = 6,
  fig.asp =  .4,
  fig.show = "hold",
  size = "tiny",
  dpi=300
)


options(max.print = 999999,
        tibble.print_max = 300,
        tibble.print_min = 3)
```




```{r setup, echo = FALSE}
path_img <-  "https://raw.githubusercontent.com/sebastiansauer/modar/master/static/images/"
```




class: inverse

<br/>
<br/>

.right[
# Hands-on data exploration using R 

<br/>

### Sebastian Sauer
]



```{r out.width = "20%", fig.align='left', echo = FALSE}
knitr::include_graphics("https://upload.wikimedia.org/wikipedia/commons/thumb/0/0e/Hochschule_f%C3%BCr_Oekonomie_%26_Management_2012_logo.svg/768px-Hochschule_f%C3%BCr_Oekonomie_%26_Management_2012_logo.svg.png")
```


.right[
.tiny[
last update: `r Sys.Date()` 
]]


---






class: inverse, center, middle

# Setup

---

class: top, left

## Overview



.Large[
.full-width[
.content-box-blue[

- Setup

- Tidyverse 101

- Data diagrams 101

- Case study
]
]
]



---


## whoami

```{r}
 system("whoami")
```



- R enthusiast

- Data analyst/scientist

- Professor at FOM Hochschule

---

## The lights are on

.center[

```{r echo = FALSE, fig.asp = 0.25}
library(leaflet)
popup = c("Sebastian", "Datanatives")
leaflet(width = "100%") %>%
  setView(lng = 12, lat = 50, zoom = 05) %>%
  addProviderTiles("NASAGIBS.ViirsEarthAtNight2012") %>%
  addMarkers(lng = c(13.4, 11.1),
             lat = c(52.5, 49.4), 
             popup = popup)
```
]


---

## Upfront preparation


### Please install the following software upfront:

- [R](https://www.r-project.org/)
- [RStudio Desktop](https://www.rstudio.com/products/rstudio/download/)

Starting RStudio will start R automatically.


<br/>
<br/>

### Please also make sure:

- Your OS is up to date
- You have internet access during the course
- You reach the next power socket (maybe better bring a power cable)


---



## You, after this workshop





```{r out.width = "70%", echo = FALSE}
knitr::include_graphics("img/kermit.gif")
```




Well, kinda off...

---




## Learning goals


.large[
.full-width[
.content-box-blue[
- Understanding basic tidyverse goals

- Applying tidyverse tools

- Visualizing data

- Basic modeling
]
]
]

---


## Intended audience



- Analysts who have used R somewhat, but who are new to the tidyverse

- Applied workers in the field of data analyses who want to learn modern techniques


<br/>
<br/>



### A basic working knowledge of R is needed to thoroughly follow the workshop

- You should know how to get R and Rstudio running (and packages installed)
- You should know some basics of R (such as object definition and indexing, using functions)
- You should have some experience with coding (not strictly necessary, but helpful)

---



## We'll use the following R packages



```{r}
pckgs <- c("nycflights13", "mosaic",  "broom", "corrr", "lubridate", "viridis", 
           "GGally", "ggmap", "pacman", "sjmisc", "leaflet", "knitr",  "tidyverse", 
           "tidyimpute", "na.tools", "checkpoint")

```


```{r echo = FALSE}
library(checkpoint)
checkpoint("2018-09-30")
```


```{r echo = FALSE}
library(nycflights13)
library(mosaic)
library(broom)
library(corrr)
library(lubridate)
library(viridis)
library(GGally)
library(ggmap)
library(sjmisc)
library(leaflet)
library(knitr)
library(tidyverse)
library(tidyimpute)
library(na.tools)
library(fontawesome)
library(icon)
```





#### Please install each missing package prior to the workshop from within R:


```{r, eval = FALSE}
install.packages("nycflights13")
```


---


## Load each package after each start of Rstudio

```{r}
library(pacman)
p_load(pckgs, character.only = TRUE)
```

Tip: Use `search()` to see loaded packages and `installed.packages()` for installed packages. 

Simpler: Check the "packages pane" in RStudio. 

---


## Data we'll use: `mtcars`


- `mtcars` is a toy dataset built into R (no need for installing).

- Data come from 1974 motor sports magazine describing some automotive.

- Columns: e.g., horsepower, weight, fuel consumption

### Load the dataset:

```{r}
data(mtcars)
```


Get help:

```{r }
?mtcars
```



---

## Data we'll use: `flights`



- `flights` is a dataset from R package `nycflights13` (package must be installed).

- Data come from flights leaving the NYC airports in 2013.

- Columns: e.g.., delay, air time, carrier name

### Load the dataset:


```{r}
data(flights, package = "nycflights13")
```

Get help:

```{r }
?flights
```



</br>
Load the data each time you open RStudio (during this workshop).


---




## RStudio running



```{r cache = TRUE, echo = FALSE, out.width="65%"}
knitr::include_graphics("http://paulyeo21.github.io/cell_bio_gen_lab/images/331f4ea3.rstudio_pane.png")
```


---


class: center, inverse, middle



# The tidyverse

---


background-image: url("https://pbs.twimg.com/media/CvzEQcfWIAAIs-N.jpg")
background-size: cover


---

## The data analysis (science) pipeline

<br/>
<br/>

```{r echo = FALSE, out.width="100%"}
knitr::include_graphics("https://d33wubrfki0l68.cloudfront.net/86cc45e87bb755a3bcecce462a6524e68d13a466/90635/images/tidy1.png")
```



---


## Get the power of the ~~uni~~ tidyverse

```{r echo = FALSE, out.width="50%"}
knitr::include_graphics("img/git-push-force.gif")
```

---


## But I love the old way ...




```{r out.width = "50%", echo = FALSE}
knitr::include_graphics("img/kid-waves.gif")
```


---




## Nice data


```{r out.width="50%", echo = FALSE}
knitr::include_graphics("https://biostat2.uni.lu/lectures/img/02/data_rectangle.png")
```


---


## Tidy data 


<br/>
<br/>
<br/>

```{r echo = FALSE, out.width="100%"}
include_graphics(paste0(path_img, "09_Praxisprobleme_der_Datenaufbereitung/06_tidy-1.png"))
```



[More Details](https://biostat2.uni.lu/lectures/lecture02_tidyverse.html#1)


---

## Dataset `mtcars`

```{r}
glimpse(mtcars)
```



---

class: inverse, center, middle

# Data wrangling


---


class: top, left

## Two tidyverse principles


<br/>
<br/>

.pull-left[
### Knock-down principle
```{r echo = FALSE}
img_file <- paste0(path_img, "07_Datenjudo/knockdown.png")
#file.exists(img_file)
include_graphics(img_file)
```

]




.pull-right[
### Pipe princriple
```{r echo = FALSE}
include_graphics(paste0(path_img, "07_Datenjudo/11_800px-Pipa_savinelli.jpg"))
```

]

---


background-image: url("https://raw.githubusercontent.com/sebastiansauer/modar/master/static/images/07_Datenjudo/knockdown.png")
background-size: contain


---


## Atoms of the knock-down principle


.Large[
- `filter()`

- `select()`

- `mutate()`

- `group_by()`
- ...
]


---



## Filtering rows with `filter()`


.pull-left[
Extract rows that meet logical criteria.
```{r out.width="40%", echo = FALSE}
knitr::include_graphics("img/filter.png")
```
]

.pull-right[
Filter table `mtcars` such that only rows remain where `cols` equal `6`

```{r echo = TRUE}
filter(mtcars, cyl == 6)
```
]



---


## `filter()` - exercises `r fontawesome::fa("dumbbell", fill = "steelblue")`


`r fontawesome::fa("dumbbell", fill = "steelblue")` Filter the automatic cars.

`r fontawesome::fa("dumbbell", fill = "steelblue")` Filter the automatic cars with more than 4 cylinders.

`r fontawesome::fa("dumbbell", fill = "steelblue")` Filter cars with  either low consumption or the the super. thirsty ones




---

## `filter()` - solutions to exercises


```{r eval  = FALSE}
data(mtcars)  # only if dataset is not yet loaded
filter(mtcars, am == 1)
filter(mtcars, cyl > 4)
filter(mtcars, mpg > 30 | mpg < 12)
```

---

## Select columns with `select()`


.pull-left[
Extract columns by name.
```{r out.width="40%", echo = FALSE}
knitr::include_graphics("img/select.png")
```
]

.pull-right[
Select the columns  `cyl` and `hp`. Discard the rest.

```{r echo = TRUE, eval = FALSE}
select(mtcars, cyl, hp)
```

```{r echo = FALSE}
select(mtcars, cyl, hp) %>% 
  head()
```
]


---

## `select()` - exercises 

`r fontawesome::fa("dumbbell", fill = "steelblue")` Select the first three columns.

`r fontawesome::fa("dumbbell", fill = "steelblue")` Select the first and third column.

`r fontawesome::fa("dumbbell", fill = "steelblue")` Select all columns containing the letter "c".
---

## `select()` - solutions to exercises 


```{r eval  = FALSE}
select(mtcars, 1:3)
select(mtcars, 1, disp)
select(mtcars, contains("c"))  # regex supported
```



---


## Add or change a column with `mutate`


.pull-left[
Apply vectorized functions to columns to create new columns.

```{r out.width="40%", echo = FALSE}
knitr::include_graphics("img/mutate.png")
```
]


.pull-right[
Define weight in kg for each car.
```{r echo = TRUE}
mtcars <- mutate(mtcars, 
                 weight_kg = wt * 2)
head(select(mtcars, wt, weight_kg))
```
]

---

## `mutate()` - exercises

`r fontawesome::fa("dumbbell", fill = "steelblue")` Compute a variable for consumption (gallons per 100 miles).

`r fontawesome::fa("dumbbell", fill = "steelblue")` Compute two variables in one mutate-call.

---

## `mutate()` - solutions to exercises
 
```{r eval  = TRUE}
mtcars <- mutate(mtcars, consumption = (1/mpg) * 100 * 3.8 / 1.6)
mtcars <- mutate(mtcars, 
                 consumption_g_per_m = (1/mpg),
                 consumption_l_per_100_k = consumption_g_per_m  * 3.8 / 1.6 * 100)
```


---


## Summarise a column with `summarise()`


.pull-left[
Apply function to summarise column to single value.
```{r out.width="40%", echo = FALSE}
knitr::include_graphics("img/summary.png")
```
]


.pull-right[
Summarise the `values` to their mean.
```{r echo = TRUE}
summarise(mtcars, 
          mean_hp = mean(hp))
```
]



---


## `summarise()` -  exercises
`r fontawesome::fa("dumbbell", fill = "steelblue")` Compute the median of consumption.

`r fontawesome::fa("dumbbell", fill = "steelblue")` Compute multiple statistics at once.

---


## `summarise()` -  solution to exercises


```{r eval  = TRUE}
summarise(mtcars, median(consumption))
summarise(mtcars, 
          consumption_md = median(consumption),
          consumption_avg = mean(consumption)
          )

```





---

## Group with `group_by()`


.pull-left[
Create "grouped" copy of table. dplyr functions will manipulate each group separately and then combine the results.

```{r out.width="40%", echo = FALSE}
knitr::include_graphics("img/group-by.png")
```
]



.pull-right[
Group cars by `am` (automatic vs. manual). Then summarise to mean in each group.

```{r echo = TRUE}
mtcars_grouped <- group_by(mtcars, am)
summarise(mtcars_grouped, mean_hp = mean(hp))
```
]

---

## `group_by()` - exercises
  

`r fontawesome::fa("dumbbell", fill = "steelblue")` Compute the median consumption, grouped by cylinder.

`r fontawesome::fa("dumbbell", fill = "steelblue")` Compute the median consumption, grouped by cylinder and `am`.



---


## `group_by()` - solution to exercises




```{r echo  = TRUE}
mtcars_grouped <- group_by(mtcars, cyl)
summarise(mtcars_grouped, mean_hp = mean(consumption))

mtcars_grouped <- group_by(mtcars, cyl,am)
mtcars_summarized <- summarise(mtcars_grouped, 
                               mean_hp = mean(consumption),
                               sd_hp = sd(consumption))  

```


---


class: center, middle, inverse

# Enter the pipe







---


background-image: url("https://www.rstudio.com/wp-content/uploads/2014/04/magrittr.png")
background-size: cover



---





## Life without the pipe operator


```{r eval = FALSE}
summarise(
  raise_to_power(
    compute_differences(data, mean), 
    2
  ),
  mean
)

```


---

## Life with the pipe operator

```{r eval = FALSE}
data %>%
  compute_differences(mean) %>% 
  raise_to_power(2) %>% 
  summarise(mean)
  
```


---


## Life with the pipe operator II

`r fontawesome::fa("lightbulb", fill = "steelblue")` Advanced.
<br/>

```{r echo = TRUE}
data <- mtcars$hp

data %>% 
  `-`(mean(data)) %>% 
  `^`(2) %>% 
  mean()


var(mtcars$hp) * (length(data)-1) / length(data)
```


---


class: center, middle, inverse
# Data diagrams

---


## Why we need diagrams


```{r echo = FALSE, out.width="70%"}
knitr::include_graphics(paste0(path_img, "11_Daten_visualisieren/02_anscombe-crop.png"))
```

---

## Anatomy of a diagram


```{r echo = FALSE, out.width = "60%"}
#diagram_file <- "/Users/sebastiansaueruser/Documents/Publikationen/In_Arbeit/Statistik__21/images/visualisieren/anatomie_en.png"
#file.exists(diagram_file)
knitr::include_graphics(paste0(path_img, "11_Daten_visualisieren/anatomie_en.png"))
```

---


## First plot with `ggplot`

```{r out.width="50%"}
mtcars %>% 
  ggplot() +  # initialize plot
  aes(x = hp, y = mpg) +  # define axes etc.
  geom_point() +  # graw points 
  geom_smooth()  # draw smoothing line
```


Notice the `+` in contrast to the pipe `%>%`.


---


## Groups and colors


```{r dpi=300}
mtcars %>% 
  ggplot(aes(x = hp, y = mpg, color = am)) +
  geom_point() +
  geom_smooth() +
  scale_color_viridis_c() +  # package "viridis" needed
  theme_bw() 
```


---


## Diagrams - exercises



`r fontawesome::fa("dumbbell", fill = "steelblue")` Plot the mean and the median for each cylinder group (dataset `mtcars`).

`r fontawesome::fa("dumbbell", fill = "steelblue")` Now add a visualization for the variation in the data.


---



## Diagrams - solutions to exercises 1


```{r}
mtcars_summarized %>% 
  ggplot() +
  aes(x = cyl, y = mean_hp, color = factor(am), 
      shape = factor(am)) +
  geom_point(size = 5)
```


---


## Diagrams - solutions to exercises 2


```{r out.width = "50%"}
mtcars_summarized %>% 
  ggplot(aes(x = cyl, color = factor(am), shape = factor(am))) +
  geom_errorbar(aes(ymin = mean_hp - sd_hp, ymax = mean_hp + sd_hp), width = .2, 
                position =  position_dodge(width=0.9)) + 
  geom_point(aes(y = mean_hp), size = 5, position =  position_dodge(width=0.9)) 
  
```


---



class: inverse, center, middle

# Case study Why are flights delayed?


---

## Know thy data

Don't forget to load it from the package via:

```{r}
data(flights)
```

A look to the help page:

```{r eval = FALSE}
?flights
```

---

class: top, left

## Glimpse data

```{r}
glimpse(flights)
```


---



## Data sanity - quantitative variables
 


.small-code[
```{r out.width = "120%", eval = TRUE}
flights %>% descr() 
```
]

---


## Data sanity - qualitative variables

```{r eval = TRUE}
flights %>%
  select_if(is.character) %>% 
  inspect()
```



---


## Distribution - quantitative variables

```{r}
flights %>% 
  ggplot(aes(x = dep_delay)) +
  geom_histogram() +
  scale_x_continuous(limits = c(-10, 60))
```

Note the long right tail ("anomaly")

---


## Deal with missing data - omit

```{r}
flights_nona <- flights %>% 
  drop_na()

nrow(flights_nona) / nrow(flights)
```


---

## Deal with missing data - replace by some value

```{r}
flights_nona2 <- flights %>% 
  mutate(dep_delay = ifelse(is.na(dep_delay),
                            mean(dep_delay, na.rm = TRUE), dep_delay))

flights_nona2 %>% 
  summarise(sum(is.na(dep_delay)))
```


---


## Deal with missing data - tidy approach


```{r results = "hide"}
flights_nona3 <- flights %>% 
  impute_all(na.mean)

flights_nona3 %>% 
  purrr::map(~sum(is.na(.)))
```

Hint: Use `package::function()` to disambiguate.


---

## Count missings per row

`r fontawesome::fa("lightbulb", fill = "steelblue")` Advanced.
<br/>
```{r}
flights %>% 
  mutate(NA_row = rowSums(is.na(.))) %>% 
  ggplot(aes(x = NA_row)) + geom_histogram()
```

---


## Descriptive statistics for delay

```{r}
flights %>% 
  drop_na() %>% 
  summarise(mean(dep_delay), median(dep_delay),
            sd(dep_delay), iqr(dep_delay))
```

---


## Descriptive statistics by origin

```{r}
flights %>% 
  drop_na() %>% 
  group_by(origin) %>% 
  summarise(mean(dep_delay), median(dep_delay),
            sd(dep_delay), iqr(dep_delay))
```


---

class: center, middle, inverse

# Start modeling


---


## Delay as a function of origin?


<br/>
<br/>
<br/>

```
delay = f(origin)
```

<br/>
<br/>

More Rish:


```
dep_delay ~ origin
```

---

## Linear models



```{r}
lm(dep_delay ~ origin, data = drop_na(flights)) %>% tidy()
```

Some as above, stated differently.


---


## Does `distance` predicts `dep_delay`?



```{r eval = FALSE}
flights %>% 
  ggplot(aes (x = distance, y = dep_delay)) +
  geom_point(alpha = .1) +
  geom_lm() +
  coord_cartesian(ylim(-10, 60))
```


---

##  Does `distance` predicts `dep_delay`?


```{r echo = FALSE}
flights %>% 
  ggplot(aes (x = distance, y = dep_delay)) +
  geom_point(alpha = .1) +
  geom_smooth() +
  coord_cartesian(ylim = c(-10, 60)) 
```


---


## Alternative visualization (binned data, code)


```{r eval = FALSE}
flights %>% 
  mutate(distance_bins = case_when(
    distance < 250 ~ 250,
    distance < 500 ~ 500,
    distance < 1000 ~ 1000,
    distance < 2000 ~ 2000,
    distance < 3000 ~ 3000,
    TRUE ~ 5000 )) %>% 
  ggplot(aes (y = dep_delay)) +
  geom_boxplot(aes(x = distance_bins, 
  group = distance_bins)) +
  geom_smooth(aes(x = distance)) +
  coord_cartesian(ylim = c(-10, 60)) 
```

Use `case_when()` for binning and recoding of data values.

---


## Alternative visualization (binned data)


```{r echo = FALSE}
flights %>% 
  mutate(distance_bins = case_when(
    distance < 250 ~ 250,
    distance < 500 ~ 500,
    distance < 1000 ~ 1000,
    distance < 2000 ~ 2000,
    distance < 3000 ~ 3000,
    TRUE ~ 5000
  )) %>% 
  ggplot(aes (y = dep_delay)) +
  geom_boxplot(aes(x = distance_bins, 
  group = distance_bins)) +
  geom_smooth(aes(x = distance)) +
  coord_cartesian(ylim = c(-10, 60)) 
```

---


## Correlation of distance and delay


```{r}
flights %>% 
  select(distance, dep_delay, origin) %>%
  group_by(origin) %>% 
  drop_na() %>% 
  summarise(cor_delay_dist = cor(dep_delay, distance))
```

---

## Delay as a function of distance


```{r}
lm(dep_delay ~ I(distance/1000) + origin, data = flights) %>% 
  tidy()
```


---

## Delay per month (code)





```{r eval = FALSE}
p1 <- flights %>% 
  group_by(origin, month, day) %>% 
  summarise(dep_delay_avg_day = mean(dep_delay, na.rm = TRUE)) %>% 
  ungroup %>% 
  mutate(dep_dt = make_date(2013, month, day)) %>% 
  ggplot(aes(x = dep_dt, y = dep_delay_avg_day, shape = origin, color = origin)) +
  geom_point(alpha = .3) +
  geom_smooth() +
  scale_color_viridis_d()
p1
```

---


## Delay per month (output)



```{r echo = FALSE, out.width= "100%"}
p1 <- flights %>% 
  group_by(origin, month, day) %>% 
  summarise(dep_delay_avg_day = mean(dep_delay, na.rm = TRUE)) %>% 
  ungroup %>% 
  mutate(dep_dt = make_date(2013, month, day)) %>% 
  ggplot(aes(x = dep_dt, y = dep_delay_avg_day, shape = origin, color = origin)) +
  geom_point(alpha = .3) +
  geom_smooth() +
  scale_color_viridis_d()
p1
```

---

## Delay per month - boxplot (code)

```{r eval = FALSE}
flights %>% 
  ggplot(aes(x = month, y = dep_delay)) +
  geom_boxplot(aes(group = month)) +
  geom_smooth() +
  coord_cartesian(ylim = c(-10, 60)) +
  scale_x_continuous(breaks = 1:12)
```


---



## Delay per month - boxplot (output)

```{r echo = FALSE}
flights %>% 
  ggplot(aes(x = month, y = dep_delay)) +
  geom_boxplot(aes(group = month)) +
  geom_smooth() +
  coord_cartesian(ylim = c(-10, 60)) +
  scale_x_continuous(breaks = 1:12)
```


---

## Is it the weekends? (code)

```{r}
flights <- flights %>%
  mutate(dow = wday(time_hour),
         weekend = case_when(
           dow %in% c(6, 7) ~ TRUE,
           TRUE ~ FALSE))

delay_dow <- 
  flights %>% 
  group_by(dow) %>% 
  drop_na() %>% 
  summarise(delay_m = mean(dep_delay),
            delay_md = median(dep_delay),
            q_05 = quantile(x = dep_delay, prob = .05),
            q_95 = quantile(x = dep_delay, prob = .95))
```

---

## Is it the weekends? (data processed)

```{r}
delay_dow %>% kable(format='html')
```


---

##Is it the weekends? (code)

```{r eval = FALSE}
flights %>% 
  ggplot(aes(x = dow)) +
  geom_boxplot(aes(group = dow, y = dep_delay, color = weekend)) +
  geom_point(data = delay_dow, aes(y = delay_m), color = "red", 
             size = 5) + 
  coord_cartesian(ylim = c(-10, 100)) +
  scale_x_continuous(breaks = 1:7) +
  geom_hline(yintercept = mean(flights$dep_delay, na.rm = TRUE), 
             linetype = "dashed") +
  geom_hline(yintercept = median(flights$dep_delay, na.rm = TRUE), 
             linetype = "dashed") +
  geom_errorbar(aes(ymin = q_05, ymax = q_95), data = delay_dow)
```


---



## Is it the weekends? (output)

```{r echo = FALSE, fig.asp = .6}
flights %>% 
  ggplot(aes(x = dow)) +
  geom_boxplot(aes(group = dow, y = dep_delay, color = weekend)) +
  geom_point(data = delay_dow, aes(y = delay_m), color = "red", 
             size = 5) + 
  coord_cartesian(ylim = c(-10, 100)) +
  scale_x_continuous(breaks = 1:7) +
  geom_hline(yintercept = mean(flights$dep_delay, na.rm = TRUE), 
             linetype = "dashed") +
  geom_hline(yintercept = median(flights$dep_delay, na.rm = TRUE), 
             linetype = "dashed") +
  geom_errorbar(aes(ymin = q_05, ymax = q_95), data = delay_dow) +
  theme_ipsum_rc() +
  scale_color_viridis_d()
```


---


## Delay per time of the day (code)

Let's check whether delays add up during the day, a popular opinion among travellers. 

```{r eval = FALSE}
flights %>% 
  select(dep_delay, hour) %>% 
  ggplot(aes(x = hour, y = dep_delay)) +
  geom_boxplot(aes(group = hour)) +
  geom_smooth(method = "lm") +
  coord_cartesian(ylim = c(-10, 60))  +
  scale_x_continuous(breaks = 1:24)
  
```


---



## Delay per time of the day (output)


```{r echo = FALSE}
flights %>% 
  select(dep_delay, hour) %>% 
  ggplot(aes(x = hour, y = dep_delay)) +
  geom_boxplot(aes(group = hour)) +
  geom_smooth(method = "lm") +
  coord_cartesian(ylim = c(-10, 60))  +
  scale_x_continuous(breaks = 1:24)
  
```


---


# Delay as function of month, hour, origin, and weekday

```{r}
lm_hour <- lm(dep_delay ~ hour + month + origin + I(dow == 7), 
              data = flights)

rsquared(lm_hour)
```


---


class: inverse, center, middle

# Geoplotting

`r fontawesome::fa("lightbulb", fill = "steelblue")` Advanced.
<br/>

---


class: top, left

## Join airport data


```{r}
data("airports")

flights_airports <-  # join destination long/lat
  flights %>% 
  left_join(airports, by = c("dest" = "faa")) %>%  
  rename(long = lon)

origin_latlong <-
  airports %>% 
  filter(faa %in% c("LGA", "JFK", "EWR")) %>% 
  rename(lat_origin = lat,
         long_origin = lon)
  
flights_airports <-  # join origin long/lat
  flights_airports %>%
  left_join(origin_latlong, by = c("origin" = "faa"))
```


---

## Dataframe for plotting (code)

```{r}
flights_airports_sum <- flights_airports %>% 
  group_by(dest, origin) %>% 
  summarise(n = n(),
            long = max(long),
            lat = max(lat),
            long_origin = max(long_origin),
            lat_origin = max(lat_origin))
```

---

## Dataframe for plotting (output)

```{r}
head(flights_airports_sum)
```

---


## Geo plot flights (code)

```{r eval = FALSE}
ggplot(data = map_data("usa")) +
  aes(x = long, y = lat, group = group) +
  geom_path(color = "grey40", size = .1) +
  geom_point(data = flights_airports_sum, 
             aes(size = n, color = n, group = NULL), alpha = .2) +
  geom_segment(data = flights_airports_sum, 
               aes(color = n, group = NULL,
                   x = long_origin, y = lat_origin,
                   xend = long, yend = lat), alpha = .5) +
  geom_text(data = flights_airports_sum %>% filter(n > 6000), 
            aes(x = long, y = lat, label = dest, group = NULL),
            color = "grey40") +  
  theme_map() +
  xlim(-130, -70) + ylim(+20, +50) +
  scale_color_viridis()
```



---

## Geo plot flights (output)

```{r echo = FALSE, out.width="80%", fig.asp = 0.618}
ggplot(data = map_data("usa")) +
  aes(x = long, y = lat, group = group) +
  geom_path(color = "grey40", size = .1) +
  geom_point(data = flights_airports_sum, 
             aes(size = n, color = n, group = NULL)) +
  geom_segment(data = flights_airports_sum, 
               aes(color = n, group = NULL,
                   x = long_origin, y = lat_origin,
                   xend = long, yend = lat)) +
  geom_text(data = flights_airports_sum %>% filter(n > 6000), 
            aes(x = long, y = lat, label = dest, group = NULL),
            color = "grey40") +  
  theme_map() +
  xlim(-130, -70) + ylim(+20, +50) +
  scale_color_viridis() +
  theme(legend.position = "bottom")
```


---



class: center, inverse, middle


# Outlook

`r fontawesome::fa("lightbulb", fill = "steelblue")` Advanced.
<br/>

---


class: top, left



## Map columns to function with `map()`


```{r echo = TRUE, out.width="20%"}
data(mtcars)
purrr::map(select(mtcars, 1:2), ~ {ggplot(mtcars, aes(x = .)) +
    geom_histogram()})
```

---



## Map TWO columns to function with `map2()`


```{r eval = FALSE}
flights %>% 
  select_if(~!is.numeric(.)) %>% 
  map2(., names(.), ~ 
         {ggplot(data = flights, aes(x = .x)) + 
             geom_bar() + labs(x = .y, title = .y)})
```

<br/>


```
Take flights dataset  
select all numeric cols  
map ggplot call to each such that  
a barplot is plotted where  
the names of each barplot is an element from the vector of col names.  
```


---

## Reshape (transform) dataframe

<br/>


```{r echo = FALSE}
knitr::include_graphics(paste0(path_img, "09_Praxisprobleme_der_Datenaufbereitung/07_tidy-dataframes-crop.png"))
```

---

## Transform dataframe for plotting

```{r echo = TRUE, fig.asp = .8, out.width="30%"}
mtcars %>% 
  select_if(is.numeric) %>% 
  gather(key = variable, value = value) %>% 
  ggplot(aes(x = value)) +
  geom_density() +
  facet_wrap(~ variable, ncol = 3, scales = "free")
```

---


## `ggplot` niceties: Themes

```{r fig.asp = .617}
library(hrbrthemes)

p2 <- p1 + theme_ipsum() + theme(legend.position = "bottom")
p2

```

May may need to install fonts upfront; see `?hrbrthemes`.
---

## ggplot niceties: Combining plots


```{r fig.width=10}
library(patchwork)

p2 + p2
```


---



class: center, inverse, left

# Resources



---


## Modern Dive

<br/>
<br/>

```{r out.width = "70%", echo = FALSE}
include_graphics("https://moderndive.com/wide_format.png")
```



[Modern Dive -- An Introduction to Statistical and Data Sciences via R
Chester Ismay and Albert Y. Kim](https://moderndive.com/index.html)

---


## R for Data Science


<br/>
<br/>

```{r out.width = "20%", echo = FALSE}
include_graphics("https://covers.oreillystatic.com/images/0636920034407/lrg.jpg")
```



[R for Data Science](https://r4ds.had.co.nz/)


---

## Moderne Datenanalyse mit R


```{r out.width="30%", echo = FALSE}
include_graphics("https://images.springer.com/sgw/books/medium/9783658215866.jpg")
```


[Moderne Datenanalyse mit R](https://www.springer.com/us/book/9783658215866)

---


## Cheatsheets





```{r echo =FALSE, out.width="50%"}
knitr::include_graphics("img/cheatsheet-dplyr.png")
```


https://www.rstudio.com/resources/cheatsheets/ 


---



## Disclaimer: There may be issues at times

```{r echo = FALSE, out.width="50%"}
knitr::include_graphics("img/overhead-bin.gif")
```

`r icon::fa("stack-overflow", size = 1)` [StackOverflow](https://stackoverflow.com/) is your friend

---



class: center, inverse, middle


class: center, inverse, middle


# Wrap-up

---

## That was quick, but it was a start

.center[
![](img/get-into-the-car.gif)
]


---

class: right, top, inverse

# Thank you

<br/>
<br/>
Sebastian Sauer
<br/>

.left[
`r icon::fa("github", size = 1)` [sebastiansauer](https://github.com/sebastiansauer)

`r icon::fa("link", size = 1)` <https://data-se.netlify.com/>

`r icon::fa("envelope", size = 1)` sebastian.sauer@fom.de

`r icon::fa("linkedin-in", size = 1)` [Sebastian Sauer](https://www.linkedin.com/in/dr-sebastian-sauer-4791762)

`r icon::fa("file", size = 1)` Get slides [here](http://data-se.netlify.com/slides/hands-on-data-exploration/handson-data-workshop_2018-11-21.html)

`r icon::fa("file-pdf", size = 1)`: Get PDF of slides [here](http://data-se.netlify.com/slides/hands-on-data-exploration/hands-on-data-exploration-using-R.pdf)

`r icon::fa("r-project", size = 1)`: Get Rmd source code of slides [here](http://data-se.netlify.com/slides/hands-on-data-exploration/handson-data-workshop_2018-11-21.Rmd)
<br/>

Licence: MIT
]




---

## Credit to

Built using R, RMarkdown, Xaringan. Thanks to the R community and the tidyverse developers. Thanks to [Yihui Xie](https://slides.yihui.name/2017-rmarkdown-UNL-Yihui-Xie.htm) and [Antoine Bichat](https://abichat.github.io/Slides/ScienceCommunicationSOTR/ScienceCommunicationSOTR.html), among others, for Xaringan inspiration. Thanks to FOM Hochschule for supporting me.

Images:

- [Data Transformation with dplyr Cheat Sheet, by RStudio](https://www.rstudio.com/resources/cheatsheets/)
- [Modern Dive, Chester Ismay and Albert Y. Kim](https://moderndive.com/)
- [Kermit typping](https://www.reddit.com/r/reactiongifs/comments/2zk2ds/mrw_i_have_a_paper_due_the_following_morning/)
- [RStudio running](http://paulyeo21.github.io/cell_bio_gen_lab/images/331f4ea3.rstudio_pane.png)
- [tidyverse](http://paulyeo21.github.io/cell_bio_gen_lab/images/331f4ea3.rstudio_pane.png)
- [Process of data analysis](https://d33wubrfki0l68.cloudfront.net/86cc45e87bb755a3bcecce462a6524e68d13a466/90635/images/tidy1.png)
- [Yoda](http://i.imgur.com/CZ4D2pf.gifv)
- [kid waves](https://media.giphy.com/media/XrDT8BuYB3I2s/giphy.gif)
- [Nice data](https://biostat2.uni.lu/lectures/img/02/data_rectangle.png)
- [tidy data](http://garrettgman.github.io/tidying/)
- [Magrittr pipe](https://www.rstudio.com/wp-content/uploads/2014/04/magrittr.png)
- [Jump car](https://wifflegif.com/gifs/598112-skill-wtf-gif)
- [Overhead locker](https://i.gifer.com/DUrV.gif)

Icons from [FontAwesome](https://fontawesome.com/icons)

---







## SessionInfo

```{r echo = FALSE}
R.Version()$version.string

si <- sessionInfo()
save(si, file = "si.RData")
```


See `si.RData` for package version (same folder as this presentation).

This document is made reproducible using [checkpoint](https://mran.microsoft.com/package/checkpoint) with day set to 2018-09-30.



