Meet the toolkit

Lecture 2

Dr. Benjamin Soltoff

Cornell University
INFO 5001 - Fall 2023

2023-08-23

Announcements

• Application exercises 00 are ungraded
• If you cannot access RStudio Workbench yet, let me know

Making INFO 5001 a success

Five tips for success

1. Complete all the preparation work before class.
2. Ask questions.
3. Do the readings.
4. Do the lab and homework assignments.
5. Don’t procrastinate and don’t let a week pass by with lingering questions.

Course FAQ

Q - Is this an intro CS course?
A - No – we assume you have completed the equivalent of CS 1110/1112

Q - What data science background does this course assume?
A - None! Sort of…

Q - Is this an intro stat course?
A - No. We presume you have taken undergraduate stats course(s)

While statistics \(\ne\) data science, they are very closely related and have tremendous of overlap.

Q - What computing language will we learn?
A - R.

Q: Why not language X?
A: Come meet with me during office hours and we can talk about it!

Course toolkit

Course operation

• Materials: info5001.infosci.cornell.edu
• Submission: Gradescope
• Discussion: GitHub Discussions
• Gradebook: Canvas

Doing data science

• Computing:
  • R
  • RStudio
  • tidyverse
  • Quarto
• Version control and collaboration:
  • Git
  • GitHub

Toolkit: Computing

Learning goals

By the end of the semester, you will…

• Construct and execute basic programs in R using elementary programming techniques and tidyverse packages (e.g. loops, conditional statements, user-defined functions).
• Implement data science workflows using common, reproducible methods and software tools.
• Apply stylistic principles of coding to generate reusable, interpretable code.
• Utilize reference documentation and debugging tools to troubleshoot problems.
• Identify and use external libraries to expand on base functions
• Apply Git and GitHub workflows for version control.

Reproducible data analysis

Reproducibility checklist

What does it mean for a data analysis to be “reproducible”?

Near-term goals:

• Are the tables and figures reproducible from the code and data?
• Does the code actually do what you think it does?
• In addition to what was done, is it clear why it was done?

Long-term goals:

• Can the code be used for other data?
• Can you extend the code to do other things?

Toolkit for reproducibility

• Scriptability \(\rightarrow\) R
• Literate programming (code, narrative, output in one place) \(\rightarrow\) Quarto
• Version control \(\rightarrow\) Git / GitHub

R and RStudio

R and RStudio

• R is an open-source statistical programming language
• R is also an environment for statistical computing and graphics
• It’s easily extensible with packages

• RStudio is a convenient interface for R called an IDE (integrated development environment), e.g. “I write R code in the RStudio IDE”
• RStudio is not a requirement for programming with R,¹ but it’s very commonly used by R programmers and data scientists

1. See VS Code

R vs. RStudio

Source: Modern Dive.

R packages

• Packages: Fundamental units of reproducible R code, including reusable R functions, the documentation that describes how to use them, and sample data¹

• As of January 2023, there are over 19,000 R packages available on CRAN (the Comprehensive R Archive Network)²

• We’re going to work with a small (but important) subset of these!

1. Wickham and Bryan, R Packages

2. CRAN contributed packages

Tour: R and RStudio

A short list (for now) of R essentials

• R is a functional language

• Functions are (most often) verbs, followed by what they will be applied to in parentheses:

  do_this(to_this)
  do_that(to_this, to_that, with_those)

• Packages are installed with the install.packages() function and loaded with the library function, once per session:

  install.packages("package_name")
  library(package_name)

R essentials (continued)

• Columns (variables) in data frames are accessed with $:

dataframe$var_name

• Object documentation can be accessed with ?

?mean

R and Python

It is not accurate to say that one programming language is inherently “better” than another, as the choice of language often depends on the specific use case and the individual’s personal preferences. However, R and Python both have their own strengths and weaknesses.

R is particularly well-suited for data analysis and visualization, and it has a large number of libraries and packages specifically designed for these tasks. R’s syntax is also designed to make it easy to manipulate and analyze data.

Python, on the other hand, is a general-purpose programming language that is widely used in a variety of fields, including web development, machine learning, and scientific computing. It has a large and active community that maintains a wide variety of libraries and packages for many different tasks. Python’s simple and easy-to-learn syntax makes it a popular choice for beginners.

Ultimately, the choice between R and Python will depend on the specific task you are trying to accomplish and your personal preferences as a developer. Both languages are powerful and have a lot to offer, and many data scientists use both languages in their work.

Source: ChatGPT

Major differences between R and Python

	R	Python
Syntax	Functional language	Object-oriented language
Statistical learning	Developed by statisticians for statistical analysis	Meh
Machine learning	tidymodels caret API interfaces to Python	scikit-learn tensorflow pytorch
Visualization	ggplot2	matplotlib + others
Package management	CRAN	pip/virtualenv/PyPI/Anaconda
Speed	Somewhat slower	Somewhat faster
Community	Academia and industry	Larger (general-purpose programming language)

tidyverse

tidyverse.org

• The tidyverse is an opinionated collection of R packages designed for data science
• All packages share an underlying philosophy and a common grammar

Quarto

Quarto

• Fully reproducible documents – each time you render the analysis is run from the beginning
• Code goes in chunks – narrative goes outside of chunks
• A visual editor for a familiar / Google docs-like editing experience
• More robust and flexible compared to Jupyter Notebooks
• Can still use the Jupyter engine to run Python natively

Tour: Quarto

Environments

Important

The environment of your Quarto document is separate from the Console!

Remember this, and expect it to bite you a few times as you’re learning to work with Quarto!

Environments

First, run the following in the console:

x <- 2
x * 3

[1] 6

All looks good, eh?

Then, add the following in an R chunk in your Quarto document

x * 3

Error in eval(expr, envir, enclos): object 'x' not found

What happens? Why the error?

How will we use Quarto?

• Every assignment / report / project / etc. is a Quarto document
• You’ll always have a template Quarto document to start with
• The amount of scaffolding in the template will decrease over the semester

What’s with all the hexes?

The Bechdel test

The Bechdel test

In order to pass the test, a movie must have

1. At least two named women in it
2. Who talk to each other
3. About something besides a man

The Bechdel test

Source: FiveThirtyEight

Your turn!

ae-00-bechdel-quarto

• Go to ae-00-bechdel-quarto and clone the repo to RStudio Workbench.
• Open and render the Quarto document bechdel.qmd, review the document, and fill in the blanks.

Warning

ae-00-bechdel-quarto is hosted on GitHub.com because we have not configured your authentication method for Cornell’s GitHub. We will do this tomorrow in lab.

Toolkit: Version control and collaboration

Git and GitHub

• Git is a version control system – like “Track Changes” features from Microsoft Word, on steroids
• It’s not the only version control system, but it’s a very popular one

• GitHub is the home for your Git-based projects on the internet – like DropBox but much, much better

• We will use GitHub (Enterprise) as a platform for web hosting and collaboration

Versioning

Source: STA 199

Versioning

with human readable messages

Source: STA 199

How we use Git and GitHub

How we use Git and GitHub

How we use Git and GitHub

How we use Git and GitHub

Git and GitHub tips

Git and GitHub tips

• There are hundreds of Git commands – you don’t have to know them all. 99% of the time you will use Git to add, commit, push, and pull.
• We will be doing Git things and interfacing with GitHub through RStudio, but if you Google for help you might come across methods for doing these things in the command line – skip that and move on to the next resource unless you feel comfortable trying it out.
• There is a great resource for working with git and R: happygitwithr.com. Some of the content in there is beyond the scope of this course, but it’s a good place to look for help.

Tour: Git + GitHub

• In discussion section
• Make sure to access Cornell’s GitHub so I can add you to the course organization on GitHub

Ithaca Farmers Market