Data Acquisition

August 17, 2023 · 5 min read

If you have any questions, feel free to comment below. Click the block can copy the code.
And if you think it's helpful to you, just click on the ads which can support this site. Thanks!

In fact, R has a large number of built-in datasets available for analysis and practice, and we can also create data in R that simulate specific distributions. In actual work, however, data analysts more often face external data from various data sources, namely data files with all kinds of extensions, such as .txt, .csv, .xlsx, .xls, and so on. Files with different extensions represent different file formats, which often causes trouble for analysts.

R provides data import tools with a wide range of applications.

1.Obtaining Built-in Datasets #

R’s built-in datasets exist in various packages, among which the base package datasets contains only datasets and no functions. This package provides nearly 100 datasets, covering medicine, nature, sociology, and other fields.

You can view them with the following command:

data(package = "datasets")

If you want to call a dataset, you can use the data( ) function. Run the following command, and R will load the iris dataset into the workspace.

data(iris)

Besides the datasets package, many other packages in R also include datasets. If they are not base packages automatically loaded after R starts, we need to install and load these packages before we can use their data. The following uses the bacteria dataset in the MASS package as an example to illustrate the process of calling data:

library(MASS)
data(bacteria)

2. Simulating Data with Specific Distributions #

R provides a series of functions that can be used for numerical simulation. These functions begin with r; commonly used ones include rnorm( ), runif( ), rbinom( ), and rpois( ), among others. For example:

# Histograms will be introduced in detail in the later visualization section
r1 <- rnorm(n = 100, mean = 0, sd = 1)
# head(r1) # Take a look at the first 5 values
hist(r1)

r2 <- runif(n = 10000, min = 0, max = 100)
hist(r2)

r3 <- rbinom(n = 80, size = 100, prob = 0.1)
hist(r3)

r4 <- rpois(n = 50, lambda = 1)
hist(r4)

3. Obtaining Data in Other Formats #

3.1 txt and csv Formats #

If the data source is an ASCII-format file created with Windows Notepad or another plain-text editor, we can use the function read.table( ) to read the data in it, which returns a data frame.

For example, suppose the data file patients.txt for the created data frame patients is stored in the current working directory; we can use the following command to read the data:

# getwd() # Get the current working directory
# Temporarily create the patients.txt data file
ID <- 1:5
sex <- c("male", "female", "male", "female", "male")
age <- c(25, 34, 38, 28, 52)
pain <- c(1, 3, 2, 2, 3)      
pain.f <- factor(pain, levels = 1:3, labels = c("mild", "medium", "severe"))   
patients <- data.frame(ID, sex, age, pain.f)
write.table(patients, "patients.txt", row.names = FALSE)

patients.data <- read.table("patients.txt", header = TRUE)
patients.data

Spreadsheet and database applications often generate delimited text files, among which .csv files use comma-separated values (Comma Separated Values). The function read.csv( ) is a variant of the function read.table( ) specifically used to read .csv files.

The default values of the parameters in the two functions read.table ( ) and read.csv ( ) are different. In the function read.table ( ), the default value of the parameter header is FALSE, which means it assumes that data, rather than variable names, begins on the first line of the file. In the function read.csv ( ), however, the default value of the parameter header is TRUE. Therefore, before reading data, it is recommended to open and examine the original file first, and then set appropriate parameters to read the data correctly.

write.csv(patients, "patients.csv", row.names=FALSE)
patients.data <- read.csv("patients.csv")
patients.data

3.2 xls or xlsx Formats #

There are many ways to read spreadsheet data. The simplest is to save the data file in Excel as a comma-separated (.csv) file and then read it into R using the method for reading .csv files described above. You can also use third-party packages (such as the openxlsx, readxl, and gdata packages) to directly read data files in xlsx or xls format.

Take the openxlsx package as an example:

library(openxlsx)
write.xlsx(patients, "patients.xlsx")
patients.data <- read.xlsx("patients.xlsx", sheet = 1)
patients.data

3.3 Importing Data from Other Statistical Software #

Sometimes we need to read data produced by other statistical software, such as SPSS, SAS, Stata, Minitab, and so on. One method is to export the data from other statistical software as a text file and then use the function read.table( ) or read.csv( ) to read the data into R. Another method is to use an extension package, such as the foreign package, whose main function is to read and write data from other statistical software.

The following uses importing an SPSS data file as an example.

Suppose the data file patients.sav is stored in the current working directory; we can use the following command to read the dataset into R:

# To save on the number of attachments, let us download it directly to the workspace
URL <- "http://download.kesci.com/qlhatmok4/patients.sav"
download.file(URL, destfile = "./patients.sav", method="curl")

library(foreign)
# The parameter `to.data.frame` in the function `read.spss( )` defaults to FALSE; if it is not set to TRUE, the returned data will be in list form.
patients.data <- read.spss("patients.sav" , to.data.frame = TRUE)
patients.data

The process of using the foreign package to import data files from software such as SAS and Stata is similar to the above; for details, please consult the documentation^[1].

4.Data Entry #

Data can be entered directly in R, but if the amount of data is large (more than 10 columns or more than 30 rows), entering data in R is not the best choice. We can choose spreadsheet software, such as Excel, to enter small-scale data.

However, if the amount of data is very large, manually entering data using spreadsheet software also has a relatively high probability of error. At this point, program software designed specifically for data entry is more suitable, such as the free software EpiData. This software not only makes it convenient to set constraints for data entry, such as range checks and automatic line wrapping, but can also add labels to each variable and variable value.

The function read.epiinfo( ) in the foreign package can directly read .rec files generated by EpiData, but it is recommended to first export the entered data as a Stata data file in EpiData, and then use the function read.dta( ) in R to read the data. The advantage of doing this is that it can preserve the attributes of variables preset in EpiData, such as variable labels and descriptions.

References

https://cran.r-project.org/web/packages/foreign/foreign.pdf ↩︎