Retrieve Data from an API for R Analysis

This is a repository with scripts that show how to use a remote API for data retrieval. The sources are mainly R but also some small scripts are available to show the basic principles from the Linux shell.

Introduction

This repository was establish during the COVID-19 pandemic for a learning resource in Wikiversity about Dynamic Report or Report Generation with KnitR

Test Usage

If you want to perform a test with your R-script to fetch a JSON record from a database API is makes sense to test the programming code if it parses a JSON correctly.

• Demo Link for Emulation of Database Query over Web for testing.

The tests encapsule debugging to the processing of a JSON. Otherwise you must distinguish if the API call was incorrect and the provided JSON could processed due to the query syntax or the parsing and processing in the R script was not working. The repository provides a correct JSON via a HTTP request and therefore an error can clearly allocated on the scripting of the parser.

R-Script to Fetch Data form API

This is the example for R code with the example URL to a JSON data in this repository. You can use this code also as KnitR code chunk.

library(rjson)
url <- "https://niebert.github.io/data4api/data/json_demo.json"
raw_data_json <- scan(url, "", sep="\n")
raw_data_json
my_dataframe <- fromJSON(paste(raw_data_json, collapse = ""))
my_dataframe

The code runs out of the box in your R-code, Shiny-WebApp or KnitR code chunk. After testing adapt the URL to your API with the appropriate parameters. The current URL is just for testing the basic functionality of the R code. The JSON is static and just for demo purpose.

Wikiversity

This repository is created in support for the Wikiversity learning resource about KniR.

Folders

The following folders are providing tools for data retrieval on the Operating System level or on the application level (e.g. in Open Source R, GNU Octave, Shiny Web-Apps with R Code, KnitR, ...

Linux - Shell Scripts

The folder linux/ contains Linux scripts that can be used for automatisation of data updates. These scripts are usually added in the crontab for daily or weekly update of data.

R-Code

The folder r_code/ contains R scripts that can be used within R/RStudio, in KnitR or in a Shiny Web-App.

Data - Demo Files

The data contains demo data with different files types for testing.

• json_demo.json contains a JSON file for testing data retrieval https://niebert.github.io/data/json_demo.json - this URL can be used as test to emulated a database query for specific data. With the R scripts provided in folder r_code/. The R-scripts use the GitHub URL to the JSON for testing. Change the URL to a real database for accessible Open Data for expanding your R-scripts for specific data analysis.

Tools

• JSON2Schema create a JSON editor from a defined JSON in 5min as AppLSAC