FingertipsR - EXTREMELY elementary question (so apologies) #65

WizardAKA · 2019-02-27T10:56:32Z

I'm trying to get some of basic functionality of FingertipsR to work e.g.

fingertips_stats()

But getting this error

Error: lexical error: invalid char in json text.
<title>Trend Micro
(right here) ------^

The fact I can't even use the most basic functions makes me think there's something wrong in the software or privleges.

Could anyone help?

sebastian-fox · 2019-02-27T10:59:41Z

Could you paste in the output of typing sessionInfo() into your console please? It shouldn't need any extra privileges for that function to work.

WizardAKA · 2019-02-27T11:02:02Z

sessionInfo()
R version 3.5.2 (2018-12-20)
Platform: i386-w64-mingw32/i386 (32-bit)
Running under: Windows 7 (build 7601) Service Pack 1

Matrix products: default

locale:
[1] LC_COLLATE=English_United Kingdom.1252 LC_CTYPE=English_United Kingdom.1252
[3] LC_MONETARY=English_United Kingdom.1252 LC_NUMERIC=C
[5] LC_TIME=English_United Kingdom.1252

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] fingertipscharts_0.0.4 fingertipsR_0.2.0

loaded via a namespace (and not attached):
[1] geojsonio_0.6.0 jsonlite_1.6 magrittr_1.5 gtable_0.2.0 class_7.3-15
[6] tidyr_0.8.2 hms_0.4.2 sp_1.3-1 stringr_1.4.0 pillar_1.3.1
[11] htmltools_0.3.6 curl_3.3 later_0.8.0 dplyr_0.8.0.1 lattice_0.20-38
[16] htmlwidgets_1.3 tidyselect_0.2.5 plyr_1.8.4 lemon_0.4.3 miniUI_0.1.1.1
[21] maptools_0.9-5 knitr_1.21 gridExtra_2.3 V8_2.0 mime_0.6
[26] pkgconfig_2.0.2 R6_2.4.0 shiny_1.2.0 digest_0.6.18 xfun_0.5
[31] colorspace_1.4-0 shinycssloaders_0.2.0 DT_0.5 stringi_1.3.1 lazyeval_0.2.1
[36] crosstalk_1.0.0 geojson_0.3.2 tibble_2.0.1 httr_1.4.0 compiler_3.5.2
[41] xtable_1.8-3 munsell_0.5.0 DBI_1.0.0 jqr_1.1.0 Rcpp_1.0.0
[46] rgdal_1.3-9 assertthat_0.2.0 readr_1.3.1 ggplot2_3.1.0 classInt_0.3-1
[51] leaflet_2.0.2 tools_3.5.2 units_0.6-2 foreign_0.8-71 httpuv_1.4.5.1
[56] e1071_1.7-0.1 scales_1.0.0 crayon_1.3.4 glue_1.3.0 purrr_0.3.0
[61] rlang_0.3.1 promises_1.0.1 rgeos_0.4-2 sf_0.7-3 grid_3.5.2

sebastian-fox · 2019-02-27T11:27:31Z

Hmmm, interesting. This is new to me and I'm not too sure how to get to the bottom of it. All of your package versions which fingertipsR depends on look ok .

Could you try closing an opening RStudio, and then run the following before loading any other package?

fingertipsR::fingertips_stats()

WizardAKA · 2019-02-27T11:30:24Z

I get exactly the same error This may take a few moments... Error: lexical error: invalid char in json text. <html><head><title>Trend Micro (right here) ------^

sebastian-fox · 2019-02-27T11:38:08Z

Thanks - I'm struggling to reproduce this.
Do you get the same error with the following?

df <- fingertipsR::indicators()

And what happens with the following?

df <- fingertipsR::fingertips_data(90366)

WizardAKA · 2019-02-27T11:45:16Z

Not sure if I can add screen grabs, but I get the same error for all.

The reason I wanted to do the screen grab is because I wasn’t sure the

(right here)-----^

Pointing at the ‘h’ in html was showing.

WizardAKA · 2019-02-27T11:48:28Z

sebastian-fox · 2019-02-27T11:52:22Z

Thank you. Does this url work in your browser?
https://fingertips.phe.org.uk/api/all_data/csv/by_indicator_id?indicator_ids=90366&child_area_type_id=102&parent_area_type_id=6&include_sortable_time_periods=yes

WizardAKA · 2019-02-27T11:54:25Z

Hi Sebastian, yes, at least I think so. It brings up a page with what looks like comma delimited data for whole bunch of indicators broken down by deprivation decile, geography etc.

And that's in Chrome. In IE it prompts to save a file with suffix ".json" which I sort of understand is a file type.

sebastian-fox · 2019-02-27T11:59:10Z

Ok, we might be getting close now...but we may not!

Can you try the following:

library(dplyr)
library(httr)
library(curl)
dataurl <- "https://fingertips.phe.org.uk/api/all_data/csv/by_indicator_id?indicator_ids=90366&child_area_type_id=102&parent_area_type_id=6&include_sortable_time_periods=yes"
df_string  <- dataurl %>%
                GET(use_proxy(ie_get_proxy_for_url(.), username = "", password = "", auth = "ntlm")) %>%
                content("text")
new_data <- read.delim(text = df_string,
                               encoding = "UTF-8",
                               sep = ",",
                               fill = TRUE,
                               header = TRUE,
                               stringsAsFactors = FALSE,
                               check.names = FALSE)

WizardAKA · 2019-02-27T12:04:31Z

This is what I'm getting

> library(dplyr)

Attaching package: ‘dplyr’

The following objects are masked from ‘package:stats’:

    filter, lag

The following objects are masked from ‘package:base’:

    intersect, setdiff, setequal, union

> library(httr)
> library(curl)

Attaching package: ‘curl’

The following object is masked from ‘package:httr’:

    handle_reset

> dataurl <- "https://fingertips.phe.org.uk/api/all_data/csv/by_indicator_id?indicator_ids=90366&child_area_type_id=102&parent_area_type_id=6&include_sortable_time_periods=yes"
> df_string  <- dataurl %>%
+   GET(use_proxy(ie_get_proxy_for_url(.), username = "", password = "", auth = "ntlm")) %>%
+   content("text")
> new_data <- read.delim(text = df_string,
+                        encoding = "UTF-8",
+                        sep = ",",
+                        fill = TRUE,
+                        header = TRUE,
+                        stringsAsFactors = FALSE,
+                        check.names = FALSE)
Error in read.table(file = file, header = header, sep = sep, quote = quote,  : 
  more columns than column names

WizardAKA · 2019-02-27T12:06:13Z

sebastian-fox · 2019-02-27T12:14:46Z

Huge can of worms!
I don't suppose any of your colleagues have the same issue?

WizardAKA · 2019-02-27T12:17:05Z

I'm literally the first person to use R in my organisation.

I did wonder if it was something to do with access privileges, or the way web access is configured. I don't know if a firewall or anything would effect this.

sebastian-fox · 2019-02-28T13:39:39Z

Hi @WizardAKA - I'm still very confused by this. Could you try running this (this is very similar to the previous chunk of code but not identical):

library(dplyr)
library(httr)
library(curl)
dataurl <- "https://fingertips.phe.org.uk/api/all_data/csv/by_indicator_id?indicator_ids=90366&child_area_type_id=102&parent_area_type_id=6&include_sortable_time_periods=yes&timestamp=12345929319412"
df_string  <- dataurl %>%
        GET(use_proxy(ie_get_proxy_for_url(.), username = "", password = "", auth = "ntlm")) %>%
        content("text")
new_data <- read.delim(text = df_string,
                       encoding = "UTF-8",
                       sep = ",",
                       fill = TRUE,
                       header = TRUE,
                       stringsAsFactors = FALSE,
                       check.names = FALSE)

Thank you

WizardAKA · 2019-02-28T13:43:29Z

Hi Sebastian,

Here's what I get -

> library(dplyr)
> library(httr)
> library(curl)
> dataurl <- "https://fingertips.phe.org.uk/api/all_data/csv/by_indicator_id?indicator_ids=90366&child_area_type_id=102&parent_area_type_id=6&include_sortable_time_periods=yes&timestamp=12345929319412"
> df_string  <- dataurl %>%
+   GET(use_proxy(ie_get_proxy_for_url(.), username = "", password = "", auth = "ntlm")) %>%
+   content("text")
> new_data <- read.delim(text = df_string,
+                        encoding = "UTF-8",
+                        sep = ",",
+                        fill = TRUE,
+                        header = TRUE,
+                        stringsAsFactors = FALSE,
+                        check.names = FALSE)
Error in read.table(file = file, header = header, sep = sep, quote = quote,  : 
  more columns than column names

WizardAKA · 2019-02-28T13:45:34Z

The error is

Error in read.table(file = file, header = header, sep = sep, quote = quote, :
more columns than column names

sebastian-fox · 2019-02-28T13:53:26Z

Thank you. Ok - I don't understand this error, but I can now replicate your original error. I'll get back to you about that soon hopefully, and then see what we can do about this one!

WizardAKA · 2019-02-28T13:54:48Z

Thanks Sebastian, really kind and appreciate the effort!

sebastian-fox · 2019-02-28T14:09:41Z

No problem. But I'm confused again!

Could you try this, which I think will fail with the same error you were originally getting:

library(dplyr)
library(httr)
library(curl)
library(jsonlite)
api_path <- "https://fingertips.phe.org.uk/api/area_types/parent_area_types"
df <- api_path %>%
        GET(use_proxy(ie_get_proxy_for_url(.), username = "", password = "", auth = "ntlm")) %>%
        content("text") %>%
        fromJSON(flatten = TRUE)

If that fails, could you run this, which I think will pass.

library(dplyr)
library(httr)
library(curl)
library(jsonlite)
api_path <- "https://fingertips.phe.org.uk/api/area_types/parent_area_types"
df <- api_path %>%
        GET(use_proxy(ie_get_proxy_for_url(.), username = "", password = "", auth = "ntlm")) %>%
        content("text", encoding = "UTF-8")
write.table(df, file = "df.txt", sep = "")

If all goes well, the last line will write a file called df.txt to your working directory. Could you attach it to this discussion? (it contains the html that you are trying to read - it shouldn't contain any personal info, but if you don't feel comfortable you can email it to me at sebastian.fox@phe.gov.uk)

Thank you

WizardAKA · 2019-02-28T14:17:54Z

The first script leads to the same error as you anticipated -

Error: lexical error: invalid char in json text.
<title>Trend Micro
(right here) ------^

The second file did pass - I will send it to you because I don't understand fully what it is. Thanks.

sebastian-fox · 2019-02-28T14:22:09Z

Super - thank you. I will respond by email but I think the issue is to do with your organisation's ransomware scanner.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FingertipsR - EXTREMELY elementary question (so apologies) #65

FingertipsR - EXTREMELY elementary question (so apologies) #65

WizardAKA commented Feb 27, 2019

sebastian-fox commented Feb 27, 2019

WizardAKA commented Feb 27, 2019

sebastian-fox commented Feb 27, 2019

WizardAKA commented Feb 27, 2019 via email •

edited

sebastian-fox commented Feb 27, 2019

WizardAKA commented Feb 27, 2019

WizardAKA commented Feb 27, 2019

sebastian-fox commented Feb 27, 2019

WizardAKA commented Feb 27, 2019 •

edited

sebastian-fox commented Feb 27, 2019

WizardAKA commented Feb 27, 2019 •

edited by sebastian-fox

WizardAKA commented Feb 27, 2019

sebastian-fox commented Feb 27, 2019

WizardAKA commented Feb 27, 2019 •

edited

sebastian-fox commented Feb 28, 2019

WizardAKA commented Feb 28, 2019 •

edited by sebastian-fox

WizardAKA commented Feb 28, 2019

sebastian-fox commented Feb 28, 2019

WizardAKA commented Feb 28, 2019

sebastian-fox commented Feb 28, 2019

WizardAKA commented Feb 28, 2019

sebastian-fox commented Feb 28, 2019

FingertipsR - EXTREMELY elementary question (so apologies) #65

FingertipsR - EXTREMELY elementary question (so apologies) #65

Comments

WizardAKA commented Feb 27, 2019

sebastian-fox commented Feb 27, 2019

WizardAKA commented Feb 27, 2019

sebastian-fox commented Feb 27, 2019

WizardAKA commented Feb 27, 2019 via email • edited

sebastian-fox commented Feb 27, 2019

WizardAKA commented Feb 27, 2019

WizardAKA commented Feb 27, 2019

sebastian-fox commented Feb 27, 2019

WizardAKA commented Feb 27, 2019 • edited

sebastian-fox commented Feb 27, 2019

WizardAKA commented Feb 27, 2019 • edited by sebastian-fox

This is what I'm getting

WizardAKA commented Feb 27, 2019

sebastian-fox commented Feb 27, 2019

WizardAKA commented Feb 27, 2019 • edited

sebastian-fox commented Feb 28, 2019

WizardAKA commented Feb 28, 2019 • edited by sebastian-fox

WizardAKA commented Feb 28, 2019

sebastian-fox commented Feb 28, 2019

WizardAKA commented Feb 28, 2019

sebastian-fox commented Feb 28, 2019

WizardAKA commented Feb 28, 2019

sebastian-fox commented Feb 28, 2019

WizardAKA commented Feb 27, 2019 via email •

edited

WizardAKA commented Feb 27, 2019 •

edited

WizardAKA commented Feb 27, 2019 •

edited by sebastian-fox

WizardAKA commented Feb 27, 2019 •

edited

WizardAKA commented Feb 28, 2019 •

edited by sebastian-fox