Add function to deal with UTF-8 characters

[karthik]

No description provided.

[hadley]

Here's something that I wrote a couple of years ago:

non_ascii <- function(x) {
  any(charToRaw(x) > 0x7F)
}

0x7F = 127 - any values higher than that imply non-ascii.

[karthik]

working on this now

[harrysouthworth]

According to it's roxygen
#' This test will check every column in a data.frame for possible unicode characters.
But doesn't it just test the column names, not the contents of the columns?
ut8 <- simplify2array(lapply(colnames(dat),non_ascii))

I just wrote the following which I /think/ check the contents of the columns. (Presumably, you do want to test the column names as well, though.) It's just a wrapper for Hadley's function.

non_ascii_cols <- function(x) {
x <- x[, sapply(x, function(X) is.character(X) | is.factor(X))]
x[, sapply(x, is.factor)] <- apply(x[, sapply(x, is.factor)], 2, as.character)

res <- sapply(x, function(X) apply(matrix(X, ncol=1), 1, function(Z) non_ascii(Z) ))
apply(res, 2, function(X) sum(X) > 0)
}