R bloggers

Syndicate content
R news and tutorials contributed by (563) R bloggers
Updated: 8 hours 49 min ago

Find the last day of the month

Fri, 2014-10-17 16:33

(This article was first published on tradeblotter » R, and kindly contributed to R-bloggers)

I have different sets of monthly data that I want to align and evaluate once a month. The different sources report the timestamp of the monthly data differently – one reports the date without the day, another as the last business day, and another as the last day of the month. For what I want to do, I’m content to align the data to the last day of the month. How do I do that in R?

The solution for this isn’t obvious in Excel or OpenOffice, either, but I thought it would be at least similarly simple in R. I’d looked around, and the best solution I’d found on r-help was something of a kludge:

index(x) = as.POSIXct(as.Date(as.yearmon(index(x)),frac=1), tz="UTC")

I said it wasn’t pretty, didn’t I? You are reading that right – convert the index first to yearmon, then to Date (with frac representing the fraction of a month with a number between 1 and 0), then finally to POSIXct.

That’s worked fine for me until I recently found strange, eight-second misalignments between my data sets that seemed to be caused by the transformation above. Timezone issues, perhaps? On Jeff Ryan’s advice to keep all my date transformations in POSIX, I finally found a better solution in r-help that had been posted by Whit Armstrong years and years ago. I’m not sure why I didn’t find it earlier, but his solution was somewhat more general than what I have been looking for. I’ve stripped his answer down a bit to the following utility function:

eom <- function(date) { # date character string containing POSIXct date date.lt <- as.POSIXlt(date) # add a month, then subtract a day: mon <- date.lt$mon + 2 year <- date.lt$year year <- year + as.integer(mon==13) # if month was December add a year mon[mon==13] <- 1 iso = ISOdate(1900+year, mon, 1, hour=0, tz=attr(date,"tz")) result = as.POSIXct(iso) - 86400 # subtract one day result + (as.POSIXlt(iso)$isdst - as.POSIXlt(result)$isdst)*3600 }

This wasn’t as simple as I was hoping for, but once functionalized it becomes simple. The premise is the same as the spreadsheet solution – add a month then subtract a day.

# Whit's example x <- seq(as.POSIXct("2001-01-10"),as.POSIXct("2005-12-10"),by="months") data.frame(before=x,after=eom(x)) before after 1 2001-01-10 2001-01-31 2 2001-02-10 2001-02-28 3 2001-03-10 2001-03-31 4 2001-04-10 2001-04-30 5 2001-05-10 2001-05-31 6 2001-06-10 2001-06-30 ... snip ...

Ah, much better. Thanks, Whit!


To leave a comment for the author, please follow the link and comment on his blog: tradeblotter » R. R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...

Categories: Methodology Blogs

Statistics doesn’t have to be so hard: simulate!

Fri, 2014-10-17 10:17

(This article was first published on Revolutions, and kindly contributed to R-bloggers)

My second-favourite keynote from yesterday's Strata Hadoop World conference was this one, from Pinterest's John Rauser. To many people (especially in the Big Data world), Statistics is a series of complex equations, but a just a little intuition goes a long way to really understanding data. John illustrates this wonderfully using an example of data collected to determine whether consuming beer causes mosquitoes to bite you more:  

 

The big lesson here, IMO, is that so many statistical problems can seem complex, but you can actually get a lot of insight by recognizing that your data is just one possible instance of a random process. If you have a hypothesis for what that process is, you can simulate it, and get an intuitive sense of how surprising your data is. R has excellent tools for simulating data, and a couple of hours spent writing code to simulate data can often give insights that will be valuable for the formal data analysis to come. 

(By the way, my favourite keynote from the connference was Amanda Cox's keynote on data visualization at the New York Times, which featured several examples developed in R. Sadly, though, it wasn't recorded.)

O'Reilly Strata: Statistics Without the Agonizing Pain

To leave a comment for the author, please follow the link and comment on his blog: Revolutions. R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...

Categories: Methodology Blogs

Add An Export Module to Shiny-app for Highcharts Figure

Fri, 2014-10-17 09:01

(This article was first published on Category: R | Huidong Tian's Blog, and kindly contributed to R-bloggers)

Highcharts is the best JavaScript chart library, I feel. By default, there is an export button which can let you save the interactive figure to PNG, JPEG, PDF and SVG format, and one important point is, you can use it in your shiny app through R package rCharts. However, there are some limits in such a way, the default exporting engine is based http, so if your shiny-app’s portal is https, then you can’t use the exporting feature (It seems Highcharts has also a https based export engine). I have spent a lot of time to configure a Highcharts export server, it works internally, but not outside. I can’t figure out the problem, but I found another solution.

The figures created by Highcharts are actually SVG element in HTML, we can easily extract that element using JavaScript, and fill it into a textInput of Shiny, when textInput received the svg element, its value changed, so it triggered Shiny server to response, then we can let Shiny-server to save the SVG element onto our server. We can convert that SVG file into the format we want, and download it use downloadHandler.

The most difficult parts to R users which not familiar with JavaScript and Linux are:

1. Extract the SVG element and fill it to textInput.

ruby // Export module function figExport() { // Select chart and get svg data; var chart = $(".shiny-html-output.rChart.highcharts.shiny-bound-output").highcharts() var svg = chart.getSVG(); var svg = svg.replace(/<g class="highcharts-button".*?g>/, "") var svg = svg.replace(/<g class="highcharts-tooltip".*?g>/, "") // Empty target object and fill with new svg data; var target = $("#svg"); target.val(""); target.val(svg); target.trigger("change"); $("#Export").attr("style", "height: 105px;") $(".ePanel").attr("style", "visibility: visible;"); $(".wPanel:has(#export)").mouseleave(function(){ $(".ePanel").attr("style", "visibility: hidden;"); $("#Export").attr("style", "height: 30px;") }); }

We can bind this function to a button: when clicking the button, browser will extract the SVG element, fill it into an textInput and tell Shiny-server that the textInput got a new value. After receiving that textInput’s value has changed, Shiny-server will save its value to a SVG file.

``` ruby p <- reactiveValues() observe({ p$tempSVG <- paste(tempfile(), “svg”, sep =”.”) p$tempPDF <- paste(tempfile(), “pdf”, sep =”.”) p$tempPNG <- paste(tempfile(), “png”, sep =”.”) if (!is.null(input$svg)) { if (nchar(input$svg)>0) { writeLines(input$svg, p$tempSVG) } } })

```

2. Convert SVG file to other formats under Linux.

There are several programs can convert svg file to other formats, I choose inkscape (sudo apt-get install inkscape), after installing it on your server, we can call it through R using system function:

``` ruby # Export figure as PDF; output$uiFigPDF <- renderUI({ downloadLink(outputId = “figPDF”, label = E2N(“Export figure as PDF”)) }) output$figPDF <- downloadHandler( filename = function() { paste(‘Highcharts-‘, Sys.Date(), ‘.pdf’, sep=’’) }, content = function(file) { system(paste(“inkscape -f”, p$tempSVG, “-A”, p$tempPDF)) file.copy(p$tempPDF, file) } )

```

That’s it!

We can also add an option to change the dimension of the figure:

``` ruby

output$uiDim <- renderUI({ textInput(inputId = “dim”, label = E2N(“Figure dimensions (W x H):”), value = “1000 x 1200”) })

Inside of ‘renderChart’ in server.r

if (!is.null(input$dim)) { H$exporting(sourceWidth = as.numeric(strsplit(input$dim, “ *x *”)[[1]][1]), sourceHeight= as.numeric(strsplit(input$dim, “ *x *”)[[1]][2]), enabled = FALSE) }

```

And to export the figure data is even simple:

``` ruby # Export figure data; output$uiFigDat <- renderUI({ downloadLink(outputId = “figDat”, label = E2N(“Export figure data”)) }) output$figDat <- downloadHandler( filename = function() { paste(‘Highcharts data-‘, Sys.Date(), ‘.csv’, sep=’’) }, content = function(file) { write.table(p$data, file, row.names = FALSE, quote = FALSE, sep = “;”, na = “”, dec = ifelse(input$lang == “no”, “,”, “.”)) } )

```

where p$data was assign before. Here is a demo, and you can find the source code on my gist.

To leave a comment for the author, please follow the link and comment on his blog: Category: R | Huidong Tian's Blog. R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...

Categories: Methodology Blogs

Design A Bilingual Shiny Application

Fri, 2014-10-17 07:25

(This article was first published on Category: R | Huidong Tian's Blog, and kindly contributed to R-bloggers)

It’s a common requirement that your Shiny application should be bilingual. I have tried several methods and finally I got one which is not so difficult to maintain.

My idea is:

  • Create a bilingual dictionary or a lookup table which contains three columns: value, English name and Norwegian name, like this one:

``` ruby value EN NO time Time Tid day Day Dag week Week Uke month Month Måned

```

  • Create a radio input with English and Norsk two options in file ui.R.

ruby radioButtons(inputId = "lang", label = "", choices = c("English" = "en", "Norsk" = "no"), selected = "English"),

  • Create a function that converts value’s name to Norsk if user choose the Norsk radio.

``` ruby ## Bilingual lookup table; LANG <- unique(read.csv(“www/lang.csv”, sep = “;”, stringsAsFactors = FALSE)) E2N <- function(en, D = LANG) { for (i in 1:length(en)) { id <- which(D$value %in% en[i]) if (length(id) == 0) { id <- which(D$EN %in% en[i]) if (length(id) == 1) { en[i] <- ifelse(input$lang == “no”, D$NO[id], D$EN[id]) } else if (length(id) > 1) { print(“Mulit-variable matched!”) } } else if (length(id) == 1) { en[i] <- ifelse(input$lang == “no”, D$NO[id], D$EN[id]) } else if (length(id) > 1) { print(“Mulit-variable matched!”) } } return(en) }

```

The ui.R file can be written as follow:

``` ruby shinyUI(pageWithSidebar( ## English or Norwegain? radioButtons(inputId = “lang”, label = “”, choices = c(“English” = “en”, “Norsk” = “no”), selected = “English”), # Select unit; sidebarPanel( uiOutput(“uiUnit”) ), mainPanel( h1(textOutput(“text1”)) ) ))

```

In server.R file, we need load the E2N function first:

``` ruby shinyServer(function(input, output, session) { LANG <- data.frame(value = c(“time”, “day”, “week”, “month”), EN = c(“Time”, “Day”, “Week”, “Month”), NO = c(“Tid”, “Dag”, “Uke”, “Maaned”), stringsAsFactors = FALSE) observe({ E2N <- function(en, D = LANG) { for (i in 1:length(en)) { id <- which(D$value %in% en[i]) if (length(id) == 0) { id <- which(D$EN %in% en[i]) if (length(id) == 1) { en[i] <- ifelse(input$lang == “no”, D$NO[id], D$EN[id]) } else if (length(id) > 1) { print(“Mulit-variable matched!”) } } else if (length(id) == 1) { en[i] <- ifelse(input$lang == “no”, D$NO[id], D$EN[id]) } else if (length(id) > 1) { print(“Mulit-variable matched!”) } } return(en) } ## Input; output$uiUnit <- renderUI({ Items <- c(“day”, “week”, “month”) selectInput(inputId = “unit”, label = E2N(“time”), choices = E2N(Items), multiple = TRUE) }) ## Output; output$text1 <- renderText({ E2N(input$unit) }) }) })

```

One shortcoming of this method is: when switching language, the application goes back to its original state. You can run above demo in R console:

``` ruby library(shiny) runGist(“https://gist.github.com/withr/3c01413b70b8a82706ab”)

```

To leave a comment for the author, please follow the link and comment on his blog: Category: R | Huidong Tian's Blog. R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...

Categories: Methodology Blogs