{"id":1406,"date":"2012-12-19T20:36:33","date_gmt":"2012-12-20T01:36:33","guid":{"rendered":"http:\/\/lukemiller.org\/?p=1406"},"modified":"2012-12-23T14:01:29","modified_gmt":"2012-12-23T19:01:29","slug":"generating-polygon-boundaries-for-plotting-simple-time-series-data-with-missing-data","status":"publish","type":"post","link":"https:\/\/lukemiller.org\/index.php\/2012\/12\/generating-polygon-boundaries-for-plotting-simple-time-series-data-with-missing-data\/","title":{"rendered":"Generating polygon boundaries for plotting simple time series data with missing data"},"content":{"rendered":"

Every so often I want to plot some data with pretty upper and lower error bounds, such as temperature data through time, perhaps with the maximum and minimum temperature range or standard error bounds for averaged data. The polygon( ) function can make those sorts of pretty plots. However, I’ll often have chunks of missing data for periods of time, so I have to break up the polygons that go with the plotted data. I could swear I wrote a function to do this several months ago, but it’s lost in a pile of other scripts, so I re-wrote a function to accomplish the task.<\/p>\n

\"A<\/a>
A simple example, where the purple\/blue polygon might represent the range around the mean value through time.<\/figcaption><\/figure>\n

The plot above shows the kind of thing I’m talking about. I might have a series of times, or dates, or just random x data, and associated y values (a mean, and upper and lower bounds around the mean). But somewhere in the data set the y-values are missing. The goal is to simplify the process of generating the x-y pair values needed for polygon( ) to plot the error values, and automatically deal with any breaks in the data set.<\/p>\n

The R script to generate the function polylims( ) is hosted on GitHub.<\/a> The function should handle datasets with any number of missing chunks, but it will also work equally well on datasets without missing points. John McKinlay pointed me to the rle( ) function shortly after I originally posted this, and it helps shrink the amount of code considerably. The modified code is shown below:<\/p>\n

\n
\n
polylims = function<\/span><\/a>(<\/span>xdata,<\/span> ydata,<\/span> ydata2)<\/span> {<\/span>\r\n\t# A function to find the beginings and endings of each run of real numbers<\/span>\r\n\t# in a set of data, in order to create polygons for plotting. The assumption<\/span>\r\n\t# is that ydata, ydata2, and xdata vectors are the same length, and that <\/span>\r\n\t# the ydata and ydata2 vectors contain NA values for missing data. The <\/span>\r\n\t# same values in ydata and ydata2 must be missing. The output will be <\/span>\r\n\t# a list of data frames of x and y values suitable for plotting polygons.<\/span>\r\n\r\n\t# Use rle function to find contiguous real numbers<\/span>\r\n\trl = rle<\/span><\/a>(<\/span>is.na<\/span><\/a>(<\/span>ydata)<\/span>)<\/span>\r\n\tstarts = vector<\/span><\/a>(<\/span>)<\/span>\r\n\tends = vector<\/span><\/a>(<\/span>)<\/span>\r\n\tindx = 1<\/span>\r\n\tfor<\/span> (<\/span>i in<\/span> 1<\/span>:<\/span>length<\/span><\/a>(<\/span>rl$<\/span>lengths)<\/span>)<\/span>{<\/span>\r\n\t\tif<\/span> (<\/span>rl$<\/span>values[<\/span>i]<\/span>)<\/span>{<\/span>\r\n\t\t\t# Value was NA, advance index without saving the numeric values<\/span>\r\n\t\t\tindx = indx +<\/span> rl$<\/span>lengths[<\/span>i]<\/span>\r\n\t\t}<\/span> else<\/span> {<\/span>\r\n\t\t\t# Value was a real number, extract and save those values<\/span>\r\n\t\t\tstarts = c<\/span><\/a>(<\/span>starts,<\/span>indx)<\/span>\r\n\t\t\tends = c<\/span><\/a>(<\/span>ends,<\/span> (<\/span>indx +<\/span> rl$<\/span>lengths[<\/span>i]<\/span> -<\/span> 1<\/span>)<\/span>)<\/span>\r\n\t\t\tindx = indx +<\/span> rl$<\/span>lengths[<\/span>i]<\/span>\r\n\t\t}<\/span>\t\r\n\t}<\/span>\r\n\r\n\t# At this point the lengths of the vectors 'starts' and 'ends' should be<\/span>\r\n\t# equal, and each pair of values represents the starting and ending indices<\/span>\r\n\t# of a continuous set of data in the ydata vector.<\/span>\r\n\r\n\t# Next separate out each set of continuous ydata, and the associated xdata,<\/span>\r\n\t# and format them for plotting as a polygon.<\/span>\r\n\tpolylist = list<\/span><\/a>(<\/span>)<\/span>\r\n\tfor<\/span> (<\/span>i in<\/span> 1<\/span>:<\/span>length<\/span><\/a>(<\/span>starts)<\/span>)<\/span>{<\/span>\r\n\t\ttemp = data.frame<\/span><\/a>(<\/span>x = c<\/span><\/a>(<\/span>xdata[<\/span>starts[<\/span>i]<\/span>]<\/span>,<\/span>xdata[<\/span>starts[<\/span>i]<\/span>:<\/span>ends[<\/span>i]<\/span>]<\/span>,<\/span>\r\n\t\t\t\t\t\trev<\/span><\/a>(<\/span>xdata[<\/span>starts[<\/span>i]<\/span>:<\/span>ends[<\/span>i]<\/span>]<\/span>)<\/span>)<\/span>,<\/span>\r\n\t\t\t\ty = c<\/span><\/a>(<\/span>ydata[<\/span>starts[<\/span>i]<\/span>]<\/span>,<\/span>ydata2[<\/span>starts[<\/span>i]<\/span>:<\/span>ends[<\/span>i]<\/span>]<\/span>,<\/span>\r\n\t\t\t\t\t\trev<\/span><\/a>(<\/span>ydata[<\/span>starts[<\/span>i]<\/span>:<\/span>ends[<\/span>i]<\/span>]<\/span>)<\/span>)<\/span>)<\/span>\r\n\t\tpolylist[<\/span>[<\/span>i]<\/span>]<\/span> = temp\t\r\n\t}<\/span>\r\n\tpolylist\r\n\t# You can iterate through the items in polylist and plot them as <\/span>\r\n\t# polygons on your plot. Use code similar to the following:<\/span>\r\n\t#\tfor (i in 1:length(polylist)){<\/span>\r\n\t#\t\t\tpolygon(polylist[[i]]$x, polylist[[i]]$y, <\/span>\r\n\t#\t\t\t\tcol = rgb(0.5,0.5,0.5,0.5), border = NA)<\/span>\r\n\t#\t}<\/span>\r\n}<\/span><\/pre>\n<\/div>\n<\/div>\n
Created by Pretty R at inside-R.org<\/a><\/p>\n
Run that once, and it creates a function called polylims that you can use in your R session. Below is an example of how I use it. We’ll start by generating some junk data consisting of a time series of y-values, with upper and lower error bounds. Assemble those into a data frame, and lose some of the data in the middle of the series.<\/p>\n
\n
\n
# Create a sequence of time points<\/span>\r\ntimes = seq<\/span><\/a>(<\/span>as.POSIXct<\/span><\/a>(<\/span>'2012-12-10 00:00'<\/span>)<\/span>, <\/span>as.POSIXct<\/span><\/a>(<\/span>'2012-12-10 12:00'<\/span>)<\/span>,<\/span> \r\n\t\tby<\/span><\/a> = 300<\/span>)<\/span>\r\n# Create some fake y data, one value for each time point<\/span>\r\ny = rnorm<\/span><\/a>(<\/span>length<\/span><\/a>(<\/span>times)<\/span>,<\/span> mean<\/span><\/a> = 10<\/span>,<\/span> sd<\/span><\/a> = 0.5<\/span>)<\/span>\r\n# Create a fake upper standard error limit for each y data point<\/span>\r\nupperse = y +<\/span> 2<\/span>\r\n# Create a fake lower standard error limit for each y data point<\/span>\r\nlowerse = y -<\/span> 2<\/span>\r\n# Assemble all this into a data frame<\/span>\r\nmydata = data.frame<\/span><\/a>(<\/span>times, <\/span>y, <\/span>upperse, <\/span>lowerse)<\/span>\r\n\r\n# Now we'll \"lose\" some data in the middle of the data set.<\/span>\r\nmydata[<\/span>45<\/span>:<\/span>50<\/span>,<\/span>-<\/span>1<\/span>]<\/span> = NA<\/span><\/pre>\n<\/div>\n<\/div>\n
After doing that, I’ll call the polylims( ) function, passing it the x data and the upper\/lower error bounds. The original ‘y’ data aren’t needed here, just the upper\/lower bounds.<\/p>\n
\n
\n
# Call the polylims function, supplied with x data, and the two sets of y data<\/span>\r\n# that represent our upper and lower error boundaries here<\/span>\r\npolys = polylims(<\/span>mydata$<\/span>times, <\/span>mydata$<\/span>upperse, <\/span>mydata$<\/span>lowerse)\r\n\r\n<\/span><\/pre>\n<\/div>\n<\/div>\n
The output object ‘polys’ is a list of data frames, each containing a column x and column y, which hold the x-y pairs needed to plot each polygon. Since there is a break in the example data, ‘polys’ contains two such data frames. Continue by plotting the data.<\/p>\n
\n
\n
# Draw the basic plot<\/span>\r\nplot<\/span><\/a>(<\/span>mydata$<\/span>times,<\/span> mydata$<\/span>y,<\/span> type = 'l'<\/span>,<\/span> ylim = c<\/span><\/a>(<\/span>5<\/span>,<\/span>15<\/span>)<\/span>,<\/span> xlab = 'Time'<\/span>,<\/span>\r\n\t\tylab = 'Y'<\/span>,<\/span> las = 1<\/span>)<\/span>\r\n# Plot the polygons using the output from the polylims function<\/span>\r\nfor<\/span>(<\/span>i in<\/span> 1<\/span>:<\/span>length<\/span><\/a>(<\/span>polys)<\/span>)<\/span>{<\/span>\r\n\tpolygon<\/span><\/a>(<\/span>polys[<\/span>[<\/span>i]<\/span>]<\/span>$<\/span>x, <\/span>polys[<\/span>[<\/span>i]<\/span>]<\/span>$<\/span>y,<\/span> col<\/span><\/a> = rgb<\/span><\/a>(<\/span>0<\/span>,<\/span>0<\/span>,<\/span>1<\/span>,<\/span>0.5<\/span>)<\/span>,<\/span> border = NA<\/span>)<\/span>\r\n}<\/span>\r\n# Replot the main y line again on top of the polygon<\/span>\r\nlines<\/span><\/a>(<\/span>mydata$<\/span>times, <\/span>mydata$<\/span>y,<\/span> lwd = 2<\/span>,<\/span> col<\/span><\/a> = 'black'<\/span>)<\/span><\/pre>\n<\/div>\n<\/div>\n
The for loop in the middle steps through each of the items in the ‘polys’ list, extracts the x&y values, and uses them to draw the polygon. After drawing the polygon, it’s usually a good idea to replot the original line over top, since the polygon may obscure it.<\/p>\n","protected":false},"excerpt":{"rendered":"
Every so often I want to plot some data with pretty upper and lower error bounds, such as temperature data through time, perhaps with the maximum and minimum temperature range or standard error bounds for averaged data. The polygon( ) function can make those sorts of pretty plots. However, I’ll often have chunks of missing […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[218],"tags":[17,149,58],"class_list":["post-1406","post","type-post","status-publish","format-standard","hentry","category-r-project","tag-plotting","tag-polygon","tag-r-project"],"_links":{"self":[{"href":"https:\/\/lukemiller.org\/index.php\/wp-json\/wp\/v2\/posts\/1406","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lukemiller.org\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/lukemiller.org\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/lukemiller.org\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/lukemiller.org\/index.php\/wp-json\/wp\/v2\/comments?post=1406"}],"version-history":[{"count":6,"href":"https:\/\/lukemiller.org\/index.php\/wp-json\/wp\/v2\/posts\/1406\/revisions"}],"predecessor-version":[{"id":1413,"href":"https:\/\/lukemiller.org\/index.php\/wp-json\/wp\/v2\/posts\/1406\/revisions\/1413"}],"wp:attachment":[{"href":"https:\/\/lukemiller.org\/index.php\/wp-json\/wp\/v2\/media?parent=1406"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/lukemiller.org\/index.php\/wp-json\/wp\/v2\/categories?post=1406"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/lukemiller.org\/index.php\/wp-json\/wp\/v2\/tags?post=1406"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}