R365: Day 10 – detrend

A lot of what I do on a day to day basis is analysis of time series. Time series exist all over the place, but they are particularly relevant in biology, such as watching something grow, or concentration of a chemical in a living cell. There are lots of ways to analyze time series, and there are probably hundreds of books and thousands of scholarly articles on the subject. One of the cool things that you can do with a time series is to ‘detrend’ it. The package {pracma} has a function called ‘detrend’ that lets you do this. Detrending a time series can be useful for observing periodic fluctuations that may not have been apparent through an overall upward or downward trend.

Detrending is done by taking a straight line though the data set and setting the slope of the line to zero, thereby plotting the residuals (‘linear’ detrending). You can also detrend by removing the mean of the dataset (‘constant’ detrending).

In this example we use the JohnsonJohnson dataset that we talked about in an earlier post (https://somethinglikescience.wordpress.com/2014/02/03/r365-day-7-datasets/). First we install necessary packages and plot out the data from JohnsonJohnson that we are interested in. Notice that it has a steep upward trend.







Next we use the coredata() function to extract just the data points from the dataset. If you try to detrend using the original dataset it will give you an error (“Error in matrix(0, n, lb) : non-numeric matrix extent”).


We want to look at the actual values versus time, so we will use the index() function to index the time series. The values are quarterly, so we say


Next we want to plot out the detrended dataset, matched against the correct time:

plot(timejj,detrend(datajj),xlab=”Time”,ylab=”J&J Stock Value Gain”)

We get a graph that looks like this:


Notice the weird concave dataset? I think this is because the increase in the stock price is not linear, so detrending the data using a linear regression would not help things.

Maybe if we transform the data we can see more useful things (*NOTE* I am wandering into unknown and dangerous territory, always be careful about the assumptions you make when transforming data!). Let’s take the log of the original data and see if that makes things better:




Wow, that worked out a lot better than I thought it would!

Let’s try detrending this data and seeing if we notice periodicity.

plot(timejj,detrend(logjj),xlab=”Time”,ylab=”Detrended Log J&J Stock”)


Wow, now THIS looks kind of interesting!

I will explore this a bit more in my next post.


One comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s