Identification of cloudy days based on Net Radiation data

Question

How can I identify that a particular day is cloudy or clear sky based on the daily time series of Net Radiation (calculated at FluxNet/AMERIFlux sites)? I read somewhere that clear sky radiation has sinusoidal behavior. If we get different behavior that means it is induced by clouds. I don't know how exactly approach this problem? Here is the sample data: https://drive.google.com/file/d/1f1YfWgPbg3fxhFG1DTNH9Ex3lTDqLBZp/view?usp=sharing

Answer 1

If we take your raw data and convert the Time column from text into actual dates we can work with, this will help the analysis. We will also remove the rows with NaN values:

df <- read.csv("net_rad.csv")

df$Date <- as.POSIXct(strptime(df$Time, "%m/%d/%Y"))
df <- df[!is.nan(df$NET_RAD),]

If we plot the daily net radiation we see a clear seasonal pattern:

library(ggplot2)

ggplot(df, aes(Date, NET_RAD)) + geom_line()

We could use a seasonal decomposition to see these seasonal effects and even remove them:

net_rad_ts <- ts(df$NET_RAD, frequency = 365)
net_rad_decomp <- stats::decompose(net_rad_ts)
plot(net_rad_decomp)

We can see that as well as the seasonal variation, there is a background trend and considerable random variation within each season. We can also see that the variance appears to be higher in summer than in winter (which makes sense)

Without a clear cut-off to define what "cloudy" means in terms of the net radiation, we can simply specify that we want to identify days when the solar radiation was below average after accounting for time of year and trend over time. So we can do:

df$cloudy <- as.logical(net_rad_decomp$random < 0)

This allows us to plot the data showing the days that were unseasonably bright in yellow, and those that were unseasonably dim in gray:

ggplot(df, aes(Date, NET_RAD, colour = cloudy)) + 
  geom_point() +
  scale_color_manual(values = c("gold", "gray40")) +
  theme_bw()

This gives us a rough approximation to your answer that will have to suffice until you work out a good definition of "cloudy"