Monday, February 13, 2017

Spatial distribution of flutter in GHCN adjustment.

I posted recently on flutter in GHCN adjustment. This is the tendency of the Pairwise Homogenisation Algorithm (PHA) to produce short-term fluctuations in monthly adjustments. It arose on a recent discussion of the kerfuffle of John Bates and the Karl 2015 paper, and has been investigated by Peter O'Neill, who is currently posting on the topic. In my earlier post, I looked at the distribution of individual month adjustments, and noted that with generally zero mean, they would be heavily damped on averaging.

But I was curious about the mechanics, so here I compare the same two adjusted files (June 2015 and Feb 9 2017) collected by station. I'll show a histogram, but more interesting is the spatial distribution shown on a trackball sphere map. The histogram shows a distribution of station RMS values tapering rather rapidly toward 1°C. The map shows the flutter is strongly associated with remoteness, especially islands.

Update: I have now enabled clicking to show not only the name of the nearest station, but the RMS adjustment change there in °C. I have also adopted William's suggestion about the color scheme (white for zero, red for large).

Here is the histogram. There were some outliers, of which the greatest is Ayacucho in Peru, at about 3.3°C. I thought that was extreme, but when I looked it up, I found that almost all years got no adjusted value at all; there were 11 years with adjusted values, all differing by the same 3.3 years. I don't know how that happened, but the station would not have got into any indices, since no normal can be assigned. As you see from the plot, these extremes are rare:

So I made a WebGL globe map of the results. This is in the style of the GHCN monthly stations page. It has a triangular mesh with stations colored according to their RMS difference value, and color linearly shaded in between. I haven't shown a scale, since the colors are rather non-linear with RMS, but generally green is zero (no flutter) and red is high. The stations are shown as dots, and you can click to bring up names. You can rotate by dragging, and expand by mousing vertically with right button down.


  1. "see this post" opens inside the window, which doesn't work at all well...

    Also, if there a colour scale? I don't see one. Also, could "zero" be white? It is always nice to see the +'s and -'s, but to not have you eye attached to the 0's.

  2. William,
    Sorry about that, the link was to here. It's actually a JS issue; they sandbox off iframes, so it's hard to make things happen to the browser.

    I could fiddle with the color shading - for the moment I am just using half the default rainbow. As I mentioned in the text, I didn't include a scale because there is a rather non-linear mapping (atan) so all the colors don't get taken up by outliers. You might say that makes a scale even more important, but alas, it makes it harder. If there is interest I can make the click on stations pop up RMS values too; it means uploading more information.

  3. The high concentration of red in USA is surprizing.

  4. Nick, very interesting analysis. I'm not surprised that flutter is high with remoteness and especially remote islands. However, I notice that most of the US, with relatively very dense coverage compared to the rest of the world, also has high flutter (as Yvan also noticed). Not sure what to make of that.

  5. For Grins what I do is seperate the stations that

    A) have no adjustments
    B) have no changes to adjustments

    And then run global averages with those.

    In short use the adjustment code as a bias detector and then totally eliminate those stations.

    With BE that leaves me with about 15K stations that are never adjusted.