Wednesday, February 1, 2017

Homogenisation and Cape Town.

An old perennial in climate wars is the adjustment of land temperature data. Stations are subject to various changes, like moving, which leads to sustained jumps that are not due to climate. For almost any climate analysis that matters, these station records are taken to be representative of some region, so it is important to adjust for the effect of these events. So GHCN publishes an additional list of adjusted temperatures. They are called homogenised with the idea that as far as can be achieved, temperatures from different times are as if measured under like conditions. I have written about this frequently, eg here, here and here.

The contrarian tactic is to find some station that has been changed and beat the drum about rewriting history, or some such. It is usually one where the trend has changed from negative to positive. Since adjustment does change values, this can easily happen. I made a Google Maps gadget here which lets you see how the various GHCN gadgets are affected, and posted histograms here. This blog started its life following a classic 2009 WUWT sally here, based on Darwin. That was probably the most publicised case.

There have been others, and their names are bandied around in skeptic circles as if they were Agincourt and Bannockburn. Jennifer Marohasy has for some reason an irrepressible bee in her bonnet about Rutherglen, and I think we'll be hearing more of it soon. I have a post on that in the pipeline. One possible response is to analyse individual cases to show why the adjustments happened. An early case was David Wratt, of NIWA on Wellington, showing that the key adjustment happened with a move with a big altitude shift. I tried here to clear up Amberley. It's a frustrating task, because there is no acknowledgement - they just go on to something else. And sometimes there is no clear outcome, as with Rutherglen. Reykjavik, often cited, does seem to be a case where the algorithm mis-identified a genuine change.

The search for metadata reasons is against the spirit of homogenisation as applied. The idea of the pairwise algorithm (PHA) used by NOAA is that it should be independent of metadata and rely solely on numerical analysis. There are good reasons for this. Metedata means human intervention, with possible bias. It also inhibits reproducibility. Homogenisation is needed because of the possibility that the inhomogeneities may have a bias. Global averaging is very good at suppressing noise(see here and here), but vulnerable to bias. So identifying and removing possibly biased events is good. It comes with errors, which contribute noise. This is a good trade-off. It may also create a different bias, but because PHA is automatic, it can be tested for that on synthetic data.

So, with that preliminary, we come to Cape Town. There have been rumblings about this fromone Philip Lloyd at WUWT, most recently here. Sou dealt with it here, and Tamino touched on it here, and an earlier occurrence here. It turns out that it can be completely resolved with metadata, as I explain at WUWT here. It's quite interesting, and I have found out more, which I'll describe below the jump.

Cape Town (CT) has a long record, from about 1857. The GHCN datasheet for the station is here. I showed the graphs from it:

You can see in the top graph a sudden descent in about 1960. GHCN said this was spurious, and made the adjustment seen in the bottom graph. This had the effect of turning a neutral trend into an uptrend, which makes it red meat for WUWT. So how did this happen?

An important clue is that the new Cape Town airport was completed in 1960, and that is the current location of the station. But of course, it wasn't in 1857. In fact, the observations were taken for nearly a century from the then Royal Observatory, founded in 1820. This is notable enough that an inner suburb is named after it. But it seems that in Jan 1961, it trekked out to the new site:

How do we know, exactly? Well, the site at the observatory did not cease reporting. The record continues as another GHCN site, CAPE TOWN-SAAO:CSIR. The important clues here are:
  1. SAAO stands for South African Astronomical Observatory (its current name). CSIR is the SA govt research outfit (similar to CSIRO here), which would have been the operator. Berkeley Earth records SAAO:CSIR as a former name of Cape Town station, and says there was a move in 1861.
  2. The GHCN number of Cape Town is 14168816000 and of SAAO:CSIR is 14168816002. The last three digits were used in V2 of GHCN to mark duplicates - ie different records from what it regarded as the same place.

So I looked up the GHCN records for both locations. The record for SAAO starts in 1960, and runs parallel to Cape Town for a year, and diverges substantially in Jan 1961 - a divergence that is then maintained. Here is a table:

It looks as if the two sets of instruments were on the same site in 1960, and then separated, with Cape Town at the new airport being substantially cooler. Here now is a plot of CT unadjusted (pink), CT adjusted, and CT Observatory (since 1960):

What was objected to was the fact that unadjusted CT seemed to have no trend, while the adjusted had an uptrend. But if you consider the composite of pink to 1960 and then blue, that is the actual unadjusted record of the Observatory site. And the adjusted curve looks very similar to that record. There is further adjustment around 1890, which I haven't looked into; the shift in 1960 was the main issue.

Does it matter that they are separated by about 1.2°C? No, because both places (Obs and airport) are equally representative of the region.

Why didn't GHCN do what I did - make a full record of the Obs site, and regard the airport as a new site? Presumably the SA Met sent the composite Obs/airport record as the official CT, probably believing that the airport would be better supported in future. It seems this worked out, as the Obs record trails off in 2002.


  1. CRU data looks slightly different, but in the end they come to the same conclusion.
    Data for their Google earth interface is here:

    The historical Observatory station has seemingly moved from its original location to a city airport. It has data from 1856 to 1991:

    The new international airport has data between 1951 and 2016:

    When everything in the 5x5 degrees gridcell is merged, the two stations above and others, the result looks like this:
    No drop in 1960 and no flat trend

    I guess that the station data above is what the SA met office actually reports to CRU. CRU then uses all stations (unaltered) that have sufficient data in the reference period 1961-1990 to construct the average gridcell temps and trends.
    I hope that the links above work permanently

    1. Olof,
      That's interesting. It looks like they have done what I suggested in the graph - keep the Observatory as one record and the airport as another. It avoids adjustment, but the Observatory record finishes earlier, and as less reliable in the late stages.

      DF Malan is the old name for CT International.

    2. Luckily CRU has sufficient data (1961-1990) for the Observatory to calculate anomalies. Otherwise they would have to discard the historical part of the Cape Town temperature record.
      I wonder why CRU doesn't use first difference methods, so they could use all data records that are too short or outside the 1961-1990 window. The are a lot of new automatic stations coming up in remote places like the Arctic and Antarctic, that cant contribute to the CRU datasets right now.

      Anyway, it looks like Gistemp discard Cape Town data prior to 1942:
      I guess that the reason is a lack of nearby rural stations for doing the UHI adjustment.
      There is only one nearby "rural" station (with a brightness index below 11) that cover the early 1900ths:
      The station is 481 km away from Cape town, which may be to far to qualify for UHI adjustment. Or is it in the wrong sub-box?

      Another thing to notice is that Gistemp actually reduce the trend of Cape Town stations. Deniers complain a lot about about "GISS adjustments" and tampering with data. Do they really dislike reduction of temperature trends?

  2. Nick: Thanks for the useful post, but it raises just as many questions in my mind as it answers. The 20th century trend at Cape Town is either about 1 K/century or 2.7 K/century - depending on whether or not this breakpoint correction is appropriate or not. How sure are we that 1.7K is the correct magnitude? I averaged 36 months of your differences to get 1.7 K, but they range from 1.0 to 2.4 K. At BEST, these differences often show a large correlation with season. Even if a correction is warranted (and it probably is) there is uncertainty associated with this correction that is discarded when the new trend is used. Let's say that the uncertain in this correction means the 20th century trend is only known to an accuracy of 2.4-3.0 K/century. Some records have breakpoints identified an average of once a decade, so the uncertainty in their trends should be recognized as being the same magnitude as the trend itself.

    Then you need to ask why the SA Met chose to relocate the official station to the airport. Perhaps they recognized that the Observatory location had gradually become urbanized and they relocated to a location that was as rural as 1900. In other words, the Observatory had gradually become biased and the relocation to the airport eliminated that bias or part of it. The truth is that we don't know; we are guessing - even with documented station moves. These breakpoints are evidence of uncertainty in the record, not artifacts that we can correct with the confidence you suggest from the data above.


    1. Frank,
      "How sure are we that 1.7K is the correct magnitude?"
      I couldn't see where the 1.7K cam from. Is that the correction? From the NOAA chart, the correction made seems to be about 1.2K, which seems a bit low - I think 1.7 looks better.

      But yes, adjustment has increased the uncertainty about trend at Capetown. This is not a figure of much interest in itself. Global and regional averages are, and there what matters is the elimination of bias, which is not reduced by averaging. Noise is reduced, to such an extent that the increse in noise will have negligible effect.

      You can see that here. The adjusted trend might be in the range 2.4-3 K/cen. Without adjustment the trend would be close to zero. The uncertainty would be less, but the result is wrong.

      I think SA transferred the official station for the usual reason - international airports have guaranteed continuity. No readings - no flights. I doubt if, in 1961, they were thinking about uncertainty of trend.