Ken Stewart, July 2010 (updated)
“…getting seriously fed up with the state of the Australian data.”
(Harry the mystery programmer, in the HARRY_READ_ME file released with the Climategate files.)
He’s not the only one. In a commendable effort to improve the state of the data, the Australian Bureau of Meteorology (BOM) has created the Australian High-Quality Climate Site Network. However, the effect of doing so has been to introduce into the temperature record a warming bias of over 40%. And their climate analyses on which this is based appear to increase this even further to around 66%.
This post is the summation of what I believe is the first ever independent check on the official climate record of Australia. It is also the first ever independent check on the official record of an entire continent.
I will try to keep it simple.
Here is the official version of “the climate trends and variations in the Australian instrumental record” published for the Australian public, the government, and all the world at http://www.bom.gov.au/climate/change/aus_cvac.shtml
Time Series Graph using their handy trend tool:
0.1 degree C per decade, or 1 degree per100 years.
In the BOM website appears this explanation:
The temperature timeseries are calculated from homogeneous or “high-quality” temperature datasets developed for monitoring long-term temperature trends and variability …….. Where possible, each station record in these datasets has been corrected for data “jumps” or artificial discontinuities caused by changes in observation site location, exposure, instrumentation or observation procedure. This involves identifying and correcting data problems using statistical techniques, visual checks and station history information or “metadata”.
and
“High-quality” Australian climate datasets have been developed in which homogeneity problems have been reduced or even eliminated.
I have given a very brief summary of this process in https://kenskingdom.wordpress.com/2010/05/12/the-australian-temperature-record-part-1-queensland/
(I should point out that this method was changed somewhat by Della-Marta et al (2004) who also used a distance weighting method as well and included some urban stations and stations with much shorter records.)
Torok and Nicholls (1996), authors of the first (published) homogenization, rightly state that
“ A high-quality, long-term surface air temperature dataset is essential for the reliable investigation of climate change and variability.”
Here is the map showing the 100 currently used High Quality stations that supposedly meet this requirement:
Before my first post, I asked BOM to explain some of the odd things I had noticed in the Queensland data. Amongst others, this statement by Dr David Jones, Head of Climate Monitoring and Prediction, National Climate Centre, Bureau of Meteorology, in an email dated 25 April 2010, caught my eye:
“On the issue of adjustments you find that these have a near zero impact on the all Australian temperature because these tend to be equally positive and negative across the network (as would be expected given they are adjustments for random station changes).”
This statement has been the yardstick for this study.
Not having access to the list of stations, the metadata, the software used, or the expertise of BOM, the average citizen would normally accept the published results as they stand. However I wanted to have a closer look. Surely the results of any adjustments should be easy to compare with the previous record.
I downloaded annual mean maxima and minima for each of the sites from BOM Climate Data Online, calculated annual means and plotted these. Frequently, two or three stations (some closed) were needed for the entire record from 1910-2009, and even then there sometimes were gaps in the record- e.g. from 1957 to 1964 many stations’ data has not been digitised. (But 8 years of missing data is nothing- many stations have many years of estimated data “filled in” to create the High Quality series). I also downloaded the annual means from the High Quality page, and plotted them. I then added a linear trend for each.
I have exhaustively rechecked data and calculations in all 100 sites before compiling this summation. I have decided to amend only one, Bowen, by creating a splice by reducing early data and omitting some data, so that the trend matches that of HQ. This is on the basis of no overlap at all, but makes the plot lines roughly meet. Unsatisfactory, and Bowen should be excluded. The net effect on the Queensland and Australian trends is negligible (0.01 C).
Let’s look at Dr Jones’ assertion for the whole of Australia.
“…a near zero impact on the all Australian temperature …”
WRONG.
We can look at the record in a number of ways- here is the graph of the average raw and adjusted temperatures for all 100 stations. The discrepancy is obvious.
That’s 0.6 degrees C / 100 years for the raw data. The adjusted trend is 0.85.
Before anyone complains that anomalies give you a more accurate picture of trends across a large region, I also calculated anomalies from the 1961-1990 mean for the all Australian means (0.6 raw to 0.85 HQ increase)
and for all 100 stations (slightly different result): (0.6 raw to 0.9- 50%)
But the figure BOM publishes is 1.0C- that’s a two-thirds increase!
We can also look at the average adjustment for each station: + 0.23 degrees Celsius. (The table of all 100 stations is too large to include).
Or we can find the median adjustment (+ 0.275 C), and the range of adjustments:
So much for “these tend to be equally positive and negative across the network”.
We can also look at the “quality” of the High Quality stations.
Urban vs Non-urban:
“Please note: Stations classified as urban are excluded from the Australian annual temperature timeseries and trend map analyses. Urban stations have some urban influence during part or all of their record.” (http://www.bom.gov.au/cgi-bin/climate/hqsites/site_networks.cgi?variable=meanT&period=annual&state=aus)
In Part 1 I showed how 3 Queensland sites listed as urban by Torok and Nicholls (1996) are now non-urban. Della-Marta et al resurrected a number of others in other states.
The full list is: Cairns AMO, Rockhampton AMO, Gladstone MO, Port Hedland AMO, Roebourne, Geraldton AMO, Albany AMO, Alice Springs AMO, Strathalbyn, Mount Gambier AMO, Richmond AMO, Mildura AMO, East Sale AMO, Cashmore Airport, Launceston Airport.
15% of the network is comprised of sites that BOM is at pains to assure us are not used to create the climate record.
Long records:
“… the number of stations is much smaller if only stations currently operating and with at least 80 years of data are considered. To increase the number of long- term stations available, previously unused data were digitised and a number of stations were combined to create composite records… all stations in the dataset (were) open by 1915.” (Torok and Nicholls)
Torok wanted 80 years of data: Della-Marta et al and BOM have settled for much less. There are six stations with no data before 1930 (80 years ago), but BOM has included these. Some are truly dreadful: Woomera- 1950; Giles- 1957; Newman- 1966.
As well, many of the sites have large slabs of data missing, with the HQ record showing “estimates” to fill in the missing years.
Here is a graph of the number of stations with data available for each year.
Note that only 70% of raw data is available for 1910; 90% by 1930; another drop from 1945 to 1960; and the huge drop off in HQ data this decade!
Data comparison:
“Generally, comparison observations for longer than five years were found to provide excellent comparison statistics between the old and new sites…… Comparisons longer than two years, and sometimes between one and two years, were also found to be useful if complete and of good quality… Poor quality comparisons lasting less than two years were generally found to be of limited use.” (Della-Marta et al, 2004)
Wouldn’t “excellent comparison statistics” be essential for such an important purpose? Apparently not. There are many sites with less than five years of overlapping data from nearby stations (up to 20 km apart). A number of sites have no overlap at all.
This results in enormous gaps in the temperature record. Here is the map of the High Quality network, with sites deleted if they are (a) listed as urban in 1996 (b) sites with less than 80 years of observations (c) sites with less than 5 years of comparative data overlap- or sometimes all of the above!
The sites left are concentrated in Eastern and South-Western Australia, with an enormous gap in the centre. Check the (admittedly very aprroximate) scale.
And finally…
Claims made in the State of the Climate report produced by BOM and CSIRO in March 2010.
Since 1960 the mean temperature in Australia has increased by about 0.7 °C . The long term trend in temperature is clear…
TRUE. But the raw data shows the mean temperature since 1910 has increased only 0.6 C.
Australian average temperatures are projected to rise by 0.6 to 1.5 ºC by 2030.
REALLY? That would require between 5 and 12 times the rate of warming seen in the raw temperature record, or between 3 and 7.5 times that shown by BOM’s published figures.
Much of Australia will be drier in coming decades
MAYBE NOT. See https://kenskingdom.wordpress.com/2010/03/20/political-science-101/
Our observations clearly demonstrate that climate change is real.
TRUE- that’s what climate does.
CSIRO and the Bureau of Meteorology will continue to provide observations and research so that Australia’s responses are underpinned by science of the highest quality.
“Highest quality”? REALLY?
Conclusion
This study shows a number of problems with the Australian High Quality Temperature Sites network, on which the official temperature analyses are based. Problems with the High Quality data include:
- It has been subjectively and manually adjusted.
- The methodology used is not uniformly followed, or else is not as described.
- Urban sites, sites with poor comparative data, and sites with short records have been included.
- Large quantities of data are not available, and have been filled in with estimates.
- The adjustments are not equally positive and negative, and have produced a major impact on the Australian temperature record.
- The adjustments produce a trend in mean temperatures that is roughly a quarter of a degree Celsius greater than the raw data does.
- The warming bias in the temperature trend is over 40%, and in the anomaly trend is 50%.
- The trend published by BOM is 66.67% greater than that of the raw data.
The High Quality data does NOT give an accurate record of Australian temperatures over the last 100 years.
BOM has produced a climate record that can only be described as a guess.
The best we can say about Australian temperature trends over the last 100 years is “Temperatures have gone down and up where we have good enough records, but we don’t know enough.”
If Anthropogenic Global Warming is so certain, why the need to exaggerate?
It is most urgent and important that we have a full scientific investigation, completely independent of BOM, CSIRO, or the Department of Climate Change, into the official climate record of Australia.
I will ask Dr Jones for his response.
(Thanks to Lance for assistance with downloading data, and janama for his NSW work. Also Jo Nova for her encouragement.)
July 27, 2010 at 1:28 pm
[…] Ken Stewart has released his much awaited review of the Australian High Quality Sites. His conclusion: […]
July 27, 2010 at 4:17 pm
There it is folks, all done. Comments are welcome, but please keep on topic, courteous, and don’t waste time. There may be a delay between your comment and its appearance as I can only moderate 2 or 3 times a day. I’ve learnt from experience to moderate comments after some pretty foul language when I first started.
Have fun!
July 27, 2010 at 4:47 pm
Ken,
The main difficulty I have with this work is the extent to which temperatures were subjectively altered even before they gained the description “raw data”.
It is hard to discern precisely which data were used for the first round of adjustments. For example, there were more station shifts of recording instruments at Darwin airport than I have seen in the literature. (But that could be because I have not access to all of the literature).
This is not a criticism of your work. It simply notes an uncertainty which some might some to come back to you and say “You were inaccurate in your choice of data”.
It is hard to discern if Della -Marta adjusted original metadata sheets from the observer, or whether he further adjusted the adjustments of Torok or Torok & Nicholls. Then there are later adjustments, but of what?
If there was a clean slate revision from first principles, I have no idea if it would make your conclusions on trends understated or overstated.
The outcome of your work has to be treated seriously. Apart from the consequences of the BoM work on global warming mitigation & policy, there is another real problem. Temperature sets like these are used to calibrate proxies for temperature. If the instrumented period has errors, especially in bias, it follows that there will be errors in the proxy reconstruction. Who really know which Australian century was the hottest in 1,000 years? I believe that nobody does.
The BoM have emailed me that they are not responsible for what others do to their work once it is released. We do know that bodies like GISS and CRU and so on tend to make further “adjustments”, so Australia is likely to be contributing to a wrong global story.
However, do remember that these weather stations were set up initially for shipping and farming and many more recent ones for aviation close to airstrips. They were never really designed for the uses to which they are now being put, so the criticism, to be fair, has to be muted a little.
I would reserve my criticism for unscientific public statements made by bodies like BoM and CSIRO, which I repect in the main, but which have acquired some people with a seemingly “modern” view of science that does not stand the test of time, nor the lessons of history.
July 27, 2010 at 10:42 pm
Ken,
Great job.
Of course you were paid by ‘Big Oil’ and I want someone to agree with me. Penny, Penny Wong. Is that you.
July 28, 2010 at 2:36 pm
Well Done Ken!
A CC of this to the relevent government minister may help shake the tree.
July 28, 2010 at 4:38 pm
[…] following article courtesy of Kenskingdom (also posted over at WUWT for comment). A significant work and read. The conclusion is as pasted […]
July 28, 2010 at 10:31 pm
Ken, I left a reply at WUWT as well. But I would like to know what the full range of temperatures is for each year. In Canada the increase in the yearly mean temp since 1900 is caused by cooler summers, and not as cold winters. See cdnsurfacetemps.blogspot.com for my analysis of stations in Canada. We are seeing a 30% drop in the number of days above 30C since the 1920’s and a 50% drop in the number of days below -20C since the 1930’s.
It would be interesting to see what is causing your increase in the yearly mean. It’s very likely that your summers are not getting hotter at all, just your daily mins and winter mins are not dropping as much. A graph like I have on my site would show that clearly for Australia.
July 29, 2010 at 6:46 am
Yes that seems to be the general pattern here. The rise in means is largely due to increasing minimum temps. In large areas of south / eastern Australia maximum temps have been decreasing, and in an area of inland Victoria and NSW the means are decreasing- cooling! I’ve seen your graph and it would not be too hard to do as I have all the data now. I stuck to means as that is how the HQ data is available.
July 29, 2010 at 7:03 am
Excellent, if Canada is having a narrowing of the range of yearly temps, and so is Australia, that’s pretty certain so is the rest of the temporate world. This is vitally important as this shows that runnaway AGW cannot happen. These two together would nicely falisify AGW. Soon as you can do one would be nice to see, include upper and lower standard deviations of the max and mins too.
July 29, 2010 at 7:23 am
In Armagh they did too:
http://www.junkscience.com/MSU_Temps/Armagh_an.html
( http://www.arm.ac.uk/home.html )
September 6, 2010 at 9:03 pm
Great work Ken! I am still to go through much of the detail of the huge amount of work that you have done, but you are certainly doing a necessary job and doing it well.
I have also noticed the common effect of warmer minimum temperatures on the overall mean in Australian records. Richard and Ldlas look at the bigger picture in Canada and Northern Ireland and find similar trends.
So I thought that you may be interested in a similar analysis that I have recently completed for Nuuk (also known as Godthab) in Greenland. A record of 145 years (since 1866) shows no positive or negative trends in summer, winter or annual temperatures – except that there is a suspect period of 30 years between 1930 and 1960 when ALL of the temperatures are about 2 degrees warmer. For details have a look at http://members.westnet.com.au/brigun/godthab_temp.html
Keep up the good work. Regards, Brian
September 7, 2010 at 11:33 am
Thanks Brian, and look out for the next post about Australian cities coming v soon. Most temperature increase in Australia is due to minimum increase, but there is a large slab of the country that has experienced cooling. Cheers, Ken
July 28, 2010 at 11:54 pm
a book from 1913
http://www.archive.org/stream/climateweatherof00huntrich#page/n7/mode/2up
July 29, 2010 at 1:56 pm
Richard Wakefield
You have to be careful in interpretation of maxima and minima.
Here is a graph derived from BOM data for Darwin NT, where there is a several year overlap of two stations about 15 km apart.
http://i260.photobucket.com/albums/ii14/sherro_2008/Darwinoverlap.jpg?t=1280374384
The airport site is most different to the city site in winter minima, but I think that this might have more to do with UHI than climate theory.
To add complications, here is the comparable graph for Maximum monthly temperature anomalies. Both this and the above graph are calculated from daily data.
http://i260.photobucket.com/albums/ii14/sherro_2008/DARWINOVERLAPTmax.jpg?t=1280375616
July 29, 2010 at 11:05 pm
Geoff, interesting. But the differences may or may not be from UHI. If you look at my site I have done comparisons of stations that are quite close to each other, an hours drive appart, and they can show as much as a 5C difference on the same day. Local terrain, and proximity to lakes (which buffers the temps) plays as much of a role as UHI.
However, I have looked at numerous stations now, some in and out of built up areas, and the pattern is the same up here. Cooler summers and not as cold winters.
One thing you will see in all the graphs from here is the pronounced change in temperature trends from 1950s-1970s, which corresponds to the drop in the average of the mean temps on all the AGW graphs. That cannot be from anything other than normal pulsating variation.
July 29, 2010 at 11:09 pm
One additional comment on those graphs. You are showing average anomalies. I know that is the common practice, and I understand the logic of why temperature is displayed that way.
However, averages lose detail. Anomalies have to use averages, and anomalies also lose detail. To see what is really going on one needs to see the details, this is why I do not show just averages and rarely use anomalies. I want to show what is physically going on with the measured data, not what calculated data shows.
July 30, 2010 at 12:05 pm
Dear Ken,
What ever way you look at the data, the result is still a warming trend.
July 30, 2010 at 4:53 pm
No argument with you on that. But not as much warming as we have been led to believe. The point is the adjustments have increased the trend when they’re supposed to be neutral. Either BOM did not do quality assurance, or if they did they let the adjustments through. Is it incompetence or deliberate?
July 31, 2010 at 12:11 am
It’s only “warmer” if you consider cooler summers and not as cold winters “warmer”. I don’t. Milder winters means the planet is not losing as much heat, which is a good thing.
July 31, 2010 at 12:03 pm
Yes, a very modest warming trend over a geologically insignificant period of time.
As opposed to the CSIRO’s production (one could say fabrication) of a substantial trend over an even shorter interval.
August 1, 2010 at 1:14 am
What is the reason that BoM gives for excluding rural stations such as Alice Springs which the late John Daly stated “was a strategically located site”?
The annual mean temperature at the old post office in Alice Springs has shown little change since record keeping began in the late 1800’s until recently. Weather data is now obtain from instruments located at the airport.
BTW, have you checked out John Daly’s website “Still waiting for greenhouse” at http://www.John-Daly.com? Have you checked out the many plots by clicking on “Station Temperature Data”?
August 1, 2010 at 10:06 am
BOM actually includes Alice Springs now. Alice Springs record was warmed significantly (See Part 2- NT) by adjustments.
About to check the John Daly site.
Ken
August 6, 2010 at 9:46 pm
Richard Wakefield – summer versus winter changes. here are some working papers from 17 BOM rural sites in Australia. Steve Mac selected them as rural from some official list so I used them to avoid cherry picking. There is some infilling of missing values by me, but this would be below 1% of the data. I don’t like infilling at all, but it was a convenience to make faster graphs. Also, some of the graphs have a constant temp arbitrarily subtracted from them, for no other reason than to conveniently fill the graph space from top to bottom. I don’t like anomaly methods either.
The working material shows that for these examples, there is a bit of a trend to constant summers and warmer winters. See the file in Excel “Master Annual temps 17 stations in the second bundle. There is no pattern I can explain in the divergence or convergence over time between Tmax and Tmin. It varies all over the place with linear fits, but I don’t like linear least squares fits for this type of data either.
Sources at
http://www.geoffstuff.com/Final%20draft.doc
http://www.geoffstuff.com/Yearly%20Data%20SI%2017%20stat.xls
These are a bit like Ken’s work but they don’t look at the differences in series adjusted by different people.
If you want to see about Alice Springs reconstructions, try this graph & compare to Ken’s:
http://i260.photobucket.com/albums/ii14/sherro_2008/2010/AliceJones.jpg?t=1281095128
August 8, 2010 at 3:03 pm
Sorry for the slow moderation- Ive been away for a few days.
Geoff and Richard- warmer nights and winters are a signal of both greenhouse effect and UHI- how to distinguish? But Im also wondering whether theyre also a signal of warming generally from whatever source, especially inland tropics? I havent time to deal with this right now Richard, but hope to in the future.
Ken
August 28, 2010 at 4:25 am
So they re-designated urban sites as rural so they could be included?
These aren’t the droids you’re looking for.
September 13, 2010 at 8:16 am
[…] Kens Kingdom – Comments on the appalling Australian temperature record ; […]
November 1, 2010 at 9:51 am
Ken I’m sure you’re aware of NZ events but here’s a link to Quadrant Online with the result of some work done on NZ Temp records
Click to access nz_climate_crisis_gets_worse.pdf
conclusion:
It is almost impossible to avoid the conclusion
that NIWA’s political urge to prove a warming trend overcame its professional urge to apply objective scientific principles in
constructing a sound and defensible temperature series.
November 1, 2010 at 10:11 am
Continuing debacle. And they were audited by BOM!
November 1, 2010 at 10:36 am
Not too sure if that auditing has been done yet Ken; but as Warwick Hughes says ‘It’s a bit like asking Dracula to audit the blood bank’
January 8, 2011 at 11:16 am
[…] average temperature is still above the previous 2 decades average temperature. Yeah but Mr K it was warmer before the last 3 decades in Australia – why do you ignore that? More here. Graph above courtesy of Warwick Hughes – you can […]
April 26, 2011 at 9:24 am
This is slightly off the subject, but still related. I am also looking at Australian temperature trends and, as far as possible, I am trying to do my work independently to what has been done by Ken and others. But it seems as though we are all heading to the same conclusions!
I am comparing the raw temperature data (from the BOM website with the BOM adjusted data (for their HQ stations) – very revealing! I am also comparing with the adopted GISS estimates. Are the GISS estimates those used for the IPCC studies? Or have IPCC made their own adjustments? I can download the GISS figures but none from HADCRU, NCDC, etc.
Also, for their Australian nationwide trends, do we know if BOM use their complete 100(?) HQ stations? If so, do we know what weighting they give to each station? It would appear likely that stations in more remote locations (Darwin, Alice Springs, Giles, etc) would be given greater weighting than stations in the eastern more-populated region.
Cheers, Brian
April 26, 2011 at 2:51 pm
Ah ha! You notice the problem! Re: your questions- your guess is as good as mine. The data surely has to be gridded and area weighted which raises its own problems. Good luck.
Ken
June 25, 2011 at 12:50 am
[…] pur-lease. Even the alarmists only give about 0.5ºC rise in the last 50 years. Ken’s Kingdom suggests that even this derisory figure may be too high, exaggerated by the urban heat island […]
May 14, 2012 at 2:14 pm
[…] had become apparent that they had significant, but never admitted, problems (see my previous posts: The Australian Temperature Record Part 8: The Big Picture; Part 9: An Urban Myth, and Part 10: BOM’s “Explanations”.) Congratulations are due to the […]
May 16, 2014 at 12:35 pm
[…] years ago in 2010 I posted my analysis of Australia’s so-called High Quality (HQ) temperature dataset. This was a temperature record which the highest officers of the Bureau of Meteorology (BOM) […]
February 14, 2019 at 11:31 am
[…] back to 1910. Previous versions of our historic temperatures included “High Quality”, which I revealed in 2010 to have major flaws, not least being the strong warming bias; and ACORN-SAT 1, released in March 2012, proudly touted […]
May 20, 2019 at 7:23 pm
[…] back to 1910. Previous versions of our historic temperatures included “High Quality”, which I revealed in 2010 to have major flaws, not least being the strong warming bias; and ACORN-SAT 1, released in March 2012, proudly touted […]