TechComm Labs

There are two good posts today to help understand the brouhaha in climatology. Iowahawk has Fables of the Reconstruction (Or, How to Make Your Own Hockey Stick) that has a step by step tutorial on how to replicate Mann’s famous hockey stick graph complete with OOo calc spreadsheets.

My goal was to provide interested people with a hands-on DIY example of the basic statistical methodology underlying temperature reconstruction, at least as practiced by the leading lights of “Climate Science.” … Is there anything wrong with this methodology? Not in principle. In fact there’s a lot to recommend it. … The devil, as they say is in the details. In each of the steps there is some leeway for, shall we say, intervention.

That deals with what is called “homogenized” data where temperature records have been ‘adjusted’ to smooth out variability and compensate for known errors. The reason to do this has much to do with such things as the accuracy of measure, which surfacestations.org notes is considered to be greater than 2C for 2/3 of US surface stations, as well as the fact that temperature is measured at a point in space and time and what you are really after is atmospheric heat content.

Basil Copeland takes on the temperature measures problem in Would You Like Your Temperature Data Homogenized, or Pasteurized?. His point is that you don’t want to remove inhomogeneities but rather to just clean up the data as it is the lumps and their distribution that have meaning, too.

with temperature data, I want very much to see the natural variability in the data. And I cannot see that with linear trends fitted through homogenized data. It may be a hokey analogy, but I want my data pasteurized – as clean as it can be – but not homogenized so that I cannot see the true and full range of natural climate variability.

The problem with any of these raw temperature data cleaning problems gets back to what Iowahawk illustrated. It is fundamental to understanding the accuracy of climate studies. When you step through data manipulations, you bring in numerous assumptions and sources of error.

That is where ideas such as noted in Models for gravity and heat become interesting. Instead of temperatures being a starting point for analysis and calculation, they become an end point or verification of a model. The model describes the variables involved in determining temperature at a given point in time and space. The problem with current models is that they are very good for current conditions but their veracity rapidly degrades the farther away from current you go. That means that they are nearly useless for climatology. That, in turn, is both why climatology turns elsewhere and an illustration of just how tough the problem is to predict climate change.

Tech Report has a poll going – Poll: Gigabytes vs. gibibytes.

You may not know it, but the world of technology is split into two camps right now. For one camp, megabytes, gigabytes, and terabytes are all powers of 10. Hard drive makers are part of that clique, as is Apple, at least since the release of Mac OS X 10.6 Snow Leopard. For the other camp—the traditionalists, if you will—those same units are powers of two. On that side of the ring, we find Microsoft, memory vendors, and just about anyone who doesn’t know the difference.

The issue is whether you are counting in base 2 like a computer or base 10 like a human. Engineering has always uses thousands as a convenient grouping so measures are reflected with kilo, mega, and giga prefixes to indicate thousands, millions, or billions. It is convenient, yet maybe a bit deceiving, that 2 to the 10th power is very close to ten to the third power. So ten with a 3, 6, or 9 exponent is sorta’ like 2 with a 10, 20, and 30 exponent.

This really wasn’t an issue when data storage was measured by thousands. Whether a thousand was 1024 or 1000 didn’t make much difference. When storage went up to a million, the difference between power of 2 round off and base ten became that of 1,048,576 or 1,000,000 and when talking billions 1,073,741,824 versus 1,000,000,000. That means an error of 2.4% for thousands, 4.9% for millions, or 7.4% for billions. A couple of percent is a ‘reasonable’ error but getting towards 5% or more can be an issue. For terabytes, the error starts getting close to ten percent.

Another factor is that devices like hard drives are becoming less connected to binary addressing schemes like other memory devices. Hard drives are also the most common (cheapest) for terabyte storage, too, and where people are most likely to being trying to purchase storage space rather than addressing convenience (i.e. speed). So sales efforts have to be a bit less deceiving and that may be why a terabyte drive is advertised as TiB so a buyer knows he’s getting a 1,000,000,000,000 rather than 1,099,511,627,776 bytes of storage.

So if you’re use to MB and GB and see something like MiB or GiB, you’ll know someone is following standards and trying to be honest with you.

AJ Strata nails the core issue in How Not To Create A Historic Global Temp Index. His reference is that of keeping track of satellites. He notes the many factors that make orbit predictions longer than a week difficult. As more is learned about what influences satellite position, the model used for predicting position is improved.

This approach is in contrast to global temperature determinations where the raw data is adjusted (homogenized) in order to fit the current desired paradigm.

Basically what alarmists needed to do was not adjust data, they needed to create a thermal atmosphere model which would take into account siting characteristics both local and large. This would include distance from large bodies of water, altitude, latitude, etc. A three dimensional model that would explain why various stations have their unique siting profiles and temperature records. It would explain why temperatures near oceans fluctuate less than stations inland 100-200 miles. It would show how a global average increase of 1°C would result in a .6°C increase at high latitudes or altitudes. It would EXPLAIN the data variations in the measurements.

What this means is that the effort should not be trying to adjust raw meteorological data to create a homogenized data set than could then be subject to various statistical manipulations. Instead, create a model where you could input factors about weather station equipment and locale and have it predict a temperature that could be compared to the actual measured temperature.

In some respects, the model AJ Strata seeks is what is being used for short term weather forecasting. As far as I know, such models are not being used to qualify temperature readings in determining a global temperature index. Also, those models usually use as input an adjusted and gridded input from raw measures so they can get rather circular in their methods.

And just what is the value or meaning of a global temperature index, anyway? The earth is so large and so varied and weather phenomena so local and so small that it seems as if the effort is an attempt to average apples and oranges to be able to describe both as one thing.

Interesting. Over on Tech Reports was a thread with cautions about Neodymium magnets and then leenks cites a page about Dirk’s Accident. Dirk has pictures. He squashed a finger tip.

Dirk had an accident. Below is the X-ray showing his totally crushed finger tip. It took 1 1/2 hours of surgery to remove the shattered bones and repair the damage. Medically speaking, he crushed his right index finger distal phalange. The magnets had a 50 cm (20 inch) separation when they decided to fly together.

When you are talking about hundreds of pounds of force between chunks of metal you can hold in your hand, there is a hazard for anything that gets between them, especially if they come together over a short distance. In this case the magnets were N45 at 20 inches. When together they’d need 400 lb or more to pull them apart. That means at 20″ the pull was probably more than 40 lb still.

ouch.