A bit over ten years ago I was pulling weather station data from the web along with satellite pictures and sounding data to support fog research at SFO. I also needed historical data for pattern analysis. It was a good exercise in gaining familiarity with Java but finding good data and translating that into some usable structure was a challenge. The recent escape of a lot of stuff about climate research in England tells me that others encountered similar headaches. See Charlie Martins note about how Climategate Computer Codes Are the Real Story.
“Harry” is starting off with two large collections of data on a UNIX or UNIX-like system (forward slashes, the word “filesystem”) and only knows very generally what the data might be. He has copied it from where it was to a new location and started to work on it. Almost immediately, he notices a problem:
…
Poor Harry is in the first circle of programmer hell: the program runs fine; the output is wrong.
At least he had a means to tell if his program was working as intended or not. All I had to go on was spurious results, or what looked strange. Of course, if the idea is to find things out, not knowing what the result is to be is what it is all about.
Post a Comment