TechComm Labs

If you collect a lot of stuff in electronic form, the issue of how many copies to maintain becomes a concern. Ideally, you’d only store one copy of each photograph, album, document, or whatever. Metadata can then be used to find the same item through various routes such as file path, search words, or description.

Google encounters this problem in spades when it comes to their project to scan the worlds books. Books of the world, stand up and be counted! All 129,864,880 of you describes the problem. The problems run the gamut from the basic question of defining what is to be considered as a book through all of the classification and identification schemes being used to manage book collections.

So what does Google do? We collect metadata from many providers (more than 150 and counting) that include libraries, WorldCat, national union catalogs and commercial providers. At the moment we have close to a billion unique raw records. We then further analyze these records to reduce the level of duplication within each provider, bringing us down to close to 600 million records.

The gets boiled down in various ways such as finding errors or April fools jokes such as the turkey probe given and ISBN book number to where Google figures there are about 130 million books in the world.

Take a gander through the essay. Things that seem simple, like creating a catalog of all the world’s books, can – and usually do – have complexities you might not imagine.

It was May 31, 1886 that a two day rail gauge conformance effort began. Southern railroads changed the distance between the rails from five feet to four feet nine in order to be compatible with the Pennsylvania Railroad. PrawfsBlawg suggests Happy Uniform Gauge Day! How a 3-inch nudge destroyed American federalism.

Today is the 134th anniversary of one of our most important yet most unrecognized constitutional events: On May 31st, 1886, the operators of Southern railroads began their famous two-day conversion of all southern railroad tracks

Check the link for the story. This was just one very big step towards conformity to enhance and enable commerce. The problem was widespread. “In 1871 no less than 23 different gauges existed in the United States, ranging in width from three to six feet.”

We face similar standards development processes today. Since computing technology has become a consumer good and service, protocols for communications, data storage, and service descriptions have followed the railroad gauge uniformity history. Back in the eighties, there were many different ways of connecting computers together both in terms of the wires and also in terms of the methods. Now the methods are overwhelmingly TCP/IP and the wire is twisted pair ethernet. The process continues in the wireless regime, however, as cell phones and wifi and other approaches compete.

There will always be custom solutions for niche markets but the economic advantages of standardization are usually overwhelming. From Cargo containers to rail gauges to electrical power delivery to clothing sizes, much of our prosperity comes from being able to talk a common language and share products and services easily.

As to whether this standardization and conformity is a attack on federalism, I don’t know. I think Rick has headed out to hyperbole, with the title of his post. States, towns, and counties still exist and have not been shoved off the map. They can just communicate better and profit more from each other’s efforts.

When it’s time to get your math on, NIST has the Digital Library of Mathematical Functions up. The goal is to provide

a reference tool for researchers and other users in applied mathematics, the physical sciences, engineering, and elsewhere who encounter special functions in the course of their everyday work.

This is a reference work. That means you need to know what you are doing to be able to make best use of it. The online version has links to papers and other documents that will provide background but otherwise this is like a dictionary of mathematics.