Jeremy Mates’s Blog

United States oil import figures—13.15 million bbl/day (2004)—coupled with current oil prices—$92.79 at time of writing—mean the U.S. spends roughly $15,000 per second on oil, or about $1.25 billion annually. Despite this massive waste, the auto industry still fights against improved fuel economy standards.

Cars must be relegated from The American Way to a Necessary But Woefully Overused Evil. While the decision to drive a car may be individually rational, the combined effect of millions of drivers moving millions of miles daily is deadly:

Technorati Tags: CO2

Posted by Jeremy Mates at 15:34 | Permalink

“You can’t get here from there” — unnamed Diplomat on travel to Moldova. Connecting from Seattle difficult, as either expensive, or necessary to make an Eastern seaboard hop then European hop then finally Chişinău.

Random fun facts!

• Giving a flu (or something) 24h+ of poor sleep and high stress travel time to develop results in… the flu (or something)!
• Vehicles in Moldova with the big blue tanks run on propane.
• If the local weather turns to cloudy with rain, blame inevitably falls on the Seattleite.
• Borat was filmed in Romania, not Kazakhstan.

Not too many photos, due to the aforementioned flu (or something. Not a Doctor). Will delve through them later.

Posted by Jeremy Mates at 07:57 | Permalink

When data is being made ready for some other purpose, and the source data is messy or requires significant cleanup, there are several ways to approach the problem. Example data (fairly clean, as these things go):

| 10957 | Aardvark | | 3079 | Badger | | 10696 | Bobolink | | 1030 | Capybara | | 2659 | Cuckoo | | 10305 | Dodo | | 21473 | Emu | | 6603 | Octopus | | 14042 | Rook |

A requirement to produce comma separated data could be met with several different approaches. The first method trims away the dressing around the data: evict the spaces and vertical bars, remove the blank lines, add a comma between the number and word. The second method ignores these dressings, and extracts the data itself: match the ID number, and the word, then print these with a comma between them.

Technorati Tags: Perl, Unix

• Trim Away Method

  $ perl -i -pe 's/^\|\s+//' input $ perl -i -pe 's/\s+\|$//' input $ perl -i -pe 's/\s+\|\s+/,/' input

  Dealing with the blank lines suggests two approaches. One could treat the file as a single line, and strip runs of newlines. Or, iterate over the lines, but only print if the line is not blank:

  $ perl -i -0777 -pe 's/\n{2,}/\n/g' input

  $ perl -i -ne 'print unless m/^\s*$/' input

• Extract Data Method

  This method ignores the blank lines automatically, as they contain nothing interesting. However, if the regular expression is wrong, entries with non-matching characters may be silently omitted. A longer script might dump non-matching lines somewhere for human consideration.

  $ perl -i -nle \
  'print "$1,$2" if m/(\d+)[\s|]+(\w+)/' input

To investigate whether any special characters are present, as mentioned elsewhere on this site, use od(1) or hexdump(1). Consult the ascii(7) manual for listings of what the octal (od -b output) codes mean. In Perl, the octal codes (for example space) may be represented with a leading backslash (for example \040). I use octal only by habit. URL, in contrast, use hex to encode special characters:

$ echo "a b" | \
perl -ple 's/([^a-zA-Z0-9_.-])/sprintf"%%%02x",ord $1/eg' a%20b

Posted by Jeremy Mates at 02:47 | Permalink

There is no doubt that the grail of efficiency leads to abuse. Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.¹

Read the article by Knuth for more context and elaboration. In Perl, a common mistake is to suffix regular expressions with the /o flag. I can personally attest to this being a bad idea: as Knuth predicted, a bug later emerged due to /o being set in one of my scripts. Instead, use qr// to create the expression, and later m/$re/.

# Good to define these early on, # not buried deep in the code. # Use a better variable name. my $re = qr{e/ge}x; … if ($string =~ m/$re/) { …

Inexperienced coders should not fret long over optimizations. First, hammer out something that works. Then have someone review it, or revisit the code some time later, asking:

• Where does a profiler show time being wasted?
• Can anything make the expensive spots less so?
• Would caching of some sort help?
• Does any of the code seem strange or awkward?
• Could a comment help explain the wacky bits?
• Is there any other way to write the code?
• Can the code be reused? Scale to new uses or more input or different output formats?
• In the future, would you code it differently?
• How long would it take to fix any problems? (Managers love time estimates.)

¹ Knuth, Donald E. "Structured Programming with go to Statements." Computing Surveys Vol. 6 No. 4, December 1974: 268.

Posted by Jeremy Mates at 07:39 | Permalink

Tricky subject to photograph, what with all the moving about and manual focus and macro lens.

Technorati Tags: creepycrawlie

Posted by Jeremy Mates at 04:36 | Permalink

Summer, being tenacious, showed up today.

Technorati Tags: cat, creepycrawlie

Posted by Jeremy Mates at 23:40 | Permalink

On the Nature of Things is a fascinating read. The poem offers on the one hand profound insight into and excellent analysis of the world and human condition, and the other flawed reasoning and rampant speculation. Granted, discovering magnetism and ruling out underground winds as a source for earthquakes and volcanism took humanity a few centuries to research properly. I read a “vigorous prose translation” that predates the all-cataloging eye of the ISBN. Many a florid turn of phrase to machete through, but some gems:

But that which is immortal wills not to have its parts transposed nor any addition to be made nor one tittle to ebb away; for whenever a thing changes and quits its proper limits, this change is at once the death of that which was before. Therefore the mind, whether it is sick or whether is is altered by medicine, alike, as I have shown, gives forth mortal symptoms. So invariably is truth found to make head against false reason and to cut off all retreat from the assailant and by a two-edged refutation to put falsehood to route.¹

O hapless race of men, when that they charged the gods with such acts and coupled with them bitter wrath! What groanings did they then beget for themselves, what wounds for us, what tears for our children’s children! No act is it of piety to be often seen with veiled head to turn to a stone and approach every altar and fall prostrate on the ground and spread out the palms before the statues of the gods and sprinkle the altars with much blood of beasts and nail up vow after vow, but rather to be able to look on all things with a mind at peace.²

And, for something completely different, the entertaining classic Lord of Light.³

¹ Lucretius. On the Nature of Things. Trans. Munroe, H. A. J. New York: Washington Square, 1965. 73.

² Lucretius 151.

³ I can see why folks love Endnote. Or hyperlink stuff.