« November 2006 | Main | January 2007 »
Bouldering skills leveled out at V2 for a month or two now. Can climb a rare V3, but usually get stuck on particular moves. Worst are those that require pinching (thumbs weak?) or enough upper body strength to hold while reaching for the next piece. Probably need gym time to round out the weak muscle groups.
Winter biking hit or miss, depending on weather. Need fenders, for starters, though these would not help against the bizarre drenching rain this fall.
Technorati Tags: cycling, rock climbing
Recently discovered Teff, a quick to cook and great tasting grain. The Teff Company lists a great Tofu Vegetable Quiche recipe, among others. I altered the recipe by using dried oregano (basil grows poorly in my apartment, and the large expensive quantities available in stores most often are not worth it), and carrot, celery, and peas for the vegetables.
Technorati Tags: food
Use xpquery to apply XPath statements to XML data. xpquery requires the XML::LibXML and XML::LibXML::XPathContext modules. Good for quick extraction of data, or to easily test XPath queries. Some HTML can also be parsed, via the -p HTML option. This relies on the parsing support in libxml2, and will fail on poorly formed HTML content.
Another option for scripts: use the XML::XPath module, which offers convenience methods for accessing query results. Other XML parsing options include XSH.
Technorati Tags: Perl
Example uses of xpquery:
Testing XPath expressions. Consult the XPath reference documentation for help when experimenting.
$ xpquery foo.xml '//a' <a><b>text</b> more text</a> $ xpquery foo.xml '//a/text()' more text $ xpquery foo.xml '//a//text()' text more text $ xpquery foo.xml '//a/descendant::text()' text more text $ xpquery foo.xml '//a/*' <b>text</b> $ xpquery foo.xml '//a/descendant::*' <b>text</b>
Microsoft.com fails parsing. How unexpected!
$ xpquery -p HTML \ http://www.microsoft.com/ '//title' /tmp/xquery.B5dv6L:1: HTML parser error : Couldn't find end of Start Tag meta ‾㰠楴汴㹥楍牣獯景⁴潃灲牯瑡潩㱮琯瑩敬‾㰠敭慴栠瑴⵰? ^ ERROR: Invalid expression
Then again, so do many other sites, including my own. Parsing HTML data requires the forgiving HTML::Parser module, though most needs will be better met by a wrapper module around HTML::Parser, such as HTML::TokeParser. Ideally sites would offer XML service interfaces, to eliminate the error prone HTML scraping of data, or worse, locking data up in annoying JavaScript and Flash. </rant>
Use initidy to reformat *.ini style files. Optionally uses the Text::Autoformat module to clean up any comments present in the file. For example, the untidy input data:
[example] key = value key=value else= other longerkey = sometext
Becomes:
$ initidy < example [example] else = other key = value key = value longerkey = sometext
Technorati Tags: Perl
Since the following information is not spelled out between the use, require, and import documentation. The following example only loads the CGI module if it is available, then pollutes the current namespace with header. Somewhat equivalent to use CGI qw/header/, but different.
#!/usr/bin/perl -w use strict; eval { require CGI; }; if (! $@) { CGI->import( qw/header/ ); } print header();
The above should always work, as CGI has shipped with Perl for quite some time. Use the same pattern for other modules:
#!/usr/bin/perl eval { require Win32; }; if (! $@) { Win32->import( qw/SW_SHOWNORMAL/ ); }
If possible, skip the import, and use object oriented access methods to minimize namespace pollution. To see how use differs from require, try placing the use statements at the end of a file, or inside an END block.
#!/usr/bin/perl print header(); END { use CGI qw(header) }
Technorati Tags: Perl
Avoid the quick glob function if no lstat related data will be used, as glob performs an lstat on each matching file. Instead, use opendir and readdir to efficiently work through many filenames, or a module such as File::Find to recurse through a directory tree.
Use ktrace(1) or similar process tracing utility to see the lstat calls done by glob in Perl. See Unix Debugging Tips for more information on ktrace and similar tools.
$ ktrace perl -e 'glob("*")' $ kdump -f ktrace.out | grep -5 lstat | head 7355 perl RET __sysctl 0 7355 perl CALL fstatfs(0xd,0xbfffc564) 7355 perl RET fstatfs 0 7355 perl CALL getdirentries(0xd,0x1811200,0x1000,0x306b94) 7355 perl RET getdirentries 460/0x1cc 7355 perl CALL lstat(0xbfffc620,0xbfffca80) 7355 perl NAMI "critic.t" 7355 perl RET lstat 0 7355 perl CALL lstat(0xbfffc620,0xbfffca80) 7355 perl NAMI "encrypt-and-sign.sh"
On the other hand, the implicit lstat caches the file metadata for quick lookup with stat _:
$ perl -le 'for (glob("*")) { print "$_ ", (stat _)[9] }' …
Been working through books by Ursula K. Leguin, notably The Tombs of Atuan. Later books in the series (The Farthest Shore and Tehanu) also excellent, though the climax seemed rushed in these. The Left Hand of Darkness covers similar thoughts in a different context, though did not like the coldness of the entire world.
Also working through The Tale of the Heike. Much better than the recent abridged version (being complete, for starters), though not without downsides (such as being complete).
Updated notes on running Unix shell commands in the background with references to other options, such as nohup, screen, and disown, and better log handling with httplog.
Technorati Tags: Unix
Not sure why Unicode contains a ☃ character, but hey…
❆ ❄ ❅ ❆ ❄ ❄ ❆
Wrote up new documentation on using the openssl smime command. This page includes mime-util to help remove MIME left behind by openssl smime -decrypt or openssl smime -verify, as well as test scripts to help encrypt and decrypt S/MIME data.
Technorati Tags: cryptography
Thoughts on marking significant changes in computer systems with a somewhat formalized change notice format.
Change notices become increasingly relevant when multiple groups (Developers, Quality Assurance, Operations) manage an application. All information about the current state of systems and how they differ from the norm cannot be communicated, as the meetings would be onerous, and humans will forget things, especially at 04:20 following an hour of bad sleep. Detailed change information next to an important change, where the on call should eventually look, will clue them in, and save time wasted hunting down who made the change, or worse, enabling of jobs that should not run.
Significant changes must be commented, so another person can revert the changes to their normal state, or know why the value differs from the norm. Uncommented, the changes could be undone by someone attempting to solve another problem. This could include an application setting, such as a timeout, or whether or not special jobs run from crontab(5).
If the change will be soon reverted, include the previous values—increase timeout to support large files (was 30)—and sufficient instructions on how and when to revert the change. Periodic changes due to load, for example during the holiday season rush, may be grouped into known configuration blocks and switched between depending on the time. These blocks should use a common keyword easily searched for, or perhaps could be hosted on an wiki.
Technorati Tags: coding
Recommendations on change notice metadata and formatting: quick comments that tie a change to a user, date, and other relevant background information. Enough breadcrumbs allow those unfamilier with the change—a new on call, for example—track down the who, what, why, and when details efficiently. An example entry that illustrates the format discussed in detail below:
# jmates 2006-10-29 [SIAL #123] libfoo 1.3.6 buggy, # upgrade to 1.3.9 to avoid "failure to init errors" …
This formal specification must not be overused. Limit to trouble ticket related changes, or where significant changes require special attention.
Formalize the change notice format.
Formalizing change notice structure will help search for and reference changes. In addition to date format standardization, also mandate order of the comment (username, date, optional trouble ticket number, notes), and the format by which trouble ticket numbers are referenced. Without standardization, humans will refer to the same trouble ticket number in a variety of formats that may be difficult to search. Mandating a format then allows tools to be written. The following perl command would print any lines matching a username timestamp ticket number change notice:
$ perl -ne 'print if m/\w+\s+\d{4}-\d\d-\d\d\s+\[SIAL/' *
And from there changes made by particular people, in particular timeframes, or by ticket number. An extended format might use a common comment prefix, or additional keywords to assist searches.
Include a username or e-mail address.
Avoid initials: these may conflict between different users, and most sites will not have an initial to person dictionary available. Instead, use your username, or if usernames are not unique, e-mail address. Anyone interested in the change should be able to find the changer without wasting time querying entire groups or digging through version control logs for the culprit.
Timestamp the change.
Include the date of a change, especially if no trouble ticket number or reference to other dated material exists. Several weeks may pass before problems from a bad change become apparent, and a date helps search for changes made around when historical logs show the first signs of trouble. Standarize the date format to simplify searching. I favor ISO 8601 style YYYY-MM-DD time stamps. Do not include HH:MM:SS data: too much detail, and exact time stamps of the change can be derived from other sources, such as a trouble ticket or version control log.
Reference related trouble tickets or incident numbers.
Trouble ticket numbers should be included as they lead to background information. Organizations should make trouble tickets available for permanent reference, indexed or searchable by trouble ticket number.
Include URL with caution.
Links to permanent blog entries are okay; avoid using WikiWords, as these may change without notice. Only use URL shortening services if the shortened links are permanent enough. URL may change, so should be used with caution. Another option: include terms searchable for on a wiki, not an exact reference. In the event of no hits, as a post mortem task update the wiki to include appropriate terms.
Wiki pages for projects can be referenced, as these are unlikely to change, and will serve up plenty of background information, such as group contact information, relevant external partners, and change schedules. On the other hand, project pages need not be referenced if a group maintains a summary Wiki page detailing all ongoing work.
Include as much relevant metadata as possible.
If possible, also include version numbers, as these will assist future searches. The change comment libfoo 1.3.6 buggy, upgrade to 1.3.9 to avoid init errors will match searching on the 1.3.6 version number, if another code tree has a similar problem that another fixed months ago. Sample error messages or log snippets on subseqent lines will also help.
