Motivated by Neil Gaiman’s Anansi Boys, I’ve started reading the Jamaica Anansi Stories by Martha Warren Bechwith. Go give’em a whirl!
Get stock quotes into a spreadsheet
This code stnippet grabs a .CSV file from Yahoo! Finance with selected quotes. This is very useful to keep an eye in your portfolio from within OpenOffice.org Calc, Gnumeric, Kspread,Microsoft Excel or similar program using your very own models.
To use it edit the paths and the symbols in the script, run it periodically from cron(8) to get $DEST/quotes.csv, import the file once and reference it from your own spreadsheet.
#!/bin/sh
TMP=/home/user/tmp
DEST=/home/user/prj/mba/dinero/portafolio
wget -q -O $TMP/quotes.csv ‘http://finance.yahoo.com/d/quotes.csv?s=^MXX+^DJI+^IXIC+MXN=X+AMXL.MX+ARA.MX+BIMBOA.MX+CIEB.MX+FEMSAUBD.MX+GFBBB.MX+GMODELOC.MX+TELMEXL.MX+TLEVISACPO.M+WALMEXV.MX&f=sl1d1t1c1ohgv&e=.csv’
echo ‘”SYMBOL”,”VALUE”,”A”,”B”,”C”,”D”,”E”,”F”,”G”,”H”‘ > $DEST/quotes.csv
cat $TMP/quotes.csv >> $DEST/quotes.csv
TMP=/home/user/tmp
DEST=/home/user/prj/mba/dinero/portafolio
wget -q -O $TMP/quotes.csv ‘http://finance.yahoo.com/d/quotes.csv?s=^MXX+^DJI+^IXIC+MXN=X+AMXL.MX+ARA.MX+BIMBOA.MX+CIEB.MX+FEMSAUBD.MX+GFBBB.MX+GMODELOC.MX+TELMEXL.MX+TLEVISACPO.M+WALMEXV.MX&f=sl1d1t1c1ohgv&e=.csv’
echo ‘”SYMBOL”,”VALUE”,”A”,”B”,”C”,”D”,”E”,”F”,”G”,”H”‘ > $DEST/quotes.csv
cat $TMP/quotes.csv >> $DEST/quotes.csv
Mitigate the .WMF vulnerability with Exim, Squid and SquidGuard
Unless you’ve been on holiday leave you’ve probably heard about the WMF vulnerability by now. Everything seems to indicate that Microsoft won’t take action to patch this gaping hole before January 9th, so here are a few measures to be on the safe side.
1. Before you do anything else, go ahead an unregister SHIMGVW.DLL
C:\> regsvr32 -u %windir%\system32\shimgvw.dll
Put this in a login script, and just for good measure go ahead an run it in every Windows box you’re responsible for. Heck, run it even in those you’re *not* responsible for as well.
2. I already got quite a few .WMF attachments on the spamtraps. F-Secure has a very interesting specimen and a lot to say about it. So the next step is to block them in exim.conf. Enable the acl_check_content ACL and make sure that you have a rule like this one:
deny message = This message contains an unwanted file extension ($found_extension)
demime = scr:vb:vbs:vbe:js:jse:reg:bat:lnk:pif:hlp:dll:com:rar:wmf
demime = scr:vb:vbs:vbe:js:jse:reg:bat:lnk:pif:hlp:dll:com:rar:wmf
3. SquidGuard can filter URLs that match a given regular expression. Add these regexes to a local-blocks/expressions or similar file:
http://.*\.(scr|vb|vbs|vbe|jse|reg|bat|lnk|pif|hlp|com|rar|wmf)($|\?)
ftp://.*\.(scr|vb|vbs|vbe|jse|reg|bat|lnk|pif|hlp|com|rar|dll|js|wmf)($|\?)
ftp://.*\.(scr|vb|vbs|vbe|jse|reg|bat|lnk|pif|hlp|com|rar|dll|js|wmf)($|\?)
Note that the .dll and .js extensions aren’t blocked for HTTP. That’s because the lovely IIS uses the .dll suffix for its extensions, and you can’t block JavaScript for HTTP either unless you want to break 90% of the Internet for all your local clients. I firmly belive that blocking them for FTP is just fine, tough.
Update 20060102 125530: Jeremy Gaddis shares a squid recipe.
Of course, this only applies if you use exim and Squid+SquidGuard -which by the way are all excellent Open Source products- but the same principle applies to any other mail and proxy servers. If you implement all three recommendations you should be fine. User should not get .WMF files through email or the Web. Even if a clueless user catches it though some other means (IM, external e-mail account, a *ack!* floppy disk/CD-ROM/flash drive, a helpful colleage, etc.) the REGSVR32 workaround should keep the exploit at bay. If you definitely need to work with WMFs, there are other alternatives as outlined by Richard Bejtlich. Just don’t hold your breath for a Microsoft-backed patch.
<rant>And about Microsoft’s “swift” response to this issue: The next time that someone gives me the line about not using Open Source because there’s nobody to take responsibility for problems, I’ll puch him in the face without further warning.</rant>
countdown
This small script is an alternative to sleep(1) that gives a visual clue to the user about the remaining seconds in the delay
(Download)
There are numerous instances where you might want your shell scripts to sleep(1) giving the user a clue about what’s going on, but just to relate to a previous example, let’s see how this can be used to throttle file leeching:
$ for i in `count 1 10 %02d`; do wget http://…./file-$i.pdf; countdown 30; done
Anansi Boys
It took a bit longer than I originally expected, but last night I finally finished reading Anansi Boys and I just have this to say: Neil Gaiman is a master storyteller.
Neil has just weaved a great story from a few seemingly loose strands in American Gods and -I swear- it is a solid web that he traps the reader into. The characters are solid and well developed over time, and by the end of the book they jump right off the page. The situations are very well constructed and some scenes have a frail, dreamlike quality. I can certainly relate to some aspects of the situations that Fat Charlie goes through, most certainly to those bits about parents being involuntarily embarrasing entities. As it is written, the characters’ ultimate destinies are very much like melodies that intertwine gradually and evolve into a great, powerful song. This song flows naturally, armonically, unavoidably. The musical bridges are in their right place, and in retrospect I can see that every note is there for a reason. The end resonates loudly, like a sustained note in a song I that I feel that I’ve heard before. And I probably have: Even now, Neil is one of the few authors that has the strange gift of haunting my sleep with their stories.
Neil: while you’re conceivably baking in the sun and healing from that nasty cold in some heavenly island in the Caribbean after which you shaped St. Andrews, I just have to thank you for taking me wide awake to that mysterious land at the beginning of time that we ordinary humans only visit in our sleep.
count
count is a minimalist perl script in the spirit of seq(1) but with a simpler syntax. It only counts in increments of 1, but -on the other hand- it knows how to count down.
(Download)
For instance, “count 10 1 %03d” will count down from 10 to 0 padding with zeroes to three digits. “count 0 15 %x” will count in hex. If you omit the format string it will default to “%d” (decimal, no padding).
count is very useful -among other things- for file leeching:
$ for i in `count 1 10 %02d`; do wget http://…./file-$i.pdf; done
de64
Base64 is used to encode binary data in printable ASCII form. de64 is a trivial perl script to decode such strings:
(Download)
One application of de64 is decoding UTF8 LDAP attributes inside LDIF files. For instance, “cn:: Um9iZXJ0byBNYXJ0w61uZXo=” may be decoded with
$ de64 Um9iZXJ0byBNYXJ0w61uZXo=
Roberto Martínez
Roberto Martínez
(Look Randal! I’m using a CPAN module this time! ;-) ) Of course, all the heavy lifting is done by MIME::Base64 from CPAN.
Luhn algorithm in PHP
As I mentioned before, the Luhn algorithm is used to validate some interesting numbers, most notably GSM IMEIs and credit card numbers. Here’s another implementation I wrote, this time in PHP.
function luhn($str) {
$odd = !strlen($str)%2;
$sum = 0;
for($i=0;$i<strlen($str);++$i) {
$n=0+$str[$i];
$odd=!$odd;
if($odd) {
$sum+=$n;
} else {
$x=2*$n;
$sum+=$x>9?$x-9:$x;
}
}
return(($sum%10)==0);
}
$odd = !strlen($str)%2;
$sum = 0;
for($i=0;$i<strlen($str);++$i) {
$n=0+$str[$i];
$odd=!$odd;
if($odd) {
$sum+=$n;
} else {
$x=2*$n;
$sum+=$x>9?$x-9:$x;
}
}
return(($sum%10)==0);
}
(Download)
perlwhich
Are you wondering where the heck does some arcane module come from? Wonder no more, perlwhich comes to the rescue:
(Download)
Next time you need to know some module’s path just run it like this:
$ perlwhich Data::Dumper
/usr/lib/perl/5.8/Data/Dumper.pm
/usr/lib/perl/5.8/Data/Dumper.pm
If a module resides in multiple locations under @INC, perlwhich will let you know as well:
$ perlwhich Salesforce
/usr/local/lib/site_perl/Salesforce.pm
/usr/local/share/perl/5.8.4/Salesforce.pm
/usr/local/lib/site_perl/Salesforce.pm
/usr/local/share/perl/5.8.4/Salesforce.pm
Luhn algorithm in Perl
Here’s an implementation of the Luhn algorithm in perl.
(Download)
This program was designed for shell scripting, using something like
$ if ./luhn 457623486; then echo “ok”; else echo “error”; fi
ok
ok
but it should be trivial to modify it for other purposes.
I’m using this algorithm to validate GSM IMEI numbers, but the Luhn algorithm is also behind credit card numbers.
Phonetize
A small filter to phonetize STDIN into the NATO phonetic alphabet. It’s very useful to spell passwords over the phone.
#!/usr/bin/perl
use strict;
my %ALPHA = map {uc(substr($_,0,1))=>$_} qw( Alpha Bravo Charlie Delta Echo Foxtrot Golf Hotel India Juliett Kilo Lima Mike November Oscar Papa Quebec Romeo Sierra Tango Uniform Victor Whiskey X-ray Yankee Zulu );
while(<STDIN>) {
chomp;
print $_, "\n";
print join(‘ ‘,map {$ALPHA{uc($_)}||$_} (split(//,$_))),"\n";
}
use strict;
my %ALPHA = map {uc(substr($_,0,1))=>$_} qw( Alpha Bravo Charlie Delta Echo Foxtrot Golf Hotel India Juliett Kilo Lima Mike November Oscar Papa Quebec Romeo Sierra Tango Uniform Victor Whiskey X-ray Yankee Zulu );
while(<STDIN>) {
chomp;
print $_, "\n";
print join(‘ ‘,map {$ALPHA{uc($_)}||$_} (split(//,$_))),"\n";
}
Here’s the source code.
Yup, I know about Lingua::Alphabet::Phonetic::NATO but I needed a quick script without module dependencies. Yes, I haven’t been able to learn the NATO phonetic alphabet yet -unlike some geeks with too much time in their hands (you know who you are)- and yes, I know I’m lazy, but that’s why I am a perl fan anyway ;-) .
There’s more information about this alphabet atWikipedia. You migth want to donate a few bucks while you’re there.
(Mexican?) Cheese to die for
Via Neil Gaiman’s Journal, a story that demonstrates that human intelligence has limits, while human stupidity does not.
BTW, my copy of Anansi Boys is on its way… and it will not make it to Christmas, I promise ^_^ . Sugartime!
Los Sangre are coming for you
Just in time for Halloween. Last Thursday I didn’t get my fix of morning radio, so this note in cofradia.org came as something of a surprise. “Beware. Los Sangre are coming for you”. The baloney detection device went off immediately, and a quick visit to snopes.com confirmed this as a well-documented urban legend. That didn’t stop the media from spreading the word -the rumour was legitimized by an official press release, after all- and further questioning to other officials in the federal and state level confirmed the presence of the gang in the country. “We are taking it seriously” sounds good enough. After all, what government spokesperson is willing to admit that he doesn’t have a clue about whatever the hell you’re asking him about?. It doesn’t hurt to say that “the (National Police Force) is on alert” and that “(the Federal Government) is informed about the situtation”, does it?
It’s not the first time that the general insecurity in Mexico contributes to give some plausability to an incredible story, and it certainly won’t be the last. Here lawlessness defies imagination and impunity is taked for granted. Take into account that criminals can break into a government facility and take back the smuggled merchandise that’s been impounded from them, and then the rumour of a macabre gang initiation game may start to seem plausible. In a country where crime statistics makeup and official coverups are routine, if some random government official tells me to be on the lookout for a specific gang ritual, he may have a good reason for it. We forget that the work of the government agencies -what *we* actually pay them for- is to give us security, not to blow out of proportion whatever chain mail “alert” falls into their inbox. I guess that’s criminal investigation in the third world amounts to. What’s next? A press release advising the population to be on the lookout for AIDS pin pricks in movie theaters?. “Hey, it may or may not be true, but we are alert just in case. You have been warned.”. I guess that this way nobody can blame them if quantum probabilities reverse unexpectedly and the rumour just happens to be true. Heck, even a false sense of security is better than having The Government issuing bogus alerts that send the population into panic. Understandably, the population is totally psyched out. Since the story has been all over the radio and TV, everyone from laymen to well-studied intelligent people is behaving in a totally unrational way. This is your primal mind on fear. That’s what panic does for you.
Now some media outlets are waking up to the fact that this is an urban legend that’s been repeating for some time in different countries and there have been mentions of Snopes in a couple of articles. However, the undertone in some of them is unnerving: “American site dismisses a very real threat”, is my lecture of some of them. Hey, these guys should know better. Next time someone refuses to compare bloggers with journalists I might be tempted to take it as a compliment.
Now why would a government official issue a press release without further research? Because the alert came from the Interpol office in a nighbourging country, that’s why (by the way, Guatemala officials show a healty skepticism) and because the head of the state government “got the alert in his computer”. I’d really like to know *who* jumped the gun, because so far I haven’t been able to find a copy of the press release that started it all, the initial vector for this particular outbreak, although some stories point to some “DGA-PII/1548/2005″ document. We better find it quickly, because once the situation comes out as it is nobody is going to take the blame. Never mind the racial undertones. The fact that officials in other countries have fell for this urban legend before doesn’t excuse ours from crying wolf over a lie, and whoever is responsible should be removed from office at once (*Ha!*).
Now for the final outcome I can only imagine what the justification will be for falling for an obviously bogus chain mail. Already, some officials have justified the heihgtened security alert by saying that they’re trying to stop copycat killers or outright pranksters. Some will say that this was an obvious cartel-backed maneuver masterminded to distract the police, or some form of guerilla warfare. Some will even talk about cyberterrorism. Or blame the media. Or Interpol. Or Guatemala. Or play semantics. I bet that others will say that this might be a conspiracy to destabilize the country — as if we needed further help with that. I’ll just stick to a corollary of Ockham’s razor in this: Never attribute to malice that which is adequately explained by stupidity.
Search for victims of Hurricane Stan
The government of the State of Chiapas has recently published on the web a list of the shelter where each victim of Hurricane Stan is right now, so their relatives can find out where they are and how to get in touch with them.
It is a good idea and undoubtel it was done in a hurry, but it is poorly implemented: the list is published in two formats: a 1.2MB PDF file and a 5.6MB HTML file (in the site the file size is listed at22MB!) that were exported directly from Microsoft Excel. If you are looking for a relative whose fate remains uncertain, it is kind of heartless to force you to download a 8MB PDF Viewer or a 5.6MB file to even start looking for your loved ones. Even more if the server doesn’t support content negotiation for GZIP compression, which would shrink the file to 262KB — under 5% of the original size! (bzip2 compression is even more efficient, the compressed file would be half that size). I will spare you of the rant about Microsoft’s disgusting HTML format.
Well, I downloaded the file and wrote a small perl script to clean up that file and import the records into a SQL database. It even does Soundex translation for improved accuracy thanks to Text::Soundex. Perl is Beautiful. If you can, you may want to publish that database on your own site to give people an opportunity to find their relatives, or point them to navegando.net where the search will be kept indifinitely and the database will be refreshed as needed.
BTW, my relatives reported while Stan was still in full force and all of them turned out unharmed.
ITESM Artwork
A couple of months ago I created a page with some hi-res ITESM artwork. According to the logs it’s been moderately successful, and it shows up pretty high on some google searches. I might even dig up some stuff that is lying around in CD-ROMs and even in a couple of Bernoulli disks. If you find this resource useful please let me know.
TiVo in Mexico HOWTO
Live in Mexico? Have a TiVo? Read the TiVo-in-Mexico HOWTO to transform that glorified VCR into a real Personal Video Recorder.
Update 20051009: I just bought a TiVo 2 unit and it’s on its way, so I’ll update the HOWTO within a few weeks.
Note: This post was originally listed under the static TiVO page, but I’ll start managing the TiVo section with WordPress.
Changing TiVo’s timezone and date/time
To change your TiVo’s timezone and date/time follow these steps:
- Get timezoneadj30.tcl. I don’t have access to tivo_canada’s archive, so I had to write my own timezoneadj30.tcl.
- Edit it and change the timezone. It seems that 0=GMT,3=EST
- Run ./timezoneadj30.tcl
# ./timezoneadj30.tcl
- Set the clock using settime, e.g.
# settime 20031123025900 # settime -rtc
- I noticed that if you set the time back, the IR blaster stops working. I’m not proud of my solution:
# reboot
Note: This post was originally listed under the static TiVO page, but I’ll start managing the TiVo section with WordPress.
TiVo Serial Cable
I found this excellent diagram for a TiVo serial cable:
TiVo serial cable schematic.
The original URL is http://www.tivohelp.com/archive/tivohelp.swiki.net/.uploads/serial/tivocable.jpg
Note: This post was originally listed under the static TiVO page, but I’ll start managing the TiVo section with WordPress.
How to make iTunes 5 ignore the Windows locale
I recently upgraded iTunes to 5.0 on Magda’s laptop and it decided to start speaking to me in Spanish. In my Thinkpad with Windows XP in english iTunes was still in english.
Magda’s laptop came pre-installed with Windows XP in spanish, but I have never had a problem installing an english version of any kind of software in it. So far, every iTunes version had asked for an installation language during setup, and stuck with that language forever. iTunes 5.0 is different: no matter what language you use during setup, it will move on to the system locale whenever you run it. I bet that somebody in Cupertino believes that this is a “feature”, but actually iTunes is ignoring an explicit user setting, a usability do-not that comes as a surprise from the UI gurus at Apple.
Just to state a fact, I’m no Malinche. I write and speak an above-average Spanish, and I do love my country and its language. But I like my english software in English, and I do not have to endure the work of some underpaid english-to-spanish translator that can’t speak either language right nor understand their respective subtleties. Besides that, the iTunes ES translation is not for Spanish but rather for Castellano, and some strings are somewhat annoying. “Party Shuffle” sounds nice. It has a festive tone to it. When iTunes is in Party Shuffle mode my subconscious thinks I’m at a party, even if it’s 2am in the morning with two deadlines to meet early in the day and a senior programmer on vacation. In comparison “Sesión aleatoria” -random session- sounds just plain boring.
Today I upgraded to 5.0.1.4 half expecting this bug to be gone, but surprisingly it is alive and well and sends best wishes to everyone back home.
A quick google search got back an interesting result on Playlist Magazine’s forums: just close iTunes, go to its installation directory under “Program Files”, find every directory named <offending language code>.lproj (for me that was es.lproj) under iTunes.Resources, iTunesHelper.Resources and iTunesMiniPlayer.Resources and nuke them out of existence. The one under iTunesHelper.Resources might complain about a DLL being in use. Just move that DLL to some other place and delete it after a reboot. The next time you run iTunes it will talk to you in English.
1st Webmasters Lounge
Webmasters México invites web designers, programmers, sysadmins and everyone who cares to attend to the 1st Webmasters Lounge, an opportunity to socialize, have a drink, interact and know each other in a cool off-line chatroom. As good as excuse as any to unplug (*ouch!*) and venture into the great outdoors (you know… the big blue room?). The appointment is this saturday, August 13th, 2005, 3pm sharp, and will last for as long as the body endures. No cover for certified geeks that show up before 8pm, and cool surprises from the sponsors. RSVP online and look for more details at Webmasters México Blog.
Update 20050910: There’s some photos at Flickr.