Moving from iPhoto to Lightroom

I've been using iPhoto since I bought my mac, but lately it seems to be getting slower and buggier... Takes ages to launch, search doesn't work properly, and my library of about 25,000 photos seems too much for it. So I decided to try Lightroom.

Unfortunately, there's no easy way to get out of iPhoto, be it for Lightroom, Picasa, or whatever... There's no simple way to export all of your events into nice subfolders! So I built my own way out!

I created a python script that reads the data from iPhoto, and uses it to create a tree structure with year/name_of_event and all the photos inside. Here's a screenshot of the script running:

To use the script, download it from here, and execute it via the terminal using:

python iphoto-to-folders.py [AlbumData.xml] [destinationDir]

[AlbumData.xml] is a file inside your iPhoto library, and can usually be located at ~/Pictures/iPhoto\ Library/AlbumData.xml

[destinationDir] is the folder where you want your photos to be copied to.

A couple of notes about the script:

• It only copies the original versions of the photos. Changes you made in iPhoto are not copied.
• It copies all the files from your iPhoto library, so it needs quite a bit of disk space.
• It may take some time to run. To copy my 25.000 photos it took 1.5 hours
• It's not supposed to mess with your iPhoto library, but do make backups!
• It has no warranty whatsoever!
• It's supposed to be tweaked! If you're comfortable with python, adapt the script to your needs.

If you have any questions or spot some problems, please submit them here. And be sure to leave a comment telling me how the script worked for you!

Converting a list of IP addresses to countries

If you have an Excel file with a column filled with IP addresses that need to be converted to countries, here's a way to do it. These instructions were tested on a Mac, but it should work fine in any environment with Python.

Start by downloading the GeoIP City database from MaxMind. They have a free version that you can download here. Download the one in binary format, and uncompress it.

Next, you need the library to access this database format. There's a pure Python library called pygeoip that you can download from google code. To install it, just uncompress it and run the installer: sudo python setup.py install

Next, you need to build a small script to convert the IP addresses to countries. Here's the script I used (note that the countries database should be in the same directory as the script).

#!/usr/bin/env python

import pygeoip, sys
gi = pygeoip.GeoIP('GeoLiteCity.dat')

for line in sys.stdin:
	rec = gi.record_by_addr(line)
	print rec['country_name']

I used this script (geo.py) by copying the IP list from excel to a plain text file (ips.txt), where you get one address per line. Then just run it with something like python geo.py < ips.text and you get a list of countries on your terminal window. Copy/paste to excel and you're done!

If you want more than just the country, just play a bit with the print line. Here's a variation I did to get the state and the city. The output is tab separated so that you can copy it easily to excel:

#!/usr/bin/env python

import pygeoip, sys
gi = pygeoip.GeoIP('GeoLiteCity.dat')

for line in sys.stdin:
	rec = gi.record_by_addr(line)
	print rec['country_name'] + '\t',
	if rec['country_code'] == 'US' and 'region_name' in rec:
		print rec['region_name'] + '\t' + rec['city']
	else:
		print '-' + '\t' + rec['city']

As a side note, I tried another database from hostip.info, but it was only able to convert about half of the IPs I threw at it, so I recommend going with the one from MaxMind...

Sending emails via Gmail with Python

Whenever I need to send files from work to home I use gmail. Usually this email message consists of one single file (that may be a tarball) and has the file name for subject. In the "Automate Everything" spirit, I decided to build a script to do this task for me.

The first thing I had to do was find out how to send an email with an attachment via gmail. It wasn't too hard to find this information around the web, but it still took me the best part of an hour. So here's a simple Python script that sends an email with an attachment:

#!/usr/bin/python

import smtplib
from email.MIMEMultipart import MIMEMultipart
from email.MIMEBase import MIMEBase
from email.MIMEText import MIMEText
from email import Encoders
import os

gmail_user = "your_email@gmail.com"
gmail_pwd = "your_password"

def mail(to, subject, text, attach):
   msg = MIMEMultipart()

   msg['From'] = gmail_user
   msg['To'] = to
   msg['Subject'] = subject

   msg.attach(MIMEText(text))

   part = MIMEBase('application', 'octet-stream')
   part.set_payload(open(attach, 'rb').read())
   Encoders.encode_base64(part)
   part.add_header('Content-Disposition',
           'attachment; filename="%s"' % os.path.basename(attach))
   msg.attach(part)

   mailServer = smtplib.SMTP("smtp.gmail.com", 587)
   mailServer.ehlo()
   mailServer.starttls()
   mailServer.ehlo()
   mailServer.login(gmail_user, gmail_pwd)
   mailServer.sendmail(gmail_user, to, msg.as_string())
   # Should be mailServer.quit(), but that crashes...
   mailServer.close()

mail("some.person@some.address.com",
   "Hello from python!",
   "This is a email sent with python",
   "my_picture.jpg")

Objects in Perl? Use Python!

I've been meaning to try Python for a while. I'm kind of a sucker for programming languages, and knowing a bunch of languages allows me to choose the right tool for the right job. Turns out that this week I had the perfect opportunity to check out Python.

The task at hand was analyzing patterns in a bunch of XML files. I had 270Mb of XML files, and the largest file was 32Mb. Because I was pretty sure I was executing the analysis more than once (errare humanum est and I want to improve the analysis in the future) I decided to use SAX to read the files.

Having decided to use SAX I then decided to use Perl to do the job. I'm pretty familiar with it and was able to quickly find a SAX sample. Besides, I had a few string matches and replacements to do, and Perl is a great language for that.

Turns out that using SAX in Perl demands that you use and define objects in Perl. And it turns out that defining objects in Perl is... well, terrible! I really hated the syntax, bless and the way attributes were defined. It all looks like a big hack! To add insult to injury, the perlSAX has a few quirks when changing handlers. This is mandatory to make your SAX code maintainable... So I dropped Perl and went for Python.

To my surprise the transition was really easy. I was able to convert my Perl code to Python very quickly, with only a few doubts now and then on specific stuff. Here's what I gained from the transformation:

• I learned Python (finally!)
  
• Better SAX handlers (the quirks that happen in Perl don't happen in Python)
• Clearer attribute access (if you have an object with a reference to an array of references and want to print it in Perl... things can get weird)
• Clearer object definition and usage (no bless!)
  
• Fewer lines of code (from ~250 to ~150)
• Same performance (I was worried about this, but both scripts took the same time to execute!)
  

I still love Perl. If I want to parse a bunch of text files, do text transformations and the like it will be my 1st choice.

But whenever I need to do something a bit more complex that requires complex data types or OO programming, from now on I'll definitely turn to Python!