Friday, June 27, 2008

Ruby for mp3 file organizing

So there we are, me and my friend Oliver, caught in a business trip. We're already bored of the only decent pub in the village, our families are a long distance away, so what's a developer to do? Code in a programming language he/she's not allowed on the job, of course! Oliver seemed interested in Ruby and I've already done a couple of small scripts with it, so we were curious to see what the fuss is about.

It goes without saying that if you want to learn to program (in a particular language) you should not rely too much on books. The only way is to find a task you want to have automated, and then code it using your language of choice. Surely, you must pick the task (or the language) carefully, since not all languages are suitable for all tasks.

One of the things which Oliver has struggled with was organizing all of his podcasts in his player, sorted neatly by directories of author and title. Having found both a hammer and a nail, we were ready to start pounding.

After a bit of research we found the mp3info and id3tag Ruby libraries. id3tag had different fields for ID3v1 and ID3v2 data and didn't have write support (not that we needed it). mp3info didn't have ID3v2.2 support, but I found an interesting link about ID3 internals- the format of the fields was something that could be useful.

After a while our pair programming session has reached a milestone- our script works. It doesn't seem very modular though, so we spend some time making classes and discussing what is the responsibility of each class. Should there be a manager-class? Or should the objects manage themselves? I go with the second approach, and here's the result:


# Class for handling information of the mp3 file
class Mp3File
attr_reader :title, :artist, :album

def initialize filename
@artist = @album = "unknown"
@filename = filename
@title = File.basename(filename, ".mp3")

read_attributes
end

def title
sanitize(@title)
if @title == "unknown" then @title = File.basename(@filename, ".mp3") end
@title
end

def read_attributes
begin
Mp3Info.open(@filename) do |mp3info|
(@title, @artist, @album) = %w{title artist album}.collect { |attrib|
begin
(result = mp3info.tag.send(attrib)).empty? ? "unknown" : result
rescue
"unknown"
end
}
end
rescue
end
end

def sanitize str
str.tr_s!("?'","_")
end

def transfer(newPath)
newPath = eval('"' + newPath + '"')
FileUtils.mkdir_p File.dirname(newPath)
FileUtils.cp @filename, newPath
end
end


This is the class which is initialized with the location of the file and then extracts information about the artist, title and track name. The read_attributes method is meant to show off our new knowledge about the dynamic nature of Ruby- we build a list of methods to invoke on the Mp3Info object, and if no meaningful result, return "unknown". Finally, as the class knows about the current location and mp3 meta-info, it has a method for copying the file to a new location. The new path is passed as a template, where the #@artist, #@album, #@title are substituted with the value of these fields.


class Mp3List

attr_reader :files

def files
@files.map {|file| Mp3File.new(file) }
end

def initialize(sourcePath, days = 7)
@sourcePath = sourcePath
@days = days
@files = read_new
end

def read_new
Dir["#@sourcePath/**/*.mp3"].find_all do |path|
test(?M, path) > (Time.now - (@days * 60 * 60 * 24))
end
end

def to_s
@files.inspect
end
end


Here comes the class, which represents a list of mp3 files in a certain directory (and subdirectories), which satisfies some criteria- in this case, how long ago the files were created (modified). Could it be made more general? Certainly, but in a 80-line script? Maybe next time.


list = Mp3List.new("/home/whoami/Music", 730)
list.files.each do |mp3|
#~ puts "Processing #{filename}"
mp3.transfer('/tmp/music/#@artist/#@album/#@title.mp3')
end


What's left was an example of how to use these classes. Seems good to me- and best of all, it works.

The only thing left was to prepare a patch for the mp3info library for ID3v2.2 support. I actually implemented one (still not incorporated in base), and it also initializes the common fields with either the v2 or v1 data, whatever present (v2 still has precedence, if both are present).

Conclusions from our short session:

  • Ruby is neat for quick hack jobs

  • mp3info does not provide an exhaustive ID3 handling support, but is good enough and workable

  • Pair programming might not be smooth from the start, but you will learn a lot about yourself

  • Organizing your music can sometimes take longer than total time spent looking for your tracks

  • You should choose your business trip accomodation place carefully if you can

Wednesday, June 11, 2008

command-not-found handler not only in Ubuntu

One of Ubuntu's strengths is that it pays attention to the little things. One of these things, which were important to me lately, was the command_not_found_handler in bash.

In layman's terms, when you open a terminal in Ubuntu and type a command, which is not on your system, it looks in a database and in a helpful way suggests that you can get this command if you install a certain package. I was using apt-file to provide the same functionality for a while, but this is both easier and more informative (apt-file scans for substrings of the whole path, so spits out a lot of false positives, which are not commands).

Now as happy as I am in using Ubuntu on my personal machine, I wanted to have this at work, too. I wanted to make it easy for my colleagues who forget to ssh/sudo before executing a command and prepend an appropriate string if the command doesn't work on this machine. Ubuntu can accomplish this trick thanks to a patch of bash, which adds a hook, called command_not_found_handler, but OpenSUSE/SLES is installed at work and this function is missing.

I had the feeling I can do this in bash only, and then I found this blog entry. Only I didn't quite like how it's implemented, though I liked the idea. Benjamin (the blog author) suggested that the check if the last command was successful is done at each prompt, so it is evaluated even if you press enter on an empty line. Besides, to obtain the last command, the whole history was retrieved only to extract the last entry.

The reason Benjamin's hook was done like that is that history expansion didn't work in noninteractive shells. I also tried to enable it by using set -H, but with no luck.

I thought that using a trap would be better than a prompt command. There was a trap, which would be evaluated only on error, so my first draft was based on the trap ERR hook. Then I would check if the exit status of the last command was 127 ("command not found" for bash) and... use the command. By chance I found out quickly about BASH_COMMAND, which in traps is evaluated to the currently executing command. But after a lot of failed attempts I found out that it only worked with the DEBUG trap.

So I used the DEBUG trap and evaluated the last command- if it could be resolved using type, then it is found. Slower, but more accurate. And it worked!


trap 'if ! type -t $BASH_COMMAND >/dev/null; then echo Do something with $BASH_COMMAND; fi' DEBUG


Now my colleagues are happily executing commands, which are only found on remote servers, oblivious to the fact that this command is transparently executing on a host different than their own.

Tuesday, June 10, 2008

Java Portable Apps


portable
–adjective

  1. capable of being transported or conveyed: a portable stage.

  2. easily carried or conveyed by hand: a portable typewriter.

  3. Computers. (of data sets, software, etc.) capable of being used on different computer systems.

  4. ...


Dictionary.com



So how many definitions of "portable" do your applications of choice satisfy? Having grown up profesionally in the spirit of "write once, run everywhere", I've always wondered how a "portable" application could run only on Windows, as is the case with most of the PortableApps.com. Not that I have anything against portableapps.com, it's a cool idea and a very pragmatic software suite is offered.

But I happen to work on Linux and my colleagues usually work on Windows, so if I want my applications-on-a-stick to be truly portable, they need to run on any platform. I need portable in the sense of "running on any operating system". Thus an obvious choice is Java- it's usually not the first choice for a desktop application, but here its intended purpose fits the bill nicely. And I don't need all operating systems- honestly, who would lend me their Mac? Come on, it's too personal to give to anyone ;-) This means I can have a JRE for Linux and Windows on my stick (in case it's not installed), and I'm set.

Of course, the task of finding the actual applications is the difficult one. They are not all completely portable in the sense of "not modifying anything on the hard disk", but most of them could be configured to run with settings from the USB. I also needed to run all of these on machines without administrator privileges. Most (with a few exceptions) are open-source:


  • Organizer: ThinkingRock
  • An outstanding application for organizing your tasks according to the Getting Things Done method. I am now so addicted to this app that I can't leave anywhere without it- I feel lost withot my next actions list. Con- it is the least portable in the sense that it creates some .java entries, but mostly related to layout- something I can live without.
  • Editor: jEdit
  • One of my all-time favorite editors, I really doubt there's much this editor can't do. With all of its useful plugins, it doubles as a mini-IDE. I run it with a command-line option to use the settings on the USB stick and the settings include the option to download my plugins there, so I have them with me as well.
  • Mind Mapper: FreeMind
  • Another immensely useful program, FreeMind is an open-source mind-mapping application, indispensable for loads of stuff like notes, brainstorming, personal database, task-manager...
  • Outliners
  • I couldn't choose between the extensibility and import/export capabilities of JOE (Java Outline Editor) and the rich text support of Jreepad (including Textile markup), so I have both. I really wish JOE's default shortcuts were a bit less awkward, but it's a great tool even though there's no development on it for several years now.
  • File Manager: muCommander
  • A couple of years ago I would be really bothered by the idea of using a Java file manager (Java doesn't have the best OS integration, you know). Now I'm happy to use this really nice lightweight commander clone. It has transparent filesystem support for popular archives and remote protocols (FTP, Windows shares, SFTP).
  • Disk space manager: JDiskReport
  • Warning: not open-source, but free for personal use. Generates very nice pie charts about filesystem usage though, and even has a Web Start version. I still want to be able to check my disk space when I'm not online, so it has a convenient place on my USB thumb drive.
  • Media organizer: MediaSort
  • Organizes music and pictures into directories/filenames based on tags inside these media files. Pretty nifty. I used to do that with a clunky one-liner script calling jhead for JPEG files and a custom Ruby script for MP3 files, but this proved to be a nicer general solution.
  • File Synchronization: JFileSync
  • I carry this around in case I don't have rsync handy. Its GUI looks very nice for syncing directories to and from my USB key.
  • Version Control: SmartSVN
  • Version control is must-have for a developer. SmartSVN is not open-source, but has a version, which is free for personal use. This was the only Subversion client I could find, which would work on Windows without administrative privileges, and use Windows authentication.
  • Port forwarders
  • Network connectivity is important to me so I have an assortment of port forwarding applications for different purposes. JPortForwarder is a simple port forwarder (site says it's multithreaded), so I don't have to rewrite one every time. PlugProxy is a really cool way to debug network applications, as it shows the network traffic as it transparently redirects it. jzbd adds encryption to forwarding when I'm worried about security. These are not updated in a long time, but what could you improve in a port forwarder?
  • Port Scanners
  • Yeah, I know the low-level scanning options of NMap are out of Java's reach, but still. JAPS makes a fast concurrent scan of a single host, while JMap can scan a subnet (no, it's got nothing to do with Sun's post-mortem memory analysis tool).


This set of programs proved especially useful when my laptop's hard drive reached the end of its intense life. Using a Damn Small Linux to boot from the USB disk along with these applications really decreased the time I could get productive again- without a hard disk- until I got a new hard-drive. To top it off, I could boot DSL in Windows, emulated by qemu- but DSL can really deserves a blog post on its own.

Update: This blog prompted me to do some more research and I found this great thread on Javalobby about Java Desktop Apps. There's at least one new cool app I am going away with- ekspos image viewer. It even has Picasa integration!