Java Collection<Properties> as XML: Convert to CSV

This xslt renders an XML file of a java Collection instance, containing java.lang.Properties instances as csv, one row per Properties instance.

Create the xml file like so:

...
		Collection canoes = new HashSet();

		canoes.add(canoeProps1);
		canoes.add(canoeProps2);
		...
		FileOutputStream fstream = new FileOutputStream("collectedproperties.xml");

		try {
		   XMLEncoder ostream = new XMLEncoder(fstream);

		   try {
		      ostream.writeObject(canoes);
		      ostream.flush();
		   } finally {
		      ostream.close();
		   }
		} finally {
		   fstream.close();
		}
...

Copy the xslt below to a file (suggest you copy paste as formatting may hide one of the long lines in wordpress standard view). Throw it and your collectedproperties.xml at your xsl processor to get a rudimentary csv file.

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

	<!--
	a trivial xml to csv xslt for a Collection of Properties written
	as XML by java.beans.XMLEncoder
	-->
	<!-- mccalni, 18-dec-09 -->

	<xsl:output method="text" encoding="iso-8859-1"/>

	<xsl:template match="java/object/void" >
		<xsl:apply-templates select="object"/>
	</xsl:template>

	<xsl:template match="object">
		<xsl:apply-templates select="void"/><xsl:text>&#xD;</xsl:text>
	</xsl:template>

	<xsl:template match="void">
		<xsl:for-each select="*">"<xsl:value-of select="normalize-space(.)"/>",</xsl:for-each>
	</xsl:template>

</xsl:stylesheet>

F-Spot will not start on Ubuntu Karmic

Find photo.db (probably in ~/.config/f-spot/) and mv it out of the way. Restart f-spot. Import your photos. Should be good.

No idea why it happens. Haven’t worked out how to get the tags, etc back from the old photo.db yet.

Here’s a bug related to this on launchpad:

Getting MSSQL Command Line Access from ISQL on Ubuntu

Install freetds
sudo apt-get install freetds-bin

Check it works:
fsql -S -U

See how to use it (it pretty much matches isql and bcp).
man fisql
man freebcp

Engineer Victory Dance (the hard work was done the freetds posse and all those lovely linux peeps, but dance anyway, required action).

Quick Ruby On Rails

Requirement

I need to do a quick prototype of a web app for a friend that has to run initially on a windows localhost and be able to hand it over to his developer for development if it floats the necessary boats. Totally coincidentally, I need to do a prototype for a web app at work too.

Tech Choice Breakdown

I haven’t spent anytime with Ruby On Rails and precious little doing web apps of any kind. Ruby I’ve scripted with a little, any web ui’ed stuff has usually ended up in hasty perl on the intranet. RoR is something I have meant to fit in for a few years.

So, I’ve dug in the library and found the Pragmatic Programmers Agile Web Development With Rails by DT, THH et al, 2005 edition. I understand Ruby and Rails have upped a version or two since then, so I’m tempted to use the book and stick with version 1.x of Rails.

InstantRails 1.7 looks like it hits the two requirements of providing quick start up for my friend, and being version 1.x of Rails so I can use the THH book to get up to speed.

Promise to self is to upgrade once prototype 0.2.x is motted on either system, probably looking at JRuby as well.

Installing InstantRails

1. Snagged InstantRails 1.7 on advice from Pragmatic Programmer forum (slightly different topic, I know, but looks as though it may apply with my old book). Work is shaping network traffic, so took a bloody age to download.
2. Extracted the zip contents to C:\, now have a directory C:\InstantRails.
3. Opened C:\InstantRails\readme.txt and followed the instructions. One functioning server – tick

Now need to do some Use Cases and get on with it….

Quantum Gag

Q: Why did the tachyon cross the road?
A: Because it was on the other side.

Chain Letter Reply Template

I hate chain letters. The underlying conceit of a chain letter is to make the sender feel guilty or afraid for not sending it on, to make the receiver feel annoyed at the sender for involving them, and to perpetuate that cycle ad nauseam. It may seem harmless at first touch, but if you forward one you are being manipulated by the original author. Even if you just send them on but don’t feel anything, some of the people you target, or indirectly target, are going to have a bad day or two because of it, because of you. So just don’t send them on: no-one’s hurt; it doesn’t cost you anything; and you’ve saved some of your bandwidth. It wasn’t that funny or clever anyway. Just full of platitudes or bad info.

I try to kill these damn things; to stop them everytime I see one (cue rousing music, maybe some kilted people thrusting swords in the air, general cheering, and exeunt all stage left (pursued by bear)).

My standard reply to chain letters:

Hi So-andSo,

Please don’t be offended, this is no reflection on you, but please don’t include me on any chain letters. I don’t like them. I find the concept underlying them offensive and divisive. If I had a religion, they’d be against it!

The wikipedia page is pretty evenhanded about them: http://en.wikipedia.org/wiki/Chain_e-mail and this is good mickey-take: http://www.perry.com/bizarre/antichn.html

Something lovely and generally ignoring the whole chain letter thing so hopefully there’s no bitter aftertaste.

If it’s a virus/worm/hoax email of any kind, I look it up on Virus Encyclopedia, Sophos, or Hoax Slayer and email the sender with the ref and an explanation.

Twitter Archiving

Been asked a few times about how to get all a user’s tweets off twitter and onto a local file.

Activestate have a great python recipe to do it that works well. You just have to get python and BeautifulSoup onto your PC.

First install Python:

• Linux, it’s easy, just check your package manager and get python.
• Windows, it’s easy, go to activestates website a download and install ActivePython

Then install BeautifulSoup:

• Download BeautifulSoup tar
• Unzip it – you may need to untar it too. On Windows you can just use 7-zip or WinZip on the tar file as if it was a zip file. You will end up with a folder called BeautifulSoup-X.x.x.
• Open a terminal or Command Prompt and cd to the BeautifulSoup-X.x.x folder
• Type python setup.py install and hit return
• BeautifulSoup is installed

Now setup the program that gets the data and saves it for you.

• Open the activestate recipe page.
• Create a new text file in your Documents folder called as twitterArchive.py (make sure you get rid of the .txt extension)
• Edit the file and copy paste the recipe into the file:

  #!/usr/bin/python
  
  import time
  from urllib2 import urlopen
  from BeautifulSoup import BeautifulSoup
  
  # Replace USERNAME with your twitter username
  url = u'http://twitter.com/USERNAME?page=%s'
  tweets_file = open('tweets', 'w')
  
  for x in range(10*10000):
      f = urlopen(url % x)
      soup = BeautifulSoup(f.read())
      f.close()
      tweets = soup.findAll('span', {'class': 'entry-content'})
      if len(tweets) == 0:
          break
      [tweets_file.write(x.renderContents() + '\n') for x in tweets]
      # being nice to twitter's servers
      time.sleep(5)
  
  tweets_file.close()
  

• Update USERNAME as instructed in the recipe and save it
• Run the program. On Windows you can double-click it or run from a Command Prompt. On linux you’ll need to run it through python: python -c twitterArchive.py
• It will create a file called tweets in the same folder. After a while (it can take some time) the file will be filled with your tweets.

OSX: Create a Playlist From a Folder

I wanted to use this excellent command line from commandlinefu.com on the mac to create a podcasts playlist on my G1 each time I chucked some new podcast files on it. Unfortunately OSX (or BSD) aren’t exact matches to linux, so here’s my hack. It’s not elegant, too many pipes, but it works!

cd /Volumes/NIALLSG1/Music/Podcasts && find . -type f -exec stat -f "%c %N" {} \; | grep -v '\./\._' | grep -v m4v | sort -rn | sed 's/\.\//;.\//' | awk -F ';' '{print $2}' > ../podcasts.m3u

On my mounted G1 all my podcasts live in /Volumes/NIALLSG1/Music/Podcasts. I generate the podcasts.m3u to the Music folder.

The stat prints the created date as seconds from the epoch followed by the file name.

The first grep -v removes the stupid itunes symbolic links (mutter, mutter) that get copied across from the listings in iTunes, the second removes video podcasts as the Music Player doesn’t play the video.

The sort gives us the list sorted on created date descending.

The sed gives me an easy delimiter to use in the awk. A cheaty bit I know.

The awk spews out the filenames only for the write to the m3u file.

Teh usual caveats apply; your mileage may vary; be careful; etc.

Installing Laconica

Using:

• OS: $ lsb_release -a
Distributor ID: Ubuntu
Description: Ubuntu 8.10
Release: 8.10
Codename: intrepid

• php 5.2.6
• mysql server 5.0.67-0ubuntu6

Added pre-requisities php libraries:

• sudo apt-get install php5-cli
• sudo apt-get install php5-curl
• sudo apt-get install php5-mysql
• sudo apt-get install php5-gd
• sudo apt-get install php-gettext
• Check mbstring is installed: php -i | grep -i mbstring
  If you get some output, you should be OK.
• Check xmlwriter is installed: php -i | grep -i xmlwriter
  If you get some output, you should be OK.

Added nice to haves:

• sudo apt-get install php5-memcache
• sudo apt-get install php-pear
• sudo apt-get install php5-dev
• sudo pecl install mailparse
• Not sure whether you need this, but did it anyway as seemed logical at the time: sudo vim /etc/php5/conf.d/mailparse.ini
  and add:
# manual configuration for php mailparse module
# yourInitials 16-mar-09
extension=mailparse.so

• Install Sphinx Server. Not sure how/when we’ll config this, but it’s as well to get it in before we’re asked for it:
  • sudo apt get install libmysqlclient15-dev
  • wget http://sphinxsearch.com/downloads/sphinx-0.9.8.1.tar.gz
  • tar -xvf sphinx-0.9.8.1.tar.gz
  • cd sphinx-0.9.8.1/
  • ./configure
  • make
  • sudo make install
  • Check it’s in OK: search
    You should see output starting with: Sphinx 0.9.8.1-release
• sudo apt-get install php5-xcache

I suggest you also install phpMyAdmin for looking after the mysql databases if you haven’t already. I’m certainly not hard enough to do all the sql work from the command-line

Just in case restart Apache:

• sudo /etc/init.d/apache2 restart

And now, we begin. Rather marvellously Mr Prodromou has just released a new laconia update, version 0.7.2.1, which includes a “first version of a web-based installer”. I’m following the instructions from the README file in the download. Here goes…. Just followed the instructions and I have a functioning install! Mwah-ha. Mwah-ha! Mwah-ha-ha-ha-haaaaaa! I have the power. etc.

Things not working: Fancy URLs. But I’ll get ’round to that.

UPDATE: run sudo a2enmod rewrite to enable the apache rewrite engine so you can use fancy urls and then follow the instructions in the README for .htaccess.

Getting a Current Subversion Project Into Git Control

Whilst learning how to use git with svn I am documenting the way I’ve done it. I’m using bash and am starting from the directory above my working directory. The project is checked out already from an svn repo. The working directory is “efot”.

• Commit anything outstanding in your working directory to svn
• Move your working directory out of the way if you’re paranoid like me: mv efot efot.pregit
• Checkout (clone) your project from svn using git: git-svn clone http://svnServer/repos/section/trunk/efot --username=yourSvnName

• Watch what’s happening: cd efot;gitk &
• Make sure git ignores .svn and, since I use eclipse in an heterogenous env with other developers, I exclude bin/, and *.log as well:
  • vim .git/info/exclude
  • add “.svn” (without the quotes) to the bottom of the file
  • newline, add “bin” (without the quotes) to the bottom of the file
  • newline, add “*.log” (without the quotes) to the bottom of the file
  • save and quit.
• Make sure svn will ignore git files and local files I don’t commit to the trunk:cd .. && svn ps svn:ignore ".git" efot && cd efot
cd .. && svn ps svn:ignore ".classpath*" efot && cd efot

cd .. && svn ps svn:ignore ".settings" efot && cd efot

• give it a descriptive label in git: git-tag -a 20090310_efot_release_prod

Now for the paranoid amongst us, cross the i’s and dot the t’s :

• cp -R ../efot.pregit/src . ; cp -R ../efot.pregit/testsrc .;cp -R ../efot.pregit/xml;cp ../efot.pregit/.classpath . ; cp ../efot.pregit/.settings .;
• git add .
• git commit -a
• update from svn: git-svn fetch
• if any come in on the fetch, replay any git changes onto it (shouldn’t be any at this point): git-svn rebase
• commit to svn: git-svn dcommit

Job’s a good’un. I think.

Refs: Git User’s Manual, Flavio Castelli, and Git Crash Course.