Don't ever by an OCZ SSD for your MacBook! The things just won't work. I'm on a replacement of a broken OCZ 240GB SATA III Vertex 3 and even the replacement doesn't work. Completely unpredictable: the first one would beachball my MacBook to death after jumping through many hoops to get it to work in the first place.

Second one doesn't even get recognized as a disk when installed as internal drive, but works when connected through an external USB interface.

And don't even get me started about the lack of support for OCZ firmware updates for the SSD on the Mac - there's none. You're left with the choice of updating the SSD's firmware using Linux or Windows. Admittedly, I blame myself for this - should have researched the SSD firmware updates before buying...

Rails upgrade pains

Rails is cool. It really is. But why does even a minor version upgrade of Rails cost me a week in fixing all the breaks that the API changes introduce? Why does every upgrade have to break everything?

Drawing trees with Raphael.js and Ruby

It's been quite some time since I last posted (I just realise I open every post like this - guess I'm one of the laziest bloggers out there :-).

I've been busy hacking Ruby and thought I share this. For a website I'm currently building, I needed to draw organisation and process hierarchies (ie. any type of tree or forest) in the browser. the drawings need to be interactive (ie. clickable for getting a node's details).

First challenge was how to do the drawing. Generate the image server side, but then what about clickable areas in the image? Draw the image client side, but then I really didn't want to implement a tree drawing algorithm in JavaScript...

I did some research about what's out there in the wild and came across Raphael.js (kudos Dmitry, great job!). Raphael is a clean, simple graphics API for JavaScript.

But I still didn't want to implement tree drawing in JavaScript, so came to the conclusion to implement the tree-drawing in Ruby so as to generate the JavaScript.

The result is further down in this post. TreeDraw is a class to which you pass a tree or forest (I'm using acts_as_tree to get methods like children, ancestors, parent) as well as some drawing options (optional, defaults are all defined).

You can also provide a base URL (e.g. "/mycontroller/myaction/") which will be augmented with a node's ID to become a link to viewing a node's details (e.g. "/mycontroller/myaction/5").

The code looks like this:

# The TreeDraw class draws trees using the Raphael Javascript drawing library.
class TreeDraw

  include ActionView::Helpers::TextHelper
  include ERB::Util

  # Construct a new Tree drawing instance following the Reingold-Tilford-Algorithm. Parameters are:
  # - roots => The forest to draw (array of roots of the forest)
  # - link => If specified, then each box will link to the specified link with the id of the specific node appended
  # - options => a hash allowing to override the various default drawing options
  def initialize(roots, link=nil, options = {})
 @roots = roots # the roots of the tree where each node responds to methods like "children", "parent", "ancestors"
 @link = link
 init_options(options)
 @x_pos = {} # hash to hold the x positions for each node
 @x_mod = {} # hash to hold the x position modifiers for each node
 @x_cumul_mod = {}
 @y_pos = {} # hash to hold the y positions for each node
 @max_width = {} # hash to hold the max width of each root's tree
 @connectors = [] # the svg path strings for all connectors in the tree 
 @completed = {}
 @total_width = 0 # total width of drawing canvas adopted to tree size
 @total_height = @options[:box_height] + @options[:v_padding] # total height of drawing canvas adopted to tree size, initialised to height of one box                               
  end

  # Return the Javascript required to draw the tree/forest at
  # the top left of the specified dom element id
  def draw(dom_element_id)
 calculate
 return generate_code(dom_element_id)
  end

  protected

  # Return the root for the specified node
  def root_for(node)
 if @roots.member?(node)==true
   return node
 end
 node.ancestors.each do |ancestor|
   if @roots.member?(ancestor)==true
  return ancestor
   end
 end
  end

  # Generate the SVG paths for connecting the specified node with its ancestor
  def connect(node)
 offset = @options[:box_width]/2
 if node.children.size > 0
   @connectors << "M#{@x_pos[node]+offset} #{@y_pos[node]+@options[:box_height]} L#{@x_pos[node]+offset} #{@y_pos[node]+@options[:box_height]+(@options[:v_padding]/2)}"
   @connectors << "M#{@x_pos[node.children.first]+offset} #{@y_pos[node]+@options[:box_height]+(@options[:v_padding]/2)} L#{@x_pos[node.children.last]+offset} #{@y_pos[node]+@options[:box_height]+(@options[:v_padding]/2)}"
 end
 node.children.each do |child|
   @connectors << "M#{@x_pos[child]+offset} #{@y_pos[child]-(@options[:v_padding]/2)} L#{@x_pos[child]+offset} #{@y_pos[child]}"
   connect(child)
 end
  end

  # Return the level of this node in the tree
  def level(node)
 root = root_for(node)
 return (node.ancestors().size() - root.ancestors().size()) + 1
  end

  # Layer all nodes in the appropriate y-coordinates
  def layer(node)
 l = level(node)
 @y_pos[node] = ((l - 1) * @options[:box_height]) + ((l - 1) * @options[:v_padding])
 if @y_pos[node] > @total_height
   @total_height = @y_pos[node]
 end
 node.children.each do |child|
   layer(child)
 end
  end

  # Return all nodes of the tree/forest on the specified level (regardless of whether these nodes are related)
  def nodes_for_level(level, node = nil)
 nodes = []
 if (level == 1)
   return @roots
 end
 if node == nil
   @roots.each do |node|
  nodes += nodes_for_level(level, node)
   end
 else
   if level==(level(node)+1)
  nodes += node.children
   else
  node.children.each do |child|
    nodes += nodes_for_level(level, child)
  end
   end
 end
 return nodes
  end

  # Return the left sibling of this node (note that "sibling" does not mean that the two nodes
  # belong to the same ancestor)
  def left_sibling(node)
 siblings = nodes_for_level(level(node))
 if siblings
   index = siblings.index(node)
   if (index > 0)
  return siblings[index-1]
   end
 end
 return nil
  end

  # Shift the specified node with respect to its left sibling.
  def shift_beyond(node)
 x = @x_pos[node]
 x += @x_mod[node]
 node.ancestors.each do |ancestor|
   if @x_mod[ancestor]==nil
  return x
   else
  x += @x_mod[ancestor]
   end
 end
 return x
  end

  # Position all nodes in the tree/forest
  def calculate()
 @roots.each do |root|
   first_pass(root)
   second_pass(root)
   layer(root)
   connect(root)
   if @total_width < @max_width[root]
  @total_width = @max_width[root]
   end
 end
  end

  # First pass placing the nodes horizontally.
  def first_pass(node)
 # first pass is post order, depth first
 node.children.each do |child|
   first_pass(child)
 end
 mod = 0
 x_pos = 0
 children = node.children
 if (children.size == 0) # leaf node
   left_sibling = left_sibling(node)
   if left_sibling # leaf node with left sibling
  x_pos = shift_beyond(left_sibling) + @options[:box_width] + @options[:h_padding]
   end
 else # node with children
   left_most_child = children.first
   right_most_child = children.last
   x_pos = (@x_pos[left_most_child] + @x_mod[left_most_child] + @x_pos[right_most_child] + @x_mod[right_most_child])/2
   left_sibling = left_sibling(node)
   if left_sibling # node with left sibling
  mod = [(@options[:box_width] + @options[:h_padding]) - (x_pos - shift_beyond(left_sibling)), 0].max
   end
 end
 @x_pos[node] = x_pos
 @x_mod[node] = mod
 @x_cumul_mod[node] = 0
 root = root_for(node)
 @max_width[root] ||= 0
 if (@x_pos[node]+@options[:box_width]) > @max_width[root]
   @max_width[root] = @x_pos[node]+@options[:box_width]
 end
  end

  # Second pass shifting the nodes horizontally according to the position of their left siblings.
  def second_pass(node)
 @x_pos[node] +=  @x_mod[node] + @x_cumul_mod[node]
 node.children.each do |child|
   @x_cumul_mod[child] = @x_mod[node] + @x_cumul_mod[node]
 end
 node.children.each do |child|
   second_pass(child)
 end
  end

  # Once the nodes have been positioned horizontally and vertically, generate the Raphael.js Javascript
  # to draw the tree in the browser
  def generate_code(dom_element_id)
 result = ""
 result += ""
  end

  # Generate the box and text for a node.
  def generate(node)
 result = generate_box(node)
 result += generate_text(node)
 node.children.each do |child|
   result += generate(child)
 end
 return result
  end

  # Draw the box and corresponding mouseover (for domTT popups).
  def generate_box(node)
 result = "t=paper.rect(#{@x_pos[node]}, #{@y_pos[node]}, #{@options[:box_width]}, #{@options[:box_height]}, #{@options[:box_round_edge]});"
 result += "t.attr({stroke:'#{@options[:box_color]}',fill:'#{@options[:box_color]}'});"
 result += "t.node.onmouseover=function(event){domTT_activate(this, event, 'content', '#{node.description.gsub(/\n/,"\\\n").gsub(/\r/,"\\\r")}', 'styleClass', 'tooltip')};"
 result += "t=paper.rect(#{@x_pos[node]+@options[:shadow_box_offset]}, #{@y_pos[node]+@options[:shadow_box_offset]}, #{@options[:box_width]}, #{@options[:box_height]}, #{@options[:box_round_edge]});"
 result += "t.attr({fill:'#{@options[:shadow_box_color]}',stroke:'#{@options[:shadow_box_color]}'});"
 result += "t.toBack();"
 return result
  end

  # Generate the node's text in the drawing as well as a link to displaying details about the node (if a link was specified upon construction) using node's id.
  def generate_text(node)
 length = node.name.size
 result = "t=paper.text(#{@x_pos[node]}, #{@y_pos[node]}, '#{truncate(node.name, :length => @options[:max_text_length], :ommission => "...")}\\n(#{truncate((node.reference ? node.reference : ""), :length => @options[:max_text_length], :ommission => "...")})\\n#{truncate((node.person ? node.person.full_name : ""), :length => @options[:max_text_length], :ommission => "...")}');"
 result += "t.attr({'text-anchor': 'middle', 'font-size': #{@options[:font_size]}, 'font-weight': '#{@options[:font_weight]}', 'font-family': '#{@options[:font_family]}', fill:'#{@options[:text_color]}'});"
 result += "t.translate(((#{@options[:box_width]} - t.getBBox().width)/2)+t.getBBox().width/2, #{@options[:box_height]}/2);"
 if (@link)
   result += "t.hover(function(){this.attr({fill: '#{@options[:highlight_color]}'});},function(){this.attr({fill: '#{@options[:text_color]}'});});"
   result += "t.click(function(){location='#{@link}/#{node.id}'});"
 end
 return result;
  end

  # Generate the connecting lines between nodes.
  def generate_connectors
 result ="t=paper.path('"
 @connectors.each do |path|
   result += " #{path}"
 end
 result += "');t.attr({stroke:'#{@options[:line_color]}'});"
 return result
  end

  # Initialise the options using defaults or the overrides specified upon construction of this object 
  def init_options(options)
 @options = options
 @options[:box_width]  ||= 200 # width of a box representing a node
 @options[:box_height]  ||= 50 # height of a box representing a node
 @options[:tree_padding]  ||= 50 # horizontal padding between trees of the forest
 @options[:v_padding]  ||= 50 # vertical padding between all nodes of the forest
 @options[:h_padding]  ||= 40 # horizontal padding between nodes of the same tree in the forest
 @options[:box_round_edge] ||= 5 # radius of round edges (0 for normal edges)
 @options[:box_color]  ||= "#9aafe5" # fill color of a box representing a node
 @options[:highlight_color] ||= "#2e6ab1" # highlight color of the text if there's a link
 @options[:shadow_box_color] ||= "#0e509e" # color of the shadow for each box representing a node
 @options[:shadow_box_offset]||= 3 # offset of the shadow for each box representing a node
 @options[:line_color]  ||= "#0e509e" # for connectors between boxes representing nodes
 @options[:text_color]  ||= "#000" # color of text in each box representing a node
 @options[:font_size]  ||= 10
 @options[:font_weight]  ||= "normal"
 @options[:font_family]  ||= "Arial"
 @options[:max_text_length] ||= 35 # truncate any text in the box after the specified characters (appending '...')
  end

end

The simplest way to use this is in the view of a Rails application like so:

      

        <%= TreeDraw.new([@organisation], url_for(:controller=>"organisation", :action=>"details", :id=>nil)).draw("org_chart_drawing") %>
      

Where @organisation is the root of a tree (using acts_as_tree), the specified URL provides the link to the details for a node and the string specified in the draw method tells TreeDraw to place the Raphael canvas in the element with ID org_chart_drawing.

When drawing the box for a node, I am using the domTT tooltip to display a description of the detail for a node...

Have fun!

If you are using Ferret for Rails full text search, never forget to do the following:

gem install acts_as_ferret (duh!)

If this works with your Rails install - fine. Don't go on and change a winning team. If it doesn't work (e.g. when you do a rake db:migrate and you get

no such file to load - ferret


do the following

gem install ferret

and you should be fine.

Rendering XML with Rails

I had to code a webservice with Rails yesterday. Its obvious purpose was to respond with XML formatted data to an HTTP GET request. I looked at some of the examples out there and ran into trouble. I lost some time searching for a solution, so here's the summary of the solution to save others the hassle.

Here's what people recommend doing to render XML from a HTTP request in Rails:

class WebServiceController < ApplicationController

 def people
  @people = Person.find :all
  respond do |format|
   format.xml
  end
 end

end

And the corresponding people.xml.builder in /views/web_service/:

xml.instruct! :xml, :version=>"1.0"

xml.people do
  @people.each do |person|
    xml.person do
        xml.firstname person.first_name
        xml.lastname person.last_name
        xml.age person.age
    end
  end
end

When testing this, I kept getting browser errors about invalid XML. When I looked at the data that was returned from my webservice call, I realised that the XML was generated like I wanted, but it was subsequently embedded into the application layout I am using for all of the views (views/layouts/application.rhtml).

The issue was resolved by changing my controller code to

class WebServiceController < ApplicationController

 def people
  @people = Person.find :all
  render :template => 'web_service/people.xml.builder', :layout => false
 end

end

Character Encoding UTF-8 with JPA/Hibernate, MySql and Tomcat

I'm writing a little Java application using JPA (Hibernate implementation) and Spring. The application will run on Tomcat and uses MySql as the RDBMS.

The problem I had today was with the good old character encoding: I was able to store German Umlaut characters (üöä) properly in MySql, but whenever I retrieved them, they would be scrambled - regardless of whether I displayed the result on a web page or just printed it to the console.

So, the problem is: how to consistently set UTF-8 as the character encoding of choice throughout the whole stack:

• for MySql as well as for any session coming through the JDBC driver in order to ensure that any entity created by Hibernate/JPA uses UTF-8
  
• for Tomcat to make sure that any data served uses UTF-8
  

I know that you'll find a lot of material on the solution for each individual piece of software in my tech stack across the net. However, I still think it's worth to post this solution as I did not find all elements of it in one place (and don't want to search again the next time :-)

Here's what I did:

1. Ensure that MySql runs on UTF-8 as default: in the MySql configuration file my.cnf add the following in the section for mysqld:

[mysqld]
...
default-character-set=utf8
default-collation=utf8_general_ci
...

2. Configure your MySql JDBC driver connection as follows (obviously hostname, port and schema are probably different in your configuration :-):

jdbc:mysql://localhost:3306/test?useUnicode=true&connectionCollation=utf8_general_ci&characterSetResults=utf8

When configuring the above driver URL in your Spring XML context definition, don't forget to escape the Ampersand as you will get parsing errors otherwise.

3. Configure Tomcat for UTF-8 by adding the following line to your : catalina.bat or catalina.sh:

JAVA_OPTS="$JAVA_OPTS -Djavax.servlet.request.encoding=UTF-8 -Dfile.encoding=UTF-8"

Versions I am using: MySql 5.0, Tomcat 6.0.20, Spring 2.5.6, Java 6, MySql Connector 5.1.6

Happy hacking!

Been a long time since I have posted anything. Got dragged into some, well - let's say, interesting work in my day job. Done some programming anyway.

GWT-Ext or SmartClient based GUI wiht a JPA/Spring/MySQL/JAX-B/JAX-RS driven backend. And I'm using Envers for auditing - I love it.

Got fed up writing all those REST requests, so basically just set up a service with a Groovy engine behind. This allows me to just send scripts using my plain POJO domain model from the GUI without having to expose every little piece of functionality as a REST service. All the while every script execution returns a piece of XML driving the AJAX GUI. On this front I love the store-based model of SmartClient over the store based model of GWT-Ext (because of the CRUD service hooks).

Need to play around with the Groovy bit some more - but I can almost hear the critics yell "SECURITY ISSUE"!

Protecting the whole thing with CAS and aspect-driven security on the server side for method execution as well as data retrievals/updates using JPA.

Need to play with this a bit more and will provide some code here. Watch this space...