Archive for the ‘Uncategorized’ Category

Debugging JRuby apps

Let me start off by saying that I love using vim and textmate for hacking sessions, however, when it comes to getting into the nitty gritty details of a running program to figure out what’s going on, it’s nice to have a debugger.

The ruby-debug gem does not work in jruby directly, as it depends on native APIs.
There is a Java-based JRuby implementation of the fast ruby debugger rdebug.

One of my favorite editors is IntelliJ IDEA, and now that it has a Ruby plugin, it’s even better. IDEA allows you to easily navigate classes, and it does a pretty good job with a dynamic language like jruby. One of the best parts about IDEA is being able to visually debug a jruby gem or jruby on rails application directly in the IDE, so that you can step in, over or though method calls, examine variables, evaluate expressions, etc.

In order to get your jruby installation set up correctly, however, there are a few simple steps you need to take first.

1. Manually download the ruby-debug-base-0.10.3-java.gem from debug-commons.

sudo jruby -S gem install ruby-debug-base-0.10.3-java.gem

2. Leave the directory and then do the following:

sudo jruby -S gem install –-ignore-dependencies ruby-debug –version 0.10.3
sudo jruby -S gem install –-ignore-dependencies ruby-debug-ide

3. Now install columnize

sudo jruby -S gem install columnize

You should be good to go. Having an excellent tool to debug your apps could boost your productivity as well.

Advertisements

Using Proprietary C libraries from JRuby

Let’s assume you work at big enterprise company X. Your company has assigned a new project to you, and one of its only requirements is to make sure you use this library… You have no control over the source code in the library, but you know you have to use it, because it contains some very valuable data to your users. You’ve got a ton of logic wrapped up in that library, and the company you work for already has some logic built around using that library. It is key to some department’s success. This means a couple things: (1) there’s great potential for you to provide great value to your users. It also means something else: (2) it’s probably written in C, and in general, C sucks for programmer happiness. That’s why most people programmers abandoned it — but the people who wrote that library haven’t, and it isn’t being replaced by java or ruby any day soon.

Now, before you search hotjobs or the 37signals job board for a new gig, realize that you still have some level of freedom in how you choose to go about satisfying your core requirement. This freedom includes building a user interface in a more agile-friendly language or framework. Let’s, for the sake of argument, pick some of our favorite tools to use from our development warchest. Is it possible? Could we be happy performing this task that includes a cryptic old C library? I think so. Let’s pick JRuby.

So how could you go about using JRuby to hit that old C library? You could try JRuby => C through FFI, but if the structures are quite complex and nested, you might end up debugging a lot of nasty mapping issues. JRuby can communicate with Java classes quite easily, and Java already has a way of communicating with native C code that’s well established and robust. Good old JNI. So how could we build a JRuby => Java => JNI => C connection?

It’s easier than you think. For the sake of simplicity, let’s send all data from JRuby to Java in hashes or arrays, and all data coming back from the JNI layer should be hash maps.

Wouldn’t it be nice to have a JRuby process that is able to do the following?

require 'proprietary_api'
Proprietary::API.some_calculation( some, arguments, for, calculation )

It’s quite possible, and it’s not too difficult to achieve, if you know the proper steps involved.
You’ll need:
a jar (for convenience), which contains…
a java class to declare some native methods, which will be implemented in C…
and those C methods will call the library functions you want to use.

The Java Class

import java.util.*;
import org.jruby.runtime.builtin.IRubyObject;
import org.jruby.RubyString;

public class ProprietaryJavaAPI
{
  private IRubyObject rubyAPI;

  public static void init( String library ) {
    try {
      System.load( library );
    } catch ( Exception e ) {
      System.out.println( "Attempted to load " + library + ", but failed" );
    }
  }

  public ProprietaryJavaAPI(IRubyObject rubyAPI) {
    this.rubyAPI = rubyAPI;
  }

  public native Collection some_calculation( String some, String arguments, String to_the, String calculation );
}

The C code in some_calculation.c

#include 
#include 
#include "string.h"
#include "ProprietaryJavaAPI.h"
#include "proprietary_c_lib.h"

JNIEXPORT jobject JNICALL Java_ProprietaryJavaAPI_some_calculation(JNIEnv * env, jobject this, jstring some, jstring arguments, jstring to_the, jstring calculation )
{
    // here's where you call the c library and return the results... 
}

The JRuby Code that uses the Java => C layer

require 'java'

 # The jar file containing java class ProprietaryJavaAPI
require File.join( File.expand_path( File.dirname(__FILE__) ), 'proprietary_c_library_java_support' ) 

# The Java class containing definitions for native methods, like proprietary_query
include_class 'ProprietaryJavaAPI'  

 # Calling the static init method on ProprietaryJavaAPI to load the native shared object library, containing C code.
ProprietaryJavaAPI.init( File.join( File.expand_path( File.dirname(__FILE__) ), "libProprietaryAPI.so" ) ) 

module Proprietary
  class API
    class << self
      JAVA_API = ProprietaryJavaAPI.new( Proprietary::API )

      def some_calculation( *args )
         JAVA_API.some_calculation( *args )
      end
   end
end

Putting it all together
Compiling Java

    javac -target 1.6 -cp #{ ENV['JRUBY_HOME'] }/lib/jruby.jar #{Dir.glob("*.java").join(" ")}
    javah -jni ProprietaryJavaAPI
    jar cvf proprietary_c_library_java_support.jar #{Dir.glob("*.class").join(" ")}

Compiling C (Makefile)

libProprietaryJavaAPI : some_calculation.c
        gcc -m32 -fPIC -I${JAVA_HOME}/include/ -I${JAVA_HOME}/include/linux -I.. --shared -o libProprietaryJavaAPI.so -L. -lproprietary_c ProprietaryJavaAPI.c some_calculation.c 
clean :
        rm libProprietaryJavaAPI.so

Using JRuby, Warbler & Rake to deploy Rails apps to JBoss

If you’re using JRuby, you’re likely going to need a way to automate your deployment to an enterprise-class application container, such as JBoss. As the Rails & JRuby community has evolved, so have the tools available to developers for everything from testing, to security, to deployment. We’ll cover some of the key tools that you can use to turn the mundane task of application deployment into a fully automated process that will be a joy to use. We’ll start with *warbler*.

Warbler is a jruby gem that can be used to make war (web archive) files with any Rails, Merb or Rack-based application. You might be thinking you’ll have to configure the deployment with some ugly xml file (the typical “java” way)… Fortunately, that’s not true! Consistent with the Ruby & Rails way, warbler allows us to use Ruby code to configure the process. That rocks. Lets start by installing warbler and configuring it to our liking.

# Installation is easy:
jruby -S gem install warbler

# Navigate to the top level folder in your Rails app
# Display the list of available actions warbler gives you:
warble -T

# NOTE: “rake” is printed out, but you should actually type “warble” instead.

rake config         # Generate a configuration file to customize your war assembly
rake pluginize      # Unpack warbler as a plugin in your Rails application
rake war            # Create trunk.war
rake war:app        # Copy all application files into the .war
rake war:clean      # Clean up the .war file and the staging area
rake war:gems       # Unpack all gems into WEB-INF/gems
rake war:jar        # Run the jar command to create the .war
rake war:java_libs  # Copy all java libraries into the .war
rake war:public     # Copy all public HTML files to the root of the .war
rake war:webxml     # Generate a web.xml file for the webapp

# Generate the base configuration file in config/warble.rb
warble config

# Generate a war file (so simple!)
warble

If you have a Rails application, the rails gem will be packaged automatically for you. Other gems in Rails.configuration.gems will be packaged as well. If safe multi-threaded execution is detected, runtime pooling will be disabled. The app, lib, config, log, tmp and vendor directories will be placed under the .war file’s WEB-INF dirctory. The public files will be placed in the root of the .war file. Any .jar files will be placed in WEB-INF/lib. If you need to upgrade a java library that’s included in the warbler gem (for example jruby-complete-1.2.0.jar, etc…), then all you need to do is replace it in the WARBLER_HOME/lib directory.

# Let’s say you want to upgrade from JRuby 1.2 to 1.3.1.
# Find the full path to warbler:
jruby -S gem which warbler

# Let’s say the output was /Users/demmons/jruby/lib/ruby/gems/1.8/gems/warbler-0.9.14/lib/warbler.rb
# Replace your old version of jruby with a newer version:
cd /Users/jrubyist/jruby/lib/ruby/gems/1.8/gems/warbler-0.9.14/lib
cp /Users/jrubyist/downloads/jruby-complete-1.3.1.jar .
rm jruby-complete-1.2.0.jar

Now, warbler comes with some sane defaults for applications, however as soon as you introduce other gem dependencies into your application, you’ll likely need to do some basic configuration. It’s not hard — it just needs to be done. Here’s an example of a config/warble.rb I use in a production-ready application.

# Warbler web application assembly configuration file
Warbler::Config.new do |config|
  # Temporary directory where the application is staged
  # config.staging_dir = "tmp/war"

  # Application directories to be included in the webapp.
  config.dirs = %w(app config lib log script vendor tmp)

  # Additional files/directories to include, above those in config.dirs
  config.includes = FileList["Rakefile"]

  # Additional files/directories to exclude
  config.excludes = FileList["log/test.log"]

  # Additional Java .jar files to include.  Note that if .jar files are placed
  # in lib (and not otherwise excluded) then they need not be mentioned here.
  # JRuby and JRuby-Rack are pre-loaded in this list.  Be sure to include your
  # own versions if you directly set the value
  #config.java_libs += FileList["lib/java/*.jar"]

  # Loose Java classes and miscellaneous files to be placed in WEB-INF/classes.
  # config.java_classes = FileList["target/classes/**.*"]

  # One or more pathmaps defining how the java classes should be copied into
  # WEB-INF/classes. The example pathmap below accompanies the java_classes
  # configuration above. See http://rake.rubyforge.org/classes/String.html#M000017
  # for details of how to specify a pathmap.
  # config.pathmaps.java_classes << "%{target/classes/,}"

  # Gems to be packaged in the webapp.  Note that Rails gems are added to this
  # list if vendor/rails is not present, so be sure to include rails if you
  # overwrite the value
  config.gems = ["activerecord-jdbc-adapter", "jruby-openssl", "jrexml", "hpricot","ruport", "memcache-client", "uuidtools"]
  config.gems << "poiyer"
  config.gems << "charter"
  config.gems["activerecord-jdbc-adapter"] = "0.9"

  # Include gem dependencies not mentioned specifically
  #config.gem_dependencies = true

  # Files to be included in the root of the webapp.  Note that files in public
  # will have the leading 'public/' part of the path stripped during staging.
  # config.public_html = FileList["public/**/*", "doc/**/*"]

  # Pathmaps for controlling how public HTML files are copied into the .war
  # config.pathmaps.public_html = ["%{public/,}p"]

  # Name of the war file (without the .war) -- defaults to the basename
  # of RAILS_ROOT
  config.war_name = "dealanalyzer"

  # Value of RAILS_ENV for the webapp
  if ENV['ENVIRONMENT'] == 'production'
    config.webxml.rails.env = 'production' 
    puts "Building war for production rails environment!"
  else
    config.webxml.rails.env = 'development'
  end
  # Application booter to use, one of :rack, :rails, or :merb. (Default :rails)
  # config.webxml.booter = :rails

  # Control the pool of Rails runtimes. Leaving unspecified means
  # the pool will grow as needed to service requests. It is recommended
  # that you fix these values when running a production server!
  config.webxml.jruby.min.runtimes = 2
  config.webxml.jruby.max.runtimes = 8

  # JNDI data source name
  # config.webxml.jndi = 'jdbc/rails'
end

# Finally, let’s test the war creation:
warble

At this point, we have an very simple automated way of generating a war file that can be deployed to a java application server, but we don’t have a way of actually performing the deployment. If you’re like me, you definitely don’t want to manually type out the commands to ftp a file to your app server destination’s temp folder, ssh in and copy the war from the tmp folder to the deploy folder and hope you did everything in the right order. That process is prone with error, especially during production bug fixes at 3 in the morning.

In fact, your deployment process may be even more complicated. Let’s assume we have two application servers to deploy to. To make things a little simpler, let’s assume that each server is set up exactly the same (all paths, all jboss and java versions). The only difference between the two machines is the role that each server will perform. The first server is your ‘staging’ environment — this is where your app is in pre-production, and you’re testing functionality right before releasing. Once you’ve done final front-to-back testing, you’re ready to deploy to your ‘production’ environment, but you want to be sure that you are actually releasing the same version that you released to ‘staging’.

JRuby and Rake to the rescue.

Now, I realize that capistrano (http://www.capify.org/index.php/Deploying_a_java_war_to_Tomcat) could be used for this purpose as well, but we developed a working process at our company long before capistrano came to the forefront of the Ruby community, and from a pragmatic standpoint, I find our process much easier to read and comprehend what is going on.

# If you choose to adopt the following rake file for your project, you will gain a simple 2 step process for deploying to your servers:

rake deploy:to_staging ENVIRONMENT=production
rake deploy:to_prod_from_staging ENVIRONMENT=production

As you can see, the deployment is fully automated, and the deployment to production is reconciled against our deployment to staging. All you have to do is replace staging-01 and production-01 with your real server names. The JRuby library includes ‘net/ssh’ and ‘net/scp’ to make our lives very easy:

# Begin lib/tasks/deploy.rake:

gem 'warbler', '=0.9.14'
require 'warbler'
gem 'net-ssh', '=2.0.3'
require 'net/ssh'
require 'net/scp'

namespace :deploy do
  class DeploymentUtils
    def initialize( server )
      @server = server
    end

    def upload( local_file, remote_file )
      puts "Pushing #{ local_file } to #{ @server }:#{ remote_file }"
      Net::SCP.upload!( @server, 'jboss', local_file, remote_file ) end

    def download( remote_file, local_file )
      puts "Pulling #{ local_file } from #{ @server }:#{ remote_file }"
      Net::SCP.download!( @server, 'jboss', remote_file, local_file  )
    end

    def archive_war
      puts "Archiving the war on #{ @server }"
      timestamp = Time.now.strftime( "%m%d%Y-%H%M%S" )
      Net::SSH.start( @server, 'jboss' ) do |ssh|
        ssh.exec!( "cp jboss-deploy/#{WAR_NAME} war_archive/#{WAR_NAME}_#{ timestamp }" )
      end
    end

    def tail_jboss_log
      puts "Tailing jboss logs... please wait"
      sleep 10
      Net::SSH.start( @server, 'jboss' ) do |ssh|
        puts ssh.exec!( "tail -50 /home/jboss/logs/jboss.log" )
      end
    end

    def exec( cmd )
      Net::SSH.start( @server, 'jboss' ) do |ssh|
        puts ssh.exec!( cmd )
      end
    end
  end

  STAGING = 'staging-01'
  PRODUCTION = 'production-01'
  WAR_NAME = "dealanalyzer.war"
  STAGING_WAR = "fromstaging.war"
  UPLOAD_PATH = "/tmp"
  DEPLOY_PATH = "/home/jboss/jboss-deploy"

  desc "Deploy War to development servers"
  task :dev => ["war:clean", :war] do
    if ENV['JBOSS_HOME']
      cmd "cp #{RAILS_ROOT}/#{WAR_NAME} #{ENV['JBOSS_HOME']}/server/default/deploy"
    else
      puts "Please set your JBOSS_HOME. War not copied!"
    end
  end

  desc "Deploy War to staging servers"
  task :staging => ["war:clean", :war] do
    if ENV['ENVIRONMENT'] =~ /production/i
      on( STAGING ).archive_war
      on( STAGING ).upload( WAR_NAME, "#{UPLOAD_PATH}/#{WAR_NAME}" )
      on( STAGING ).exec( "mv #{UPLOAD_PATH}/#{WAR_NAME} #{DEPLOY_PATH}/#{WAR_NAME}")
      on( STAGING ).tail_jboss_log
    else
      puts "You must set the environment to production; rake deploy:staging ENVIRONMENT=production"
    end
  end

  desc "Push war from staging to prod"
  task :to_prod_from_staging do
    puts "pushing this war from #{ STAGING } to #{ PRODUCTION }..."
    on( STAGING ).exec( "ls -l jboss-deploy/#{WAR_NAME}" )
    puts "Are you sure?"

    if STDIN.gets.chomp! =~ /^Y$/i
      on( STAGING ).download( "#{DEPLOY_PATH}/#{WAR_NAME}", STAGING_WAR )
      if File.exists?( "#{ RAILS_ROOT }/#{WAR_NAME}" ) &&
         File.size( "#{ RAILS_ROOT }/#{WAR_NAME}" ) == File.size( "#{ RAILS_ROOT }/#{STAGING_WAR}" )
        on( PRODUCTION ).upload( STAGING_WAR, "#{UPLOAD_PATH}/#{WAR_NAME}" )
        on( PRODUCTION ).exec( "mv #{UPLOAD_PATH}/#{WAR_NAME} #{DEPLOY_PATH}/#{WAR_NAME}")
        on( PRODUCTION ).tail_jboss_log
     else
        puts "Either you didn't deploy from here or the war on staging is different. Cancelling!"
      end
    else
      puts "Cancelled!"
    end
  end

  private

    def cmd( cmd )
      puts "Running : #{ cmd }"
      system( cmd )
    end

    def on( server )
      DeploymentUtils.new( server )
    end
end