Creating OSGi bundles from any JAR

One of the major pains in OSGi is the creation of bundles. Peter Kriens' BND tool does a great job - it's powerful and can be used for creating the most complex OSGi bundles.

But haven't you ever wished for a simple tool at which you just throw an existing JAR file and it produces the corresponding OSGi bundle jar? Well, I have - and I have written the class below which does exactly that. And, yes, I have been found guilty of reinventing the wheel (see comment below) - a simple wrapper to BND would have achieved the same purpose except I learnt a few things along the way doing it myself :-)

This is a convenient way to quickly produce an OSGi bundle to be deployed in your OSGi runtime.

Clearly, this is not a replacement for BND or similar tools. It is a good 80% solution. Hopefully, we will soon see reasonably searchable OSGi bundle repositories come up (and I do not mean what is currently offered on the OSGi alliance site - you can't even search for bundles offering a certain class or package, to name just one use case). Hopefully, we will also see bundles offered for all of the major open source packages, not just a chosen few used in Eclipse. Ok, rant finished.

Here comes the code. [Deleted my comment about reinventing the weel from the original post - admittedly I have been found guilty].

And before I forget, this code depends on the following JARs:

• Dependency: I had to adapt it to the current version of ASM (that was simple, the source code is available). If you send me an email, I can send you my version of the code.
  
• ASM
  
• Apache Commons IO
  

import java.io.BufferedReader;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.Enumeration;
import java.util.HashMap;
import java.util.Set;
import java.util.StringTokenizer;
import java.util.jar.JarEntry;
import java.util.jar.JarFile;
import org.vafer.dependency.utils.DependencyUtils;
import org.apache.commons.io.FileUtils;

/**
* This class implements a simplistic approach to creating
* bundles from any JAR file.
*
* @author Mathias Richter
*/
public class Osgify
{

  /**
   * Return all packages imported by classes in the specified jar file.
   *
   * @param jarFileName The name of the JAR file to analyse.
   * @return A set of package names.
   *
   * @throws java.io.FileNotFoundException
   * @throws java.io.IOException
   */
  public static Set<String> getImportPackages( String jarFileName ) throws FileNotFoundException, IOException
  {
      HashMap<String, String> packages = new HashMap<String, String>();
      Set<String> dependencies = DependencyUtils.getDependenciesOfJar( new FileInputStream( jarFileName ) );
      for ( String name : dependencies )
      {
          if ( !name.startsWith( "[" ) )
          {
              int index = name.lastIndexOf( "." );
              if ( index > 0 )
                  name = name.substring( 0, index );
          } else
          {
              if ( name.length() > 2 )
              {
                  int index = name.lastIndexOf( "." );
                  if ( index > 2 )
                      name = name.substring( 2, index );
              } else
                  continue;
          }
          if ( !name.startsWith( "java." ) )
              packages.put( name, name );
      }
      return packages.keySet();
  }

  /**
   * Return a set of names of the packages declared in the specified JAR file.
   *
   * @param jarFileName The name of the JAR file to analyse.
   * @return The names of the packages declared by the classes in the JAR file.
   *
   * @throws java.io.IOException
   */
  public static Set<String> getExportPackages( String jarFileName ) throws IOException
  {
      HashMap<String, String> packages = new HashMap<String, String>();
      JarFile file = new JarFile( jarFileName );
      for ( Enumeration<JarEntry> e = file.entries(); e.hasMoreElements();)
      {
          String name = e.nextElement().getName();
          if ( name.endsWith( ".class" ) )
          {
              name = name.replaceAll( "/", "." );
              int index = name.lastIndexOf( "." );
              if ( index > 0 )
                  name = name.substring( 0, name.lastIndexOf( "." ) );
              index = name.lastIndexOf( "." );
              if ( index > 0 )
                  name = name.substring( 0, name.lastIndexOf( "." ) );
              packages.put( name, name );
          }
      }
      return packages.keySet();
  }

  /**
   * Analyse the specified JAR for its package imports and package declarations and
   * create a manifest file describing these in OSGi fashion.
   *
   * @param jarFileName The name of the JAR file for which to create an OSGi bundle manifest. May not be null.
   * @param bundleName The name of the bundle to delcrae in the manifest. May not be null.
   * @param bundleSymbolicName The symbolic name to declare in the manifest. If null, uses the bundle name.
   * @param bundleVersion The version to declare in the manifest. If null, uses "1.0.0".
   * @param bundleActivator The activator class to declare in the manifest. If null, does not specify an activator.
   * @param resolveOptional If true, all package imports will be marked with "resolution:=optional".
   *
   * @return The manifest contents as a string.
   *
   * @throws java.io.FileNotFoundException
   * @throws java.io.IOException
   */
  public static String getManifest( String jarFileName, String bundleName, String bundleSymbolicName, String bundleVersion, String bundleActivator, boolean resolveOptional ) throws FileNotFoundException, IOException
  {
      StringBuffer manifest = new StringBuffer();
      Set<String> im = getImportPackages( jarFileName );
      Set<String> ex = getExportPackages( jarFileName );
      im.removeAll( ex );
      manifest.append( "Bundle-Name: " );
      manifest.append( bundleName );
      manifest.append( "\n" );
      if ( bundleSymbolicName == null )
          bundleSymbolicName = bundleName;
      manifest.append( "Bundle-SymbolicName: " );
      manifest.append( bundleSymbolicName );
      manifest.append( "\n" );
      if ( bundleVersion == null )
          bundleVersion = "1.0.0";
      manifest.append( "Bundle-Version: " );
      manifest.append( bundleVersion );
      manifest.append( "\n" );
      if ( bundleActivator != null )
      {
          manifest.append( "Bundle-Activator: " );
          manifest.append( bundleActivator );
          manifest.append( "\n" );
      }
      manifest.append( "Import-Package: " );
      boolean previous = false;
      for ( String p : im )
      {
          if ( previous )
              manifest.append( "," );
          manifest.append( p );
          if ( resolveOptional )
              manifest.append( ";resolution:=optional" );
          previous = true;
      }
      manifest.append( "\n" );
      manifest.append( "Export-Package: " );
      previous = false;
      for ( String p : ex )
      {
          if ( previous )
              manifest.append( "," );
          manifest.append( p );
          previous = true;
      }
      manifest.append( "\n" );
      manifest.append( "\n" );
      return split( manifest.toString(), 70 );
  }

  /**
   * Format the specified source string to use a max line length as specified.
   * Lines will be broken in "manifest" fashion, ie. continuation lines start with
   * a single space.
   *
   * @param source the source String to format
   * @param maxLineLength The maximum line length in the result.
   * @return The formatted string.
   */
  protected static String split( String source, int maxLineLength )
  {
      StringBuffer result = new StringBuffer();
      StringTokenizer tokens = new StringTokenizer( source, "\n" );
      while ( tokens.hasMoreTokens() )
      {
          String line = tokens.nextToken();
          int length = line.length();
          for ( int index = 0; index < length;)
          {
              if ( index > 0 )
                  result.append( " " );
              int remainder = length - index;
              if ( remainder <= maxLineLength )
              {
                  result.append( line.substring( index, length ) );
                  index = length;
              } else
              {
                  result.append( line.substring( index, index + maxLineLength ) );
                  index += maxLineLength;
              }
              result.append( "\n" );
          }
      }
      return result.toString();
  }

  /**
   * If the process has output, return it as a String.
   *
   * @param process The process.
   * @return The process output as String.
   * @throws java.io.IOException
   */
  protected static String getOutput( Process process ) throws IOException
  {
      StringBuffer result = new StringBuffer();
      InputStream is = process.getInputStream();
      InputStreamReader isr = new InputStreamReader( is );
      BufferedReader br = new BufferedReader( isr );
      String line = null;
      while ( ( line = br.readLine() ) != null )
          result.append( line );
      is.close();
      isr.close();
      br.close();
      return result.toString();
  }

  /**
   * Create an OSGi jar from the specified JAR file using the specified bundle name.
   * The OSGi bundle jar will be created in the same directory as the original JAR.
   *
   * @param jarFileName The JAR file to analyse and produce an OSGi bundle from.
   * @param bundleName the name of the bundle.
   *
   * @throws java.io.FileNotFoundException
   * @throws java.io.IOException
   * @throws java.lang.InterruptedException
   */
  public static void osgify( String jarFileName, String bundleName ) throws FileNotFoundException, IOException, InterruptedException
  {
      osgify( jarFileName, bundleName, null, null, null, true );
  }

  /**
   * Create an OSGi jar from the specified JAR file using the specified bundle name and bundle symbolic name.
   * The OSGi bundle jar will be created in the same directory as the original JAR.
   *
   * @param jarFileName The JAR file to analyse and produce an OSGi bundle from.
   * @param bundleName the name of the bundle.
   * @param bundleSymbolicName The symbolic bundle name to use.
   *
   * @throws java.io.FileNotFoundException
   * @throws java.io.IOException
   * @throws java.lang.InterruptedException
   */
  public static void osgify( String jarFileName, String bundleName, String bundleSymbolicName ) throws FileNotFoundException, IOException, InterruptedException
  {
      osgify( jarFileName, bundleName, bundleSymbolicName, null, null, true );
  }

  /**
   * Create an OSGi jar from the specified JAR file using the specified bundle name and bundle symbolic name.
   * The OSGi bundle jar will be created in the same directory as the original JAR.     *
   *
   * @param jarFileName The name of the JAR file for which to create an OSGi bundle manifest. May not be null.
   * @param bundleName The name of the bundle to declare in the manifest. May not be null.
   * @param bundleSymbolicName The symbolic name to declare in the manifest. If null, uses the bundle name.
   * @param bundleVersion The version to declare in the manifest. If null, uses "1.0.0".
   * @param bundleActivator The activator class to declare in the manifest. If null, does not specify an activator.
   * @param resolveOptional If true, all package imports will be marked with "resolution:=optional".
   *
   * @throws java.io.FileNotFoundException
   * @throws java.io.IOException
   * @throws java.lang.InterruptedException
   */
  public static void osgify( String jarFileName, String bundleName, String bundleSymbolicName, String bundleVersion, String bundleActivator, boolean resolveOptional ) throws FileNotFoundException, IOException, InterruptedException
  {
      int index = jarFileName.lastIndexOf( File.separator );
      String targetDir = "./";
      if ( index > 0 )
          targetDir = jarFileName.substring( 0, index + 1 );
      File tempDir = new File( targetDir + ".osgify" + System.currentTimeMillis() );
      FileUtils.forceMkdir( tempDir );
      try
      {
          Process p = Runtime.getRuntime().exec( "jar xf " + jarFileName, null, tempDir );
          if ( p.waitFor() != 0 )
              throw new Error( "Could not expand '" + jarFileName + "' in temporary directory: "  + getOutput( p ) );
          File manifest = new File( tempDir.getAbsolutePath() + "/META-INF/MANIFEST.MF" );
          if ( manifest.exists() )
              FileUtils.forceDelete( manifest );
          manifest = new File( tempDir.getAbsolutePath() + "/META-INF/MANIFEST.MF" );
          String manifestContent = getManifest( jarFileName, bundleName, bundleSymbolicName, bundleVersion, bundleActivator, resolveOptional );
          FileUtils.writeStringToFile( manifest, manifestContent );
          String netJarName = jarFileName.substring( jarFileName.lastIndexOf( File.separator ) + 1, jarFileName.lastIndexOf( "." ) );
          String targetJarName = targetDir + netJarName + "_" + bundleVersion + ".jar";
          String cmd = "jar cfm " + targetJarName + " META-INF/MANIFEST.MF ./*";
          p = Runtime.getRuntime().exec( cmd, null, tempDir );
          if ( p.waitFor() != 0 )
              throw new Error( "Could not create OSGi bundle jar '" + targetJarName + "' in temporary directory: " + getOutput( p )  );
      } finally
      {
          FileUtils.forceDelete( tempDir );
      }
  }

}

Standalone Java CAS Client

There's a variety of clients for CAS. The Java-based clients (JA-SIG, Yale, see JA-SIG website) typically handle the browser-based client interaction with CAS very well through ServletFilter implementations.

Now what about programmatic authentication, i.e. achieving authentication through non-browser based applications? There exists a CAS .NET client but I did not manage to find the appropriate Java implementation. So here goes - it is based on the Apache HttpClient.

In case I missed any existing implementation achieving the same purpose, let's look at the bright side: at least now I understand the CAS protocol :-)

My CAS client works within any application. It uses the HttpClient and behaves like a browser client as CAS requires cookie support.

Here's the code:

import org.apache.commons.httpclient.Header;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpMethod;
import org.apache.commons.httpclient.HttpStatus;
import org.apache.commons.httpclient.methods.GetMethod;
import org.apache.commons.httpclient.methods.PostMethod;
import org.apache.log4j.Logger;

/**
* The CasClient allows users to programmatically login
* to CAS protected services based on the CAS 2 protocol.
* This client behaves like a browser-client in terms of
* cookie handling.<br>
*
* @author Mathias Richter
*/
public class CasClient
{
  
   public static Logger LOG = Logger.getLogger( CasClient.class  );

   public static final String LOGIN_URL_PART = "login";
   public static final String SERVICE_VALIDATE_URL_PART = "serviceValidate";
   public static final String TICKET_BEGIN = "ticket=";
   private static final String LT_BEGIN = "name=\"lt\" value=\"";
   public static final String CAS_USER_BEGIN = "<cas:user>";
   public static final String CAS_USER_END = "</cas:user>";
  
   private HttpClient fClient;
   private String fCasUrl;
  
   /**
    * Construct a new CasClient.
    *
    * @param casUrl The base URL of the CAS service to be used.
    */
   public CasClient( String casBaseUrl )
   {
       this( new HttpClient(), casBaseUrl );
   }
  
   /**
    * Construct a new CasClient which uses the specified HttpClient
    * for its HTTP calls.
    *
    * @param client
    * @param casBaseUrl
    */
   public CasClient( HttpClient client, String casBaseUrl )
   {
       fClient = client;
       fCasUrl = casBaseUrl;
   }
  
   /**
    * Authenticate the specified username with the specified password.
    * This will not yield any ticket, as no service is authenticated
    * against. This wil just set the CAS cookie in this client upon
    * successful authentication.
    *
    * @param username
    * @param password
    */
   public void authenticate( String username, String password )
   {
       authenticate( null, username, password );
   }
  
   /**
    * Validate the specified service ticket against the specified service.
    * If the ticket is valid, this will yield the clear text user name
    * of the autenticated user.<br>
    * Note that each service ticket issued by CAS can be used exactly once
    * to validate.
    *
    * @param serviceUrl
    * @param serviceTicket
    *
    * @return Clear text username of the authenticated user.
    */
   public String validate( String serviceUrl, String serviceTicket )
   {
       String result = null;
       PostMethod method = new PostMethod( fCasUrl + SERVICE_VALIDATE_URL_PART );
       method.setParameter( "service", serviceUrl );
       method.setParameter( "ticket", serviceTicket );
       try
       {
           int statusCode = fClient.executeMethod(method);
           if (statusCode != HttpStatus.SC_OK)
           {
               LOG.error( "Could not validate: " + method.getStatusLine() );
               method.releaseConnection();
           } else
           {   
               result = extractUser( new String( method.getResponseBody() ) );
           }
       } catch ( Exception x )
       {
           LOG.error( "Could not validate: " + x.toString () );
           x.printStackTrace();
       }
       method.releaseConnection();
       return result;
   }
  
   /**
    * Authenticate the specified user with the specified password against the
    * specified service.
    *
    * @param serviceUrl May be null. If a url is specified, the authentication will happen against this service, yielding a service ticket which can be validated.
    * @param username
    * @param password
    * @return A valid service ticket, if and only if the specified service URL is not null.
    */
   public String authenticate( String serviceUrl, String username, String password )
   {
       String lt = getLt( serviceUrl );
       if ( lt == null )
       {
           LOG.error( "Cannot retrieve LT from CAS. Aborting authentication for '" + username + "'" );
           return null;
       }
       String result = null;
       PostMethod method = new PostMethod( fCasUrl + LOGIN_URL_PART );
       if ( serviceUrl != null ) // optional
           method.setParameter( "service", serviceUrl );
       method.setParameter( "_eventId", "submit" );
       method.setParameter("username", username );
       method.setParameter("password", password );
       method.setParameter("lt", lt );
       method.setParameter( "gateway", "true" );
       try
       {
           fClient.executeMethod(method);
           if ( serviceUrl == null )
           {
               if ( extractLt( new String( method.getResponseBody() ) ) != null ) // if CAS does not return a login page with an LT authentication was successful
               {
                   LOG.error( "Authentication for '" +  username + "' unsuccessful" );
                   if ( LOG.isDebugEnabled() )
                       LOG.debug( "Authentication for '" + username + "' unsuccessful." );
               } else
               {
                   if ( LOG.isDebugEnabled() )
                       LOG.debug( "Authentication for '" + username + "' unsuccessful." );
               }
           } else
           {
               Header h = method.getResponseHeader( "Location" );
               if ( h != null )
                   result = extractServiceTicket( h.getValue() );
               if ( result == null )
                   LOG.error( "Authentication for '" + username + "' unsuccessful." );
           }
       } catch ( Exception x )
       {
           LOG.error( "Could not authenticate'" + username + "':" + x.toString () );
       }
       method.releaseConnection();
       return result;
   }
  
   /**
    * Helper method to extract the user name from a "service validate" call to CAS.
    *
    * @param data Response data.
    * @return The clear text username, if it could be extracted, null otherwise.
    */
   protected String extractUser( String data )
   {
       String user = null;
       int start = data.indexOf( CAS_USER_BEGIN  );
       if ( start >= 0 )
       {
           start += CAS_USER_BEGIN.length();
           int end = data.indexOf( CAS_USER_END );
           if ( end > start )
               user = data.substring( start, end );
           else
               LOG.warn( "Could not extract username from CAS validation response. Raw data is: '" + data + "'" );
       } else
       {
           LOG.warn( "Could not extract username from CAS validation response. Raw data is: '" + data + "'" );
       }
       return user;
   }
  
   /**
    * Helper method to extract the service ticket from a login call to CAS.
    *
    * @param data Response data.
    * @return The service ticket, if it could be extracted, null otherwise.
    */
   protected String extractServiceTicket( String data )
   {
       String serviceTicket = null;
       int start = data.indexOf( TICKET_BEGIN  );
       if ( start > 0 )
       {
           start += TICKET_BEGIN.length();
           serviceTicket = data.substring( start );
       }
       return serviceTicket;
   }
  
   /**
    * Helper method to extract the LT from a login form from CAS.
    *
    * @param data Response data.
    * @return The LT, if it could be extracted, null otherwise.
    */
   protected String extractLt( String data )
   {
       String token = null;
       int start = data.indexOf( LT_BEGIN  );
       if ( start < 0 )
       {
           LOG.error( "Could not obtain LT token from CAS: LT Token not found in response." );
       } else
       {
           start += LT_BEGIN.length();
           int end = data.indexOf( "\"", start );
           token = data.substring( start, end );
       }       
       return token;
   }
  
   /**
    * This method requests the original login form from CAS.
    * This form contains an LT, an initial token that must be
    * presented to CAS upon sending it an authentication request
    * with credentials.<br>
    * If a service URL is provided (which is optional), this method
    * will post the URL such that CAS authenticates against the
    * specified service when a subsequent authentication request is
    * sent.
    *
    * @param serviceUrl
    * @return The LT token if it could be extracted from the CAS response.
    */
   protected String getLt( String serviceUrl )
   {
       String lt = null;
       HttpMethod method = null;
       if ( serviceUrl == null )
           method = new GetMethod( fCasUrl + LOGIN_URL_PART );
       else
       {
           method = new PostMethod( fCasUrl + LOGIN_URL_PART );
           ( ( PostMethod ) method ).setParameter( "service", serviceUrl );
       }
       try
       {
           int statusCode = fClient.executeMethod(method);
           if (statusCode != HttpStatus.SC_OK)
           {
               LOG.error( "Could not obtain LT token from CAS: " + method.getStatusLine() );
               method.releaseConnection();
           } else
           {
               Object o = method.getResponseHeaders() ;
               return extractLt( new String( method.getResponseBody() ) );
           }
       } catch ( Exception x )
       {
           LOG.error( "Could not obtain LT token from CAS: " + x.toString () );
       }
       method.releaseConnection();
       return lt;
   }
  
}

Making Spring Dynamic Modules work

Spring Dynamic Modules (Spring support for OSGi) is a great thing. Just in case I am not the only one who struggled to get the required bundles straight, here goes. These are the bundles that I had to import to make Spring DM work on the Equinox OSGi implementation:

aopalliance.osgi-1.0-SNAPSHOT.jar
backport-util-concurrent.osgi-3.1-SNAPSHOT.jar
jcl104-over-slf4j-1.4.3.jar
log4j.osgi-1.2.15-SNAPSHOT.jar
slf4j-api-1.4.3.jar
slf4j-log4j12-1.4.3.jar
spring-aop-2.5.1.jar
spring-beans-2.5.1.jar
spring-context-2.5.1.jar
spring-core-2.5.1.jar
spring-osgi-core-1.0.jar
spring-osgi-extender-1.0.jar
spring-osgi-io-1.0.jar

Install these and then make sure that you install a bundle which contains the LOG4J configuration of your choice. This is described here, so I won't go over that again.

BTW, the version numbers on the above bundles might change, if you use later versions of Spring DM and their dependencies.

How to launch an OSGi runtime in your application with Equinox

Here's a useful code snippet for launching an OSGi runtime from within your application. This is useful where you want to use OSGi in your own application as a plug-in mechanism.

I use Equinox, the Eclipse OSGi implementation.

import org.eclipse.osgi.baseadaptor.BaseAdaptor;
import org.eclipse.osgi.framework.adaptor.FrameworkAdaptor;
import org.eclipse.osgi.framework.internal.core.OSGi;

...

/**
* Launch the OSGi framework.
*
* @param installArea Directory path to where the OSGi bundles remain.
* @param configArea Direcotory path to where OSGi should store its configuration                data.
*/
public void launchOsgi( String installArea, String configArea )
{
    System.setProperty( "osgi.install.area", installArea );
    System.setProperty( "osgi.configuration.area", configArea );
    FrameworkAdaptor adaptor = new BaseAdaptor( null );
    OSGi osgiInstance = new OSGi( adaptor );
    osgiInstance.launch();
}

...

The installation area points to a location in the file system where you want to make your OSGi runtime believe it is "installed", i.e. where it would look for bundles if you used the install file:mybundle_1.0.0.jar command.

The config area points to the location where you want your OSGi runtime to store configuration information about bundles you install while it is running. As you probably know, OSGi stores information about every bundle you install such that upon the next restart, you will not have to install all bundles from scratch again.

Instead of setting system properties, which is admittedly a bit cumbersome, you could pass parameters to the BaseAdaptor constructor, but I havent figured out the parameter names yet :-)

From the OSGi instance you have created you can use osgiInstance.getBundleContext() to install and manage bundles.

That's basically all you need to build an OSGi-based plugin mechanism into your application.

Well, almost. Of course you will need to write some code to detect new bundles in the install area or other locations (at start time of your application or dynamically at runtime of your application). You could use functionality like Peter Kriens' FileInstall bundle, for example. Equinox has something similar but I have not looked at it yet :-)

I might cover this topic in one of my future posts.