practical benefits of Subversion

I recently worked with a legacy version control system and found myself missing Subversion instantly. As a consultant, I often work with clients that are using in-house processes/software to manage project artifacts.  I've grown used to many of SVN's features and just assumed everyone was using something comparable these days. Here are my notes on some of the practical benefits of using Subversion that I think apply to most projects...

core features/benefits

• atomic commits
• one transaction to add/update/delete files
• promotes grouping related changes together in one commit to easily identify and revert
• full revision history retained for files that are renamed, copied, moved or removed
• no file locking (by design)
• file locking is extremely counter-productive (especially for large teams in multiple timezones)
• SVN's automatic merging and conflict resolution tools make locking unnecessary
• working copy of files are always writable and all changes are local until committed
  
• allows you to easily revert unwanted changes at commit time
• fast and flexible update/commits
• all tasks can be done from any level easily (updates, commits, merges, reverts, etc)
• branching and tagging are cheap operations
• simple to create a tag/branch and merge between them
• ease of access to the commit log
• provides high visibility to changes, file history, integrated file comparison
• ease of reverting changes to a file or an entire commit
• easy to setup email triggers (on checkins to keep team informed of changes, etc)
• integrates with everything
• windows explorer integration via Tortoise
• with leading IDEs (Eclipse, IntelliJ)
• Agile/continuous integration tools (Atlassian, Jira, Fisheye, TeamCity, Hudson, etc.)
• various websites to provide project/committer stats (Ohloh, etc)
• ease of setup and administration

benefits in practice (using Windows and Tortoise)

• syncing up my local environment
• open Windows Explorer and navigate to the root directory of my project
• right-click and choose 'TortoiseSVN->Commit'
• sounds counter-intuitive, but this show me a quick view of uncommitted local changes (aka 'mock commit')
• reminds me what I was working on
• lets me right-click on any files and choose 'compare with base' to get an instant diff
• once I'm recalibrated to the project and any in-progress changes, I cancel the commit...
• update my local working code
  
• switch to another branch (if necessary)
  
• if I need to work on another branch than what is currently selected, then I can switch to it easily
• right-click on project and choose 'TortoiseSVN->Switch'
• enter the SVN URL of the project I need to switch to
• click 'OK' to sync up with my local files
• otherwise, right-click and choose 'SVN Update'
  
• this syncs up my local working directory with the latest changes in the repository
• again, I can right-click on any files to compare them my local copy (to see what has changed)
• any conflicts are marked (rare)
• I can choose to either resolve it now or later, choose to use my version, or choose to use the repository version
• working on bug fixes/enhancements
  
• work on a single feature at a time
• make all related changes, write any necessary unit tests and test locally (of course)
• once complete, I commit the change...
• flip back to Explorer (can also be done with Eclipse SVN plugins)
• right-click on the project and choose 'TortoiseSVN->SVN Update' (optional)
• only necessary if you are working with a large development team or its been a while since I last updated from SVN
• this will insure that there weren't any other changes that might affect my commit
• right-click on the base project directory and choose 'TortoiseSVN->SVN Commit'
• dialogue box comes up to review my changes
• right-click on any files in question, revert as necessary
• check/uncheck the files I want to commit
• enter a quick comment referencing the bug/enhancement worked on and a description of the change
• click 'OK' to commit the changes
• working on major refactoring or large feature development
• I don't want this to affect the trunk, so I create a new development branch
• right-click on the project root and choose 'TortoiseSVN->Branch/Tag'
• in the dialogue box
• enter a new directory for your branch (generally /branches/<branch name>)
• enter a comment about the purpose of the branch
• click 'OK'...
• switch your local working directory to use the branch you just created (see instructions above)
• that is it...you now have a development branch (whole process takes about 30 seconds unless you have a large project)

summary
Overall, I think Subversion provides many productivity/quality control benefits and can even prove cost-effective to switch to mid-project.  Given that most open source Java development projects use SVN, I feel pretty confident that it continues to be a solid choice for the most projects.

managing Camel routes with JMX APIs

here is a quick example of how to programmatically access Camel MBeans to monitor and manipulate routes...

first, get a connection to a JMX server (assumes localhost, port 1099, no auth)
note, always cache the connection for subsequent requests (can cause memory utilization issues otherwise)

JMXServiceURL url = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi");
JMXConnector jmxc = JMXConnectorFactory.connect(url);
MBeanServerConnection server = jmxc.getMBeanServerConnection();

use the following to iterate over all routes and retrieve statistics (state, exchanges, etc)...

ObjectName objName = new ObjectName("org.apache.camel:type=routes,*");
List<ObjectName> cacheList = new LinkedList(server.queryNames(objName, null));
for (Iterator<ObjectName> iter = cacheList.iterator(); iter.hasNext();)
{
    objName = iter.next();
    String keyProps = objName.getCanonicalKeyPropertyListString();
    ObjectName objectInfoName = new ObjectName("org.apache.camel:" + keyProps);
    String routeId = (String) server.getAttribute(objectInfoName, "RouteId");
    String description = (String) server.getAttribute(objectInfoName, "Description");
    String state = (String) server.getAttribute(objectInfoName, "State");
    ...
}

use the following to execute operations against a Camel route (stop,start, etc)

ObjectName objName = new ObjectName("org.apache.camel:type=routes,*");
List<ObjectName> cacheList = new LinkedList(server.queryNames(objName, null));
for (Iterator<ObjectName> iter = cacheList.iterator(); iter.hasNext();)
{
    objName = iter.next();
    String keyProps = objName.getCanonicalKeyPropertyListString();
    if(keyProps.contains(routeID))
    {
        ObjectName objectRouteName = new ObjectName("org.apache.camel:" + keyProps);
        Object[] params = {};
        String[] sig = {};
        server.invoke(objectRouteName, operationName, params, sig);
        return;
    }
}

summary

These APIs can easily be used to build a web or command line based tool to support remote Camel management features. All of these features are available via the JMX console and Camel does provide a web console to support some management/monitoring tasks.

See these pages for more information...
http://camel.apache.org/camel-jmx.html
http://camel.apache.org/web-console.html

managing ActiveMQ with JMX APIs

here is a quick example of how to programmatically access ActiveMQ MBeans to monitor and manipulate message queues...

first, get a connection to a JMX server (assumes localhost, port 1099, no auth)
note, always cache the connection for subsequent requests (can cause memory utilization issues otherwise)

JMXServiceURL url = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi");
JMXConnector jmxc = JMXConnectorFactory.connect(url);
MBeanServerConnection conn = jmxc.getMBeanServerConnection();

then, you can execute various operations such as addQueue, removeQueue, etc...

String operationName="addQueue";
String parameter="MyNewQueue";
ObjectName activeMQ = new ObjectName("org.apache.activemq:BrokerName=localhost,Type=Broker");
if(parameter != null) {
    Object[] params = {parameter};
    String[] sig = {"java.lang.String"};
    conn.invoke(activeMQ, operationName, params, sig);
} else {
    conn.invoke(activeMQ, operationName,null,null);
}

also, you can get an ActiveMQ QueueViewMBean instance for a specified queue name...

ObjectName activeMQ = new ObjectName("org.apache.activemq:BrokerName=localhost,Type=Broker");
BrokerViewMBean mbean = (BrokerViewMBean) MBeanServerInvocationHandler.newProxyInstance(conn, activeMQ,BrokerViewMBean.class, true);

for (ObjectName name : mbean.getQueues()) {
    QueueViewMBean queueMbean = (QueueViewMBean)
           MBeanServerInvocationHandler.newProxyInstance(mbsc, name, QueueViewMBean.class, true);

    if (queueMbean.getName().equals(queueName)) {
        queueViewBeanCache.put(cacheKey, queueMbean);
        return queueMbean;
    }
}

then, execute one of several APIs against the QueueViewMBean instance...

queue monitoring - getEnqueueCount(), getDequeueCount(), getConsumerCount(), etc...

queue manipulation - purge(), getMessage(String messageId), removeMessage(String messageId), moveMessageTo(String messageId, String destinationName), copyMessageTo(String messageId, String destinationName), etc...

summary
The APIs can easily be used to build a web or command line based tool to support remote ActiveMQ management features. That being said, all of these features are available via the JMX console itself and ActiveMQ does provide a web console to support some management/monitoring tasks.

See these pages for more information...

http://activemq.apache.org/jmx-support.html
http://activemq.apache.org/web-console.html

Camel exception handling overview

Here are some notes on adding Camel (v2.3) exception handling to a JavaDSL route.  There are various approaches/options available.  These notes cover the important distinctions between approaches...

default handling
The default mode uses the DefaultErrorHandler strategy which simply propagates any exception back to the caller and ends the route immediately.  This is rarely the desired behavior, at the very least, you should define a generic/global exception handler to log the errors and put them on a queue for further analysis (during development, testing, etc).

onException(Exception)
    .to("log:GeneralError?level=ERROR")
    .to("activemq:GeneralErrorQueue");

try-catch-finally
This approach mimics the Java for exception handling and is designed to be very readable and easy to implement.  It inlines the try/catch/finally blocks directly in the route and is useful for route specific error handling.

from("direct:start")
    .doTry()
        .process(new MyProcessor())
    .doCatch(Exception.class)
        .to("mock:error");
    .doFinally()
        .to("mock:end");

onException
This approach defines the exception clause separately from the route.  This makes the route and exception handling code more readable and reusable.  Also, the exception handling will apply to any routes defined in its CamelContext.

from("direct:start")
    .process(new MyProcessor())
    .to("mock:end");

onException(Exception.class)
    .to("mock:error");

handled/continued
These APIs provide valuable control over the flow.   Adding handled(true) tells Camel to not propagate the error back to the caller (should almost always be used).  The continued(true) tells Camel to resume the route where it left off (rarely used, but powerful).  These can both be used to control the flow of the route in interesting ways, for example...

from("direct:start")
    .process(new MyProcessor())
    .to("mock:end");

//send the exception back to the client (rarely used, clients need a meaningful response)
onException(ClientException.class)
    .handled(false)    //default
    .log("error sent back to the client");

//send a readable error message back to the client and handle the error internally
onException(HandledException.class)
    .handled(true)
    .setBody(constant("error"))
    .to("mock:error");

//ignore the exception and continue the route (can be dangerous, use wisely)
onException(ContinuedException.class)
    .continued(true);

using a processor for more control
If you need more control of the handler code, you can use an inline Processor to get a handle to the exception that was thrown and write your own handle code...

onException(Exception.class)
    .process(new Processor() {
         public void process(Exchange exchange) throws Exception {
               Exception exception = (Exception) exchange.getProperty(Exchange.EXCEPTION_CAUGHT);
               //log, email, reroute, etc.
         }
    });

Summary
Overall, the exception handling is very flexible and can meet almost any scenario you can come up with.  For the sake of focusing on the basics, many advanced features haven't been covered here.
  
For more details, see these pages on Camel's site...

http://camel.apache.org/error-handling-in-camel.html
http://camel.apache.org/try-catch-finally.html
http://camel.apache.org/exception-clause.html

Servicemix 4 development environment setup

Here is a quick guide to setting up a development environment for working on Apache Servicemix projects...

Java 1.6

• download the JDK and run installation
• add the /jdk/bin directory to your system path
• create JAVA_HOME environment variable pointing to the root JDK directory

Maven 2.1

• download binary and unzip to a local directory
• edit the /bin/mvn.bat file to increase the Maven heap size (MAVEN_OPTS=-Xmx512m)
• add the /maven/bin directory to your system path

Eclipse 3.4

• download Eclipse and run installation
• open Eclipse
• create a new workspace
• create new project or import any existing projects 
• setup m2eclipse plugin
• go to Help->Software Updates->Available Software tab
• then click 'Add Site' and add 'http://m2eclipse.sonatype.org/update/ and call it 'm2eclipse'
• select the m2clipse components to install, click finish and restart Eclipse
• go to Windows->Preferences->Maven->Installations
• set the user settings to your maven settings.xml file
• verify that this setup the M2_REPO variable to point to your local repo under Java->Build Path->Classpath Variables
• go to Windows->Preferences->Maven->Archetypes
• click 'Add Remote Catalog'
• catalog file = http://repo.fusesource.com/maven2
• name = Fuse Catalog
• click OK

Fuse ESB 4.2 (aka ServiceMix)

• donwload Fuse ESB 4.2 and run installation

basic development process

• create a new project using an archetype (Maven template) that best matches your application needs
• from Eclipse
• File->New->Maven->Maven Project
• check 'use default workspace location' and click next
• choose the 'Fuse Catalog' and enter a desire filter
• choose an archetype
• click next
• enter a group id of 'com.<company>.<project>'
• enter an artifact of '<component>'
• enter a package name of 'com.<company>.<project>.<component>'
• click finish 
• note: I have seen issues with some archetypes in Eclipse, if this happens, use the command line approach instead
• from command line
• mvn archetype:create
  -DarchetypeGroupId=org.apache.servicemix.tooling
  -DarchetypeArtifactId=servicemix-osgi-camel-archetype
  -DarchetypeVersion=2010.01.0-fuse-01-00
  -DremoteRepositories=http://repo.fusesource.com/maven2
  -DgroupId=com.mycompany.project1
  -DartifactId=component1
  -Dversion=0.0.1-SNAPSHOT
•  build the project
• in Eclipse
• right click on the project, go to Run As->Maven install
• from command line
• run 'mvn install'
• project JAR will be created under the /target directory and deployed it to your local Maven repository
• deploying the project to Fuse ESB 4.2
• run /<smx>/bin/servicemix to start the server
• make sure any required features/bundles have been installed
• activemq, camel, etc
• from the karaf console, install your project bundle
• osgi:install -s mvn:com.mycompany.project1/component1
• assuming that worked, you now have a working project
• customize project and repeat build/deploy process

monitoring

• watch the servicemix log file
• tail -f /<smx>/data/log/karaf.log
•  JConsole
• <jdk>/bin/jconsole.exe
• provides remote/local JVM monitoring
• remote connection uri = service:jmx:rmi:///jndi/rmi://:1099/jmxrmi
• default user/password = karaf/karaf
• VisualVM
• <jdk>/bin/jvisualvm.exe
• provides remote/local JVM monitoring (slightly better than JConsole)

other recommended software (for Windows)

• cygwin  - adds basic Linux command support for Windows
• PowerCMD - tabbed command prompts...trial/not free
• kdiff3 - excellent file/directory comparison tool
• SoapUI - for testing HTTP endpoints

If you know of other configurations or tools that are relevant, let me know...

OSGi bundle communication options

There are various approaches to communicating between bundles...each having its own benefits/tradeoffs. Here are my (in progress) notes on evaluating the options with regards to Fuse ESB 4.2...

OSGi Service Registry
This is the most common (and well documented) approach from a pure OSGi perspective. Simple XML can be used to both register APIs as services and access other bundle's services. The service registry then provides an abstraction layer and an event framework to support dynamic binding, reacting to service lifecycle changes, etc. The whiteboard pattern can also help better manage this interaction in more complex/demanding scenarios.

OSGi Import-Package/Export-Package
This approach is more direct and relies on OSGi bundle manifests to tell the classloader which classes to export (make available to other bundles) or import (find classes in other bundles). This has the effect of making any imported classes look like any other class in a bundle.

However, this tends to tightly couple bundles together and can cause various classloader/versioning issues when bundles get out of sync.

JMS
An ActiveMQ broker is provided with Fuse ESB and is easily setup/configured. JMS endpoints can then easily be setup using Camel to wrap access to POJO Java code, etc. This can easily serve as a bridge between bundles or even VMs/machines entirely. This approach povides the standard JMS benefits (guaranteed messaging, loose coupling, high performance, failover/load balancing, etc).

Some initial performance concerns usually arise when considering use of persistent XML messaging. Most performance concerns can be mitigated by varying the JMS QoS or even using serialized object messages instead of XML, etc.

HTTP (SOAP, REST)
Another classic approach is to provide HTTP endpoints to interact with a bundle. This provides a loosely coupled strategy bound to either a SOAP WSDL or REST request/response format. Camel components can be used to setup these endpoints and wrap requests to POJO Java code, etc.

Unfortunately, these endpoints aren't the easiest to configure/test and often impose a significant overhead in terms of marshaling request/response data around.

NMR
Though its roots were in JBI/XML based messaging, it has evolved to provide a more generic/configurable approach to bundle communication. It can be configured to be to be synchronous or asynchronous.  Also, the camel-nmr component makes it easy to define NMR endpoints to route data between bundles without the need to use JMS brokers or more complex HTTP definitions.

See this post from Adrian Trenaman for more details on the new NMR...

VM
VM uses asynchronous SEDA to send messages between entities in the same virtual machine.  This has the benefit of using a seperate thread for a producer and consumer.  
More to come on this...I'm still working out the differences between VM and NMR in OSGi.

Distributed OSGi (DOSGi)
This implements the remove services functionality using SOAP web services.  Its very new and I'm still coming up to speed on it...(thanks Ashwin for pointing this out)

More to come on this...see these page for now.

http://cxf.apache.org/distributed-osgi.html
http://osgithoughts.blogspot.com/2010/07/cxf-distributed-osgi-12-is-out.html

summary
This is clearly still a work in progress.  There are a lot of options and I'm still trying to get my head around the trade-offs of each approach (hence this page).  As time permits, I'll try to expand on these more...

If anyone has had good/bad experiences with these techniques or others, let me know...

Fuse ESB 4.2 quick start guide

Here are some notes of getting started using Fuse ESB 4.2 (aka ServiceMix 4).

For full documentation, click here

install/config

• prerequisites
• required software - JDK 1.5+
• recommended software - Maven 2.1+, Eclipse 3.4+
• see this guide for more details
• download Fuse ESB 4.2 here and follow the installation instructions

basic structure

• /bin - executable files
• /etc - configuration files
• /deploy - used to deploy bundles
• /data - runtime files

basic configuration

• edit the "org.apache.felix.karaf.features.cfg" file to set necessary startup features (camel, activemq, etc)

start server

• cd /[fuse]/bin
• run 'servicemix' (starts server and karaf shell)

start shell against a running server

• ssh -p 8101 karaf@localhost (default password:karaf)

frequently used commands (for a full list click here)

• managing features
• features:list
• features:listUrl
• features:uninstall [name]
• features:install [name]
• managing bundles
• osgi:uninstall [id]
• osgi:install -s file:/bundles/example-1.0.jar
• osgi:install -s mvn:com.company/example/1.0
• osgi:update [id] (refresh from install location)
• osgi:refresh [id] (reinitialize the bundle)
• monitoring
• osgi:list (list all bundles)
• osgi:list | grep test
• osgi:headers (view bundle header info)
• osgi:headers | grep test
• log:display
• log:set [level] [log package] (log:set DEBUG com.foo.bar)
• shutdown (stop servicemix)

other tips

• monitor log file
• tail -f /[fuse]/data/log/karaf.log
• deploy bundles
• osgi:install or move bundle to the [fuse]/deploy directory

Spring property-placeholder with OSGi

Here are some notes on adding support for properties files to a project. I have it deployed as an OSGI bundle on Fuse ESB 4.2 (aka Apache ServiceMix), but it can be used in various containers/configurations. Anyways, here are the steps...

first, add these required Maven dependencies to your project...
<dependency>
  <groupid>org.springframework.osgi</groupid>
  <artifactid>spring-osgi-core</artifactid>
  <version>1.2.0</version>
</dependency>
<dependency>
  <groupid>org.springframework</groupid>
  <artifactid>spring-context</artifactid>
  <version>2.5.6.SEC01</version>
</dependency>

then, add the following spring XML..
<beans xmlns:context="http://www.springframework.org/schema/context"
            xmlns:osgi="http://www.springframework.org/schema/osgi"
            xmlns:osgix="http://www.springframework.org/schema/osgi-compendium"
            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xmlns="http://www.springframework.org/schema/beans"
            xsi:schemalocation="
                http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
                http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
                http://www.springframework.org/schema/osgi http://www.springframework.org/schema/osgi/spring-osgi.xsd
                http://www.springframework.org/schema/osgi-compendium http://www.springframework.org/schema/osgi-compendium/spring-osgi-compendium.xsd">
    <osgix:cm-properties id="props" persistent-id="my.props">
        <prop key="property.name">default value</prop>
    </osgix:cm-properties>
    <context:property-placeholder properties-ref="props"></context:property-placeholder>
    <bean id="myBean" lass="com.company.MyClass">
        <property name="property" value="${property.name}"></property>
    </bean>
</beans>  

finally, create a properties file...
add a file called "my.props.cfg" to your container's classpath (/fuse/etc directory, etc) that contains key=value line separated properties...

tips on using this in practice...

• the filename must match the <osgix:cm-properties> persistent-id attribute and end with ".cfg". also, avoid using dashes in the filename...
• this will NOT work in unit tests, you'll need to override your spring XML file under /src/test/resources, remove references to osgi-compendium and hardcode the properties

another option altogether...use the SystemClassLoader
put your properties files in the /servicemix/conf/ directory (doesn't exist by default).  This is in a bundle's classpath and can be loaded as follows...
       
        try {
            Properties prop = new Properties();
            ClassLoader sysClassLoader = ClassLoader.getSystemClassLoader();
            InputStream is = sysClassLoader.getResourceAsStream("my.properties");
            prop.load(is);
        } catch (Exception e) {
            logger.error("error getting props->" + e);
        }

This won't leverage the osgi benefits, but is simple and will work for most cases...