Smooks v1.3… Available for download

We were slow about doing it, but eventually… Smooks v1.3 (final) has hit the shelves!!

Download and try it out.

Thanks to everyone that contributed to this release.  As well as 65+ bug fixes, we’ve added a number of new features and improvements:

  1. Greatly improved XPath support in selectors. This allows you to write more complex selectors with XPath syntax. See here. See JIRA.
  2. Support for Fixed Length Field (FLF) messages. See JIRA.
  3. Java Binding Improvements:
    • Ability to bind raw data values into the bean context. See JIRA.
    • Support for factory methods for bean creation. See JIRA.
    • Make bind data available to Expression Base Bindings. See JIRA.
  4. String manipulation functions for tweaking CSV and FLF fields before forwarding to Smooks components. See JIRA.
  5. Support premature Filter termination. See JIRA.
  6. Simple fragment splitter. See JIRA.
  7. CSV Header Validation. See JIRA.

See here for a full list of Fixes and Features

Apache Camel + Smooks

I did a little playing with Camel over the weekend… had a go at integrating Smooks.

I think those using Camel may find it interesting as another option.  The integration was simple enough and can be seen in this svn workspace (including tests etc).  With better knowledge of Camel, I’m sure there’s a better way of integrating it, but at least it’s a start.

At the moment, you can map String or Java results into the out-message of the Camel Exchange, so you can do templating or java binding (not exposing validation results yet) of XML and non-XML data.  So you can do lots of the normal Smooks type processing.

The aspect I think most Camel users might find interesting is the potential wrt splitting and routing of huge messages (XML and non-XML).  Basically… being able to split and route huge messages as you stream them i.e. not the two step method that seems to be there at the moment – 1) split out all messages 2) route all messages.

Lets look at a simple example, where we have an XML message stream containing customer account activity logs (or… could be CSV formatted).  We want to route each log event to JMS queues based on nationality.

The RouteBuilder config (can also configure the SmooksProcessor via Java ala DSL) …

public class MyRouteBuilder extends RouteBuilder {

    public void configure() throws Exception {

        from("file:src/data/logs?noop=true").
            process(new SmooksProcessor("/configs/log-split.xml", this));

        from("jms:queue:ireland").process(new LogProcessor("ie"));
        from("jms:queue:greatbritain").process(new LogProcessor("gb"));
    }
}

And then the Smooks “log-split.xml” config…

<smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"
          xmlns:jb="http://www.milyn.org/xsd/smooks/javabean-1.3.xsd"
          xmlns:camel="http://www.milyn.org/xsd/smooks/camel-1.4.xsd">

    <!-- Create a new ActivityLogEvent for each log entry... -->
    <jb:bean beanId="logEvent" class="com.acme.activity.ActivityLogEvent"
                                  createOnElement="activity/event">
        <jb:value property="transactionId" data="event/@txId" />
        <jb:value property="customerId" data="event/customer/@customerId" />
        <jb:value property="nationality" data="event/customer/countryCode" />
        <!-- etc... -->
    </jb:bean>

    <!-- Route GB based customer activity... -->
    <camel:router beanId="logEvent" toEndpoint="jms:queue:greatbritain"
                                 routeOnElement="activity/event">
        <condition>logEvent.nationality == 'GB'</condition>
    </camel:router>

    <!-- Route Irish based customer activity... -->
    <camel:router beanId="logEvent" toEndpoint="jms:queue:ireland"
                                 routeOnElement="activity/event">
        <condition>logEvent.nationality == 'IE'</condition>
    </camel:router>

</smooks-resource-list>

The <jb:bean> config creates an ActivityLogEvent bean instance for every activity log. The following <camel:router> configs conditionally route the ActivityLogEvent bean instances to the specified JMS endpoints based on the country-code on the event log.  Of course, using Smooks Templating, you could apply a template to the ActivityLogEvent to produce some other data format and route that instead (a simple template generated String bean bound into the Smooks BeanContext).

The size of the event log is irrelevant here – could be GBs.  Only one ActivityLogEvent instance is in memory at any given time (a new instance created for each event).  The routing is performed as the event log is being streamed i.e. not batched up for routing in a later step.  Assumption here is that there’s a way of telling the Camel File component to forward the file as a stream/reader.

The current implementation is available in SVN here.  Since I’m not overly familiar with Camel, I’m sure there are things we could do better/different.  Please let us know!!!

At JBoss, we’ve been making some good progress with the JBoss Tools Smooks Editor.  The following are two rough Screencasts illustrating two use cases that extend from each other.

EDI to Java

In this Screencast, we bind EDI Purchase Order data (but can also be XML, JSON etc) into an “Order” object model (Java).  This Screencast was created a few weeks ago, using the v3.1.0.CR1 release of JBoss Tools.

To enlarge, press screen icon (bottom right)

EDI to XML (via XSD)

In this Screencast, we extend the example shown in the above Screencast, applying a template to the “Order” object model to produce an XML Purchase Order.  The XML Purchase Order is defined in an XML Schema (XSD), but you’ll see how the editor makes it very easy to work with the XSD.  This Screencast was created recently, using one of the Nightly Builds of JBoss Tools.

To enlarge, press screen icon (bottom right)

Status

We are currently working towards the v3.1.0.CR2 release of JBoss Tools, which is due to be released in Feb (not sure of exact date).  There are still a number of issues with the editor.  We are working on these and will be resolving many of them for the 3.1.0.CR2 release.

We have purposely restricted the 3.1.0 version of the editor:

  1. Supported Use Cases:
    • Java Mapping/Binding:  Drag & Drop mapping/binding of XML, JSON, CSV and EDI data to Java.
    • CSV and XML Templating:  Drag & Drop templating of Java Mappings (see #1 above) to produce CSV or XML (XML via XSD or XML Sample).
  2. Supported Integration:
    • JBoss Tools ESB Editor: Will have basic open and edit integration with the ESB SmooksAction.
  3. Supported Versions:
    • Smooks v1.2 Configurations:  The Editor will only support Smooks v1.2 configuration namespaces.  Therefore, if you have configurations using config namespaces from earlier (or newer v1.3) versions of Smooks, the editor will not open the configurations.

So, the 3.1.0 version of the editor will not support:

  1. Direct Source to Result drag & drop templating e.g. XML to XML, EDI to XML, XML to CSV etc.  We hope to support this in a future release (next one hopefully), but for now (in the v3.1.0 release), you need to define a Java Object model to use as the canonical form of the data being mapped i.e. v3.1.0 templating works as Source -> Java -> Result.
  2. A number of features supported by the Smooks Runtime, such as Persistence, Validation, Splitting & Routing etc.  We will be adding support for these in future releases.
  3. Smooks v1.1 (and before) configuration namespaces that have newer versions in Smooks v1.2.  For example, Smooks v1.2 added a v1.2 javabean config namespace.  It is this version of the javabean config namespace that is supported by the editor.  The v1.1 javabean config namespace is not supported.
  4. Smooks v1.3 (yet to be released) configuration namespaces.

Try it Out…

Please take the editor for a spin and let us know what you think.  It can be downloaded (standalone, or as part of JBoss Tools) from the JBoss Tools Downloads PagePlease log any issues you find with the editor (against the “smooks” component), if not already logged.

Smooks v1.2.4 Released

We’ve just made a Smooks v1.2.4 patch release (Download).

This contains:

  1. Some performance enhancements (reader pooling),
  2. Some bug fixes and,
  3. One small new feature to support Basic Fragment Splitting and Routing.  Right… we normally only do fixes in patch releases, but I wanted to get this into the JBoss SOA Platform before it released, so we made an exception for this one.

I would like to introduce the new factory feature in the Javabean cartridge of the up coming Smooks 1.3 release. This new features makes it possible to use a static factory method or a factory object to instantiate objects with the Javabean cartridge.


Continue reading…

Of late, JBoss have been making more of a push on their Smooks Eclipse Tooling within JBoss Tools.  Prior to a few weeks ago, Dart Peng was “more or less” left to his own devices on this work, without much help from myself or anyone else.  More recently, Brian Fitzpatrick (JBoss) and myself have been more actively contributing to the great work being done by Dart.

Brian has been blogging about this work here on the JBoss Tools blog. Please take a look and contribute your throughs and ideas!!!

SOPERA have created a new Open Source project on Google called “soperadi-smooks“.  It is (or is hopefully going to be) a Smooks component for Talend.

Cool!!!!

Smooks can produce “output” in a number of different ways.  It can produce multiple sets of results, as well as output fragments of data to multiple different endpoints (ESB Service Endpoints, Files, JMS Destinations, DBs etc… and in different formats), all in a single filtering pass of a message stream.

For more on this, please see the Smooks Wiki…

Smooks v1.2 has just been released (download).  This release adds a number of very interesting features and enhancements to Smooks, most notably:

  1. EDIFACT Java Compiler (EJC):  EJC greatly simplifies the process of binding EDI data to a Java object graph.  EJC is similar to JAXBs XJC, accept for EDI messages.  This is just the first of a number of developer optimization features we are in the process of adding in the area of EDI message processing.
  2. Entity Persistence Framework Support:  Reuse your Entity Persistence resources (like Hibernate, iBATIS or any JPA compatible resource) to persist or enrich messages of any format (EDI, XML etc).  Access a database and use it’s query language, or CRUD methods, for reading and writing to the database.
  3. Validation:  Perform Rule based message fragment validation on messages of any format (EDI, XML etc).  Supports Data Field level validation using regular expressions, or Business Rule compliance validation using MVEL expressions.
  4. Simplified CSV Processing:  Bind CSV records to Java objects in seconds using a much simplified XML configuration, or using a very simple programmatic API.
  5. Improved Programmatic APIs:  Prior to Smooks v1.2, programmatic configuration was not one of Smooks’ strengths.   In Smooks v1.2 we’ve made significant improvements in this area.  Most Smooks features can now be utilised through Java, without writing a line of XML.

As well as these new features, Smooks v1.2 includes numerous bug fixes.  Please download Smooks v1.2 and let us know what you think.  We hope you find it useful!!

Data Validation with Smooks v1.2

Smooks 1.2 adds support for message data Validation as one of its new features. This new feature allows you to perform strong field and fragment validation on not just XML data, but also on EDI, JSON, CSV etc. It currently supports Regex and MVEL rules bases (Drools to follow). Regex rules allow you to perform low level field format validation, while MVEL rules allow you to perform Business rules validation at a fragment/message level.


Continue reading…