Apache Camel - Enterprise Integration Patterns
Apache Camel - Enterprise Integration Patterns
Apache Camel - Enterprise Integration Patterns
Camel empowers you to define routing and mediation rules in a variety of domain-
specific languages, including a Java-based Fluent API, Spring or Blueprint XML
Configuration files, and a Scala DSL. This means you get smart completion of routing
rules in your IDE, whether in a Java, Scala or XML editor.
Apache Camel uses URIs to work directly with any kind of Transport or messaging
model such as HTTP, ActiveMQ, JMS, JBI, SCA, MINA or CXF, as well as pluggable
Components and Data Format options. Apache Camel is a small library with minimal
dependencies for easy embedding in any Java application. Apache Camel lets you work
with the same API regardless which kind of Transport is used - so learn the API once
and you can interact with all the Components provided out-of-box.
Apache Camel provides support for Bean Binding and seamless integration with
popular frameworks such as Spring, Blueprint and Guice. Camel also has extensive
support for unit testing your routes.
The following projects can leverage Apache Camel as a routing and mediation engine:
Apache ServiceMix - a popular distributed open source ESB and JBI container
Apache ActiveMQ - a mature, widely used open source message broker
Apache CXF - a smart web services suite (JAX-WS and JAX-RS)
Apache Karaf - a small OSGi based runtime in which applications can be
deployed
Apache MINA - a high-performance NIO-driven networking framework
If you are new to Camel you might want to try the Getting Started in the User Guide
before attempting to implement these patterns.
The EIP icons library is available as a Visio stencil file adapted to render the icons with
the Camel color : sand. Download it here for your presentation, functional and technical
analysis documents. The original EIP stencil is also available in OpenOffice 3.x Draw
(thanks to Marco Garbelini) , Microsoft Visio, or Omnigraffle.
Messaging Systems
Messaging Channels
Point to Point How can the caller be sure that exactly one receiver will
Channel receive the document or perform the call?
Publish
How can the sender broadcast an event to all interested
Subscribe
receivers?
Channel
Dead Letter What will the messaging system do with a message it
Channel cannot deliver?
Guaranteed How can the sender make sure that a message will be
Delivery delivered, even if the messaging system fails?
What is an architecture that enables separate applications
to work together, but in a de-coupled fashion such that
Message Bus
applications can be easily added or removed without
affecting the others?
Message Construction
Return Address How does a replier know where to send the reply?
Message Routing
Message Transformation
Messaging Endpoints
System Management
Camel supports the Message Channel from the EIP patterns. The Message Channel is an
internal implementation detail of the Endpoint interface and all interactions with the
Message Channel are via the Endpoint interfaces.
Message
Message Endpoint
Message
Camel supports the Message from the EIP patterns using the Message interface.
To support various message exchange patterns like one way Event Message and
Request Reply messages Camel uses an Exchange interface which has a pattern
property which can be set to InOnly for an Event Message which has a single inbound
Message, or InOut for a Request Reply where there is an inbound and outbound
message.
Here is a basic example of sending a Message to a route in InOnly and InOut modes
Requestor Code
//InOnly
getContext().createProducerTemplate().sendBody("direct:startInOnly",
"Hello World");
//InOut
String result = (String)
getContext().createProducerTemplate().requestBody("direct:startInOut", "Hello World");
<route>
<from uri="direct:startInOut"/>
<inOut uri="bean:process"/>
</route>
Camel supports the Pipes and Filters from the EIP patterns in various ways.
With Camel you can split your processing across multiple independent Endpoint
instances which can then be chained together.
You can create pipelines of logic using multiple Endpoint or Message Translator
instances as follows
Though pipeline is the default mode of operation when you specify multiple outputs in
Camel. The opposite to pipeline is multicast; which fires the same message into each of
its outputs. (See the example below).
In Spring XML you can use the <pipeline/> element as of 1.4.0 onwards
<route>
<from uri="activemq:SomeQueue"/>
<pipeline>
<bean ref="foo"/>
<bean ref="bar"/>
<to uri="activemq:OutputQueue"/>
</pipeline>
</route>
In the above the pipeline element is actually unnecessary, you could use this...
<route>
<from uri="activemq:SomeQueue"/>
<bean ref="foo"/>
<bean ref="bar"/>
<to uri="activemq:OutputQueue"/>
</route>
Its just a bit more explicit. However if you wish to use <multicast/> to avoid a pipeline -
to send the same message into multiple pipelines - then the <pipeline/> element comes
into its own.
<route>
<from uri="activemq:SomeQueue"/>
<multicast>
<pipeline>
<bean ref="something"/>
<to uri="log:Something"/>
</pipeline>
<pipeline>
<bean ref="foo"/>
<bean ref="bar"/>
<to uri="activemq:OutputQueue"/>
</pipeline>
</multicast>
</route>
In the above example we are routing from a single Endpoint to a list of different
endpoints specified using URIs. If you find the above a bit confusing, try reading about
the Architecture or try the Examples
Message Router
The Message Router from the EIP patterns allows you to consume from an input
destination, evaluate some predicate then choose the right output destination.
The following example shows how to route a request from an input queue:a endpoint to
either queue:b, queue:c or queue:d depending on the evaluation of various Predicate
expressions
from("seda:a")
.choice()
.when(header("foo").isEqualTo("bar"))
.to("seda:b")
.when(header("foo").isEqualTo("cheese"))
.to("seda:c")
.otherwise()
.to("seda:d");
}
};
Using the Spring XML Extensions
<camelContext errorHandlerRef="errorHandler"
xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="seda:a"/>
<choice>
<when>
<xpath>$foo = 'bar'</xpath>
<to uri="seda:b"/>
</when>
<when>
<xpath>$foo = 'cheese'</xpath>
<to uri="seda:c"/>
</when>
<otherwise>
<to uri="seda:d"/>
</otherwise>
</choice>
</route>
</camelContext>
If you use a choice without adding an otherwise, any unmatched exchanges will be
dropped by default.
Message Translator
Camel supports the Message Translator from the EIP patterns by using an arbitrary
Processor in the routing logic, by using a bean to perform the transformation, or by
using transform() in the DSL. You can also use a Data Format to marshal and
unmarshal messages in different encodings.
Using the Fluent Builders
You can transform a message using Camel's Bean Integration to call any method on a
bean in your Registry such as your Spring XML configuration file as follows
from("activemq:SomeQueue").
beanRef("myTransformerBean", "myMethodName").
to("mqseries:AnotherQueue");
from("direct:start").process(new Processor() {
public void process(Exchange exchange) {
Message in = exchange.getIn();
in.setBody(in.getBody(String.class) + " World!");
}
}).to("mock:result");
from("direct:start").transform(body().append("
World!")).to("mock:result");
Use Spring XML
You can also use Spring XML Extensions to do a transformation. Basically any
Expression language can be substituted inside the transform element as shown below
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="direct:start"/>
<transform>
<simple>${in.body} extra data!</simple>
</transform>
<to uri="mock:end"/>
</route>
</camelContext>
<route>
<from uri="activemq:Input"/>
<bean ref="myBeanName" method="doTransform"/>
<to uri="activemq:Output"/>
</route>
You can also use Templating to consume a message from one destination, transform it
with something like Velocity or XQuery and then send it on to another destination. For
example using InOnly (one way messaging)
from("activemq:My.Queue").
to("velocity:com/acme/MyResponse.vm").
to("activemq:Another.Queue");
from("activemq:My.Queue").
to("velocity:com/acme/MyResponse.vm");
Message Endpoint
Camel supports the Message Endpoint from the EIP patterns using the Endpoint
interface.
When using the DSL to create Routes you typically refer to Message Endpoints by their
URIs rather than directly using the Endpoint interface. Its then a responsibility of the
CamelContext to create and activate the necessary Endpoint instances using the
available Component implementations.
Message
Messaging Channels
Point to Point Channel
Camel supports the Point to Point Channel from the EIP patterns using the following
components
Camel supports the Publish Subscribe Channel from the EIP patterns using for example
the following components:
JMS for working with JMS Topics for high performance, clustering and load balancing
XMPP when using rooms for group communication
SEDA for working with SEDA in the same CamelContext which can work in pub-sub, but
allowing multiple consumers.
VM as SEDA but for intra-JVM.
Camel supports the Dead Letter Channel from the EIP patterns using the
DeadLetterChannel processor which is an Error Handler.
Unlike the Default Error Handler that does not have a dead letter queue. So
whenever an Exchange could not be processed the error is propagated back to the
client.
Notice: You can adjust this behavior of whether the client should be notified or not
with the handled option.
Redelivery
The RedeliveryPolicy defines how the message is to be redelivered. You can customize
things like
Once all attempts at redelivering the message fails then the message is forwarded to the
dead letter queue.
Handled on Dead Letter Channel was introduced in Camel 2.0, this feature does not
exist in Camel 1.x
When all attempts of redelivery have failed the Exchange is moved to the dead letter
queue (the dead letter endpoint). The exchange is then complete and from the client
point of view it was processed. As such the Dead Letter Channel have handled the
Exchange.
<bean id="myDeadLetterErrorHandler"
class="org.apache.camel.builder.DeadLetterChannelBuilder">
<property name="deadLetterUri" value="jms:queue:dead"/>
<property name="redeliveryPolicy" ref="myRedeliveryPolicyConfig"/>
</bean>
<bean id="myRedeliveryPolicyConfig"
class="org.apache.camel.processor.RedeliveryPolicy">
<property name="maximumRedeliveries" value="3"/>
<property name="redeliveryDelay" value="5000"/>
</bean>
The Dead Letter Channel above will clear the caused exception
(setException(null)), by moving the caused exception to a property on the
Exchange, with the key Exchange.EXCEPTION_CAUGHT. Then the Exchange is moved to
the "jms:queue:dead" destination and the client will not notice the failure.
About moving Exchange to dead letter queue and using the original
message
from("jms:queue:order:input")
.to("bean:validateOrder")
.to("bean:transformOrder")
.to("bean:handleOrder");
The route listen for JMS messages and validates, transforms and handle it. During this
the Exchange payload is transformed/modified. So in case something goes wrong and
we want to move the message to another JMS destination, then we can configure our
Dead Letter Channel with the useOriginalBody option. But when we move the
Exchange to this destination we do not know in which state the message is in. Did the
error happen in before the transformOrder or after? So to be sure we want to move the
original input message we received from jms:queue:order:input. So we can do this
by enabling the useOriginalMessage option as shown below:
.useOriginalMessage().mamimumRedeliveries(5).redeliverDelay(5000);
Then the messages routed to the jms:queue:dead is the original input. If we want to
manually retry we can move the JMS message from the failed to the input queue, with
no problem as the message is the same as the original we received.
OnRedelivery
When Dead Letter Channel is doing redeliver its possible to configure a Processor that
is executed just before every redelivery attempt. This can be used for the situations
where you need to alter the message before its redelivered. See below for sample.
onException and onRedeliver
In Camel 2.0 we also added support for per onException to set a onRedeliver. That means
you can do special on redelivery for different exceptions, as opposed to onRedelivery set
on Dead Letter Channel can be viewed as a global scope.
In Camel 2.0 redelivery is disabled by default, as opposed to Camel 1.x in which Dead
Letter Channel is configured with maximumRedeliveries=5.
The maximum redeliver delay ensures that a delay is never longer than the value,
default 1 minute. This can happen if you turn on the exponential backoff.
The maximum redeliveries is the number of re delivery attempts. By default Camel will
try to process the exchange 1 + 5 times. 1 time for the normal attempt and then 5
attempts as redeliveries.
Setting the maximumRedeliveries to a negative value such as -1 will then always
redelivery (unlimited).
Setting the maximumRedeliveries to 0 will disable any re delivery attempt.
Camel will log delivery failures at the DEBUG logging level by default. You can
change this by specifying retriesExhaustedLogLevel and/or retryAttemptedLogLevel.
See ExceptionBuilderWithRetryLoggingLevelSetTest for an example.
In Camel 2.0 you can turn logging of stack traces on/off. If turned off Camel will still
log the redelivery attempt. Its just much less verbose.
The idea is to set groups of ranges using the following syntax: limit:delay;limit
2:delay 2;limit 3:delay 3;...;limit N:delay N
5:1000
10:5000
20:20000
Redelivery attempt number 1..4 = 0 millis (as the first group start with 5)
Redelivery attempt number 5..9 = 1000 millis (the first group)
Redelivery attempt number 10..19 = 5000 millis (the second group)
Redelivery attempt number 20.. = 20000 millis (the last group)
Note: The first redelivery attempt is 1, so the first group should start with 1 or higher.
There is no requirement that the next delay should be higher than the previous. You can
use any delay value you like. For example with delayPattern=1:5000;3:1000 we
start with 5 sec delay and then later reduce that to 1 second.
Redelivery header
When Camel routes messages it will decorate the Exchange with a property that
contains the last endpoint Camel send the Exchange to:
This information is updated when Camel sends a message to any endpoint. So if it exists
its the last endpoint which Camel send the Exchange to.
When for example processing the Exchange at a given Endpoint and the message is to
be moved into the dead letter queue, then Camel also decorates the Exchange with
another property that contains that last endpoint:
String failedEndpointUri =
exchange.getProperty(Exchange.FAILURE_ENDPOINT, String.class);
This allows for example you to fetch this information in your dead letter queue and use
that for error reporting.
This is useable if the Camel route is a bit dynamic such as the dynamic Recipient List so
you know which endpoints failed.
Notice: These information is kept on the Exchange even if the message was
successfully processed by a given endpoint, and then later fails for example in a local
Bean processing instead. So beware that this is a hint that helps pinpoint errors.
from("activemq:queue:foo")
.to("http://someserver/somepath")
.beanRef("foo");
Now suppose the route above and a failure happens in the foo bean. Then the
Exchange.TO_ENDPOINT and Exchange.FAILURE_ENDPOINT will still contain the value
of http://someserver/somepath.
Samples
The following example shows how to configure the Dead Letter Channel configuration
using the DSL
errorHandler(deadLetterChannel("seda:errors").maximumRedeliveries(2).u
seExponentialBackOff());
In Camel 1.6.0 we added support directly in Dead Letter Channel to set a Processor that
is executed before each redelivery attempt.
When Dead Letter Channel is doing redeliver its possible to configure a Processor that
is executed just before every redelivery attempt. This can be used for the situations
where you need to alter the message before its redelivered.
exchange.getIn().setBody(body + count);
Guaranteed Delivery
Camel supports the Guaranteed Delivery from the EIP patterns using among others the
following components:
Camel supports the Message Bus from the EIP patterns. You could view Camel as a
Message Bus itself as it allows producers and consumers to be decoupled.
Folks often assume that a Message Bus is a JMS though so you may wish to refer to the
JMS component for traditional MOM support.
Also worthy of note is the XMPP component for supporting messaging over XMPP
(Jabber)
Of course there are also ESB products such as Apache ServiceMix which serve as full
fledged message busses.
You can interact with Apache ServiceMix from Camel in many ways, but in particular
you can use the NMR or JBI component to access the ServiceMix message bus directly.
Message Construction
Event Message
Camel supports the Event Message from the EIP patterns by supporting the Exchange
Pattern on a Message which can be set to InOnly to indicate a oneway event message.
Camel Components then implement this pattern using the underlying transport or
protocols.
The default behaviour of many Components is InOnly such as for JMS, File or SEDA
Related
See the related Request Reply message.
If you are using a component which defaults to InOut you can override the Exchange
Pattern for an endpoint using the pattern property.
foo:bar?exchangePattern=InOnly
From 2.0 onwards on Camel you can specify the Exchange Pattern using the dsl.
from("mq:someQueue").
inOnly("mq:anotherQueue");
Request Reply
Camel supports the Request Reply from the EIP patterns by supporting the Exchange
Pattern on a Message which can be set to InOut to indicate a request/reply. Camel
Components then implement this pattern using the underlying transport or protocols.
For example when using JMS with InOut the component will by default perform these
actions
Related
See the related Event Message message
You can explicitly force an endpoint to be in Request Reply mode by setting the
exchange pattern on the URI. e.g.
jms:MyQueue?exchangePattern=InOut
You can specify the exchange pattern in DSL rule or Spring configuration.
Correlation Identifier
Camel supports the Correlation Identifier from the EIP patterns by getting or setting a
header on a Message.
When working with the ActiveMQ or JMS components the correlation identifier header
is called JMSCorrelationID. You can add your own correlation identifier to any
message exchange to help correlate messages together to a single conversation (or
business process).
The use of a Correlation Identifier is key to working with the Camel Business Activity
Monitoring Framework and can also be highly useful when testing with simulation or
canned data such as with the Mock testing framework
Some EIP patterns will spin off a sub message, and in those cases, Camel will add a
correlation id on the Exchange as a property with they key Exchange.CORRELATION_ID,
which links back to the source Exchange. For example the Splitter, Multicast, Recipient
List, and Wire Tap EIP does this.
Return Address
Camel supports the Return Address from the EIP patterns by using the JMSReplyTo
header.
For example when using JMS with InOut the component will by default return to the
address given in JMSReplyTo.
Requestor Code
getMockEndpoint("mock:bar").expectedBodiesReceived("Bye World");
template.sendBodyAndHeader("direct:start", "World", "JMSReplyTo",
"queue:bar");
<route>
<from uri="activemq:queue:foo"/>
<transform>
<simple>Bye ${in.body}</simple>
</transform>
</route>
<route>
<from uri="activemq:queue:bar?disableReplyTo=true"/>
<to uri="mock:bar"/>
</route>
For a complete example of this pattern, see this junit test case
Message Routing
Content Based Router
The Content Based Router from the EIP patterns allows you to route messages to the
correct destination based on the contents of the message exchanges.
The following example shows how to route a request from an input seda:a endpoint to
either seda:b, seda:c or seda:d depending on the evaluation of various Predicate
expressions
from("seda:a")
.choice()
.when(header("foo").isEqualTo("bar"))
.to("seda:b")
.when(header("foo").isEqualTo("cheese"))
.to("seda:c")
.otherwise()
.to("seda:d");
}
};
For further examples of this pattern in use you could look at the junit test case
Message Filter
The Message Filter from the EIP patterns allows you to filter messages
The following example shows how to create a Message Filter route consuming
messages from an endpoint called queue:a which if the Predicate is true will be
dispatched to queue:b
from("seda:a")
.filter(header("foo").isEqualTo("bar"))
.to("seda:b");
}
};
You can of course use many different Predicate languages such as XPath, XQuery, SQL
or various Scripting Languages. Here is an XPath example
from("direct:start").
filter().xpath("/person[@name='James']").
to("mock:result");
from("direct:start")
.filter().method(MyBean.class,
"isGoldCustomer").to("mock:result").end()
.to("mock:end");
For further examples of this pattern in use you could look at the junit test case
Using stop
Stop is a bit different than a message filter as it will filter out all messages and end the
route entirely (filter only applies to its child processor). Stop is convenient to use in a
Content Based Router when you for example need to stop further processing in one of
the predicates.
In the example below we do not want to route messages any further that has the word
Bye in the message body. Notice how we prevent this in the when predicate by using the
.stop().
from("direct:start")
.choice()
.when(body().contains("Hello")).to("mock:hello")
.when(body().contains("Bye")).to("mock:bye").stop()
.otherwise().to("mock:other")
.end()
.to("mock:result");
The Message Filter EIP will add a property on the Exchange which states if it was
filtered or not.
The property has the key Exchannge.FILTER_MATCHED which has the String value of
CamelFilterMatched. Its value is a boolean indicating true or false. If the value is
true then the Exchange was routed in the filter block.
Using This Pattern
If you would like to use this EIP Pattern then please read the Getting Started, you may
also find the Architecture useful particularly the description of Endpoint and URIs.
Then you could try out some of the Examples first before trying this pattern out.
Dynamic Router
The Dynamic Router from the EIP patterns allows you to route messages while
avoiding the dependency of the router on all possible destinations while maintaining its
efficiency.
Beware
You must ensure the expression used for the dynamicRouter such as a bean, will return
null to indicate the end. Otherwise the dynamicRouter will keep repeating endlessly.
Options
Default
Name Description
Value
From Camel 2.5 the Dynamic Router will set a property (Exchange.SLIP_ENDPOINT)
on the Exchange which contains the current endpoint as it advanced though the slip.
This allows you to know how far we have processed in the slip. (It's a slip because the
Dynamic Router implementation is based on top of Routing Slip).
Java DSL
from("direct:start")
// use a bean as the dynamic router
.dynamicRouter(method(DynamicRouterTest.class, "slip"));
Which will leverage a Bean to compute the slip on-the-fly, which could be implemented
as follows:
/**
* Use this method to compute dynamic where we should route next.
*
* @param body the message body
* @return endpoints to go, or <tt>null</tt> to indicate the end
*/
public String slip(String body) {
bodies.add(body);
invoked++;
if (invoked == 1) {
return "mock:a";
} else if (invoked == 2) {
return "mock:b,mock:c";
} else if (invoked == 3) {
return "direct:foo";
} else if (invoked == 4) {
return "mock:result";
}
/**
* Use this method to compute dynamic where we should route next.
*
* @param body the message body
* @param properties the exchange properties where we can store state
between invocations
* @return endpoints to go, or <tt>null</tt> to indicate the end
*/
public String slip(String body, @Properties Map<String, Object>
properties) {
bodies.add(body);
// get the state from the exchange properties and keep track how
many times
// we have been invoked
int invoked = 0;
Object current = properties.get("invoked");
if (current != null) {
invoked = Integer.valueOf(current.toString());
}
invoked++;
// and store the state back on the properties
properties.put("invoked", invoked);
if (invoked == 1) {
return "mock:a";
} else if (invoked == 2) {
return "mock:b,mock:c";
} else if (invoked == 3) {
return "direct:foo";
} else if (invoked == 4) {
return "mock:result";
}
You could also store state as message headers, but they are not guaranteed to be
preserved during routing, where as properties on the Exchange are. Although there was
a bug in the method call expression, see the warning below.
Spring XML
<bean id="mySlip"
class="org.apache.camel.processor.DynamicRouterTest"/>
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="direct:start"/>
<dynamicRouter>
<!-- use a method call on a bean as dynamic router -->
<method ref="mySlip" method="slip"/>
</dynamicRouter>
</route>
<route>
<from uri="direct:foo"/>
<transform><constant>Bye World</constant></transform>
<to uri="mock:foo"/>
</route>
</camelContext>
@DynamicRouter annotation
You can also use the @DynamicRouter annotation, for example the Camel 2.4 example
below could be written as follows. The route method would then be invoked repeatedly
as the message is processed dynamically. The idea is to return the next endpoint uri
where to go. Return null to indicate the end. You can return multiple endpoints if you
like, just as the Routing Slip, where each endpoint is separated by a delimiter.
@Consume(uri = "activemq:foo")
@DynamicRouter
public String route(@XPath("/customer/id") String customerId,
@Header("Location") String location, Document body) {
// query a database to find the best match of the endpoint
based on the input parameteres
// return the next endpoint uri, where to go. Return null to
indicate the end.
}
}
The simplest way to implement this is to use the RecipientList Annotation on a Bean
method to determine where to route the message.
@Consume(uri = "activemq:foo")
@RecipientList
public List<String> route(@XPath("/customer/id") String
customerId, @Header("Location") String location, Document body) {
// query a database to find the best match of the endpoint
based on the input parameteres
...
}
}
In the above we can use the Parameter Binding Annotations to bind different parts of
the Message to method parameters or use an Expression such as using XPath or
XQuery.
The method can be invoked in a number of ways as described in the Bean Integration
such as
POJO Producing
Spring Remoting
Bean component
Recipient List
The Recipient List from the EIP patterns allows you to route messages to a number of
dynamically specified recipients.
The recipients will receive a copy of the same Exchange, and Camel will execute them
sequentially.
Options
Default
Name Description
Value
The following example shows how to route a request from an input queue:a endpoint to
a static list of destinations
Using Annotations
You can use the RecipientList Annotation on a POJO to create a Dynamic Recipient
List. For more details see the Bean Integration.
from("seda:a")
.multicast().to("seda:b", "seda:c", "seda:d");
}
};
Usually one of the main reasons for using the Recipient List pattern is that the list of
recipients is dynamic and calculated at runtime. The following example demonstrates
how to create a dynamic recipient list using an Expression (which in this case it extracts
a named header value dynamically) to calculate the list of endpoints which are either of
type Endpoint or are converted to a String and then resolved using the endpoint URIs.
from("seda:a")
.recipientList(header("foo"));
}
};
The above assumes that the header contains a list of endpoint URIs. The following takes
a single string header and tokenizes it
from("direct:a").recipientList(
header("recipientListHeader").tokenize(","));
Iteratable value
The dynamic list of recipients that are defined in the header must be iteratable such as:
java.util.Collection
java.util.Iterator
arrays
org.w3c.dom.NodeList
Camel 1.6.0: a single String with values separated with comma
any other type will be regarded as a single value
For further examples of this pattern in use you could look at one of the junit test case
In Spring DSL you can set the delimiter attribute for setting a delimiter to be used if
the header value is a single String with multiple separated endpoints. By default Camel
uses comma as delimiter, but this option lets you specify a customer delimiter to use
instead.
<route>
<from uri="direct:a" />
<!-- use comma as a delimiter for String based values -->
<recipientList delimiter=",">
<header>myHeader</header>
</recipientList>
</route>
Note: In Java DSL you use the tokenizer to archive the same. The route above in Java
DSL:
from("direct:a").recipientList(header("myHeader").tokenize(","));
In Camel 2.1 its a bit easier as you can pass in the delimiter as 2nd parameter:
from("direct:a").recipientList(header("myHeader"), "#");
Sending to multiple recipients in parallel
The Recipient List now supports parallelProcessing that for example Splitter also
supports. You can use it to use a thread pool to have concurrent tasks sending the
Exchange to multiple recipients concurrently.
from("direct:a").recipientList(header("myHeader")).parallelProcessing(
);
<route>
<from uri="direct:a"/>
<recipientList parallelProcessing="true">
<header>myHeader</header>
</recipientList>
</route>
The Recipient List now supports stopOnException that for example Splitter also
supports. You can use it to stop sending to any further recipients in case any recipient
failed.
from("direct:a").recipientList(header("myHeader")).stopOnException();
<route>
<from uri="direct:a"/>
<recipientList stopOnException="true">
<header>myHeader</header>
</recipientList>
</route>
Note: You can combine parallelProcessing and stopOnException and have them
both true.
The Recipient List now supports ignoreInvalidEndpoints which the Routing Slip
also supports. You can use it to skip endpoints which is invalid.
from("direct:a").recipientList(header("myHeader")).ignoreInvalidEndpoi
nts();
<route>
<from uri="direct:a"/>
<recipientList ignoreInvalidEndpoints="true">
<header>myHeader</header>
</recipientList>
</route>
Then lets say the myHeader contains the following two endpoints
direct:foo,xxx:bar. The first endpoint is valid and works. However the 2nd is
invalid and will just be ignored. Camel logs at INFO level about, so you can see why
the endpoint was invalid.
You can now use you own AggregationStrategy with the Recipient List. However its
not that often you need that. What its good for is that in case you are using Request
Reply messaging then the replies from the recipient can be aggregated. By default
Camel uses UseLatestAggregationStrategy which just keeps that last received reply.
What if you must remember all the bodies that all the recipients send back, then you can
use your own custom aggregator that keeps those. Its the same principle as with the
Aggregator EIP so check it out for details.
from("direct:a")
.recipientList(header("myHeader")).aggregationStrategy(new
MyOwnAggregationStrategy())
.to("direct:b");
<route>
<from uri="direct:a"/>
<recipientList strategyRef="myStrategy">
<header>myHeader</header>
</recipientList>
<to uri="direct:b"/>
</route>
<bean id="myStrategy"
class="com.mycompany.MyOwnAggregationStrategy"/>
A thread pool is only used for parallelProcessing. You supply your own custom
thread pool via the ExecutorServiceStrategy (see Camel's Threading Model), the
same way you would do it for the aggregationStrategy. By default Camel uses a
thread pool with 10 threads (subject to change in a future version).
from("activemq:queue:test").recipientList().method(MessageRouter.class
, "routeTo");
When you use a Bean then do not also use the @RecipientList annotation as this will
in fact add yet another recipient list, so you end up having two. Do not do like this.
@RecipientList
public String routeTo() {
String queueName = "activemq:queue:test2";
return queueName;
}
}
Well you should only do like that above (using @RecipientList) if you route just route
to a Bean which you then want to act as a recipient list.
So the original route can be changed to:
from("activemq:queue:test").bean(MessageRouter.class, "routeTo");
Which then would invoke the routeTo method and detect its annotated with
@RecipientList and then act accordingly as if it was a recipient list EIP.
Using timeout
If you use parallelProcessing then you can configure a total timeout value in millis.
Camel will then process the messages in parallel until the timeout is hit. This allows you
to continue processing if one message is slow. For example you can set a timeout value
of 20 sec.
For example in the unit test below you can see we multicast the message to 3
destinations. We have a timeout of 2 seconds, which means only the last two messages
can be completed within the timeframe. This means we will only aggregate the last two
which yields a result aggregation which outputs "BC".
from("direct:start")
.multicast(new AggregationStrategy() {
public Exchange aggregate(Exchange oldExchange, Exchange
newExchange) {
if (oldExchange == null) {
return newExchange;
}
String body =
oldExchange.getIn().getBody(String.class);
oldExchange.getIn().setBody(body +
newExchange.getIn().getBody(String.class));
return oldExchange;
}
})
.parallelProcessing().timeout(250).to("direct:a", "direct:b",
"direct:c")
// use end to indicate end of multicast route
.end()
.to("mock:result");
from("direct:a").delay(1000).to("mock:A").setBody(constant("A"));
from("direct:b").to("mock:B").setBody(constant("B"));
from("direct:c").to("mock:C").setBody(constant("C"));
Timeout in other EIPs
This timeout feature is also supported by Splitter and both multicast and
recipientList.
/**
* A timeout occurred
*
* @param oldExchange the oldest exchange (is <tt>null</tt> on
first aggregation as we only have the new exchange)
* @param index the index
* @param total the total
* @param timeout the timeout value in millis
*/
void timeout(Exchange oldExchange, int index, int total, long
timeout);
This allows you to deal with the timeout in the AggregationStrategy if you really
need to.
Timeout is total
The timeout is total, which means that after X time, Camel will aggregate the messages
which has completed within the timeframe. The remainders will be cancelled. Camel will
also only invoke the timeout method in the TimeoutAwareAggregationStrategy
once, for the first index which caused the timeout.
Splitter
The Splitter from the EIP patterns allows you split a message into a number of pieces
and process them individually
Options
Default
Name Description
Value
stopOnException false
Camel 2.2: Whether or not to stop continue processing
immediately when an exception occurred. If disable, then
Camel continue splitting and process the sub-messages
regardless if one of them failed. You can deal with exceptions
in the AggregationStrategy class where you have full control
how to handle that.
Exchange properties
The following properties are set on each Exchange that are split:
Examples
The following example shows how to take a request from the queue:a endpoint the split
it into pieces using an Expression, then forward each piece to queue:b
from("seda:a")
.split(body(String.class).tokenize("\n"))
.to("seda:b");
}
};
The splitter can use any Expression language so you could use any of the Languages
Supported such as XPath, XQuery, SQL or one of the Scripting Languages to perform
the split. e.g.
from("activemq:my.queue").split(xpath("//foo/bar")).convertBodyTo(Stri
ng.class).to("file://some/directory")
Using the Spring XML Extensions
<camelContext errorHandlerRef="errorHandler"
xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="seda:a"/>
<split>
<xpath>/invoice/lineItems</xpath>
<to uri="seda:b"/>
</split>
</route>
</camelContext>
For further examples of this pattern in use you could look at one of the junit test case
You can use the tokenizer expression in the Spring DSL to split bodies or headers using
a token. This is a common use-case, so we provided a special tokenizer tag for this.
In the sample below we split the body using a @ as separator. You can of course use
comma or space or even a regex pattern, also set regex=true.
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="direct:start"/>
<split>
<tokenize token="@"/>
<to uri="mock:result"/>
</split>
</route>
</camelContext>
Splitting the body in Spring XML is a bit harder as you need to use the Simple language
to dictate this
<split>
<simple>${body}</simple>
<to uri="mock:result"/>
</split>
What the Splitter returns
If you want to execute all parts in parallel you can use special notation of split() with
two arguments, where the second one is a boolean flag if processing should be parallel.
e.g.
In Camel 2.0 the boolean option has been refactored into a builder method
parallelProcessing so its easier to understand what the route does when we use a
method instead of true|false.
Stream based
You can split streams by enabling the streaming mode using the streaming builder
method.
from("direct:streaming").split(body().tokenize(",")).streaming().to("a
ctivemq:my.parts");
You can also supply your custom splitter to use with streaming like this:
import static
org.apache.camel.builder.ExpressionBuilder.beanExpression;
from("direct:streaming")
.split(beanExpression(new MyCustomIteratorFactory(),
"iterator"))
.streaming().to("activemq:my.parts")
StAX component
The Camel StAX component can also be used to split big XML files in a streaming mode. See
more details at StAX.
<orders>
<order>
<!-- order stuff here -->
</order>
<order>
<!-- order stuff here -->
</order>
...
<order>
<!-- order stuff here -->
</order>
</orders>
Now to split this big file using XPath would cause the entire content to be loaded into
memory. So instead we can use the Tokenizer language to do this as follows:
from("file:inbox")
.split().tokenizeXML("order").streaming()
.to("activemq:queue:order");
Notice the tokenizeXML method which will split the file using the tag name of the child
node, which mean it will grab the content between the <order> and </order> tags
(incl. the tokens). So for example a splitted message would be as follows:
<order>
<!-- order stuff here -->
</order>
If you want to inherit namespaces from a root/parent tag, then you can do this as well by
providing the name of the root/parent tag:
<route>
<from uri="file:inbox"/>
<split streaming="true">
<tokenize token="order" inheritNamespaceTagName="orders"
xml="true"/>
<to uri="activemq:queue:order"/>
</split>
</route>
from("file:inbox")
.split().tokenizeXML("order", "orders").streaming()
.to("activemq:queue:order");
You can customize the underlying ThreadPoolExecutor used in the parallel splitter. In
the Java DSL try something like this:
from("activemq:my.queue")
.split(xPathBuilder).parallelProcessing().executorService(pool)
.to("activemq:my.parts");
In the route we define the Expression as a method call to invoke our Bean that we have
registered with the id mySplitterBean in the Registry.
from("direct:body")
// here we use a POJO bean mySplitterBean to do the split of
the payload
.split().method("mySplitterBean", "splitBody")
.to("mock:result");
from("direct:message")
// here we use a POJO bean mySplitterBean to do the split of
the message
// with a certain header value
.split().method("mySplitterBean", "splitMessage")
.to("mock:result");
And the logic for our Bean is as simple as. Notice we use Camel Bean Binding to pass
in the message body as a String object.
/**
* The split body method returns something that is iteratable such
as a java.util.List.
*
* @param body the payload of the incoming message
* @return a list containing each part splitted
*/
public List<String> splitBody(String body) {
// since this is based on an unit test you can of cause
// use different logic for splitting as Camel have out
// of the box support for splitting a String based on comma
// but this is for show and tell, since this is java code
// you have the full power how you like to split your messages
List<String> answer = new ArrayList<String>();
String[] parts = body.split(",");
for (String part : parts) {
answer.add(part);
}
return answer;
}
/**
* The split message method returns something that is iteratable
such as a java.util.List.
*
* @param header the header of the incoming message with the name
user
* @param body the payload of the incoming message
* @return a list containing each part splitted
*/
public List<Message> splitMessage(@Header(value = "user") String
header, @Body String body) {
// we can leverage the Parameter Binding Annotations
// http://camel.apache.org/parameter-binding-annotations.html
// to access the message header and body at same time,
// then create the message that we want, splitter will
// take care rest of them.
// *NOTE* this feature requires Camel version >= 1.6.1
List<Message> answer = new ArrayList<Message>();
String[] parts = header.split(",");
for (String part : parts) {
DefaultMessage message = new DefaultMessage();
message.setHeader("user", part);
message.setBody(body);
answer.add(message);
}
return answer;
}
}
This sample shows how you can split an Exchange, process each splitted message,
aggregate and return a combined response to the original caller using request/reply.
The route below illustrates this and how the split supports a aggregationStrategy to
hold the in progress processed messages:
/**
* We just handle the order by returning a id line for the order
*/
public String handleOrder(String line) {
LOG.debug("HandleOrder: " + line);
return "(id=" + ++counter + ",item=" + line + ")";
}
/**
* We use the same bean for building the combined response to send
* back to the original caller
*/
public String buildCombinedResponse(String line) {
LOG.debug("BuildCombinedResponse: " + line);
return "Response[" + line + "]";
}
}
And our custom aggregationStrategy that is responsible for holding the in progress
aggregated message that after the splitter is ended will be sent to the
buildCombinedResponse method for final processing before the combined response
can be returned to the waiting caller.
/**
* This is our own order aggregation strategy where we can control
* how each splitted message should be combined. As we do not want to
* loos any message we copy from the new to the old to preserve the
* order lines as long we process them
*/
public static class MyOrderStrategy implements AggregationStrategy {
if (oldExchange == null) {
// the first time we aggregate we only have the new
exchange,
// so we just return it
return newExchange;
}
HandleOrder: A
HandleOrder: B
Aggregate old orders: (id=1,item=A)
Aggregate new order: (id=2,item=B)
HandleOrder: C
Aggregate old orders: (id=1,item=A);(id=2,item=B)
Aggregate new order: (id=3,item=C)
BuildCombinedResponse: (id=1,item=A);(id=2,item=B);(id=3,item=C)
Response to caller:
Response[(id=1,item=A);(id=2,item=B);(id=3,item=C)]
The Splitter will by default continue to process the entire Exchange even in case of one
of the splitted message will thrown an exception during routing.
For example if you have an Exchange with 1000 rows that you split and route each sub
message. During processing of these sub messages an exception is thrown at the 17th.
What Camel does by default is to process the remainder 983 messages. You have the
chance to remedy or handle this in the AggregationStrategy.
But sometimes you just want Camel to stop and let the exception be propagated back,
and let the Camel error handler handle it. You can do this in Camel 2.1 by specifying
that it should stop in case of an exception occurred. This is done by the
stopOnException option as shown below:
from("direct:start")
.split(body().tokenize(",")).stopOnException()
.process(new MyProcessor())
.to("mock:split");
<route>
<from uri="direct:start"/>
<split stopOnException="true">
<tokenize token=","/>
<process ref="myProcessor"/>
<to uri="mock:split"/>
</split>
</route>
The Splitter will by default not share unit of work between the parent exchange and
each splitted exchange. This means each sub exchange has its own individual unit of
work.
For example you may have an use case, where you want to split a big message. And you
want to regard that process as an atomic isolated operation that either is a success or
failure. In case of a failure you want that big message to be moved into a dead letter
queue. To support this use case, you would have to share the unit of work on the
Splitter.
errorHandler(deadLetterChannel("mock:dead").useOriginalMessage()
.maximumRedeliveries(3).redeliveryDelay(0));
from("direct:start")
.to("mock:a")
// share unit of work in the splitter, which tells Camel to
propagate failures from
// processing the splitted messages back to the result of the
splitter, which allows
// it to act as a combined unit of work
.split(body().tokenize(",")).shareUnitOfWork()
.to("mock:b")
.to("direct:line")
.end()
.to("mock:result");
from("direct:line")
.to("log:line")
.process(new MyProcessor())
.to("mock:line");
Now in this example what would happen is that in case there is a problem processing
each sub message, the error handler will kick in (yes error handling still applies for the
sub messages). But what doesn't happen is that if a sub message fails all redelivery
attempts (its exhausted), then its not moved into that dead letter queue. The reason is
that we have shared the unit of work, so the sub message will report the error on the
shared unit of work. When the Splitter is done, it checks the state of the shared unit of
work and checks if any errors occurred. And if an error occurred it will set the exception
on the Exchange and mark it for rollback. The error handler will yet again kick in, as the
Exchange has been marked as rollback and it had an exception as well. No redelivery
attempts is performed (as it was marked for rollback) and the Exchange will be moved
into the dead letter queue.
Using this from XML DSL is just as easy as you just have to set the shareUnitOfWork
attribute to true:
<camelContext errorHandlerRef="dlc"
xmlns="http://camel.apache.org/schema/spring">
<!-- define error handler as DLC, with use original message enabled
-->
<errorHandler id="dlc" type="DeadLetterChannel"
deadLetterUri="mock:dead" useOriginalMessage="true">
<redeliveryPolicy maximumRedeliveries="3" redeliveryDelay="0"/>
</errorHandler>
<route>
<from uri="direct:start"/>
<to uri="mock:a"/>
<!-- share unit of work in the splitter, which tells Camel to
propagate failures from
processing the splitted messages back to the result of the
splitter, which allows
it to act as a combined unit of work -->
<split shareUnitOfWork="true">
<tokenize token=","/>
<to uri="mock:b"/>
<to uri="direct:line"/>
</split>
<to uri="mock:result"/>
</route>
</camelContext>
Implementation of shared unit of work in Camel 2.x
The Camel team had to introduce a SubUnitOfWork to keep API compatible with the
current UnitOfWork in Camel 2.x code base. So in reality the unit of work is not shared as
a single object instance. Instead SubUnitOfWork is attached to their parent, and issues
callback to the parent about their status (commit or rollback). This may be refactored in
Camel 3.0 where larger API changes can be done.
Aggregator
This applies for Camel version 2.3 or newer. If you use an older version then use
this Aggregator link instead.
The Aggregator from the EIP patterns allows you to combine a number of messages
together into a single message.
Aggregator options
Reference to lookup a
aggregationRepositoryRef
aggregationRepository in the Registry.
About AggregationStrategy
The AggregationStrategy is used for aggregating the old (lookup by its correlation
id) and the new exchanges together into a single exchange. Possible implementations
include performing some kind of combining or delta processing, such as adding line
items together into an invoice or just using the newest exchange and removing old
exchanges such as for state tracking or market data prices; where old values are of little
use.
Notice the aggregation strategy is a mandatory option and must be provided to the
aggregator.
Here are a few example AggregationStrategy implementations that should help you
create your own custom strategy.
When aggregation Exchanges at some point you need to indicate that the aggregated
exchanges is complete, so they can be send out of the aggregator. Camel allows you to
indicate completion in various ways as follows:
Notice that all the completion ways are per correlation key. And you can combine them
in any way you like. It's basically the first which triggers that wins. So you can use a
completion size together with a completion timeout. Only completionTimeout and
completionInterval cannot be used at the same time.
Notice the completion is a mandatory option and must be provided to the aggregator. If
not provided Camel will thrown an Exception on startup.
Persistent AggregationRepository
The aggregator provides a pluggable repository which you can implement your own
org.apache.camel.spi.AggregationRepository.
If you need persistent repository then you can use either Camel HawtDB or SQL
Component components.
Examples
See some examples from the old Aggregator which is somewhat similar to this new
aggregator.
Using completionTimeout
In this example we want to aggregate all incoming messages and after 3 seconds of
inactivity we want the aggregation to complete. This is done using the
completionTimeout option as shown:
from("direct:start")
// aggregate all exchanges correlated by the id header.
// Aggregate them using the BodyInAggregatingStrategy strategy
which
// and after 3 seconds of inactivity them timeout and complete the
aggregation
// and send it to mock:aggregated
.aggregate(header("id"), new
BodyInAggregatingStrategy()).completionTimeout(3000)
.to("mock:aggregated");
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="direct:start"/>
<aggregate strategyRef="aggregatorStrategy"
completionTimeout="3000">
<correlationExpression>
<simple>header.id</simple>
</correlationExpression>
<to uri="mock:aggregated"/>
</aggregate>
</route>
</camelContext>
<bean id="aggregatorStrategy"
class="org.apache.camel.processor.BodyInAggregatingStrategy"/>
Using TimeoutAwareAggregationStrategy
Using completionSize
In this example we want to aggregate all incoming messages and when we have 3
messages aggregated (in the same correlation group) we want the aggregation to
complete. This is done using the completionSize option as shown:
from("direct:start")
// aggregate all exchanges correlated by the id header.
// Aggregate them using the BodyInAggregatingStrategy strategy
which
// and after 3 messages has been aggregated then complete the
aggregation
// and send it to mock:aggregated
.aggregate(header("id"), new
BodyInAggregatingStrategy()).completionSize(3)
.to("mock:aggregated");
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="direct:start"/>
<aggregate strategyRef="aggregatorStrategy"
completionSize="3">
<correlationExpression>
<simple>header.id</simple>
</correlationExpression>
<to uri="mock:aggregated"/>
</aggregate>
</route>
</camelContext>
<bean id="aggregatorStrategy"
class="org.apache.camel.processor.BodyInAggregatingStrategy"/>
Using completionPredicate
In this example we want to aggregate all incoming messages and use a Predicate to
determine when we are complete. The Predicate can be evaluated using either the
aggregated exchange (default) or the incoming exchange. We will so both situations as
examples. We start with the default situation as shown:
from("direct:start")
// aggregate all exchanges correlated by the id header.
// Aggregate them using the BodyInAggregatingStrategy strategy
which
// and when the aggregated body contains A+B+C then complete the
aggregation
// and send it to mock:aggregated
.aggregate(header("id"), new
BodyInAggregatingStrategy()).completionPredicate(body().contains("A+B+
C"))
.to("mock:aggregated");
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="direct:start"/>
<aggregate strategyRef="aggregatorStrategy">
<correlationExpression>
<simple>header.id</simple>
</correlationExpression>
<completionPredicate>
<simple>${body} contains 'A+B+C'</simple>
</completionPredicate>
<to uri="mock:aggregated"/>
</aggregate>
</route>
</camelContext>
<bean id="aggregatorStrategy"
class="org.apache.camel.processor.BodyInAggregatingStrategy"/>
And the other situation where we use the eagerCheckCompletion option to tell Camel
to use the incoming Exchange. Notice how we can just test in the completion predicate
that the incoming message is the END message:
from("direct:start")
// aggregate all exchanges correlated by the id header.
// Aggregate them using the BodyInAggregatingStrategy strategy
// do eager checking which means the completion predicate will use
the incoming exchange
// which allows us to trigger completion when a certain exchange
arrived which is the
// END message
.aggregate(header("id"), new BodyInAggregatingStrategy())
.eagerCheckCompletion().completionPredicate(body().isEqualTo("END"))
.to("mock:aggregated");
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="direct:start"/>
<aggregate strategyRef="aggregatorStrategy"
eagerCheckCompletion="true">
<correlationExpression>
<simple>header.id</simple>
</correlationExpression>
<completionPredicate>
<simple>${body} == 'END'</simple>
</completionPredicate>
<to uri="mock:aggregated"/>
</aggregate>
</route>
</camelContext>
<bean id="aggregatorStrategy"
class="org.apache.camel.processor.BodyInAggregatingStrategy"/>
In this example we want to aggregate all incoming messages and after a period of
inactivity we want the aggregation to complete. The period should be computed at
runtime based on the timeout header in the incoming messages. This is done using the
completionTimeout option as shown:
from("direct:start")
// aggregate all exchanges correlated by the id header.
// Aggregate them using the BodyInAggregatingStrategy strategy
which
// and the timeout header contains the timeout in millis of
inactivity them timeout and complete the aggregation
// and send it to mock:aggregated
.aggregate(header("id"), new
BodyInAggregatingStrategy()).completionTimeout(header("timeout"))
.to("mock:aggregated");
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="direct:start"/>
<aggregate strategyRef="aggregatorStrategy">
<correlationExpression>
<simple>header.id</simple>
</correlationExpression>
<completionTimeout>
<header>timeout</header>
</completionTimeout>
<to uri="mock:aggregated"/>
</aggregate>
</route>
</camelContext>
<bean id="aggregatorStrategy"
class="org.apache.camel.processor.BodyInAggregatingStrategy"/>
Note: You can also add a fixed timeout value and Camel will fallback to use this value
if the dynamic value was null or 0.
In this example we want to aggregate all incoming messages based on a dynamic size
per correlation key. The size is computed at runtime based on the mySize header in the
incoming messages. This is done using the completionSize option as shown:
from("direct:start")
// aggregate all exchanges correlated by the id header.
// Aggregate them using the BodyInAggregatingStrategy strategy
which
// and the header mySize determines the number of aggregated
messages should trigger the completion
// and send it to mock:aggregated
.aggregate(header("id"), new
BodyInAggregatingStrategy()).completionSize(header("mySize"))
.to("mock:aggregated");
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="direct:start"/>
<aggregate strategyRef="aggregatorStrategy">
<correlationExpression>
<simple>header.id</simple>
</correlationExpression>
<completionSize>
<header>mySize</header>
</completionSize>
<to uri="mock:aggregated"/>
</aggregate>
</route>
</camelContext>
<bean id="aggregatorStrategy"
class="org.apache.camel.processor.BodyInAggregatingStrategy"/>
Note: You can also add a fixed size value and Camel will fallback to use this value if
the dynamic value was null or 0.
Resequencer
The Resequencer from the EIP patterns allows you to reorganise messages based on
some comparator. By default in Camel we use an Expression to create the comparator;
so that you can compare by a message header or the body or a piece of a message etc.
Batch resequencing collects messages into a batch, sorts the messages and sends
them to their output.
Stream resequencing re-orders (continuous) message streams based on the detection
of gaps between messages.
By default the Resequencer does not support duplicate messages and will only keep the
last message, in case a message arrives with the same message expression. However in
the batch mode you can enable it to allow duplicates.
Batch Resequencing
The following example shows how to use the batch-processing resequencer so that
messages are sorted in order of the body() expression. That is messages are collected
into a batch (either by a maximum number of messages per batch or using a timeout)
then they are sorted in order and then sent out to their output.
This is equvalent to
from("direct:start")
.resequence(body()).batch()
.to("mock:result");
The batch-processing resequencer can be further configured via the size() and
timeout() methods.
from("direct:start")
.resequence(body()).batch().size(300).timeout(4000L)
.to("mock:result")
This sets the batch size to 300 and the batch timeout to 4000 ms (by default, the batch
size is 100 and the timeout is 1000 ms). Alternatively, you can provide a configuration
object.
from("direct:start")
.resequence(body()).batch(new BatchResequencerConfig(300, 4000L))
.to("mock:result")
So the above example will reorder messages from endpoint direct:a in order of their
bodies, to the endpoint mock:result.
Typically you'd use a header rather than the body to order things; or maybe a part of the
body. So you could replace this expression with
resequencer(header("mySeqNo"))
for example to reorder messages using a custom sequence number in the header
mySeqNo.
You can of course use many different Expression languages such as XPath, XQuery,
SQL or various Scripting Languages.
Allow Duplicates
In the batch mode, you can now allow duplicates. In Java DSL there is a
allowDuplicates() method and in Spring XML there is an allowDuplicates=true
attribute on the <batch-config/> you can use to enable it.
Reverse
In the batch mode, you can now reverse the expression ordering. By default the order is
based on 0..9,A..Z, which would let messages with low numbers be ordered first, and
thus also also outgoing first. In some cases you want to reverse order, which is now
possible.
It's now much easier to use the Resequencer to resequence messages from JMS queues
based on JMSPriority. For that to work you need to use the two new options
allowDuplicates and reverse.
from("jms:queue:foo")
// sort by JMSPriority by allowing duplicates (message can have
same JMSPriority)
// and use reverse ordering so 9 is first output (most important),
and 0 is last
// use batch mode and fire every 3th second
.resequence(header("JMSPriority")).batch().timeout(3000).allowDuplicat
es().reverse()
.to("mock:result");
The Resequencer EIP will from Camel 2.9 onwards throw a CamelExchangeException
if the incoming Exchange is not valid for the resequencer - ie. the expression cannot be
evaluated, such as a missing header. You can use the option ignoreInvalidExchanges
to ignore these exceptions which means the Resequencer will then skip the invalid
Exchange.
from("direct:start")
.resequence(header("seqno")).batch().timeout(1000)
// ignore invalid exchanges (they are discarded)
.ignoreInvalidExchanges()
.to("mock:result");
Stream Resequencing
The next example shows how to use the stream-processing resequencer. Messages are
re-ordered based on their sequence numbers given by a seqnum header using gap
detection and timeouts on the level of individual messages.
The stream-processing resequencer can be further configured via the capacity() and
timeout() methods.
from("direct:start")
.resequence(header("seqnum")).stream().capacity(5000).timeout(4000L)
.to("mock:result")
This sets the resequencer's capacity to 5000 and the timeout to 4000 ms (by default, the
capacity is 1000 and the timeout is 1000 ms). Alternatively, you can provide a
configuration object.
from("direct:start")
.resequence(header("seqnum")).stream(new
StreamResequencerConfig(5000, 4000L))
.to("mock:result")
By default, the stream resequencer expects long sequence numbers but other sequence
numbers types can be supported as well by providing a custom expression.
}
from("direct:start").resequence(new
MyFileNameExpression()).stream().timeout(100).to("mock:result");
Further Examples
For further examples of this pattern in use you could look at the batch-processing
resequencer junit test case and the stream-processing resequencer junit test case
The Composed Message Processor from the EIP patterns allows you to process a
composite message by splitting it up, routing the sub-messages to appropriate
destinations and the re-aggregating the responses back into a single message.
Example
In this example we want to check that a multipart order can be filled. Each part of the
order requires a check at a different inventory.
<route>
<from uri="seda:aggregate"/>
<aggregate strategyRef="myOrderAggregatorStrategy"
completionTimeout="1000">
<correlationExpression>
<simple>header.orderId</simple>
</correlationExpression>
<to uri="mock:result"/>
</aggregate>
</route>
To do this we split up the order using a Splitter. The Splitter then sends individual
OrderItems to a Content Based Router which checks the item type. Widget items get
sent for checking in the widgetInventory bean and gadgets get sent to the
gadgetInventory bean. Once these OrderItems have been validated by the
appropriate bean, they are sent on to the Aggregator which collects and re-assembles the
validated OrderItems into an order again.
When an order is sent it contains a header with the order id. We use this fact when we
aggregate, as we configure this .header("orderId") on the aggregate DSL to
instruct Camel to use the header with the key orderId as correlation expression.
Scatter-Gather
The Scatter-Gather from the EIP patterns allows you to route messages to a number of
dynamically specified recipients and re-aggregate the responses back into a single
message.
In this example we want to get the best quote for beer from several different vendors.
We use a dynamic Recipient List to get the request for a quote to all vendors and an
Aggregator to pick the best quote out of all the responses. The routes for this are defined
as:
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="direct:start"/>
<recipientList>
<header>listOfVendors</header>
</recipientList>
</route>
<route>
<from uri="seda:quoteAggregator"/>
<aggregate strategyRef="aggregatorStrategy"
completionTimeout="1000">
<correlationExpression>
<header>quoteRequestId</header>
</correlationExpression>
<to uri="mock:result"/>
</aggregate>
</route>
</camelContext>
So in the first route you see that the Recipient List is looking at the listOfVendors
header for the list of recipients. So, we need to send a message like
Map<String, Object> headers = new HashMap<String, Object>();
headers.put("listOfVendors", "bean:vendor1, bean:vendor2,
bean:vendor3");
headers.put("quoteRequestId", "quoteRequest-1");
template.sendBodyAndHeaders("direct:start", "<quote_request
item=\"beer\"/>", headers);
@Produce(uri = "seda:quoteAggregator")
private ProducerTemplate quoteAggregator;
<bean id="aggregatorStrategy"
class="org.apache.camel.spring.processor.scattergather.LowestQuoteAggr
egationStrategy"/>
<bean id="vendor1"
class="org.apache.camel.spring.processor.scattergather.MyVendor">
<constructor-arg>
<value>1</value>
</constructor-arg>
</bean>
<bean id="vendor2"
class="org.apache.camel.spring.processor.scattergather.MyVendor">
<constructor-arg>
<value>2</value>
</constructor-arg>
</bean>
<bean id="vendor3"
class="org.apache.camel.spring.processor.scattergather.MyVendor">
<constructor-arg>
<value>3</value>
</constructor-arg>
</bean>
Each bean is loaded with a different price for beer. When the message is sent to each
bean endpoint, it will arrive at the MyVendor.getQuote method. This method does a
simple check whether this quote request is for beer and then sets the price of beer on the
exchange for retrieval at a later step. The message is forwarded on to the next step using
POJO Producing (see the @Produce annotation).
At the next step we want to take the beer quotes from all vendors and find out which
one was the best (i.e. the lowest!). To do this we use an Aggregator with a custom
aggregation strategy. The Aggregator needs to be able to compare only the messages
from this particular quote; this is easily done by specifying a correlationExpression
equal to the value of the quoteRequestId header. As shown above in the message
sending snippet, we set this header to quoteRequest-1. This correlation value should
be unique or you may include responses that are not part of this quote. To pick the
lowest quote out of the set, we use a custom aggregation strategy like
if (oldExchange.getIn().getBody(int.class) <
newExchange.getIn().getBody(int.class)) {
return oldExchange;
} else {
return newExchange;
}
}
}
Finally, we expect to get the lowest quote of $1 out of $1, $2, and $3.
camel-spring/src/test/java/org/apache/camel/spring/processor/scattergather/
camel-spring/src/test/resources/org/apache/camel/spring/processor/scattergather/scatter-
gather.xml
You can lock down which recipients are used in the Scatter-Gather by using a static
Recipient List. It looks something like this
from("direct:start").multicast().to("seda:vendor1", "seda:vendor2",
"seda:vendor3");
from("seda:vendor1").to("bean:vendor1").to("seda:quoteAggregator");
from("seda:vendor2").to("bean:vendor2").to("seda:quoteAggregator");
from("seda:vendor3").to("bean:vendor3").to("seda:quoteAggregator");
from("seda:quoteAggregator")
.aggregate(header("quoteRequestId"), new
LowestQuoteAggregationStrategy()).to("mock:result")
A full example of the static Scatter-Gather configuration can be found in the Loan
Broker Example.
Routing Slip
The Routing Slip from the EIP patterns allows you to route a message consecutively
through a series of processing steps where the sequence of steps is not known at design
time and can vary for each message.
Options
Default
Name Description
Value
Example
The following route will take any messages sent to the Apache ActiveMQ queue
SomeQueue and pass them into the Routing Slip pattern.
from("activemq:SomeQueue").routingSlip("headerName");
Messages will be checked for the existance of the "headerName" header. The value of
this header should be a comma-delimited list of endpoint URIs you wish the message to
be routed to. The Message will be routed in a pipeline fashion (i.e. one after the other).
Note: In Camel 1.x the default header name routingSlipHeader has been
@deprecated and is removed in Camel 2.0. We feel that the DSL needed to express, the
header it uses to locate the destinations, directly in the DSL to not confuse readers. So
the header name must be provided.
From Camel 2.5 the Routing Slip will set a property (Exchange.SLIP_ENDPOINT) on
the Exchange which contains the current endpoint as it advanced though the slip. This
allows you to know how far we have processed in the slip.
The Routing Slip will compute the slip beforehand which means, the slip is only
computed once. If you need to compute the slip on-the-fly then use the Dynamic Router
pattern instead.
Configuration options
Here we set the header name and the URI delimiter to something different.
The Routing Slip now supports ignoreInvalidEndpoints which the Recipient List
also supports. You can use it to skip endpoints which is invalid.
from("direct:a").routingSlip("myHeader").ignoreInvalidEndpoints();
<route>
<from uri="direct:a"/>
<routingSlip headerName="myHeader"
ignoreInvalidEndpoints="true"/>
</route>
Then lets say the myHeader contains the following two endpoints
direct:foo,xxx:bar. The first endpoint is valid and works. However the 2nd is
invalid and will just be ignored. Camel logs at INFO level about, so you can see why
the endpoint was invalid.
Expression supporting
The Routing Slip now supports to take the expression parameter as the Recipient List
does. You can tell the camel the expression that you want to use to get the routing slip.
from("direct:a").routingSlip(header("myHeader")).ignoreInvalidEndpoint
s();
<route>
<from uri="direct:a"/>
<!--NOTE from Camel 2.4.0, you need to specify the expression
element inside of the routingSlip element -->
<routingSlip ignoreInvalidEndpoints="true">
<header>myHeader</header>
</routingSlip>
</route>
Further Examples
For further examples of this pattern in use you could look at the routing slip test cases.
Throttler
The Throttler Pattern allows you to ensure that a specific endpoint does not get
overloaded, or that we don't exceed an agreed SLA with some external service.
Options
Default
Name Description
Value
Examples
Using the Fluent Builders
from("seda:a").throttle(3).timePeriodMillis(10000).to("log:result",
"mock:result");
So the above example will throttle messages all messages received on seda:a before
being sent to mock:result ensuring that a maximum of 3 messages are sent in any 10
second window. Note that typically you would often use the default time period of a
second. So to throttle requests at 100 requests per second between two endpoints it
would look more like this...
from("seda:a").throttle(100).to("seda:b");
For further examples of this pattern in use you could look at the junit test case
In Camel 2.8 onwards you must set the maximum period as an Expression as shown
below where we use a Constant expression:
<route>
<from uri="seda:a"/>
<!-- throttle 3 messages per 10 sec -->
<throttle timePeriodMillis="10000">
<constant>3</constant>
<to uri="mock:result"/>
</throttle>
</route>
Dynamically changing maximum requests per perio d
<route>
<from uri="direct:expressionHeader"/>
<throttle timePeriodMillis="500">
<!-- use a header to determine how many messages to throttle
per 0.5 sec -->
<header>throttleValue</header>
<to uri="mock:result"/>
</throttle>
</route>
Asynchronous delaying
You can let the Throttler use non blocking asynchronous delaying, which means Camel
will use a scheduler to schedule a task to be executed in the future. The task will then
continue routing. This allows the caller thread to not block and be able to service other
messages etc.
from("seda:a").throttle(100).asyncDelayed().to("seda:b");
Sampling Throttler
A sampling throttler allows you to extract a sample of the exchanges from the traffic
through a route.
It is configured with a sampling period during which only a single exchange is allowed
to pass through. All other exchanges will be stopped.
Options
Default
Name Description
Value
Samples the message every N'th message. You can only use
messageFrequency
either frequency or period.
Samples the message every N'th period. You can only use
samplePeriod 1
either frequency or period.
You use this EIP with the sample DSL as show in these samples.
from("direct:sample")
.sample()
.to("mock:result");
from("direct:sample-configured")
.sample(1, TimeUnit.SECONDS)
.to("mock:result");
from("direct:sample-configured-via-dsl")
.sample().samplePeriod(1).timeUnits(TimeUnit.SECONDS)
.to("mock:result");
from("direct:sample-messageFrequency")
.sample(10)
.to("mock:result");
from("direct:sample-messageFrequency-via-dsl")
.sample().sampleMessageFrequency(5)
.to("mock:result");
<route>
<from uri="direct:sample"/>
<sample samplePeriod="1" units="seconds">
<to uri="mock:result"/>
</sample>
</route>
<route>
<from uri="direct:sample-messageFrequency"/>
<sample messageFrequency="10">
<to uri="mock:result"/>
</sample>
</route>
<route>
<from uri="direct:sample-messageFrequency-via-dsl"/>
<sample messageFrequency="5">
<to uri="mock:result"/>
</sample>
</route>
And since it uses a default of 1 second you can omit this configuration in case you also
want to use 1 second
<route>
<from uri="direct:sample"/>
<!-- will by default use 1 second period -->
<sample>
<to uri="mock:result"/>
</sample>
</route>
If you would like to use this EIP Pattern then please read the Getting Started, you may
also find the Architecture useful particularly the description of Endpoint and URIs.
Then you could try out some of the Examples first before trying this pattern out.
Delayer
The Delayer Pattern allows you to delay the delivery of messages to some destination.
The Delayer in Camel 1.x works a bit differently than Camel 2.0 onwards.
In Camel 2.0 the expression is a value in millis to wait from the current time, so the
expression should just be 3000.
However in both Camel 1.x and 2.0 you can use a long value for a fixed value to
indicate the delay in millis.
See the Spring DSL samples for Delayer in Camel 1.x vs. Camel 2.0.
Using Delayer in Java DSL
See this ticket: https://issues.apache.org/jira/browse/CAMEL-2654
Options
Default
Name Description
Value
You can of course use many different Expression languages such as XPath, XQuery,
SQL or various Scripting Languages. You can just delay things a fixed amount of time
from the point at which the delayer receives the message. For example to delay things 2
seconds
delayer(2000)
The above assume that the delivery order is maintained and that the messages are
delivered in delay order. If you want to reorder the messages based on delivery time,
you can use the Resequencer with this pattern. For example
from("activemq:someQueue").resequencer(header("MyDeliveryTime")).delay
("MyRedeliveryTime").to("activemq:aDelayedQueue");
<bean id="myDelayBean"
class="org.apache.camel.processor.MyDelayCalcBean"/>
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="seda:a"/>
<delay>
<header>MyDelay</header>
</delay>
<to uri="mock:result"/>
</route>
<route>
<from uri="seda:b"/>
<delay>
<constant>1000</constant>
</delay>
<to uri="mock:result"/>
</route>
<route>
<from uri="seda:c"/>
<delay>
<method ref="myDelayBean" method="delayMe"/>
</delay>
<to uri="mock:result"/>
</route>
</camelContext>
<delayer>
<delayTime>3000</delayTime>
</expression>
</delayer>
The empty tag </expression> is needed to fulfill the XSD validation as its an optional
element and we use JAXB annotations to generated the XSD in Camel and some
combinations is hard to auto generate with optional elements.
For further examples of this pattern in use you could look at the junit test case
Asynchronous delaying
You can let the Delayer use non blocking asynchronous delaying, which means Camel
will use a scheduler to schedule a task to be executed in the future. The task will then
continue routing. This allows the caller thread to not block and be able to service other
messages etc.
from("activemq:queue:foo").delay(1000).asyncDelayed().to("activemq:aDe
layedQueue");
From Spring XML
<route>
<from uri="activemq:queue:foo"/>
<delay asyncDelayed="true">
<constant>1000</constant>
</delay>
<to uri="activemq:aDealyedQueue"/>
</route>
You can use an expression to determine when to send a message using something like
this
from("activemq:foo").
delay().method("someBean", "computeDelay").
to("activemq:bar");
The Load Balancer Pattern allows you to delegate to one of a number of endpoints using
a variety of different load balancing policies.
Policy Description
The exchanges are selected from in a round robin fashion. This is a well known
Round Robin
and classic policy, which spreads the load evenly.
Topic Topic which sends to all destinations (rather like JMS Topics)
Failover Camel 2.0: In case of failures the exchange will be tried on the next endpoint.
Camel 2.5: The weighted load balancing policy allows you to specify a processing
Weighted load distribution ratio for each server with respect to the others. In addition to
Round-Robin the weight, endpoint selection is then further refined using round-robin
distribution based on weight.
Camel 2.5: The weighted load balancing policy allows you to specify a processing
Weighted load distribution ratio for each server with respect to others.In addition to the
Random weight, endpoint selection is then further refined using random distribution
based on weight.
Camel 2.8: From Camel 2.8 onwards the preferred way of using a custom Load
Custom
Balancer is to use this policy, instead of using the @deprecated ref attribute.
Round Robin
The round robin load balancer is not meant to work with failover, for that you should
use the dedicated failover load balancer. The round robin load balancer will only
change to next endpoint per message.
The round robin load balancer is stateful as it keeps state of which endpoint to use next
time.
<camelContext id="camel"
xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="direct:start"/>
<loadBalance>
<roundRobin/>
<to uri="mock:x"/>
<to uri="mock:y"/>
<to uri="mock:z"/>
</loadBalance>
</route>
</camelContext>
The above example loads balance requests from direct:start to one of the available
mock endpoint instances, in this case using a round robin policy.
For further examples of this pattern look at this junit test case
Failover
from("direct:start")
// here we will load balance if IOException was thrown
// any other kind of exception will result in the Exchange as
failed
// to failover over any kind of exception we can just omit the
exception
// in the failOver DSL
.loadBalance().failover(IOException.class)
.to("direct:x", "direct:y", "direct:z");
You can specify multiple exceptions to failover as the option is varargs, for instance:
from("direct:foo").
loadBalance().failover(IOException.class, MyOtherException.class)
.to("direct:a", "direct:b");
Failover can also be used from Spring DSL and you configure it as:
<route errorHandlerRef="myErrorHandler">
<from uri="direct:foo"/>
<loadBalance>
<failover>
<exception>java.io.IOException</exception>
<exception>com.mycompany.MyOtherException</exception>
</failover>
<to uri="direct:a"/>
<to uri="direct:b"/>
</loadBalance>
</route>
from("direct:start")
// Use failover load balancer in stateful round robin mode
// which mean it will failover immediately in case of an exception
// as it does NOT inherit error handler. It will also keep
retrying as
// its configured to newer exhaust.
.loadBalance().failover(-1, false, true).
to("direct:bad", "direct:bad2", "direct:good",
"direct:good2");
<route>
<from uri="direct:start"/>
<loadBalance>
<!-- failover using stateful round robin,
which will keep retrying forever those 4 endpoints until
success.
You can set the maximumFailoverAttempt to break out after
X attempts -->
<failover roundRobin="true"/>
<to uri="direct:bad"/>
<to uri="direct:bad2"/>
<to uri="direct:good"/>
<to uri="direct:good2"/>
</loadBalance>
</route>
Disabled inheritErrorHandler
You can configure inheritErrorHandler=false if you want to failover to the next
endpoint as fast as possible. By disabling the Error Handler you ensure it does not intervene
which allows the failover load balancer to handle failover asap. By also enabling
roundRobin mode, then it will keep retrying until it success. You can then configure the
maximumFailoverAttempts option to a high value to let it eventually exhaust (give up)
and fail.
In many enterprise environments where server nodes of unequal processing power &
performance characteristics are utilized to host services and processing endpoints, it is
frequently necessary to distribute processing load based on their individual server
capabilities so that some endpoints are not unfairly burdened with requests. Obviously
simple round-robin or random load balancing do not alleviate problems of this nature. A
Weighted Round-Robin and/or Weighted Random load balancer can be used to address
this problem.
The weighted load balancing policy allows you to specify a processing load distribution
ratio for each server with respect to others. You can specify this as a positive processing
weight for each server. A larger number indicates that the server can handle a larger
load. The weight is utilized to determine the payload distribution ratio to different
processing endpoints with respect to others.
Disabled inheritErrorHandler
As of Camel 2.6, the Weighted Load balancer usage has been further simplified, there is no
need to send in distributionRatio as a List<Integer>. It can be simply sent as a delimited
String of integer weights separated by a delimiter of choice.
In Camel 2.5
In Camel 2.5
// round-robin
from("direct:start")
.loadBalance().weighted(true, distributionRatio)
.to("mock:x", "mock:y", "mock:z");
//random
from("direct:start")
.loadBalance().weighted(false, distributionRatio)
.to("mock:x", "mock:y", "mock:z");
// round-robin
from("direct:start")
.loadBalance().weighted(true, "4:2:1"
distributionRatioDelimiter=":")
.to("mock:x", "mock:y", "mock:z");
//random
from("direct:start")
.loadBalance().weighted(false, "4,2,1")
.to("mock:x", "mock:y", "mock:z");
<route>
<from uri="direct:start"/>
<loadBalance>
<weighted roundRobin="false" distributionRatio="4-2-1"
distributionRatioDelimiter="-" />
<to uri="mock:x"/>
<to uri="mock:y"/>
<to uri="mock:z"/>
</loadBalance>
</route>
You can use a custom load balancer (eg your own implementation) also.
from("direct:start")
// using our custom load balancer
.loadBalance(new MyLoadBalancer())
.to("mock:x", "mock:y", "mock:z");
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="direct:start"/>
<loadBalance>
<!-- refer to my custom load balancer -->
<custom ref="myBalancer"/>
<!-- these are the endpoints to balancer -->
<to uri="mock:x"/>
<to uri="mock:y"/>
<to uri="mock:z"/>
</loadBalance>
</route>
</camelContext>
Notice in the XML DSL above we use <custom> which is only available in Camel 2.8
onwards. In older releases you would have to do as follows instead:
<loadBalance ref="myBalancer">
<!-- these are the endpoints to balancer -->
<to uri="mock:x"/>
<to uri="mock:y"/>
<to uri="mock:z"/>
</loadBalance>
To implement a custom load balancer you can extend some support classes such as
LoadBalancerSupport and SimpleLoadBalancerSupport. The former supports the
asynchronous routing engine, and the latter does not. Here is an example:
The Multicast allows to route the same message to a number of endpoints and process
them in a different way. The main difference between the Multicast and Splitter is that
Splitter will split the message into several pieces but the Multicast will not modify the
request message.
Options
Default
Name Description
Value
Example
The following example shows how to take a request from the direct:a endpoint , then
multicast these request to direct:x, direct:y, direct:z.
from("direct:a").multicast().parallelProcessing().to("direct:x",
"direct:y", "direct:z");
In case of using InOut MEP, an AggregationStrategy is used for aggregating all reply
messages. The default is to only use the latest reply message and discard any earlier
replies. The aggregation strategy is configurable:
from("direct:start")
.multicast(new MyAggregationStrategy())
.parallelProcessing().timeout(500).to("direct:a", "direct:b",
"direct:c")
.end()
.to("mock:result");
The Multicast will by default continue to process the entire Exchange even in case one
of the multicasted messages will thrown an exception during routing.
For example if you want to multicast to 3 destinations and the 2nd destination fails by
an exception. What Camel does by default is to process the remainder destinations. You
have the chance to remedy or handle this in the AggregationStrategy.
But sometimes you just want Camel to stop and let the exception be propagated back,
and let the Camel error handler handle it. You can do this in Camel 2.1 by specifying
that it should stop in case of an exception occurred. This is done by the
stopOnException option as shown below:
from("direct:start")
.multicast()
.stopOnException().to("direct:foo", "direct:bar",
"direct:baz")
.end()
.to("mock:result");
from("direct:foo").to("mock:foo");
from("direct:bar").process(new MyProcessor()).to("mock:bar");
from("direct:baz").to("mock:baz");
<route>
<from uri="direct:start"/>
<multicast stopOnException="true">
<to uri="direct:foo"/>
<to uri="direct:bar"/>
<to uri="direct:baz"/>
</multicast>
<to uri="mock:result"/>
</route>
<route>
<from uri="direct:foo"/>
<to uri="mock:foo"/>
</route>
<route>
<from uri="direct:bar"/>
<process ref="myProcessor"/>
<to uri="mock:bar"/>
</route>
<route>
<from uri="direct:baz"/>
<to uri="mock:baz"/>
</route>
The Multicast will copy the source Exchange and multicast each copy. However the
copy is a shallow copy, so in case you have mutateable message bodies, then any
changes will be visible by the other copied messages. If you want to use a deep clone
copy then you need to use a custom onPrepare which allows you to do this using the
Processor interface.
Notice the onPrepare can be used for any kind of custom logic which you would like to
execute before the Exchange is being multicasted.
Animal
public Animal() {
}
@Override
public String toString() {
return id + " " + name;
}
}
Then we can create a deep clone processor which clones the message body:
AnimalDeepClonePrepare
Then we can use the AnimalDeepClonePrepare class in the Multicast route using the
onPrepare option as shown:
from("direct:start")
.multicast().onPrepare(new
AnimalDeepClonePrepare()).to("direct:a").to("direct:b");
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="direct:start"/>
<!-- use on prepare with multicast -->
<multicast onPrepareRef="animalDeepClonePrepare">
<to uri="direct:a"/>
<to uri="direct:b"/>
</multicast>
</route>
<route>
<from uri="direct:a"/>
<process ref="processorA"/>
<to uri="mock:a"/>
</route>
<route>
<from uri="direct:b"/>
<process ref="processorB"/>
<to uri="mock:b"/>
</route>
</camelContext>
<!-- the on prepare Processor which performs the deep cloning -->
<bean id="animalDeepClonePrepare"
class="org.apache.camel.processor.AnimalDeepClonePrepare"/>
<!-- processors used for the last two routes, as part of unit test -->
<bean id="processorA"
class="org.apache.camel.processor.MulticastOnPrepareTest$ProcessorA"/>
<bean id="processorB"
class="org.apache.camel.processor.MulticastOnPrepareTest$ProcessorB"/>
Notice the onPrepare option is also available on other EIPs such as Splitter, Recipient
List, and Wire Tap.
Loop
The Loop allows for processing a message a number of times, possibly in a different
way for each iteration. Useful mostly during testing.
Default mode
Notice by default the loop uses the same exchange throughout the looping. So the result
from the previous iteration will be used for the next (eg Pipes and Filters). From Camel 2.8
onwards you can enable copy mode instead. See the options table for more details.
Options
Default
Name Description
Value
Camel 2.8: Whether or not copy mode is used. If false then the same
Exchange will be used for each iteration. So the result from the previous
copy false iteration will be visible for the next iteration. Instead you can enable copy
mode, and then each iteration restarts with a fresh copy of the input
Exchange.
Exchange properties
For each iteration two properties are set on the Exchange. Processors can rely on these
properties to process the Message in different ways.
Property Description
Examples
The following example shows how to take a request from the direct:x endpoint, then
send the message repetitively to mock:result. The number of times the message is sent
is either passed as an argument to loop(), or determined at runtime by evaluating an
expression. The expression must evaluate to an int, otherwise a
RuntimeCamelException is thrown.
from("direct:a").loop(8).to("mock:result");
Use expression to determine loop count
from("direct:b").loop(header("loop")).to("mock:result");
from("direct:c").loop().xpath("/hello/@times").to("mock:result");
<route>
<from uri="direct:a"/>
<loop>
<constant>8</constant>
<to uri="mock:result"/>
</loop>
</route>
<route>
<from uri="direct:b"/>
<loop>
<header>loop</header>
<to uri="mock:result"/>
</loop>
</route>
For further examples of this pattern in use you could look at one of the junit test case
from("direct:start")
// instruct loop to use copy mode, which mean it will use a copy
of the input exchange
// for each loop iteration, instead of keep using the same
exchange all over
.loop(3).copy()
.transform(body().append("B"))
.to("mock:loop")
.end()
.to("mock:result");
However if we do not enable copy mode then "mock:loop" will receive "AB", "ABB",
"ABBB", etc. messages.
from("direct:start")
// by default loop will keep using the same exchange so on the 2nd
and 3rd iteration its
// the same exchange that was previous used that are being looped
all over
.loop(3)
.transform(body().append("B"))
.to("mock:loop")
.end()
.to("mock:result");
<route>
<from uri="direct:start"/>
<!-- enable copy mode for loop eip -->
<loop copy="true">
<constant>3</constant>
<transform>
<simple>${body}B</simple>
</transform>
<to uri="mock:loop"/>
</loop>
<to uri="mock:result"/>
</route>
Message Transformation
Content Enricher
Camel supports the Content Enricher from the EIP patterns using a Message Translator,
an arbitrary Processor in the routing logic or using the enrich DSL element to enrich the
message.
You can use Templating to consume a message from one destination, transform it with
something like Velocity or XQuery and then send it on to another destination. For
example using InOnly (one way messaging)
from("activemq:My.Queue").
to("velocity:com/acme/MyResponse.vm").
to("activemq:Another.Queue");
from("activemq:My.Queue").
to("velocity:com/acme/MyResponse.vm");
Here is a simple example using the DSL directly to transform the message body
from("direct:start").setBody(body().append("
World!")).to("mock:result");
In this example we add our own Processor using explicit Java code
from("direct:start").process(new Processor() {
public void process(Exchange exchange) {
Message in = exchange.getIn();
in.setBody(in.getBody(String.class) + " World!");
}
}).to("mock:result");
Finally we can use Bean Integration to use any Java method on any bean to act as the
transformer
from("activemq:My.Queue").
beanRef("myBeanName", "myMethodName").
to("activemq:Another.Queue");
For further examples of this pattern in use you could look at one of the JUnit tests
TransformTest
TransformViaDSLTest
<route>
<from uri="activemq:Input"/>
<bean ref="myBeanName" method="doTransform"/>
<to uri="activemq:Output"/>
</route>
enrich
pollEnrich
enrich is using a Producer to obtain the additional data. It is usually used for Request
Reply messaging, for instance to invoke an external web service.
pollEnrich on the other hand is using a Polling Consumer to obtain the additional data.
It is usually used for Event Message messaging, for instance to read a file or download a
FTP file.
Enrich Options
Default
Name Description
Value
The endpoint uri for the external service to enrich from. You must
uri
use either uri or ref.
Refers to the endpoint for the external service to enrich from. You
ref
must use either uri or ref.
from("direct:resource")
...
The content enricher (enrich) retrieves additional data from a resource endpoint in
order to enrich an incoming message (contained in the original exchange). An
aggregation strategy is used to combine the original exchange and the resource
exchange. The first parameter of the AggregationStrategy.aggregate(Exchange,
Exchange) method corresponds to the the original exchange, the second parameter the
resource exchange. The results from the resource endpoint are stored in the resource
exchange's out-message. Here's an example template for implementing an aggregation
strategy:
Using this template the original exchange can be of any pattern. The resource exchange
created by the enricher is always an in-out exchange.
<camelContext id="camel"
xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="direct:start"/>
<enrich uri="direct:resource" strategyRef="aggregationStrategy"/>
<to uri="direct:result"/>
</route>
<route>
<from uri="direct:resource"/>
...
</route>
</camelContext>
from("direct:start")
.enrich("direct:resource")
.to("direct:result");
In the route above the message sent to the direct:result endpoint will contain the
output from the direct:resource as we do not use any custom aggregation.
<route>
<from uri="direct:start"/>
<enrich uri="direct:resource"/>
<to uri="direct:result"/>
</route>
The pollEnrich works just as the enrich however as it uses a Polling Consumer we
have 3 methods when polling
receive
receiveNoWait
receive(timeout)
PollEnrich Options
Default
Name Description
Value
The endpoint uri for the external service to enrich from. You must
uri
use either uri or ref.
Refers to the endpoint for the external service to enrich from. You
ref
must use either uri or ref.
Timeout in millis when polling from the external service. See below
timeout 0
for important details about the timeout.
You can pass in a timeout value that determines which method to use
if timeout is -1 or other negative number then receive is selected
if timeout is 0 then receiveNoWait is selected
otherwise receive(timeout) is selected
Example
In this example we enrich the message by loading the content from the file named
inbox/data.txt.
from("direct:start")
.pollEnrich("file:inbox?fileName=data.txt")
.to("direct:result");
<route>
<from uri="direct:start"/>
<pollEnrich uri="file:inbox?fileName=data.txt"/>
<to uri="direct:result"/>
</route>
If there is no file then the message is empty. We can use a timeout to either wait
(potentially forever) until a file exists, or use a timeout to wait a certain period.
<route>
<from uri="direct:start"/>
<pollEnrich uri="file:inbox?fileName=data.txt" timeout="5000"/>
<to uri="direct:result"/>
</route>
Content Filter
Camel supports the Content Filter from the EIP patterns using one of the following
mechanisms in the routing logic to transform content from the inbound message.
Message Translator
invoking a Java bean
Processor object
A common way to filter messages is to use an Expression in the DSL like XQuery, SQL
or one of the supported Scripting Languages.
from("direct:start").setBody(body().append("
World!")).to("mock:result");
from("direct:start").process(new Processor() {
public void process(Exchange exchange) {
Message in = exchange.getIn();
in.setBody(in.getBody(String.class) + " World!");
}
}).to("mock:result");
For further examples of this pattern in use you could look at one of the JUnit tests
TransformTest
TransformViaDSLTest
<route>
<from uri="activemq:Input"/>
<bean ref="myBeanName" method="doTransform"/>
<to uri="activemq:Output"/>
</route>
You can also use XPath to filter out part of the message you are interested in:
<route>
<from uri="activemq:Input"/>
<setBody><xpath
resultType="org.w3c.dom.Document">//foo:bar</xpath></setBody>
<to uri="activemq:Output"/>
</route>
Claim Check
The Claim Check from the EIP patterns allows you to replace message content with a
claim check (a unique key), which can be used to retrieve the message content at a later
time. The message content is stored temporarily in a persistent store like a database or
file system. This pattern is very useful when message content is very large (thus it
would be expensive to send around) and not all components require all information.
It can also be useful in situations where you cannot trust the information with an outside
party; in this case, you can use the Claim Check to hide the sensitive portions of data.
Example
In this example we want to replace a message body with a claim check, and restore the
body at a later step.
from("direct:start").to("bean:checkLuggage", "mock:testCheckpoint",
"bean:dataEnricher", "mock:result");
<route>
<from uri="direct:start"/>
<pipeline>
<to uri="bean:checkLuggage"/>
<to uri="mock:testCheckpoint"/>
<to uri="bean:dataEnricher"/>
<to uri="mock:result"/>
</pipeline>
</route>
The example route is pretty simple - its just a Pipeline. In a real application you would
have some other steps where the mock:testCheckpoint endpoint is in the example.
The message is first sent to the checkLuggage bean which looks like
This bean stores the message body into the data store, using the custId as the claim
check. In this example, we're just using a HashMap to store the message body; in a real
application you would use a database or file system, etc. Next the claim check is added
as a message header for use later. Finally we remove the body from the message and
pass it down the pipeline.
The next step in the pipeline is the mock:testCheckpoint endpoint which is just used
to check that the message body is removed, claim check added, etc.
To add the message body back into the message, we use the dataEnricher bean which
looks like
This bean queries the data store using the claim check as the key and then adds the data
back into the message. The message body is then removed from the data store and
finally the claim check is removed. Now the message is back to what we started with!
camel-core/src/test/java/org/apache/camel/processor/ClaimCheckTest.java
Normalizer
Camel supports the Normalizer from the EIP patterns by using a Message Router in
front of a number of Message Translator instances.
Example
This example shows a Message Normalizer that converts two types of XML messages
into a common format. Messages in this common format are then filtered.
.when().xpath("/employee").to("bean:normalizer?method=employeeToPerson
")
.when().xpath("/customer").to("bean:normalizer?method=customerToPerson
")
.end()
.to("mock:result");
In this case we're using a Java bean as the normalizer. The class looks like this
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="direct:start"/>
<choice>
<when>
<xpath>/employee</xpath>
<to uri="bean:normalizer?method=employeeToPerson"/>
</when>
<when>
<xpath>/customer</xpath>
<to uri="bean:normalizer?method=customerToPerson"/>
</when>
</choice>
<to uri="mock:result"/>
</route>
</camelContext>
<bean id="normalizer"
class="org.apache.camel.processor.MyNormalizer"/>
Sort
Sort can be used to sort a message. Imagine you consume text files and before
processing each file you want to be sure the content is sorted.
Sort will by default sort the body using a default comparator that handles numeric
values or uses the string representation. You can provide your own comparator, and
even an expression to return the value to be sorted. Sort requires the value returned from
the expression evaluation is convertible to java.util.List as this is required by the
JDK sort operation.
Options
Default
Name Description
Value
In the route below it will read the file content and tokenize by line breaks so each line
can be sorted.
from("file://inbox").sort(body().tokenize("\n")).to("bean:MyServiceBea
n.processLine");
from("file://inbox").sort(body().tokenize("\n"), new
MyReverseComparator()).to("bean:MyServiceBean.processLine");
In the route below it will read the file content and tokenize by line breaks so each line
can be sorted.
<route>
<from uri="file://inbox"/>
<sort>
<simple>body</simple>
</sort>
<beanRef ref="myServiceBean" method="processLine"/>
</route>
Camel 2.6 or older
<route>
<from uri="file://inbox"/>
<sort>
<expression>
<simple>body</simple>
</expression>
</sort>
<beanRef ref="myServiceBean" method="processLine"/>
</route>
<route>
<from uri="file://inbox"/>
<sort comparatorRef="myReverseComparator">
<simple>body</simple>
</sort>
<beanRef ref="MyServiceBean" method="processLine"/>
</route>
<bean id="myReverseComparator"
class="com.mycompany.MyReverseComparator"/>
Camel 2.6 or older
<route>
<from uri="file://inbox"/>
<sort comparatorRef="myReverseComparator">
<expression>
<simple>body</simple>
</expression>
</sort>
<beanRef ref="MyServiceBean" method="processLine"/>
</route>
<bean id="myReverseComparator"
class="com.mycompany.MyReverseComparator"/>
Besides <simple>, you can supply an expression using any language you like, so long
as it returns a list.
If you would like to use this EIP Pattern then please read the Getting Started, you may
also find the Architecture useful particularly the description of Endpoint and URIs.
Then you could try out some of the Examples first before trying this pattern out.
Validate
You can use the validate DSL with all kind of Predicates and Expressions. Validate
evaluates the Predicate/Expression and if it is false a PredicateValidationException
is thrown. If it is true message processing continues.
The route below will read the file contents and validate them against a regular
expression.
from("file://inbox")
.validate(body(String.class).regex("^\\w{10}\\,\\d{2}\\,\\w{24}$"))
.to("bean:MyServiceBean.processLine");
Validate is not limited to the message body. You can also validate the message header.
from("file://inbox")
.validate(header("bar").isGreaterThan(100))
.to("bean:MyServiceBean.processLine");
from("file://inbox")
.validate(simple("${in.header.bar} == 100"))
.to("bean:MyServiceBean.processLine");
To use validate in the Spring DSL, the easiest way is to use simple expressions.
<route>
<from uri="file://inbox"/>
<validate>
<simple>${body} regex ^\\w{10}\\,\\d{2}\\,\\w{24}$</simple>
</validate>
<beanRef ref="myServiceBean" method="processLine"/>
</route>
The XML DSL to validate the message header would looks like this:
<route>
<from uri="file://inbox"/>
<validate>
<simple>${in.header.bar} == 100</simple>
</validate>
<beanRef ref="myServiceBean" method="processLine"/>
</route>
Messaging Endpoints
Messaging Mapper
Camel supports the Messaging Mapper from the EIP patterns by using either Message
Translator pattern or the Type Converter module.
See also
Message Translator
Type Converter
CXF for JAX-WS support for binding business logic to messaging & web services
Pojo
Bean
Event Driven Consumer
Camel supports the Event Driven Consumer from the EIP patterns. The default
consumer model is event based (i.e. asynchronous) as this means that the Camel
container can then manage pooling, threading and concurrency for you in a declarative
manner.
Message
Message Endpoint
Polling Consumer
Camel supports implementing the Polling Consumer from the EIP patterns using the
PollingConsumer interface which can be created via the
Endpoint.createPollingConsumer() method.
ConsumerTemplate
The template supports the 3 operations above, but also including convenient methods
for returning the body, etc consumeBody.
The example from above using ConsumerTemplate is:
And you can provide the body type as a parameter and have it returned as the type:
With the Spring DSL we can declare the consumer in the CamelContext with the
consumerTemplate tag, just like the ProducerTemplate. The example below illustrates
this:
<camelContext xmlns="http://camel.apache.org/schema/spring">
<!-- define a producer template -->
<template id="producer"/>
<!-- define a consumer template -->
<consumerTemplate id="consumer"/>
<route>
<from uri="seda:foo"/>
<to id="result" uri="mock:result"/>
</route>
</camelContext>
Then we can get leverage Spring to inject the ConsumerTemplate in our java class. The
code below is part of an unit test but it shows how the consumer and producer can work
together.
@ContextConfiguration
public class SpringConsumerTemplateTest extends
AbstractJUnit38SpringContextTests {
@Autowired
private ProducerTemplate producer;
@Autowired
private ConsumerTemplate consumer;
@EndpointInject(ref = "result")
private MockEndpoint mock;
In this sample we use a Timer to schedule a route to be started every 5th second and
invoke our bean MyCoolBean where we implement the business logic for the Polling
Consumer. Here we want to consume all messages from a JMS queue, process the
message and send them to the next queue.
from("timer://foo?period=5000").bean(cool, "someBusinessLogic");
from("activemq:queue.foo").to("mock:result");
Quite a few inbound Camel endpoints use a scheduled poll pattern to receive messages
and push them through the Camel processing routes. That is to say externally from the
client the endpoint appears to use an Event Driven Consumer but internally a scheduled
poll is used to monitor some kind of state or resource and then fire message exchanges.
Since this a such a common pattern, polling components can extend the
ScheduledPollConsumer base class which makes it simpler to implement this pattern.
There is also the Quartz Component which provides scheduled delivery of messages
using the Quartz enterprise scheduler.
PollingConsumer
Scheduled Polling Components
o ScheduledPollConsumer
o Atom
o File
o FTP
o iBATIS
o JPA
o Mail
o MyBatis
o Quartz
o SNMP
o AWS-S3
o AWS-SQS
ScheduledPollConsumer Options
Defaul
Option Description
t
A pluggable
org.apache.camel.PollingConsumerPollStrate
gy allowing you to provide your custom implementation
to control error handling usually occurred during the
poll operation before an Exchange have been created
pollStrategy
and being routed in Camel. In other words the error
occurred while the polling was gathering information, for
instance access to a file network failed so Camel cannot
access it to scan for files. The default implementation will
log the caused exception at WARN level and ignore it.
Camel 2.9: If the polling consumer did not poll any files,
sendEmptyMessageWhenIdl
false you can enable this option to send an empty message
e
(no body) instead.
begin
void begin(Consumer consumer, Endpoint endpoint)
begin (Camel 2.3)
boolean begin(Consumer consumer, Endpoint endpoint)
commit
void commit(Consumer consumer, Endpoint endpoint)
commit (Camel 2.6)
void commit(Consumer consumer, Endpoint endpoint, int
polledMessages)
rollback
boolean rollback(Consumer consumer, Endpoint endpoint, int
retryCounter, Exception e) throws Exception
In Camel 2.3 onwards the begin method returns a boolean which indicates whether or
not to skipping polling. So you can implement your custom logic and return false if
you do not want to poll this time.
In Camel 2.6 onwards the commit method has an additional parameter containing the
number of message that was actually polled. For example if there was no messages
polled, the value would be zero, and you can react accordingly.
The most interesting is the rollback as it allows you do handle the caused exception
and decide what to do.
Notice that we are given the Consumer as a parameter. We could use this to restart the
consumer as we can invoke stop and start:
from("file://inbox/?pollStrategy=#myPoll").to("activemq:queue:inbox")
Competing Consumers
Camel supports the Competing Consumers from the EIP patterns using a few different
components.
from("jms:MyQueue?concurrentConsumers=5").bean(SomeBean.class);
or in Spring DSL
<route>
<from uri="jms:MyQueue?concurrentConsumers=5"/>
<to uri="bean:someBean"/>
</route>
If you would like to use this EIP Pattern then please read the Getting Started, you may
also find the Architecture useful particularly the description of Endpoint and URIs.
Then you could try out some of the Examples first before trying this pattern out.
Message Dispatcher
Camel supports the Message Dispatcher from the EIP patterns using various
approaches.
You can use a component like JMS with selectors to implement a Selective Consumer
as the Message Dispatcher implementation. Or you can use an Endpoint as the Message
Dispatcher itself and then use a Content Based Router as the Message Dispatcher.
See Also
JMS
Selective Consumer
Content Based Router
Endpoint
Selective Consumer
The Selective Consumer from the EIP patterns can be implemented in two ways
The first solution is to provide a Message Selector to the underlying URIs when
creating your consumer. For example when using JMS you can specify a selector
parameter so that the message broker will only deliver messages matching your criteria.
The other approach is to use a Message Filter which is applied; then if the filter matches
the message your consumer is invoked as shown in the following example
from("seda:a")
.filter(header("foo").isEqualTo("bar"))
.process(myProcessor);
}
};
<camelContext errorHandlerRef="errorHandler"
xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="seda:a"/>
<filter>
<xpath>$foo = 'bar'</xpath>
<process ref="myProcessor"/>
</filter>
</route>
</camelContext>
Durable Subscriber
Camel supports the Durable Subscriber from the EIP patterns using the JMS component
which supports publish & subscribe using Topics with support for non-durable and
durable subscribers.
Another alternative is to combine the Message Dispatcher or Content Based Router with
File or JPA components for durable subscribers then something like SEDA for non-
durable.
from("activemq:topic:foo?clientId=1&durableSubscriptionName=bar1").to(
"mock:result1");
from("activemq:topic:foo?clientId=2&durableSubscriptionName=bar2").to(
"mock:result2");
<route>
<from
uri="activemq:topic:foo?clientId=1&durableSubscriptionName=bar1"/>
<to uri="mock:result1"/>
</route>
<route>
<from
uri="activemq:topic:foo?clientId=2&durableSubscriptionName=bar2"/>
<to uri="mock:result2"/>
</route>
Here is another example of JMS durable subscribers, but this time using virtual topics
(recommended by AMQ over durable subscriptions)
from("activemq:queue:Consumer.1.VirtualTopic.foo").to("mock:result1");
from("activemq:queue:Consumer.2.VirtualTopic.foo").to("mock:result2");
<route>
<from uri="activemq:queue:Consumer.1.VirtualTopic.foo"/>
<to uri="mock:result1"/>
</route>
<route>
<from uri="activemq:queue:Consumer.2.VirtualTopic.foo"/>
<to uri="mock:result2"/>
</route>
See Also
JMS
File
JPA
Message Dispatcher
Selective Consumer
Content Based Router
Endpoint
Idempotent Consumer
The Idempotent Consumer from the EIP patterns is used to filter out duplicate
messages.
The Idempotent Consumer essentially acts like a Message Filter to filter out duplicates.
Camel will add the message id eagerly to the repository to detect duplication also for
Exchanges currently in progress.
On completion Camel will remove the message id from the repository if the Exchange
failed, otherwise it stays there.
MemoryIdempotentRepository
FileIdempotentRepository
HazelcastIdempotentRepository (Available as of Camel 2.8)
JdbcMessageIdRepository (Available as of Camel 2.7)
JpaMessageIdRepository
Options
The following example will use the header myMessageId to filter out duplicates
from("seda:a")
.idempotentConsumer(header("myMessageId"),
MemoryIdempotentRepository.memoryIdempotentRepository(200))
.to("seda:b");
}
};
The above example will use an in-memory based MessageIdRepository which can
easily run out of memory and doesn't work in a clustered environment. So you might
prefer to use the JPA based implementation which uses a database to store the message
IDs which have been processed
from("direct:start").idempotentConsumer(
header("messageId"),
jpaMessageIdRepository(lookup(JpaTemplate.class),
PROCESSOR_NAME)
).to("mock:result");
In the above example we are using the header messageId to filter out duplicates and
using the collection myProcessorName to indicate the Message ID Repository to use.
This name is important as you could process the same message by many different
processors; so each may require its own logical Message ID Repository.
For further examples of this pattern in use you could look at the junit test case
The following example will use the header myMessageId to filter out duplicates
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="direct:start"/>
<idempotentConsumer messageIdRepositoryRef="myRepo">
<!-- use the messageId header as key for identifying
duplicate messages -->
<header>messageId</header>
<!-- if not a duplicate send it to this mock endpoint -->
<to uri="mock:result"/>
</idempotentConsumer>
</route>
</camelContext>
You can now set the skipDuplicate option to false which instructs the idempotent
consumer to route duplicate messages as well. However the duplicate message has been
marked as duplicate by having a property on the Exchange set to true. We can leverage
this fact by using a Content Based Router or Message Filter to detect this and handle
duplicate messages.
For example in the following example we use the Message Filter to send the message to
a duplicate endpoint, and then stop continue routing that message.
Filter duplicate messages
from("direct:start")
// instruct idempotent consumer to not skip duplicates as we will
filter then our self
.idempotentConsumer(header("messageId")).messageIdRepository(repo).ski
pDuplicate(false)
.filter(property(Exchange.DUPLICATE_MESSAGE).isEqualTo(true))
// filter out duplicate messages by sending them to someplace
else and then stop
.to("mock:duplicate")
.stop()
.end()
// and here we process only new messages (no duplicates)
.to("mock:result");
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="direct:start"/>
<!-- we do not want to skip any duplicate messages -->
<idempotentConsumer messageIdRepositoryRef="myRepo"
skipDuplicate="false">
<!-- use the messageId header as key for identifying
duplicate messages -->
<header>messageId</header>
<!-- we will to handle duplicate messages using a filter -
->
<filter>
<!-- the filter will only react on duplicate messages,
if this property is set on the Exchange -->
<property>CamelDuplicateMessage</property>
<!-- and send the message to this mock, due its part
of an unit test -->
<!-- but you can of course do anything as its part of
the route -->
<to uri="mock:duplicate"/>
<!-- and then stop -->
<stop/>
</filter>
<!-- here we route only new messages -->
<to uri="mock:result"/>
</idempotentConsumer>
</route>
</camelContext>
from("direct:in").idempotentConsumer(header("messageId"),
idempotentRepo).to("mock:out");
You have to define how long the repository should hold each message id (default is to
delete it never). To avoid that you run out of memory you should create an eviction
strategy based on the Hazelcast configuration. For additional information see camel-
hazelcast.
See this little tutorial, how setup such an idempotent repository on two cluster nodes
using Apache Karaf.
Transactional Client
Camel recommends supporting the Transactional Client from the EIP patterns using
spring transactions.
Transaction Oriented Endpoints (Camel Toes) like JMS support using a transaction for
both inbound and outbound message exchanges. Endpoints that support transactions
will participate in the current transaction context that they are called from.
Configuration of Redelivery
The redelivery in transacted mode is not handled by Camel but by the backing system (the
transaction manager). In such cases you should resort to the backing system how to
configure the redelivery.
You should use the SpringRouteBuilder to setup the routes since you will need to setup
the spring context with the TransactionTemplates that will define the transaction
manager configuration and policies.
<bean id="jmsTransactionManager"
class="org.springframework.jms.connection.JmsTransactionManager">
<property name="connectionFactory" ref="jmsConnectionFactory" />
</bean>
<bean id="jmsConnectionFactory"
class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="tcp://localhost:61616"/>
</bean>
Then you look them up and use them to create the JmsComponent.
PlatformTransactionManager transactionManager =
(PlatformTransactionManager) spring.getBean("jmsTransactionManager");
ConnectionFactory connectionFactory = (ConnectionFactory)
spring.getBean("jmsConnectionFactory");
JmsComponent component =
JmsComponent.jmsComponentTransacted(connectionFactory,
transactionManager);
component.getConfiguration().setConcurrentConsumers(1);
ctx.addComponent("activemq", component);
Transaction Policies
Outbound endpoints will automatically enlist in the current transaction context. But
what if you do not want your outbound endpoint to enlist in the same transaction as
your inbound endpoint? The solution is to add a Transaction Policy to the processing
route. You first have to define transaction policies that you will be using. The policies
use a spring TransactionTemplate under the covers for declaring the transaction
demarcation to use. So you will need to add something like the following to your spring
xml:
<bean id="PROPAGATION_REQUIRED"
class="org.apache.camel.spring.spi.SpringTransactionPolicy">
<property name="transactionManager" ref="jmsTransactionManager"/>
</bean>
<bean id="PROPAGATION_REQUIRES_NEW"
class="org.apache.camel.spring.spi.SpringTransactionPolicy">
<property name="transactionManager" ref="jmsTransactionManager"/>
<property name="propagationBehaviorName"
value="PROPAGATION_REQUIRES_NEW"/>
</bean>
Once created, you can use the Policy objects in your processing routes:
from("activemq:queue:foo").policy(requirenew).to("activemq:queue:bar")
;
OSGi Blueprint
If you are using OSGi Blueprint then you most likely have to explicit declare a policy
and refer to the policy from the transacted in the route.
<bean id="required"
class="org.apache.camel.spring.spi.SpringTransactionPolicy">
<property name="transactionManager" ref="jmsTransactionManager"/>
<property name="propagationBehaviorName"
value="PROPAGATION_REQUIRED"/>
</bean>
<route>
<from uri="activemq:queue:foo"/>
<transacted ref="required"/>
<to uri="activemq:queue:bar"/>
</route>
In this sample we want to ensure that two endpoints is under transaction control. These
two endpoints inserts data into a database.
The sample is in its full as a unit test.
First of all we setup the usual spring stuff in its configuration file. Here we have defined
a DataSource to the HSQLDB and a most importantly
the Spring DataSoruce TransactionManager that is doing the heavy lifting of ensuring
our transactional policies. You are of course free to use any
of the Spring based TransactionMananger, eg. if you are in a full blown J2EE container
you could use JTA or the WebLogic or WebSphere specific managers.
We use the required transaction policy that we define as the PROPOGATION_REQUIRED
spring bean. And as last we have our book service bean that does the business logic
and inserts data in the database as our core business logic.
<!-- policy for required transaction used in our Camel routes -->
<bean id="PROPAGATION_REQUIRED"
class="org.apache.camel.spring.spi.SpringTransactionPolicy">
<property name="transactionManager" ref="txManager"/>
<property name="propagationBehaviorName"
value="PROPAGATION_REQUIRED"/>
</bean>
In our Camel route that is Java DSL based we setup the transactional policy, wrapped as
a Policy.
As its a unit test we need to setup the database and this is easily done with Spring
JdbcTemplate
And our core business service, the book service, will accept any books except the
Donkeys.
public BookService() {
}
And lastly the rollback condition since the 2nd book is a Donkey book:
In this sample we want to listen for messages on a queue and process the messages with
our business logic java code and send them along. Since its based on a unit test the
destination is a mock endpoint.
This time we want to setup the camel context and routes using the Spring XML syntax.
Since the rest is standard XML stuff its nothing fancy now for the reader:
<bean id="poolConnectionFactory"
class="org.apache.activemq.pool.PooledConnectionFactory">
<property name="maxConnections" value="8"/>
<property name="connectionFactory" ref="jmsConnectionFactory"/>
</bean>
<bean id="jmsConnectionFactory"
class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL"
value="vm://localhost?broker.persistent=false&broker.useJmx=false"
/>
</bean>
<bean id="jmsTransactionManager"
class="org.springframework.jms.connection.JmsTransactionManager">
<property name="connectionFactory" ref="poolConnectionFactory"/>
</bean>
<bean id="jmsConfig"
class="org.apache.camel.component.jms.JmsConfiguration">
<property name="connectionFactory" ref="poolConnectionFactory"/>
<property name="transactionManager" ref="jmsTransactionManager"/>
<property name="transacted" value="true"/>
<property name="concurrentConsumers" value="1"/>
</bean>
<bean id="activemq"
class="org.apache.activemq.camel.component.ActiveMQComponent">
<property name="configuration" ref="jmsConfig"/>
</bean>
<bean id="PROPAGATION_REQUIRED"
class="org.apache.camel.spring.spi.SpringTransactionPolicy">
<property name="transactionManager" ref="jmsTransactionManager"/>
</bean>
Our business logic is set to handle the incomming messages and fail the first two times.
When its a success it responds with a Bye World message.
And our unit test is tested with this java code. Notice that we expect the Bye World
message to be delivered at the 3rd attempt.
mock.assertIsSatisfied();
Camel 1.x - Spring based configuration
In Camel 1.4 we have introduced the concept of configuration of the error handlers
using spring XML configuration. The sample below demonstrates that you can
configure transaction error handlers in Spring XML as spring beans. These can then be
set as global, per route based or per policy based error handler. The latter has been
demonstrated in the samples above. This sample is the database sample configured in
Spring XML.
Notice that we have defined two error handler, one per route. The first route uses the
transaction error handler, and the 2nd uses no error handler at all.
</camel:camelContext>
The following snippet is the Spring XML configuration to setup the error handlers in
pure spring XML:
<!-- the transaction error handle we refer to from the route -->
<bean id="transactionErrorHandler"
class="org.apache.camel.spring.spi.TransactionErrorHandlerBuilder">
<property name="transactionTemplate" ref="PROPAGATION_REQUIRED"/>
</bean>
DelayPolicy (@deprecated)
This allows a simple redelivery interval that can be configured for development mode or
light production to avoid a rapid redelivery strategy that can exhaust a system that
constantly fails.
We strongly recommend that you configure the backing system for correct
redelivery policy in your environment.
In this sample we want to ensure that two endpoints is under transaction control. These
two endpoints inserts data into a database.
The sample is in its full as a unit test.
First of all we setup the usual spring stuff in its configuration file. Here we have defined
a DataSource to the HSQLDB and a most importantly
the Spring DataSoruce TransactionManager that is doing the heavy lifting of ensuring
our transactional policies. You are of course free to use any
of the Spring based TransactionMananger, eg. if you are in a full blown J2EE container
you could use JTA or the WebLogic or WebSphere specific managers.
Then we are ready to define our Camel routes. We have two routes: 1 for success
conditions, and 1 for a forced rollback condition.
This is after all based on a unit test. Notice that we mark each route as transacted using
the transacted tag.
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="direct:okay"/>
<!-- we mark this route as transacted. Camel will lookup the
spring transaction manager
and use it by default. We can optimally pass in arguments
to specify a policy to use
that is configured with a spring transaction manager of
choice. However Camel supports
convention over configuration as we can just use the
defaults out of the box and Camel
that suites in most situations -->
<transacted/>
<setBody>
<constant>Tiger in Action</constant>
</setBody>
<bean ref="bookService"/>
<setBody>
<constant>Elephant in Action</constant>
</setBody>
<bean ref="bookService"/>
</route>
<route>
<from uri="direct:fail"/>
<!-- we mark this route as transacted. See comments above. -->
<transacted/>
<setBody>
<constant>Tiger in Action</constant>
</setBody>
<bean ref="bookService"/>
<setBody>
<constant>Donkey in Action</constant>
</setBody>
<bean ref="bookService"/>
</route>
</camelContext>
That is all that is needed to configure a Camel route as being transacted. Just remember
to use the transacted DSL. The rest is standard Spring XML to setup the transaction
manager.
Camel 2.0 - JMS Sample
In this sample we want to listen for messages on a queue and process the messages with
our business logic java code and send them along. Since its based on a unit test the
destination is a mock endpoint.
First we configure the standard Spring XML to declare a JMS connection factory, a
JMS transaction manager and our ActiveMQ component that we use in our routing.
<bean id="jmsConnectionFactory"
class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL"
value="vm://localhost?broker.persistent=false&broker.useJmx=false"
/>
</bean>
And then we configure our routes. Notice that all we have to do is mark the route as
transacted using the transacted tag.
<camelContext xmlns="http://camel.apache.org/schema/spring">
<!-- disable JMX during testing -->
<jmxAgent id="agent" disabled="true"/>
<route>
<!-- 1: from the jms queue -->
<from uri="activemq:queue:okay"/>
<!-- 2: mark this route as transacted -->
<transacted/>
<!-- 3: call our business logic that is myProcessor -->
<process ref="myProcessor"/>
<!-- 4: if success then send it to the mock -->
<to uri="mock:result"/>
</route>
</camelContext>
<bean id="myProcessor"
class="org.apache.camel.component.jms.tx.JMSTransactionalClientTest$My
Processor"/>
Transaction error handler
When a route is marked as transacted using transacted Camel will automatic use the
TransactionErrorHandler as Error Handler. It supports basically the same feature set as the
DefaultErrorHandler, so you can for instance use Exception Clause as well.
<bean id="PROPAGATION_REQUIRED"
class="org.apache.camel.spring.spi.SpringTransactionPolicy">
<property name="transactionManager" ref="txManager"/>
<property name="propagationBehaviorName"
value="PROPAGATION_REQUIRED"/>
</bean>
<bean id="PROPAGATION_REQUIRES_NEW"
class="org.apache.camel.spring.spi.SpringTransactionPolicy">
<property name="transactionManager" ref="txManager"/>
<property name="propagationBehaviorName"
value="PROPAGATION_REQUIRES_NEW"/>
</bean>
Then in the routes you use transacted DSL to indicate which of these two propagations
it uses.
from("direct:mixed")
// using required
.transacted("PROPAGATION_REQUIRED")
// all these steps will be okay
.setBody(constant("Tiger in Action")).beanRef("bookService")
.setBody(constant("Elephant in Action")).beanRef("bookService")
// continue on route 2
.to("direct:mixed2");
from("direct:mixed2")
// tell Camel that if this route fails then only rollback this
last route
// by using (rollback only *last*)
.onException(Exception.class).markRollbackOnlyLast().end()
// using a different propagation which is requires new
.transacted("PROPAGATION_REQUIRES_NEW")
// this step will be okay
.setBody(constant("Lion in Action")).beanRef("bookService")
// this step will fail with donkey
.setBody(constant("Donkey in Action")).beanRef("bookService");
Notice how we have configured the onException in the 2nd route to indicate in case of
any exceptions we should handle it and just rollback this transaction.
This is done using the markRollbackOnlyLast which tells Camel to only do it for the
current transaction and not globally.
See Also
Messaging Gateway
Camel has several endpoint components that support the Messaging Gateway from the
EIP patterns.
Components like Bean and CXF provide a a way to bind a Java interface to the message
exchange.
However you may want to read the Using CamelProxy documentation as a true
Messaging Gateway EIP solution.
Another approach is to use @Produce which you can read about in POJO Producing
which also can be used as a Messaging Gateway EIP solution.
See Also
Bean
CXF
Using CamelProxy
POJO Producing
Spring Remoting
Service Activator
Camel has several endpoint components that support the Service Activator from the EIP
patterns.
Components like Bean, CXF and Pojo provide a a way to bind the message exchange to
a Java interface/service where the route defines the endpoints and wires it up to the
bean.
In addition you can use the Bean Integration to wire messages to a bean using
annotation.
See Also
Bean
Pojo
CXF
System Management
Detour
The Detour from the EIP patterns allows you to send messages through additional steps
if a control condition is met. It can be useful for turning on extra validation, testing,
debugging code when needed.
Example
from("direct:start").choice()
.when().method("controlBean", "isDetour").to("mock:detour").end()
.to("mock:result");
Using the Spring XML Extensions
<route>
<from uri="direct:start"/>
<choice>
<when>
<method bean="controlBean" method="isDetour"/>
<to uri="mock:detour"/>
</when>
</choice>
<to uri="mock:result"/>
</split>
</route>
whether the detour is turned on or off is decided by the ControlBean. So, when the
detour is on the message is routed to mock:detour and then mock:result. When the
detour is off, the message is routed to mock:result.
camel-core/src/test/java/org/apache/camel/processor/DetourTest.java
Wire Tap
Wire Tap (from the EIP patterns) allows you to route messages to a separate location
while they are being forwarded to the ultimate destination.
Streams
If you Wire Tap a stream message body then you should consider enabling Stream caching
to ensure the message body can be read at each endpoint. See more details at Stream
caching.
Options
Default
Name Description
Value
The Wire Tap uses a thread pool to process the tapped messages. This thread pool will
by default use the settings detailed at Threading Model. In particular, when the pool is
exhausted (with all threads utilized), further wiretaps will be executed synchronously by
the calling thread. To remedy this, you can configure an explicit thread pool on the Wire
Tap having either a different rejection policy, a larger worker queue, or more worker
threads.
WireTap node
Camel's Wire Tap node supports two flavors when tapping an Exchange:
-With the traditional Wire Tap, Camel will copy the original Exchange and set its
Exchange Pattern to InOnly, as we want the tapped Exchange to be sent in a fire and
forget style. The tapped Exchange is then sent in a separate thread so it can run in
parallel with the original.
-Camel also provides an option of sending a new Exchange allowing you to populate it
with new values.
From Camel 2.3 onwards the Expression or Processor is pre-populated with a copy of
the original Exchange, which allows you to access the original message when you
prepare a new Exchange to be sent. You can use the copy option (enabled by default) to
indicate whether you want this. If you set copy=false, then it works as in Camel 2.2 or
older where the Exchange will be empty.
Below is the processor variation. This example is from Camel 2.3, where we disable
copy by passing in false to create a new, empty Exchange.
from("direct:start")
.wireTap("direct:foo", false, new Processor() {
public void process(Exchange exchange) throws Exception {
exchange.getIn().setBody("Bye World");
exchange.getIn().setHeader("foo", "bar");
}
}).to("mock:result");
from("direct:foo").to("mock:foo");
Here is the Expression variation. This example is from Camel 2.3, where we disable
copy by passing in false to create a new, empty Exchange.
from("direct:start")
.wireTap("direct:foo", false, constant("Bye World"))
.to("mock:result");
from("direct:foo").to("mock:foo");
<route>
<from uri="direct:start2"/>
<wireTap uri="direct:foo" processorRef="myProcessor"/>
<to uri="mock:result"/>
</route>
Here is the Expression variation, where the expression is defined in the body tag:
<route>
<from uri="direct:start"/>
<wireTap uri="direct:foo">
<body><constant>Bye World</constant></body>
</wireTap>
<to uri="mock:result"/>
</route>
This variation accesses the body of the original message and creates a new Exchange
based on the Expression. It will create a new Exchange and have the body contain "Bye
ORIGINAL BODY MESSAGE HERE"
<route>
<from uri="direct:start"/>
<wireTap uri="direct:foo">
<body><simple>Bye ${body}</simple></body>
</wireTap>
<to uri="mock:result"/>
</route>
Further Example
For another example of this pattern, refer to the wire tap test case.
If you send a new message using Wire Tap, then you could only set the message body
using an Expression from the DSL. If you also need to set headers, you would have to
use a Processor. In Camel 2.8 onwards, you can now set headers as well in the DSL.
Java DSL
from("direct:start")
// tap a new message and send it to direct:tap
// the new message should be Bye World with 2 headers
.wireTap("direct:tap")
// create the new tap message body and headers
.newExchangeBody(constant("Bye World"))
.newExchangeHeader("id", constant(123))
.newExchangeHeader("date", simple("${date:now:yyyyMMdd}"))
.end()
// here we continue routing the original messages
.to("mock:result");
XML DSL
The XML DSL is slightly different than Java DSL in how you configure the message
body and headers using <body> and <setHeader>:
<route>
<from uri="direct:start"/>
<!-- tap a new message and send it to direct:tap -->
<!-- the new message should be Bye World with 2 headers -->
<wireTap uri="direct:tap">
<!-- create the new tap message body and headers -->
<body><constant>Bye World</constant></body>
<setHeader
headerName="id"><constant>123</constant></setHeader>
<setHeader
headerName="date"><simple>${date:now:yyyyMMdd}</simple></setHeader>
</wireTap>
<!-- here we continue routing the original message -->
<to uri="mock:result"/>
</route>
Log
Camel provides many ways to log processing a message. Here is just some examples:
You can use the Log component which logs the Message content.
You can use the Tracer which trace logs message flow.
You can also use a Processor or Bean and log from Java code.
You can use the log DSL.
And in Camel 2.2 you can use the log DSL which allows you to use Simple language
to construct a dynamic message which gets logged.
For example you can do
from("direct:start").log("Processing ${id}").to("bean:foo");
Which will construct a String message at runtime using the Simple language. The log
message will by logged at INFO level using the route id as the log name. By default a
route is named route-1, route-2 etc. But you can use the routeId("myCoolRoute")
to set a route name of choice.
The log DSL have overloaded methods to set the logging level and/or name as well.
from("direct:start").log(LoggingLevel.DEBUG, "Processing
${id}").to("bean:foo");
For example you can use this to log the file name being processed if you consume files.
In Spring DSL its also easy to use log DSL as shown below:
<route id="foo">
<from uri="direct:foo"/>
<log message="Got ${body}"/>
<to uri="mock:foo"/>
</route>
The log tag has attributes to set the message, loggingLevel and logName. For
example:
<route id="baz">
<from uri="direct:baz"/>
<log message="Me Got ${body}" loggingLevel="FATAL"
logName="cool"/>
<to uri="mock:baz"/>
</route>
<route id="baz">
<from uri="direct:baz"/>
<log message="Me Got ${body}" loggingLevel="FATAL"
logName="cool" marker="myMarker"/>
<to uri="mock:baz"/>
</route>