Tuesday, November 17, 2015

RabbitMQ and its nuances

I happened to work on AMQP/AMQPS implementation for one of our large server farms.
One side of the RMQ( Rabbit MQ) is in java and other side is in go lang,

Java RMQ Libraries

  • https://www.rabbitmq.com/api-guide.html
The most noticeable challenge was the amount of thread created when using this library, read below.
I had to set up a Executor

import com.google.common.util.concurrent.ThreadFactoryBuilder;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;


private final ThreadFactory threadFactory = new ThreadFactoryBuilder().setNameFormat("RmqExecutor-pool-%d").setDaemon(true).build();

private ExecutorService rmqExecutorService = Executors.newFixedThreadPool(1,threadFactory);

connection = factory.newConnection(rmqExecutorService);

With above code of creating a ThreadFactory and assigning this ExecutorService into    

Advanced Connection options

Consumer thread pool

Consumer threads (see Receiving below) are automatically allocated in a new ExecutorService thread pool by default. If greater control is required supply an ExecutorService on the newConnection() method, so that this pool of threads is used instead. Here is an example where a larger thread pool is supplied than is normally allocated:
ExecutorService es = Executors.newFixedThreadPool(20);
Connection conn = factory.newConnection(es);
Both Executors and ExecutorService classes are in the java.util.concurrent package.
When the connection is closed a default ExecutorService will be shutdown(), but a user-suppliedExecutorService (like es above) will not be shutdown(). Clients that supply a custom ExecutorService must ensure it is shutdown eventually (by calling its shutdown() method), or else the pool’s threads may prevent JVM termination.
The same executor service may be shared between multiple connections, or serially re-used on re-connection but it cannot be used after it is shutdown().
Use of this feature should only be considered if there is evidence that there is a severe bottleneck in the processing of Consumer callbacks. If there are no Consumer callbacks executed, or very few, the default allocation is more than sufficient. The overhead is initially minimal and the total thread resources allocated are bounded, even if a burst of consumer activity may occasionally occur.

No comments:

Post a Comment

bee-social