Tuesday, November 17, 2015

Setting up golang proto-gen-go

Here is the step by step directions:
  1. Download protoc-win32.zip from https://developers.google.com/protocol-buffers/docs/downloads
  2. Unzip and add location of the protoc.exe to your PATH environment variable
  3. Run `protoc --version` from command prompt to verify
  4. Verify the your GOPATH environment variable is set
  5. Run `go get -u github.com/golang/protobuf/protoc-gen-go` from command prompt. This should install the binary to %GOPATH%/bin
  6. Add `%GOPATH%/bin` to your PATH environment variable
  7. Open a new command prompt, navigate to your .proto file, run `protoc --go_out=. *.proto` 
NOTE: if you are running from a text editor or ide, you may need to reboot after modifying your environment variables

RabbitMQ and its nuances

I happened to work on AMQP/AMQPS implementation for one of our large server farms.
One side of the RMQ( Rabbit MQ) is in java and other side is in go lang,

Java RMQ Libraries

  • https://www.rabbitmq.com/api-guide.html
The most noticeable challenge was the amount of thread created when using this library, read below.
I had to set up a Executor

import com.google.common.util.concurrent.ThreadFactoryBuilder;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;


private final ThreadFactory threadFactory = new ThreadFactoryBuilder().setNameFormat("RmqExecutor-pool-%d").setDaemon(true).build();

private ExecutorService rmqExecutorService = Executors.newFixedThreadPool(1,threadFactory);

connection = factory.newConnection(rmqExecutorService);

With above code of creating a ThreadFactory and assigning this ExecutorService into    

Advanced Connection options

Consumer thread pool

Consumer threads (see Receiving below) are automatically allocated in a new ExecutorService thread pool by default. If greater control is required supply an ExecutorService on the newConnection() method, so that this pool of threads is used instead. Here is an example where a larger thread pool is supplied than is normally allocated:
ExecutorService es = Executors.newFixedThreadPool(20);
Connection conn = factory.newConnection(es);
Both Executors and ExecutorService classes are in the java.util.concurrent package.
When the connection is closed a default ExecutorService will be shutdown(), but a user-suppliedExecutorService (like es above) will not be shutdown(). Clients that supply a custom ExecutorService must ensure it is shutdown eventually (by calling its shutdown() method), or else the pool’s threads may prevent JVM termination.
The same executor service may be shared between multiple connections, or serially re-used on re-connection but it cannot be used after it is shutdown().
Use of this feature should only be considered if there is evidence that there is a severe bottleneck in the processing of Consumer callbacks. If there are no Consumer callbacks executed, or very few, the default allocation is more than sufficient. The overhead is initially minimal and the total thread resources allocated are bounded, even if a burst of consumer activity may occasionally occur.

Tuesday, July 14, 2015

GO programming Collection

My recent exploration of writing GO programs, i had to refer below

http://www.goinggo.net
http://play.golang.org/
https://www.ardanlabs.com/hardcore-go
https://godoc.org/github.com/capnm/sysinfo
https://plus.google.com/+WilliamKennedy/posts


Friday, June 5, 2015

Compiling protobuf

How to build protobuf

git clone https://github.com/google/protobuf.git
sudo apt-get update
sudo apt-get install autoconf
sudo apt-get install Libtool
./autogen
./configure
make

Build protobuf jars

protoc --java_out=src/main/java -I../src \
     ../src/google/protobuf/descriptor.proto

Wednesday, May 6, 2015

Panamax


  • Panamax is a open source project from Centurylinklabs
  • It provides an user interface to manage docker containers in a host.
  • As i tried now, it comes as a vagrant box image , which has CoreOS + panamax installed on it, few ports are opened to reach out this VM from host
  • Some good are
    • Panamax has a User Interface it lets you see list of containers co-hosted in this VM
    • See stats of containers - hooked up with google/CAdvisor
    • Search docker.io repo for new images - BUT when tried to Add Image it does not work
  • Lot of bad are
    • There are too many github small projects under centurylinklabs and i am not sure what this guys are trying to do? Looks too amateur to me .
    • Code base is too buggy
    • Open Issues are not answered

Docker aka Container management

I am recently playing more with dockers and containers as my job demand it.

Jotting down some useful details here on my learning for general public and for my self records as well.

What is docker?
A Startup company in San Francisco, California , created a simple linux tool to manage linux containers.

Why one need docker?
Like VMWare's virtualization technology is for virtualization .Docker is the answer to provide next generation light weight virtualization technology. Docker is made to ease things out here.

What is a Container?
Container exist for a long time in linux world, it is a instance of another linux inside a linux operating system.

Why we need a Operating System inside another Operating System?
Virtualization is all about making efficient use of  "COMPUTE", "STORAGE" and "NETWORKING" . As new software applications or upgrade are rolled out every second in data centers for modern needs like (social networking, chatting, video, etc... ), a big chunk of compute power is needed. For ease of management of software applications in data centers a smallest possible workload bundle is needed. For which a host ( LINUX OS) should be capable of running multiple smaller guest LINUX OS ( which sounds like the virtual machine concept , but not : containers are super light weight compared to virtual machines.

Tuesday, January 6, 2015

Making sense of SDN

I came across this three part video in youtube, which was recorded back in August 2012, i see it is interesting and useful.






OpenSSL User Interface

An excellent tool for OpenSSL (working version) tool i came across http://sourceforge.net/projects/opensslui/

  • It has a installable windows UI
  • It uses OpenSSL equivalent command to create and manage CA certs, Keys for PKI



References

bee-social