Wednesday, December 9, 2015

Docker Monitoring Tools

I came across recently this article on monitoring options for docker world

http://rancher.com/comparing-monitoring-options-for-docker-deployments/

A Quick Summary looks like below

  1. Docker Stats
  2. CAdvisor ( https://github.com/google/cadvisor)
  3. Scout
  4. Datadog ( Cloud based )
  5. Prometheus - http://prometheus.io 
    1. container exporter : https://github.com/docker-infra/container_exporter
    2. https://hub.docker.com/r/prom/prometheus/
    3. Prometheus Dashboard Builder
Prometheus' main distinguishing features as compared to other monitoring systems are:
  • multi-dimensional data model (timeseries defined by metric name and set of key/value dimensions)
  • flexible query language to leverage this dimensionality
  • no dependency on distributed storage; single server nodes are autonomous
  • timeseries collection happens via a pull model over HTTP
  • pushing timeseries is supported via an intermediary gateway
  • targets are discovered via service discovery or static configuration
  • multiple modes of graphing and dashboarding support
  • federation support coming soon

Inspiring IOT

Came across a neat open source project for IOT platform, a framework to deal with interactions between IOT agents and server.
http://www.kaaproject.org
https://github.com/kaaproject/kaa


KAA Server is a server component written purely in JAVA

  • spring MVC
  • NoSQL
  • Jenkins
  • ...

KAA Client
https://github.com/kaaproject/kaa/tree/master/client/client-multi

Tuesday, November 24, 2015

nodejs based web UI Shell Emulator

* Install nodejs in ubuntu
& sudo apt-get update
& sudo apt-get install nodejs
& sudo apt-get install npm
*git clone https://github.com/chjj/tty.js/
& nodejs install tty.js
 & cd bin
&./tty.js --daemonize --port 3000

References :
https://www.digitalocean.com/community/tutorials/how-to-set-up-a-node-js-application-for-production-on-ubuntu-14-04

GPG Errors in ubuntu

W: GPG error: https://apt.dockerproject.org ubuntu-trusty Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY F76221572C52609D

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys F76221572C52609D
sudo apt-get update

Tuesday, November 17, 2015

Setting up golang proto-gen-go

Here is the step by step directions:
  1. Download protoc-win32.zip from https://developers.google.com/protocol-buffers/docs/downloads
  2. Unzip and add location of the protoc.exe to your PATH environment variable
  3. Run `protoc --version` from command prompt to verify
  4. Verify the your GOPATH environment variable is set
  5. Run `go get -u github.com/golang/protobuf/protoc-gen-go` from command prompt. This should install the binary to %GOPATH%/bin
  6. Add `%GOPATH%/bin` to your PATH environment variable
  7. Open a new command prompt, navigate to your .proto file, run `protoc --go_out=. *.proto` 
NOTE: if you are running from a text editor or ide, you may need to reboot after modifying your environment variables

RabbitMQ and its nuances

I happened to work on AMQP/AMQPS implementation for one of our large server farms.
One side of the RMQ( Rabbit MQ) is in java and other side is in go lang,

Java RMQ Libraries

  • https://www.rabbitmq.com/api-guide.html
The most noticeable challenge was the amount of thread created when using this library, read below.
I had to set up a Executor

import com.google.common.util.concurrent.ThreadFactoryBuilder;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;


private final ThreadFactory threadFactory = new ThreadFactoryBuilder().setNameFormat("RmqExecutor-pool-%d").setDaemon(true).build();

private ExecutorService rmqExecutorService = Executors.newFixedThreadPool(1,threadFactory);

connection = factory.newConnection(rmqExecutorService);

With above code of creating a ThreadFactory and assigning this ExecutorService into    

Advanced Connection options

Consumer thread pool

Consumer threads (see Receiving below) are automatically allocated in a new ExecutorService thread pool by default. If greater control is required supply an ExecutorService on the newConnection() method, so that this pool of threads is used instead. Here is an example where a larger thread pool is supplied than is normally allocated:
ExecutorService es = Executors.newFixedThreadPool(20);
Connection conn = factory.newConnection(es);
Both Executors and ExecutorService classes are in the java.util.concurrent package.
When the connection is closed a default ExecutorService will be shutdown(), but a user-suppliedExecutorService (like es above) will not be shutdown(). Clients that supply a custom ExecutorService must ensure it is shutdown eventually (by calling its shutdown() method), or else the pool’s threads may prevent JVM termination.
The same executor service may be shared between multiple connections, or serially re-used on re-connection but it cannot be used after it is shutdown().
Use of this feature should only be considered if there is evidence that there is a severe bottleneck in the processing of Consumer callbacks. If there are no Consumer callbacks executed, or very few, the default allocation is more than sufficient. The overhead is initially minimal and the total thread resources allocated are bounded, even if a burst of consumer activity may occasionally occur.

Tuesday, July 14, 2015

GO programming Collection

My recent exploration of writing GO programs, i had to refer below

http://www.goinggo.net
http://play.golang.org/
https://www.ardanlabs.com/hardcore-go
https://godoc.org/github.com/capnm/sysinfo
https://plus.google.com/+WilliamKennedy/posts


Friday, June 5, 2015

Compiling protobuf

How to build protobuf

git clone https://github.com/google/protobuf.git
sudo apt-get update
sudo apt-get install autoconf
sudo apt-get install Libtool
./autogen
./configure
make

Build protobuf jars

protoc --java_out=src/main/java -I../src \
     ../src/google/protobuf/descriptor.proto

Wednesday, May 6, 2015

Panamax


  • Panamax is a open source project from Centurylinklabs
  • It provides an user interface to manage docker containers in a host.
  • As i tried now, it comes as a vagrant box image , which has CoreOS + panamax installed on it, few ports are opened to reach out this VM from host
  • Some good are
    • Panamax has a User Interface it lets you see list of containers co-hosted in this VM
    • See stats of containers - hooked up with google/CAdvisor
    • Search docker.io repo for new images - BUT when tried to Add Image it does not work
  • Lot of bad are
    • There are too many github small projects under centurylinklabs and i am not sure what this guys are trying to do? Looks too amateur to me .
    • Code base is too buggy
    • Open Issues are not answered

bee-social