Friday, May 20, 2016

golang CI

I came across these important tools and marking it here for references, here

Golang tiny test framework
  • https://github.com/nbio/st
Golang excellent way to un-marshal data stream ( JSON) into struct
  • https://github.com/mitchellh/mapstructure

Wednesday, May 4, 2016

consul , vault , registrator

All these products are from hashicorp.

consul : a ready made datacenter tool for service discovery ( finding applications coming up on nodes on data center) .

  • Each data center MUST run atleast one consul server
  • Each node MUST run a consul agent

How to run a simple cluster of consul servers ( as docker containers )



    • docker run -d --name consul-node-1 -h consul-node-1 -p 0.0.0.0:53:8600/udp -p 8500:8500 -p 8400:8400 gliderlabs/consul-server -node consul-node-1 -bootstrap -advertise 10.0.0.2 -client 0.0.0.0 
    • JOIN_IP="$(docker inspect -f '{{.NetworkSettings.IPAddress}}' consul-node-1)" docker run -d --name consul-node-2 -h consul-node-2 gliderlabs/consul-server -node consul-node-2 -join $JOIN_IP
    • docker run -d --name consul-node-3 -h consul-node-3 gliderlabs/consul-server -node consul-node-3 -join $JOIN_IP 
    • docker run -d --name consul-node-4 -h consul-node-4 gliderlabs/consul-server -node consul-node-4 -join $JOIN_IP

     docker ps
    CONTAINER ID        IMAGE                      COMMAND                  CREATED             STATUS              PORTS                                                                                                          NAMES
    8b1fb4cd3c09        gliderlabs/consul-server   "/bin/consul agent -s"   16 minutes ago      Up 16 minutes       8300-8302/tcp, 8400/tcp, 8500/tcp, 8301-8302/udp, 8600/tcp, 8600/udp                                           consul-node-3
    8c5a59c2f878        gliderlabs/consul-server   "/bin/consul agent -s"   18 minutes ago      Up 18 minutes       8300-8302/tcp, 8400/tcp, 8500/tcp, 8301-8302/udp, 8600/tcp, 8600/udp                                           consul-node-2
    cdf06afbe5fc        gliderlabs/consul-server   "/bin/consul agent -s"   20 minutes ago      Up 20 minutes       0.0.0.0:8400->8400/tcp, 8300-8302/tcp, 8600/tcp, 8301-8302/udp, 0.0.0.0:8500->8500/tcp, 0.0.0.0:53->8600/udp   consul-node-1



    I am writing this page to collect all my findings and share with team.
    Hope it helps, happy reading!!!
    I am working on
    • setting up hashicorp's consul and vault . 
    • Write a vault client as part of PEP
    • How to start consul locally in your VM ( i have a ubuntu on my parallels )
      • Each data center MUST run atleast one consul server
      • Each node MUST run a consul agent
      • Running consul-server as a docker container
        • docker run -d --name consul-node-1 -h consul-node-1 -p 0.0.0.0:53:8600/udp -p 8500:8500 -p 8400:8400 gliderlabs/consul-server -node consul-node-1 -bootstrap -advertise 10.0.0.2 -client 0.0.0.0
      • Running consul-server as a standalone binary on your host
      • This starts hashicorp's consul , UI : http://127.0.0.1:8500/ui
      • Running a consul cluster of docker containers
        • docker run -d --name consul-node-1 -h consul-node-1 -p 0.0.0.0:53:8600/udp -p 8500:8500 -p 8400:8400 gliderlabs/consul-server -node consul-node-1 -bootstrap -advertise 10.0.0.2 -client 0.0.0.0 
        • JOIN_IP="$(docker inspect -f '{{.NetworkSettings.IPAddress}}' consul-node-1)" docker run -d --name consul-node-2 -h consul-node-2 gliderlabs/consul-server -node consul-node-2 -join $JOIN_IP
        • docker run -d --name consul-node-3 -h consul-node-3 gliderlabs/consul-server -node consul-node-3 -join $JOIN_IP 
        • docker run -d --name consul-node-4 -h consul-node-4 gliderlabs/consul-server -node consul-node-4 -join $JOIN_IP
    • How to start a vault pointing to above consul
      • vault can be downloaded directly as a packaged binary ( 64-bit )
      • a simple vault configuration could be as below 
        • backend "consul" { address = "127.0.0.1:8500" path = "vault" advertise_addr = "http://127.0.0.1" scheme = "http" } listener "tcp" { address = "0.0.0.0:8200" tls_disable = 1 }
      • starting vault server
        • export VAULT_ADDR=http://127.0.0.1:8200
          • vault status Error checking seal status: Error making API request. URL: GET http://127.0.0.1:8200/v1/sys/seal-status Code: 400. Errors: * server is not yet initialized
            ubuntu$ vault init 
            Key 1: 5fea1643a59b41cea982ce684baf2399851d3423483a1deccfb59a22d315875901 
            Key 2: bdf7d80022697c44e9d50c5318cae72926f69008c1aef3b02f82d4f44c438eb502 
            Key 3: ad7e6ca87d8bb3df70549f2062fafd6532ce0c2cbb4543b960eb515fb60fe6b703 
            Key 4: d5127290d248b68774a554053ae11bbdae22b41ed3d185480166d7b3fd53df1904 
            Key 5: c59bc6388daa791ced24c77640d101f1ba1a283aa93a35414e0f5218071fb71b05 
            Initial Root Token: 97a7735a-0b7d-7a40-b51e-779904e85d8b Vault initialized with 5 keys and a key threshold of 3. Please securely distribute the above keys. When the Vault is re-sealed, restarted, or stopped, you must provide at least 3 of these keys to unseal it again. Vault does not store the master key. Without at least 3 keys, your Vault will remain permanently sealed.
      • Writing secrets into vault
        • vault has to be authenticated and unsealed before any operation can be done 
        •  $vault auth 97a7735a-0b7d-7a40-b51e-779904e85d8b
        • $vault write secret value=secret
    • References

    bee-social