Public Cloud Monitoring and Diagnostic tools

As i was exploring tools on the microsoft azure and actively deploying applications.
Around 300 some VMs and lots of network constructs with VLANs and subnets.

I was looking for a one stop shop / single pane of glass tool for public cloud ( azure ) and local DC server applications , and hybrid monitoring tool.


I see the eco-system has good momentum and lots of thrid party applications claiming the

• https://stackify.com/
• http://www.cerebrata.com/products/azure-diagnostics-manager
• http://cloudmonix.com/ ( lots of visualization graphs and drill down ability )
• https://www.unigma.com/

Some other tools

• https://www.upguard.com/features, I personally like these features 
• Configuration search
• Scalable Anomaly Detection
• Time-lapse Collaboration View
• Visual Difference at its core
• https://www.opsclarity.com/product/features/
• Automated Topology Discovery
• Infrastructure Host map 
• APIs
• Identity and Access : https://jumpcloud.com/daas-product/index
• User Management 
• Device Management 
• SSO
• Active Directory Bridge
• LDAP-aas
• MFA
• https://www.moogsoft.com/product/
• Real-time anomaly detection 
• Closed loop remediation
• Production workbench

cadvisor , influx and grafanna

1.sudo docker run -d -p 8083:8083 -p 8086:8086 --expose 8090 --expose 8099 --name influxsrv tutum/influxdb

2.sudo docker run --volume=/:/rootfs:ro --volume=/var/run:/var/run:rw --volume=/sys:/sys:ro --volume=/var/lib/docker/:/var/lib/docker:ro --publish 8080:8080 --detach=true --link influxsrv:influxsrv --name=cadvisor google/cadvisor:latest -storage_driver_db=influxdb -storage_driver_host=influxsrv:8086

3.sudo docker run -d -p 3000:3000 -e INFLUXDB_HOST=localhost -e INFLUXDB_PORT=8086 -e INFLUXDB_NAME=cadvisor -e INFLUXDB_USER=root -e INFLUXDB_PASS=root --link influxsrv:influxsrv --name grafana grafana/grafana

# git repo for remote TTY over the web # git clone https://github.com/krishnasrinivas/wetty

4. sudo docker run --name term -p 3000 -dt nathanleclaire/wetty

Linux Process Management - S6

http://skarnet.org/software/s6/overview.html

S6 is a  process management tool , built from ground up. Based upon reliable libraries

https://skarnet.org/software/s6/why.html

Purpose : To manage any process inside a operating system , restart if needed.

Some examples of s6 programs meant to be used in run scripts:

• The s6-log program is a long-lived process. It is meant to be executed into by a ./log/run script: it will be supervised, and will process what it reads on its stdin (i.e. the output of the ./run daemon).
• The s6-envdir program is a short-lived process that will update its current environment according to what it reads in a given directory, then execute into the rest of its command line. It is meant to be used in a run script to adjust the environment with which the final daemon will be executed into.
• Similarly, the s6-softlimit program adjusts its resource limits, then executes into the rest of its command line: it is meant to set the resources the final daemon will have access to.
• The s6-applyuidgid program, part of the s6-*uidgid family, drops root privileges before executing into the rest of its command line: it is meant to be used in run scripts that need root privileges when starting but do not need it for the execution of the long-lived process.
• s6-ipcserverd is a daemon that listens to a Unix socket and spawns a program for every connection. It is meant to be supervised, so it should be used in a run script, and it's also meant to be a flexible super-server that you can use for different applications: so it is a building block that may appear in several of your run scripts defining local services.

opendoor

A home flipping startup ,

CASB

CASB (Cloud Access Security Broker ) and the players
CSP ( Cloud Service Providers) Salesforce, GCP ,etc...
UEBA ( User Behavior Anlaytics )

• Palerra ( acquired by oracle )
• Perspecys ( Blue Coat )
• Netskope in campbell, CA raised $132 m
• Skyhigh Networks
• Bit Glass ( Campbell, CA ) 
• Cloud lock ( Waltham, MA)
• Elastica ( Blue Coat )
• Cipher Cloud
• Fire Layers  ( Acquired by ProofPoint )

References

http://www.esecurityplanet.com/network-security/slideshows/cloud-access-security-brokers-9-vendors-you-need-to-know.html

http://www.wsj.com/articles/cloud-security-startup-skyhigh-networks-raises-a-down-round-1474677341

https://www.skyhighnetworks.com/cloud-security-blog/forrester-ranks-casb-vendors-in-wave-report/

http://www.bitglass.com/casb-cloud-access-security-broker

http://info.skyhighnetworks.com/WP-Forrester-Wave-Q4-2016-Banner-Cloud-v2.html?Source=Website&LSource=Website

Standalone kubernetes cluster on OSX

Plan to bring up standalone kubernetes on core-os using vagrant on OSX

https://coreos.com/kubernetes/docs/latest/kubernetes-on-vagrant-single.html

Ran in to an issue where i have to un-install vagrant and re-install with new one

Also got an vagrant ssh issue which was resolved by this thread
https://github.com/mitchellh/vagrant/issues/8013
[ this issue was OSX specific]

Happy 2017 and devops coverage

Happy New Year 2017

Coverage on devops tools and where are we with data center monitoring

https://www.itcentralstation.com/categories/event-monitoring

Apart from here I found some which are very apt

• http://www.alertwoo.com/
• https://www.moogsoft.com/
• https://www.upguard.com/ ( i personally like the CSTAR rating which this tool provides and security analytics tool ), check their chrome plug-in
• https://jumpcloud.com/

Github

• https://www.youtube.com/watch?v=NST3u-GjjFw

The world of NPM application life cycle management

How to demonize an NPM app , firstly why to demonize?
Well, an app process could have a life cycle and a support for its management, like init.d, upstart, etc..

NPM comes has its nice facility tool called "PM2"
https://github.com/Unitech/pm2

It is super cool , it restarts the app, every time it crashes, kill -9 <pid> what ever.
I ran into a situation , where i forgot that i orchestrated my NPM app with this PM2 tool, and could not find a reason why i have a port conflict.
I found that i have this app running, tracked the parent process, which is re-spinning this app, on my kill -9.

And have to do a "$ pm2 stopall"

Application secrets management

All applications uses secrets

• user name / password
• database connection strings
• TLS Certificate / Key
• Application specific content , etc...

Configurations of any kind (.ini, .yaml, .xml, etc ..), even the OS configuration for application tuning, MUST be versioned and most of the time the packaging bundles the application with specific configuration,  this is not a good practice.

Storing secrets in build time is a big flaw. Likewise storing secrets in ENV's is also a security issue.

Secrets better be on a separate service, from which applications pull specific version of relevant secret data.

References
AWS, EC2 Container
https://blogs.aws.amazon.com/security/post/Tx2B3QUWAA7KOU/How-to-Manage-Secrets-for-Amazon-EC2-Container-Service-Based-Applications-by-Usi

https://infinum.co/the-capsized-eight/articles/hiding-secrets-in-vault

Why secrets store / vault do not have user interface???

https://feedback.azure.com/forums/170024-additional-services/suggestions/12370134-please-consider-user-interface-for-azure-key-vault

Azure Key Vault ( KV)

https://blogs.technet.microsoft.com/kv/

Android Key Store
https://developer.android.com/training/articles/keystore.html

Distelli
https://www.distelli.com/blog/keeping-your-application-secrets-safe
https://www.distelli.com/docs/kb/using-secure-package-with-secrets


Update:
Referenfce : https://changelog.com/podcast/239

Go Lang 1.7 released

https://golang.org/doc/go1.7

Golang 1.7 released

static single assignment SSA low level machine language support.

More details in link