Posting Kubernetes Events to Slack

This is part 4 in a 6-part series on containers, microservices, and container orchestration to help tech and business professionals of all levels understand and embrace this transformative technology and the best practices processes to maximize your investment in it.

Here is an outline:

  1. Intro: Why Containers and Microservices Matters to Business – Executive thought leadership perspective – Coming Soon!
  2. Getting started with Kubernetes – How to start with a POC, weave k8s into your existing cicd pipelines, build a new pipeline
  3. Intermediate level post – Ready to kick the tires? K8s, Ansible, Terraform media/entertainment enterprise case study
  4. Advanced tips and tricks – Take things further with “Tapping into Kubernetes Events“, “Posting Kubernetes Events to Slack“, and “Chatbots for Chatops set up on Gcloud w Container engine
  5. Scaling Docker and Kubernetes in production to thousands of nodes on Kubernetes in one of the largest consumer web properties – Coming Soon!

As a follow-on from my earlier post, I want to chat some more about the things you could do with the kubernetes-sniffer go app we created. Once we were able to detect pods in the cluster, handler functions were called when a new pod was created or an existing pod was removed. These handler functions were just printing out to the terminal in our last example, but when you start thinking about it a bit more, you could really do anything you want with that info. We could post pod info to some global registry of systems, we could act upon the metadata for the pods in some way, or we could do something fun like post it to Slack as a bot. Which option do you think I chose?

Setting Up Slack

In order to properly communicate with Slack, you will need to set up an incoming webhook.

  • Incoming webhooks are an app you add to Slack. You can find the app here.

  • Once this is done, you can configure a new hook. In the “Add Configuration” page, simply select the Slack channel you would like to post to.

  • On the next page, save the Webhook URL that is supplied to you and edit the information about your bot as necessary. I added a Kubernetes logo and changed his name to “k8s-bot”.

Posting To Slack

So with our webhook setup, we are now ready to post to our channel when events occur in the Kubernetes cluster. We will achieve this by adding a new function “notifySlack”.

  • Add the “notifySlack” method to your k8s-sniffer.go file above the “podCreated” and “podDeleted” functions:
  • Update the url variable with your correct Webhook URL.

  • Notice that the function takes an interface and a string as input. This allows us to pass in the pod object that is caught by the handlers, as well as a string indicating whether that pod was added or deleted.

  • With this method in place, it’s dead simple to update our handler functions to call it instead of outputting to the terminal. Update “podCreated” and “podDeleted” to look like the following:

  • The full file will now look like:

Posted Up

Alright, now when we fire up our go application, we will see posts to our channel in Slack. Remember, the first few will happen quickly, as the store of our pods is populated.

  • Run with go run k8s-sniffer.go

  • View the first few posts to Slack:

  • Try scaling down an RC to see the delete: kubectl scale rc test-rc --replicas=0


Author: Spencer Smith, Cloud Engineer, Solinea