CICD and Continuous Communication (CICDCC)

This is a presentation for test driven infrastructure utilizing continuous integration and delivery tools and realtime communication with slack.

madisonhub/mean-skel


Just like most CICD systems, I can push code and have it build it.  Whats cool with slack is that I can get notifications, in realtime, from GitLab and GitLab-CI.

The gitlab links will take you directly to the project and commit diffs.  Clicking on the gitlab-ci links will allow you to watch build output, in realtime, as well as build charts and graphs.

Since gitlab-ci shows the build is passing, its likely the build will work if done in a production container.  Ill create a container and add it to my salt-master environment.

# docker run -d --name mean-skel -h mean-skel.docker01 -t docker-minion

# docker inspect mean-skel | grep IPAddress
        "IPAddress": "172.17.0.14",

Now that I know the IP address of the container, I need to add it to my servers pillar and tell nagios to reload.

/srv/pillar/servers.sls:

  mean-skel:
    ip: 172.17.0.14
    node_type: container
    nagioscfg:
      use: linux-server
      parents: docker01

salt 'nagios*' state.sls nagios.running container

I am now monitoring it, in realtime.

By default, I monitor for salt-minion process and users.  Since the new container isn't in its final highstate, I am getting notifications.

Since I decided not to automatically install the nrpe agent, I am getting those errors.  Notice that "CRITICAL" is highlighted yellow.  That is a custom filter I am looking for specifically in the alert channel on slack.  If I am not actively in slack, I will get notified on my phone with the slack app of critical events in my infrastructure.

salt 'mean-skel*' state.sls nagios.nrpe container

Slack has notified me that services have come back.  Taking a look at the Tactical Overview in nagios, I can see that everything is OK.

Whats next?  What isn't shown here is the salt-master interface and its hooks into slack for the state of minions.  That is next.  Stay tuned.