Docker is an emerging platform to build and deploy software using lightweight, pared-down virtual machines known as containers. By delivering easy-to-provision recipes for developers and bit-for-bit compatibility between environments, Docker is a popular solution to solve continuous delivery in modern infrastructure.
Like virtual machines before them, containers require a new monitoring strategy. Monitoring containerized applications and environments has also moved beyond just the operations team, or at the very least has opened up to more roles, as the DevOps movement encourages cross-team accountability and end-to-end responsibility. ‘You build it, you own it’.
What should matter to you
Modern monitoring solutions should be able to visualize a Dockerized landscape including its dependencies into an end-to-end view (from hardware level to business process level) that’s easy to analyze.
If you’re considering a change and want to be in control of your containerized landscape, here are some questions to ask:
Do you know which processes are running on which containers?
Do you know which metrics and events matter the most for your business?
Are you able to monitor dependencies and their impact?
When you upgrade or change something, what the impact will be?
How to integrate container monitoring with existing monitoring of dependent parts of the stack?
Container monitoring innovation starts with recognizing that a new approach needs to align with the dramatic technology changes over the last decade.
Setup Docker monitoring in StackState
Traditional monitoring tools require an agent that runs on the same OS as the application. However, in a microservices environment, it is not desirable to run a monitoring agent inside a container. Not only because it adds a lot of overhead to the lightweight structures that containers are. More importantly: because it breaks the concept of process isolation, where each container runs only a single well-bounded process.
To set up Docker monitoring in StackState, you have to follow these 4 steps:
Ensure Docker is running on the host.
Install the agent as described in for your host OS.
Add the agent user to the docker group:
usermod -a -G docker sts-agenCreate docker_daemon.yaml by copying the example file in the agent conf.d directory. If you have a standard install of Docker on your host, there shouldn’t be anything you need to change to get the integration to work.
Topology discovery
After installing the StackState agent, it will automatically discover your topology and keep track of what is running where. With the topology discovery feature, you can now continuously monitor your critical business processes and services, no matter how dynamic and complex the underlying infrastructure may be.
But, why do you need this topology at all, and what purpose does it serve? Well, it's more than just a nice graphical visualization. Let me explain it:
Containers add an extra layer of complexity between hosts and applications.
With legacy systems, it's enough to know which hosts could be running a specific type of service. However, with containers, services run on multiple hosts and the containers can easily be moved around by orchestration tools such as Mesos or Kubernetes. Insight into the relationship between 'new IT' and 'old IT' has become a necessity to move forward.
Containers depend on each other.
It is important to cluster containers together and provide a holistic overview of how one group of containers depends and talks to another group. This makes it easy to know exactly how your microservices (and monoliths) architecture is set up and how it influences other parts of the stack. This allows you to find weak spots and improve enterprise architecture.
Troubleshooting across containers is very hard.
Troubleshooting is hard. Visualizing the topology and dependencies makes it easier. The topology visualization is a very useful model to reason about what breaks, what broke first, and where actually breakages come from. Automating the troubleshooting process will significantly reduce the time and energy spent on manual root-cause analysis.
Conclusion
Our experience shows that adopting Docker and orchestration tools like Kubernetes and Mesos will have a huge impact on your current monitoring strategy.
You’re going to need a modern monitoring solution that is able to visualize a Dockerized landscape including its dependencies into an end-to-end view (from hardware level to business process level) that’s easy to analyze. When looking at a container monitoring solution, the solution must be simple and scalable. It should have native support for Docker, Kubernetes and Mesos and give insight into the dependencies between Dockerized environments and the rest of your IT and Business landscape.
