use Infrastructure-as-a-Service for production applications. Most are SaaS companies with large Amazon Web Services deployments (typically hundreds or thousands of EC2 instances, $25k-$1M/month in AWS spend).
We define the categories as follows:
● Application Performance – Tools that provide visibility into the run-time application or business transactions (see here).
● Availability – Tools that provide “outside-in” visibility into uptime of application resources.
● Log Analysis – Tools that provide easy access and visibility into information contained in application log files.
● System Monitoring – Tools that provide “inside-out” visibility into the health, capacity, and utilization of instances.
● IaaS Monitoring – Tools that provide insights into users’ cloud infrastructure resources and configuration.
● Alerting – Tools that provide unified workflow for notifications that are generated in other management tools.
● Event Tracking – Tools that provide a unified repository and interface for reporting on application errors and events.
Satisfaction with the current state of affairs varies dramatically. By and large, those who manage mature cloud operations view their current web of open source and custom tools as satisfactory—but quickly acknowledge that the journey has been more difficult and costly than they ever anticipated. Newer cloud users are worse off; they feel the pain of not having the right management infrastructure in place (in the form of performance issues, unplanned downtime, and cost surprises) but lack the people and experience to build and integrate the portfolio of tools that are required today.
The more concerning implication of our research is that “cloud-powered” companies across the maturity spectrum are losing out on opportunities to innovate and serve their customers due to ineffective cloud management. They build and support IT applications to manage their infrastructure. They extend and integrate open-source monitoring systems. They build custom metric data warehouses and visualization tools. And still, they struggle to deliver performance and availability that are on par with legacy dedicated infrastructure.
One of the people we interviewed, Brendan Schwartz, CTO and co-founder of Wistia, said it best: “We moved to the cloud to spend more time on dev and less time on ops. Unfortunately, we are a long way from that reality.”