The importance of Discovery in Application Performance Management

As I read articles and blogs on Application Performance Management (APM), I see a lot of discussions on some very important topics such as:

  • SaaS vs. on-premise
  • Predictive Analytics/Dynamic thresholding
  • Log Analytics
  • Agentless vs. Agent-based monitoring
  • Transaction Tracking
  • And more…

One area that is often overlooked is the importance of discovery.   If you don’t understand the environment that you are monitoring, it is virtually impossible to effectively monitor the business applications.   Think about a cloud based business application where additional servers are added to provide additional capacity or an environment where CPU cores are added to a Virtual Machine. The application behavior, including the performance, will change.   In addition, you must ensure that the new servers are being monitored appropriate. There are multiple aspects of discovery that are important as described below.

First, you must ensure that all resources are being monitored.   This can be accomplished in a few different ways. You can tie your APM tools in with your provisioning process.   As new servers and applications are provisioned, monitoring is provisioned at the same time. This process must include configuring the appropriate thresholds based on the workload or the use of predictive analytics to detect problems. Another option is to use a dedicated discovery product. These tools can be very valuable for customers without a strong change control process.   As I meet with customers, I frequently find that many servers are not even being monitored.

Another alternative is to have application aware monitoring tools.   By using tools that can monitor end-to-end Transaction Tracking, you are able to automatically discover the application topology and any changes in the topology as traffic flows change. For example, if a new Application Server is added to a cluster, the traffic flow to that node is automatically discovered.

Finally, you can use a purpose built discovery tool.   One of the benefits of tools that are designed for discovery and inventory is that they typically gather a much richer set of data about the servers, hypervisors, switches, and applications being used.   In addition, they provide reporting capabilities including change detection.   This can be invaluable when debugging a problem. You can compare what’s changed within the last week or compare a system against a golden image.

Any of the three approaches will work to discover your business applications and relationships.   The key is to know what you have and to ensure that monitoring is in place and setup with the right thresholds (or use analytics).

Advertisements

Some of my favorite recent blogs, articles, and videos on APM

I have compiled some of my favorite recent blogs, articles, and video on Application Performance Management (APM).  I thought a consolidated list would be valuable to people reading my blog. 

 

Articles/White Papers

 

Blogs

Videos/Webcasts:

Information on IBM’s SaaS and IT Analytics portfolio

I wanted to let people know about a couple of recent and upcoming events around IBM’s SaaS and IT Analytics portfolio.   First, I wanted to let people know that IBM just released new monitoring agents for Python, PHP, and PostgreSQL for our SaaS based Application Performance Management.  You can go online to http://ibmserviceengage.com and get a live demonstration of the new capabilities.  IBM ServiceEngage also allows you to request a free 30 day trial of the SaaS Application Performance Monitoring capabilities.

In addition, I want to let you know that we’ll also be discussing and demonstrating these new monitoring agents as part of a weekly webcast series on IBM’s Application Performance Management and IT Analytics portfolio.  Upcoming events include information on the new monitoring agents as well as sessions on our IT Analytics capabilities.   The following website includes information about the upcoming events as well as recordings of the previous sessions.   Go to https://ibm.biz/BdxUGq.

Weekly APM and IT Analytics webcasts

I wanted to remind people about the weekly Application Performance Management (APM) and IT Analytics webcasts that IBM is hosting.   The following website contains a list of upcoming webcasts as well as recordings of the previous webcasts.    Upcoming webcasts include:

  • MongoDB and Ruby on Rails monitoring
  • PHP, Python, and PostgreSQL monitoring
  • The value of combining Log Analytics and Application Performance Management (APM)

Go to the following link to get details:   https://ibm.biz/BdxUGq

Thoughts on IBM’s recently announced Application Performance Management SaaS offering

IBM recently announced a SaaS offering for their Application Performance Management (APM) product.  By providing a SaaS offering along with an on-premise solution, customers have a lot of flexibility in their monitoring solutions.   In addition, the SaaS offering introduced some application domain coverage that many customers might find useful.  In this post, I’ll describe some of the key aspects that I think customers will find valuable.

IBM’s APM solution now offers three options.   While these solutions are all slightly different in terms of the infrastructure, they reuse many of the same components and offer a consistent look and feel.   In addition, they reuse the same monitoring agents which is key and I’ll talk more about that later.

  • There is a traditional enterprise scalable solution based on the IBM Tivoli Monitoring infrastructure.   This offers a highly robust monitoring solution with the key features that a centralized monitoring team is looking for.  It provides coverage for all key platforms including mainframe and distributed platforms from a single infrastructure, it covers dozens of hypervisors, applications, middleware, response time monitoring, transaction tracking, and more.  The solution is feature rich and tightly integrated with IBM’s IT Analytics products for doing Log Analytics/Event analytics and Search as well as multi-variant predictive analytics on performance metrics.
  • IBM offers a lightweight on-premise monitoring solution.   While the features/functions provided by the light weight architecture are not as robust as the IBM Tivoli Monitoring based solution, the time to value is incredibly fast.  The solution can be installed in 10 or 15 minutes without the use of product manuals.  The Agents are installed using native packaging such as RPM’s on Linux making time to value much faster.
  • Finally, IBM now offers a SaaS solution.  For customers or organizations within a customer that want an Application Performance Management solution, but don’t want to be responsible for maintaining the infrastructure, the SaaS offering is an ideal fit.   In addition to not having to manage the infrastructure, customers can benefit from the monthly subscription based pricing.   Included in the SaaS offering are some new monitoring capabilities for monitoring MongoDB, MySQL, and Ruby on Rails.  The SaaS offering has a free Demo where you can logon to a website and try out the software using applications that are running on IBM’s SoftLayer cloud.  In addition, a free 30 day trial is offered so that customers can try out the SaaS offering with minimal investment in time before deciding whether to purchase the product online or via their IBM sales rep.

Now, let’s talk about some of the key benefits of having 3 options for customers.  First, all three offerings have a consistent user interface that can be leveraged by an operations team, an infrastructure management team, lines of business, and application developers.  This is important because all of the organizations can look at the same metrics and use the same terminology when debugging a problem.  In addition, all three solutions leverage the same Agent-based and Agent-less monitoring capabilities so that the metrics viewed by the various teams are identical.   Having worked with hundreds of customers around the world, I can’t stress enough the importance of consistent data.   If the operations team is looking at “Heap Utilization”, but the subject matter experts are looking at a metric called “Heap Usage”, it can cause confusion and waste time in resolving a problem.  In addition, when looking at different tools, the metrics may be using a different scale.  For example, one tool may be using KBytes while another is using MBytes.   One tool may be using transactions/second while another may be using transactions/minute.    Some tools use a rolling average while others use a point in time snapshot of the metrics.   This can cause tremendous confusing between the various teams trying to identify, isolate, and resolve a problems with an application.  By using IBM’s tools, customers get the flexibility of 3 options, but the metric data is the same.

If you want to learn more about IBM’s Application Performance Management tools and the recently announced SaaS offering, visit the IBM ServiceEngage website at:  http://ibmserviceengage.com.  You will find a few products listed including the Application Performance Management solution.

Recommendations on Customers with Enterprise wide APM solutions that also want best of breed for SMEs

I was up in New York visiting one of our large enterprise customers.   Some interesting topics came up that I think apply to most customers.   I thought I would post a blog on the  key points of the discussion.

Many customers struggle with the challenge of deciding between a single enterprise solution vs. best of breed solutions for each application/infrastructure domain.   In addition, I often see customers that want centralized control of all servers, but that conflicts with the desires of the lines of business and subject matter experts.   In this particular case, we had a customer with an enterprise solution using IBM’s Tivoli Monitoring Agents in Autonomous mode, but a PowerVM team that wanted to use IBM’s SmartCloud Monitoring product in a more traditional deployment to monitor their PowerVM systems. We also had discussions of possibly using a SaaS solution to monitor their PowerVM systems.

Customers have several options when dealing with this type of problem.   Up-line management can mandate a single solution and force all teams to use the centralized solution.   But, that usually doesn’t work well except in small organizations.  Customers can certainly choose best of breed for each domain, but that’s extremely costly.   First, the operations team must be trained.   Second, customers must negotiate contracts with 10 or 15 different vendors.  Often times, customers underestimate the cost of contract negotiations.  Purchasing departments and legal teams are quite expensive.    Negotiating with one vendor is certainly much cheaper than initiating 10 or 15 unique negotiations.

In this case, IBM was able to offer the customer a solution that provided multiple monitoring options for the different teams.   At the enterprise level, they chose autonomous monitoring without a monitoring framework/infrastructure.  This solution can be easily integrated with either IBM’s upcoming SaaS offering or IBM SmartCloud Monitoring solution.   Essentially, the customer can use the same set of monitoring agents in different ways depending on the desires of each team.   This allows the customer to deploy a single set of monitoring agents, negotiate with one vendor, and reduce the training costs for the operations team.

There will always be the need for 1-off solutions that can be integrated at the Event Management or Service Desk.   But, it’s much more efficient if customers can deploy a solution from a single vendor that can provide all of their monitoring needs.