Open source APM: How to choose the best solution

The following post has been authored by Paolo Sala.

Today’s APM (Application Performance Management) open source solutions market is extremely dynamic. At Moviri we are always keen on offering the most effective and up-to-date solutions for our customers, so we want to share with you a step-by-step process to review APM solutions against a real-life scenario. Even if scenarios are slightly different from one another, it’s possible to adapt and use a similar approach as the one exemplified below.

The scenario

Our “imaginary customer” is a company that is evaluating open source APM solutions. Let’s assume that they already have some commercial monitoring solution in place and expect to migrate them over an open platform. The goal is to produce an objective scouting analysis useful to the managerial decision-making process.

The approach

We followed a structured approach to get the job done:

1. We created an evaluation framework to compare different monitoring solutions based on actual real-life needs.
2. We mapped the hypothetical APM corporate solutions already in use, as it simplifies the understanding of what advantages current solutions may have (mainly in term of product support) and it helps to highlight the differences.
3. We applied the framework to a shortlist APM open source solutions and we obtained a comparative table that maps out strengths and weakness of each of them.

The framework

The evaluation framework has been developed to cover five main dimensions that may also be divided in smaller sub-dimensions as needed:

• Total Cost: Includes the information related to the solution’s pricing model. It should include not only costs directly derived from acquiring the solution such as license costs, hardware etc. but also indirect costs derived from its installation, use and support, such as, for example, development and customization costs.
• Scalability:  Includes information related to the ability to scale of the solution. We have included this dimension since modern IT Infrastructures are very dynamic and can potentially grow at very fast rates, so monitoring solution should scale easily as well.
• Core Functionality:  It’s one of the core dimension of the analysis.  Includes information about features strictly related to monitoring and details about the solution’s ability to monitor specific kind of objects (e.g. server metrics, application-specific KPI…).
• Additional Features:  Includes data warehousing, alert & reporting capabilities, data extraction capabilities (both from API or GUI), data post-processing (e.g. SLA calculations). This dimension is very important cause consider out-of-the-box functionality.
• Integration & Correlation: Monitoring solution are not designed to work in isolation. To get the best from their use, collected information should be easy to exchange among system and to correlate to one another.

Each dimension is further detailed in specific sub-dimensions to better describe each feature, capability and function.

Once the framework is defined, it can be applied first on the APM corporate solutions currently in use and the to open source solutions, following these steps:

• 1. Compiling an easy-to-read product sheet, that describes the main characteristics of the product as they relate to each sub-dimension.
• 2. Score each sub-dimension with a numerical score (e.g. from 0 to 5), and calculate each dimension’s score as an average of the sub-dimensions scores.
• 3. Create a score matrix to assess the details for each dimensions, with rows representing the each dimension and products in the columns.

Results

One of the great aspect of this analysis is that it maps open source solutions capabilities against real-life risks and costs in an enterprise environment. Moreover, breaking down the analysis in well-defined dimensions helps to evaluate on several layers, each suitable to a different kind of stakeholders.

Finally, by combining the deliverables, it’s easy to get a clear overview of all the solutions under evaluation and to retrieve the desired level of detail. Specifically the matrix achieves the first goal while the product sheet achieves the second.

At Moviri, we are always looking at new ways to help our customer to evaluate, select and implement productive and cost-effective solutions for their most complex IT problems. As well in the open source software market.

 

Image CC by David Martín :: Suki_ ::/Flick