Moviri blog

Performance Data Reporting – Velocity Amsterdam

velocity_amsterdam

 

 

 

 

 

In this post, I’ll go through some interesting hints around the “performance data reporting”. Even if this is not a fancy or trending topic the solutions proposed address issues that are quite common (and often underestimated) for most performance engineer. Due to the increase of complexity, data sources and therefore KPI over last years, to assess or just to monitor a service (web or mobile application) became a nightmare: find the “rotten apple” is definitely not an easy task.

The two speeches that I picked-up provide different point of view to the “performance data reporting”. From one side there is the pure theoretical & mathematical approach provided by Heinrich Hartmann (Chief Data Scientist at Circonus – @heinrichhartman) in Statistics for Engineers. On the other side, we have the customer centric view offered by Simon Hearne (Senior web performance technical consultant at NCC Group – @simonhearne) in Beyond the waterfall – meaningful web performance visualizations.

One of the most interesting step of the Heinrich speech has been on a quite common problem: how can we provide a synthetic view of the available data without losing relevant information? Or, as he more accurately said – given a bunch of data points how can we:

  • characterize the distribution in one or two values;
  • have a characterization robust to outliers.

The following charts provide a clear example to this kind of problem: peak erosion caused by aggregation.

1st graph2nd Graph

 

 

 

 

 

 

 

In the two graph, the same peak is shown with height 0.08 and 0.03 due to the aggregation applied to chart the same data over different time spans.

 

Two the solutions presented by Heinrich:

1- Histogram Aggregation

3rd Graph

 

 

 

2- Max Aggregation

4th Graph

 

 

 

 

A second interesting hint provided by the data scientist is about the use and abuse of the standard deviation as typical displacement from the mean value. Std. dev. is popular because everybody learns about it at school and it has extremely nice mathematical properties as:

  • 68% of data falls within 1 std-dev of the mean;
  • 95% falls within 2 std-dev of the mean;
  • 99.7 falls within 3 std-dev of the mean.

Problem is “this is utter nonsense” since all these properties are true only for normally distributed data! When was the last time that you check about your data distribution? Moreover std. dev. is definitely not good to measure outliers.

So how can we represent the displacement from the mean value? Heinrich suggest to keep in mind that we have other options… do you know the mean abs. dev.?

EDIT 01/27/2016: As suggested by Heinrich Hartmann in the comments,  the best alternative to std. deviation measures are provided by quantiles (e.g. 99%-quantile) or quantile-based measures like the inter quartile range (span between 75% and 25% percentile).

 

As anticipated, the speech from Simon Hearne, provides a more customer centric approach, based on the assumption that as performance consultant: our job is to find the best visualizations to support our story; in order to present and share our findings with not technical people in an easy and understandable way.

The following chart provides a Simons analysis of the common data visualization approaches for web performance data.

 

5th Graph

 

 

 

 

 

In his opinion what are we missing is an intuitive way to provide live info about the performance of a whole site.

His solution is the “rum wall” (following chart), a data visualization based on the common google analytics data. Each box of the wall represents a site’s page: the length of the box is the page exit rate, the height is the page views count, while the color is given by the relative performances (resp. time increases from green to red).

 

6th Graph

 

 

 

 

 

 

A “rum wall” is hence able to show if and where you have to focus your analysis: the page linked to a big red box has for sure a problem!

If you use google analytics in your site you can try the rum wall using the following link: rumwall.webperf.tools …. but don’t forget to thanks Simon for sharing his ideas and tools!

Attend the Velocity 2015 conference in Amsterdam has been an amazing experience. In just 3 days I collected tons of information and ideas directly from the most innovative and smart people active on topics like web operations, performance and DevOps. If you want to build & deliver sites, apps and services that are fast, scalable, resilient, and highly available to be at the Velocity conference is a must!

Moviri at Velocity Conference 2015 – (Part II)

velocity_amsterdam

 

Build Resilient System at Scale

(Here you can find the 1st part of the post)

There were so many interesting sessions that I wished I was able to travel across space and time to attend them all, but I’m not quite there yet, so I had to pick some. I mostly focused on sessions in the Performance and Metrics & Monitoring categories and would like to spend few words about those I enjoyed the most.

  • Extreme web performance for mobile devices by Maximiliano Firtman (@firt). I totally agree with him on “responsive design” is a mean and not the end goal for user experience. I believe any of us has at least once used a awesome-looking website with terrible performance, and I also believe that the overall experience was unsatisfactory for the majority of us. He also gave very convincing numbers about how complex is to have consistent performance given the extreme diversity in mobile device configurations.
  • Measuring the performance of single page web applications by Nic Jansma (@nicj) and Philip Tellis (@bluesmoon). Frameworks are nice and speed up coding, custom framework are even nicer because grant total control, but can turn into a nightmare when it come to troubleshooting performance problems. Especially when the framework you’ve coded against override the standard navigation timing and resource timing API and your monitoring tool is ineffective and leaves you in the dark.
  • Building real-time metrics pipelines by Samantha Quiñones (@ieatkillerbees). Publicly stating you’ve failed is never easy, and saying you failed three times is even harder, unless you understand the meaning behind “fail fast, fail cheap” mantra. She understands it very well and now her system is live, meets performance expectation, and most importantly generate business value for her company.
  • Alert overload: How to adopt a microservices architecture without being overwhelmed with noise by Sarah Wells (@sarahjwells). She definitely is not afraid of the (in)famous “microservices deathstar” (simply google it for some examples), and not because she’s a Jedi (well, maybe she is) but because she defined and thoroughly executed a strategy to address the challenges of monitoring such architectures.

As a Movirian, Velocity Conference is a great place where to learn from peers, to bring them your experience from successful projects and to have fun.

Moviri at Velocity Conference 2015 – (Part I)

velocity_amsterdam

Build Resilient System at Scale

Amsterdam installment of 2015 Velocity Conference was all about building end-to-end optimization through rapid development, continuous deployment, deft instrumentation, and automated feedback loops. DevOps together with Microservices, Container, and Optimization were the words most used during speeches and keynotes. Well, no surprises here if you think that those IT organizations that embraced their underlying concepts emerged as clear leader and innovators (Netflix, Uber, Facebook, and the like).

Keynotes were inspiring and fun, here are some takeaways:

  • A team of three people can move all your services to a new datacenter overnight from hotel wifi connection, provided you have the right bottle of whiskey (Genesis: Terraforming a New Home for Firefox Crash Reporter – JP Schneider & Daniel Maher);
  • Most probably your next customer will come from the Asia given the projected population will be 5 billion by 2050 and forecasted online retail spending for China only will be 1 trillion USD by 2019 (Ensuring a High Performing Web for the Next Billion People – Bruce Lawson);
  • Conversational computing which you can think of as the evolution of current personal digital assistants like Siri (Apple), Iris (Android), Cortana (Windows) is fast progressing although you can only either fly a drone or have it take a picture, but not at the same time (A Day in the Life: An Immersive Data Experience – David Boloker).

 

Looking at tools in the APM domain, there are two clear trends driving their roadmaps: 1) ease of deployment and administration so that you no longer need to ask your precious systems, dba, and middleware experts to take care of them (much more true for SaaS deployment options); 2) increase of capabilities and features coming from the Analytics domain with the goal of making it easier to slice and dice data and better monitor what matters.

This will probably lead to all-in-one offering fitting for small organization or project with low complexity, while big organizations will keep building their best of breed solution out of market offerings.

If you like to learn more about the amazing breakout sessions,  take a look at the 2nd part.

Moviri at Performance & Capacity 2015 CMG Conference

 

Alamo

The Alamo Mission, part of the San Antonio Missions UNESCO World Heritage Site in San Antonio, Texas.

We had the pleasure of sponsoring and attending the Performance & Capacity 2015 in San Antonio (TX), the 41th US National Computer Measurement Group (CMG) conference.

The Computer Measurement Group is a not-for-profit, worldwide organization of IT professionals committed to sharing information and best practices focused on ensuring the efficiency and scalability of IT service delivery to the enterprise through measurement, quantitative analysis and forecasting.

What we like the most about CMG conferences is the unique opportunity to get in touch with top capacity and performance professionals, working on different industries and organizations around the globe. It is a pleasure and a privilege to sit together and share experiences and ideas about how to approach actual performance and capacity planning problems! Plus, it is also a great occasion for networking with our customers and prospects, getting to know each other in person and spending some relaxing time outside of the conference hotel.

As Moviri, we have a long tradition of attending CMG conferences to share our best practices and constantly keeping up to date with the advancements of our capacity and performance field.

However, this year was special for us as we not only managed to get a paper approved for this year conference (second year in a row!), but our Stefano Doni had the honor of receiving  the conference Best Paper award! We feel especially delighted for this important recognition and wish a huge thanks to all the referees that spent their time reviewing our paper and providing very useful feedbacks! We’re proud of this international award, we think it speaks for itself about the quality of what Moviri delivers.

Plus, Renato Bonomini managed to deliver an outstanding vendor tool talk on “Business Capacity Modeling with BMC TrueSight Capacity Optimization”, regarded as one of the most appreciated sessions by the attendees – great work Renato!

As usual, we also had a vendor booth which enabled us to present our offerings to US prospects and talk with customers & BMC partners. Thank you Jon!

FlyerBest Paper: Lessons from Capacity Planning a Java Enterprise Application: How to Keep Capacity Predictions on Target and Cut CPU Usage by 5x

What is this work all about? Suppose you need to estimate if your user facing service will be able to cope with the increased traffic of your most important business campaign. The stakes are high, you can’t be wrong or the business will be severely impacted. Chances are that your capacity predictions will fall short and the service will *crash* well before the estimated capacity will be reached. In this talk, we described actionable methodologies that we developed over years of capacity planning business critical Java enterprise applications.

Overall, the session has been very well received, we got a lot of questions and interesting feedbacks – thank you!

If you did not have a chance of attending the session, you can sign up for an upcoming webinar on this topic where we will present it live.

 

Webinar-Button_banner_post

 

 CMG Session Highlights

As always, CMG is packed with many interesting sessions, selecting and summarizing the best ones is no easy task! Here are two of the most interesting sessions we have attended.

Hadoop super scaling

Outstanding talk by the great Neil Gunther on Hadoop scalability. Hadoop is the most common widely deployed big data framework. Despite its popularity, its performance characteristics are not well understood and the myth of linear (or even super-linear!) scalability is commonly thought among big data developers. Neil shed some light on the real Hadoop scalability and debunked the super-linear scalability myth.

Identify the cause of high latencies in Storage Traces Using Workload Decomposition and Feature Selection

Very good talk by Daniel Myers, a PhD and CS professor who also worked on modeling storage systems @ Google. The proposed methodology allows the identification of the real cause of high latency in storage systems using machine learning. For example, the method can identify which factors (among any) contributes the most to high storage latency, for example high IOPS, high randomness, burstiness and so on. The method was tested against production storage traces collected from Microsoft servers and was able to provide interesting results: nice work!

Conclusions

It’s been a real pleasure to attend this year CMG conference. It was packed with super interesting sessions, we enjoied networking with a lot with customers and partners and were delighted to share an award-winning Moviri-developed capacity planning methodology, hopefully contributing to advancing the capacity planning field with a new actionable approach.

Are you puzzled by our findings and would you like to know if the same is happening to your infrastructure and Java applications? Feel free to get in touch with us, we’re here to help you!

MoviriPayoff_II

Movirian Stefano Doni awarded 2015 Computer Measurement Group Best Paper Award

CMG-15-Social-Media-Banners-Banner

Managing the capacity and efficiency of business-critical Java applications is challenging. At the next Computer Measurement Group’s 41th International Conference Performance & Capacity 2015 from November 2nd to November 5th in San Antonio, Texas, Moviri will give you actionable solutions.

We are delighted and honored to receive the prestigious 2015 Computer Measurement Group Best Paper Award. The award is CMG’s recognition for papers judged exemplary by industry peers and reflecting the best of the best offered at the conference. 

 Our capacity wizard and award recipient Stefano Doni will be a key speaker on Wednesday November 4th with a talk entitled Lessons from Capacity Planning a Java Enterprise Application: How to Keep Capacity Predictions on Target and Cut CPU Usage by 5x.

In this work Stefano proposes actionable methodologies and key metrics that enable you to

  • Highlight the hidden bottleneck of many Java applications
  • Devise a business-oriented capacity model that represents Java memory bottlenecks
  • Detect unsound memory usage patterns and anticipate memory leaks
  • Uncover a well kept secret – the garbage collector drives the CPU usage of your servers (not your business!) and how to fix it
  • Show how the garbage collector might be your first scalability bottleneck.

We will be also there with our booth.  So don’t miss the chance to meet us in person !

BMC Technology Day in Chicago March 2015: a great event in a great venue

BMC Tech Day33
Andrea Gallo and Renato Bonomini of Moviri report from the Willis Tower, which hosted the BMC Technology Day 2015 in Chicago on March 11th. BMC Software showcased the updated TrueSight portfolio and for Moviri it was the perfect opportunity to discuss trending technologies and solutions with customers and partners.

The event at the Willis Tower (formerly Sears Tower and for 25 years the tallest building in the world) was organized in five different rooms, one for each BMC portfolio pillar in the distributed world , where each room hosted several sessions where attendees shared success stories and use cases. Herb Van Hook, BMC’s VP and Deputy CTO, and Tony Seba, lecturer from Stanford University, delivered the opening keynote. Similarly to BMC Engage 2014 in Orlando, the keynote focused on the new “Digitalization era” and on what is everyone’s position on these new disruptive technologies. No matter what you or the market think, the future is now!

Moviri at BMC Tech Day Chicago

The break-out sessions included discussions around several topics – let’s summarize the hot ones:

  • Less click, more value – Ease of access to the important information, tailored to customer needs (the Vodafone experience with SmartIT).
  • Power is nothing without control – infrastructure continuously growing in dimension and complexity, with companies aiming to achieve clearer visibility of its composition and status.
  • Disruptive Benefits with IT Cost Transparency – how BMC TrueSight Capacity Optimization helped HCSC.
  • What it takes to be a Capacity Planner – the journey from a simple spreadsheets to a full-blown and mature Business Aware Capacity Management system and the skills that help in the job.

Two sessions in particular saw Moviri being mentioned several times in Performance Management & Analytics:

Session 2: Customer Case Study Health Care Services Corp (HCSC)

Our friend Ben Davies from Health Care Services Corporation set the motto of the day to “suck less” while describing the journey that enabled cost transparency at HCSC. The key business benefits, as Ben mentioned, were achieved with Moviri’s help to get onto the right direction and avoid “shooting ourselves in the head”, to the point where HSCS scaled the process from “1 to 150 applications in 90secs”.

BMC Tech Day15

Session 4: Proactively Adapting to Changes in IT – Customer and Product Mgmt. Q&A Panel

Moviri joined a very interesting Q&A Panel, moderated by Clem Chang of BMC Software, where participants were able to share their thoughts on the topic ‘Future trends of IT’. Renato, together with BMC software consultants and product managers, had the chance to share the vision and discuss how to proactively adapt to an ever changing IT world. The hottest topics discussed were:

  • Big Data & Cloud – How can IT Operations and Capacity Managers support these technologies? Moviri is in the front line and we are ready to announce new solutions for capacity of big-data.
  • A single console to rule them all – Is there a plan to merge all of the different consoles into a unified platform? Having an holistic view is the greatest help to connect the dots and stay ahead of application failures. Furthermore, the introduction of IT Data Analytics is designed to streamline operations to find and resolve outages causes as quickly as possible.
  • “We can’t find good capacity planners!” – a few organizations attending the panel pointed out that it’s very hard to find capacity planners with any level experience.

The Moviri team in Chicago shared what some of the key skills and common profiles of good capacity specialists are. That is, after all, what Moviri does best. If you are looking for some solid capacity skills, make sure you take a minute to talk to our experts.

BMC Tech Day38

Moviri and the Digitalization Era: 2014 BMC Engage Conference report

BMC Engage 2014

It was show time this October in Orlando for the 2014 BMC Engage Global User Conference!

We were present at the event both as sponsors and as speakers, learning about and building upon the renewed and refreshed BMC value proposition (our partners introduced officially their new brand identity and logo), showcasing some of our success cases and strengthening relationships and developing a vision of the future with and among our customers.

Capacity Management took the spotlight in 4 customer-centered sessions. During each of them, Moviri’s role was highlighted, confirming our leading role as partner-of-choice in BMC’s Performance and Availability Ecosystem. Citrix, HCSC and Sanofi explained how they make sure to fulfill critical business initiatives with our help, while Northern Trust identified the supported integration with SCOM – provided by Moviri – as one of the key driver in the decision to adopt TrueSight Capacity Optimization.

The Digitalization Era and Disruptive Technologies

bmc ceo

The main protagonists of the opening and closing keynotes, BMC’s CEO Bob Beauchamp and Tony Seba, lecturer in entrepreneurship, disruption and clean energy from Stanford University, introduced a vision that contains the seeds of a very exciting idea – or scary, depending on your perspective: the IT industry is again on the verge of a new era, the Digitalization Era, and considering how information technologies have been at the core of all market disruptions over the last decades it’s quite hard to confidently predict what we’ll all be talking about in three years.

What it takes to win then? Well, we can outline the key success factors as:

  • Embracing a constantly growing holistic approach, expanding visibility beyond the the four walls of the Datacenter
  • Unleashing and successfully leveraging the business opportunities coming from IoT, Cloud Computing and Big Data. While we’re approaching the Zettabyte of data milestone, the market is waiting for guidance on a renewed set of best practices to use in conjunction to “what it used to be
  • Consumerization, empowerment and automation. Architectures, hardware devices and applications keep becoming more and more powerful and complex – take for instance the spreading of Converged Architectures. It is critical to provide the market with automated, integrated, intuitive and robust solutions to ensure IT professionals spend time taking decisions rather than understanding how to tackle data.

B0Ppjj0IMAILTIa.jpg-large

How does Moviri fit in this picture? Looks like our efforts to stay on the edge of the curve most likely will pay off.  Let’s review some highlights of what we showcased at Engage.

Sanofi’s Transformation Journey

2014-10-14 23.51.54

Chris Wimer, Sanofi Global Capacity Manager, and I talked about how Capacity Management supported the ambitious IT Transformation program of consolidation, harmonization and automation in line with Sanofi’s renewed business strategy.

image14

The key takeaways for the audience:

  • Rely on experts — and the experts are Moviri. ☺
  • Clearly define goals and challenges at the very start.
  • Perform a rock-solid assessment and analysis of the as-is, identify KPIs to measure the effectiveness of the process, review and prioritize the available data sources…then start implementing.
  • Think business-oriented: business value realization always comes first. That’s why processes and data streams need to be standardized, analytics need to be clear and accessible, areas of improvement must be measurable, priorities and service levels must be agreed upon.
  • Be holistic: as we have shown, one of the key success factors has been the capability to centralize IT and Business information under the same roof
  • Focus the reporting efforts on risk mitigation, provide improved visibility and reduce costs.
  • Leverage business-aware analysis, understand IT services as a whole, report and optimize capacity in terms of actual and predicted business demand… and make it all work by taking advantage of Splunk and Moviri’s connector for TrueSight Capacity Optimization!

image10

Accounting and Chargeback in Citrix…

image11

Troy Hall, Citrix Senior Business Analyst, and Moviri’s Renato Bonomini took the lead sharing best practices and recommendations on how to set up an effective Accounting and Chargeback (ACB) process with TrueSight Capacity Optimization.

One of Renato’s quotes stood out: “Not all servers were created equal”.  With that he referred to the business value of performing ACB and achieve improved cost visibility, making sure to have a fair allocation and showback of costs and resources.

As Renato explained, any ACB process is made up by 3 components:

  • Identify the costs, which is usually the most painful point – as confirmed by Troy – since reliable data is generally not available and needs to be estimated.
  • Allocate costs, choosing from a variety of possible models taking in account KPIs such as Simplicity, Fairness, Predictability, Controllability, Complexity.
  • Chargeback costs to the final consumer.

image12

Knowing the theory is not enough: a critical success factor is to exercise ACB as an Iterative-V process:

  • Start with a model that makes sense considering the state-of-art information.
  • Run the analysis, check results and evaluate deviations from expectations.
  • Adjust cost rates and allocation models, working closely with finance.
  • Re-iterate the process.

image13

Troy outlined the most important positive outcomes in adopting ACB:

  • Increased satisfaction of business units, more educated IT decisions.
  • Resources are freed up and reallocated to more critical business objectives.
  • Ability to quickly reconfigure models and automatically generate auditable results.
  • The effort required for monthly showback reporting is minimal, saving precious time and providing detail for even greater IT resources savings.

… and again ACB in HCSC!

Similar concepts were also part of the session held by another of our most important and beloved customers, HCSC. Rick Kickert, IT Planning Senior Manager, and Ben Davies, Senior IT Business Consultant, discussed how ACB helped achieving cost transparency at HCSC.

image02

Rick focused on the need to drive competitive advantage via efficiencies and cost-effectiveness through IT optimization, simplification, and agile infrastructure & applications, closing his remarks with some figures about the results:

image04

Ben delighted the audience with some real-life success tips. The concept of “useful, consistent lie” resonated in particular with the audience, as Ben’s way to explain that it’s wrong to expect to earn value from ACB only when perfect cost models are in place. The iterative-V approach – accept inaccuracy, but be more consistent and useful after each iteration – is the most sensible choice.

image05

End even if we did not have the chance to present together, Ben took his time to give Moviri some credit and depict his experience working with us. Here’s a couple of quotes:

“If smart were sexy, and it is, the Moviri guys would be super models”,

and

“the best capacity results come from Moviri, BMC, then Moviri, Moviri, Moviri, Moviri.”

Hadoop Capacity, SAP Hana and more

Engage has not only been about conference sessions, but also and more importantly, about welcoming IT professionals at our booth, discussing about Performance & Availability and presenting our solutions, some of which we announced and premiered at Engage. The hottest topics have been:

  • New connectors for TrueSight Capacity Optimization: we are working on complete solutions – integrations and reports – for two of the most important platforms available in the market: Hadoop Cloudera and SAP Hana.
  • Moviri Managed BCO: our managed service offering for BMC Capacity Optimization (BCO) customers designed to maximize the return on their BCO investment.

Let me recall the seed of an idea I started this post with: the Digitalization Era and Disruptive Technologies. Well, we believe Hadoop can play a relevant role in the emerging market of Big Data. Of course as a source of data; but especially for capacity managers also as a service that needs to be planned, optimized and aligned to real business demands.

We’ve seen a few proposals of Hadoop capacity analysis best practices being published on the web during the last few months, and our feeling has frankly been that “this is not quite enough… we could do way better” – again, quite: we also had outstanding conversations!

Engage gave us the chance to announce our intent to take it to the next level. To us, a capacity analysis has always meant more than reporting on “how much CPU I’m using and how the trend looks like”.  How about dependencies and correlations? How about achieving an understanding of the true causes of resource consumption? How about being effective in identifying both hardware and software bottlenecks using a holistic approach? How about proactively planning based on actual demand information, rather than trending data?

It surely sound ambitious: that’s why we’ve been so excited in hearing so much good feedback on what we’ve shown, and we’re thrilled that many customers have agreed to help us moving forward by working together on a trial version of the final product, to test and refine our intuitions and use-case design on the field.

Want to know more or join us? Let’s Engage!

To learn more about Moviri’s work with BMC and our capacity management track record, you can directly contact one of our experts.

image06

Saipem’s session at Splunk .conf2014: Security and Compliance with Moviri

splunk-conf2014-5150-0-s-307x512We recently joined our partners at Splunk at the global annual Splunk .conf2014 conference, which took place on October 6-9, 2014 in Las Vegas.

We had the pleasure to share the stage there with one of our key clients and Oil & Gas contract industry leader Saipem.

Saipem had a prominent role at the event, with a session led by Ugo Salvi and Luca Mazzocchi –  respectively Saipem CIO and CISO on “Splunk at Saipem: Security and Compliance in the Oil and Gas Industry”.

 

Why Splunk at Saipem

In his presentation, Mr. Salvi outlined how Saipem introduced Splunk in the IT ecosystem. Back in 2012, the company combined log management and compliance to establishing dashboards and an automatic alerting system to meet SOX and privacy compliance regulations. Thanks to the agility of the product and Moviri’s expertise, Saipem successfully achieved unprecedented flexibility and time to market and decided to build upon the early success.

When, at the end of 2012, in two instances Saipem’s competitors where it by business-disrupting malware attacks, Saipem started focusing on a few questions: are backup policies effectively in place? Can we restore business operations — Saipem’s yearly revenues exceed $10 billion), in case an attack is successful? The answers are now with what Saipem calls “Backup inspector”, a new application based on Splunk that has enabled Saipem to enforce policies across the enterprise for all the backups of all the relevant applications.

In 2014, Saipem has progressed even further by setting up a new SIEM (Security Information and Event Management) system using Splunk and the Enterprise Security app to identify, address and investigate security threats. Meanwhile, Saipem, convinced of SPlunk’s reliability as a source of information for IT and business (e.g. license usage or distribution of accounts around the world), is looking at possible future applications such as:

  • Industrial systems (SCADA, supervisory control and data acquisition)
  • APM, Monitoring control room and troubleshooting
  • IT and Business reporting

SCADA data as a new opportunity

During the Q&A that followed Saipem’s session, most questions were related to SCADA data. Mr Salvi pointed out that Saipem is going through a POC, waiting for the opportunity to have these devices available from their operations. Saipem is also working with R&D to understand how to monitor pipeline stress and stretching, using fiber optic technology. Splunk can correlate these industrial and pipeline data with other metrics. Another key challenge regarding SCADA data is investigating possible threats due to maintenance activities, such as for example the VPN that is opened to perform maintenance on systems on offshore vessels.

Key takeaways

splunk conf teamII

For Moviri, working with Saipem and Splunk has been a long and rewarding process. And after this first few years of these implementations, Mr. Salvi made it a point to share Saipem’s key takeaways:

  • Splunk has replaced and continues to successfully replace other tools within the Saipem IT ecosystem.
  • The challenge of digitization and IT in an enterprise setting that, like Oil & Gas, is by its very nature rooted in the analog, industrial world presents great opportunities.
  • It has been a long journey for Splunk at Saipem (SOX in 2012, Backup Inspector in 2013, SIEM in 2014)… and it is far from being over!!

To learn more about Moviri’s Splunk capabilities, visit our Analytics and Security services or talk to our Experts.

 

 

 

VMware vForum 2014 and the Future of the Data Center Operational Model

At the recent vForum conference in Milan, where several Moviri experts were in attendance, VMware unveiled the results of a survey of more than 1,800 IT executives. The key findings highlight the increasing gap between the needs of the business and what IT is actually able to deliver.

IT is slowing business down

Two-thirds of IT decision makers say that there is an average gap of about four months between what the business expects and what IT can provide. The exponential growth in business expectations is increasingly unsustainable for traditional IT management. The IT challenges in the Mobile-Cloud era, as defined by VMware, require for example real-time data analysis, continuous delivery or resource deployments in hours, if not in minutes. This is not achievable with old resource management defined by hardware-driven infrastructures.

The VMware’s answer

The answer, according to VMware, comes from the so called Software-Defined Data Center (SDDC). VMware’s vision for IT infrastructure extends virtualization concepts such as abstraction, pooling, and automation to all of the data center’s resources and services to achieve IT as a service (ITaaS). In a SDDC, all elements of the infrastructure (networking, storage, CPU and security) are virtualized and delivered as a service, in order to bring IT at the Speed of Business.

VMware IT at the Speed of Business

Nowadays, enterprises invest in average only 30% of their IT budget in innovation. The reasons include manual device management, slow and non-automated provisioning, production workloads handled via email and everything else IT needs to perform just to “keep the lights on”. According to VMware, SDDC could help enterprises save 30% of capex and up to 60% of opex, allowing the investment in innovation to reach 50% of the IT budget and thus increasing market competitiveness.

VMware NSX release

VMware has drawn inspiration from great players in infrastructure innovation like Amazon, Facebook, Netflix or Google and has developed products for each technologic silo: vSphere for x86 virtualization, VSAN for storage and the just recently released NSX for network virtualization, the big news of this year.

The VMware NSX network virtualization platform provides the critical third pillar of VMware’s SDDC architecture. NSX delivers for networking what VMware has already delivered for compute and storage using network virtualization concepts.

Network and Server Virtualization

In much the same way that server virtualization allows to manage virtual machines, network hypervisor enables virtual networks to be handled without requiring any reconfiguration of the physical network.

Network virtualization overview

With network virtualization, the functional equivalent of a network hypervisor reproduces the complete set of Layer 2 to Layer 7 networking services (e.g., switching, routing, access control, firewalling and load balancing) in software. The result fundamentally transforms the data center network operational model, reduces network provisioning and simplifies network operations.

Since networking is no more just connecting machines, but rather delivering services like enable balancing, manage firewall rules or route planning, the first impression is that network virtualization, thanks to the combination of OpenFlow capabilities and experienced companies like VMware or Cisco, will have a similar revolutionary impact on the network, as server virtualization has had on servers.

As for x86 hypervisors, network hypervisors do not replace but enhance and add features on top of physical layers. They do not make connectivity available, they provide services and improve datacenter network agility. Physical connectivity is still required, but complex connections are no longer a requirements because everything can be handled at the software level.

Network virtualization (NV) in a nutshell, it’s a tunnel. Rather than physically connecting two domains in a network, NV creates a connection through the existing network to connect two domains. NV is helpful because it saves the time required to physically wire up each new domain connection, especially for new virtual machines. This is valuable because companies don’t have to change what they have already done. They get a new instrument to virtualize their infrastructure and make changes on top of the existing infrastructure.

Network Virtualization

The key benefits of NV could be summarized in:

  • Ability to easily overcome VLAN limits to support scalability network requirements.
  • Each application can have its own network and security policy via NV traffic isolation improving multi-tenancy.
  • No need to touch Layer 1 for the majority of requests.
  • Improved performance for VM-to-VM traffic within the same server or rack due to the fact that traffic is handled by the virtual switch because all the hops to the physical layer are just skipped.
  • NV management tools represent a single point of configuration, monitoring and troubleshooting in large virtualized data centers.

However, there are some disadvantages:

  • The new workload coming with NV features is now handled by hypervisors’ kernel and not from dedicated hardware.
  • Performance degradation and network traffic increase by tunnel-header overhead.

Conclusion

Current adoption of NV technology is in its very early stages with a few organizations in production and more communications service providers and enterprises in pilot implementations. Running NV software like NSX as an overlay to existing network infrastructure provides a relatively easy way to manage VM networking challenges. As a result, we expect NV adoption to strongly increase during the next two years in order to close the gap with SDDC and speed up IT to meet business demands, as suggested by VMware.

(Images courtesy of VMware vForum 2014)

 

Insights from the DCIM leading edge at DatacenterDynamics Converged 2014

Claudio Bellia and I had the pleasure to attend the DatacenterDynamics Converged conference for the third year. DCD Converged is a one-day, peer-led data center conference that gathers IT and Facility professionals.

What I like the most about this conference are the case study sessions. During these sessions, organizations present real-life initiatives about how they managed to improve data center efficiency and save a good amount of energy – good for the environment and company cash. In the process, they share interesting internal details, sometime previously undisclosed, about the company data centers.

The DCD conference traditional audience are facility designers and operators. However, over the years I have noticed increasingly relevant IT sessions, which demonstrates a growing recognition that, in addition to facilities, also servers, storage and networks management offers large potential optimization opportunities.

Here’s some highlights from the two sessions I have found the most interesting.

Telecom Italia and CNR Case Study: Energy Consumption and Cost Reduction through Data Center Workload Consolidation

The case study highlighted how Telecom Italia (the largest italian telecommunication operator) is saving significant amounts of energy and costs thanks to initiatives specifically targeted at the “IT part” of the data center through servers, storage and workload consolidation.

The session started off by showing the global medium-term trends that are driving the enterprise IT evolution. Besides the usual suspects, such as cloud computing, big data and open source software, two lesser talked-about trends are the adoption of commodity hardware (no big news here) and IT efficiency, which can be summarized as proper server selection with energy efficiency in mind and, at the micro level, using knowledge of server workloads to perform consolidations and improve capacity planning (a relatively new concept). As IT optimization experts, in Moviri we wholeheartedly believe in IT Efficiency as as a major source of innovation, energy and cost savings, available to organizations todays and in the future.

The next key point was about technology refresh initiatives that Telecom Italia has performed to take advantage of the evolution in servers such as adoption of virtualization and more powerful and efficient CPUs and storage such as thin-provisioning and autotiering, to optimize the usage of the existing resources and to slash energy bills. Traditional capacity management approaches too often can be summarized as: “do not buy new hardware until the installed capacity (read: sunk investment) is fully used”. At Moviri we believe this mantra has become obsolete, as current data center cost structures are very different from 20 years ago. Energy costs are impacting data center TCO in important ways (20% and rising) and proper technology refresh and server selection are paramount to achieving energy and costs saving, while reducing footprint and increasing processing capacity too!

Another point is related to server processor selection. Despite CPUs increasing computing power and capacity over time (courtesy of Moore’s Law), what is often overlooked is which processor provides the best fit from a performance and cost perspective. Telecom Italia highlighted how newer Intel CPUs, if properly selected, can be a source of cost and energy savings. I can add that, in my experience, I have seen how different CPU performance and price greatly vary among different models on the market, so equipping the entire server fleet with the same, standard CPU will guarantee unused capacity, unnecessarily high acquisition (CAPEX) and energy (OPEX) costs. As performance is workload dependent, a proper characterization of datacenter workloads is paramount to really understand what are the requirements and consequently make the best investment.

Finally, the session focused on the adoption of an emerging paradigm called Intelligent Workload Management: managing the workloads in dynamic, virtualized environment, to achieve increased utilization levels, reduce stranded capacity and save costs. Telecom Italia adopted this concept by implementing two products: Intel Datacenter Manager and Eco4Cloud. The former product enables a fine-grained collection of power and thermal metrics directly from the servers. The latter is an automated workload consolidation solution, designed by a CNR (Italian National Research Council) spin-off company, that can pack virtual machines into fewer servers and hibernate the others. This resulted in at least 20% energy savings (gigawatt-hours!) and clearly highlights how a data center infrastructure management solution (DCIM) is important to optimize data center efficiency and capacity.

IMG_3198

Case Study: Eni Green Data Center – Why 360° Monitoring?

This case study, presented by Eni (the largest italian gas utility), highlighted the energy efficiency design and operation of the newest company data center.

The first interesting data point relates to facility efficiency (a.k.a. Power Usage Effectiveness or PUE) and to where the power goes (read: is wasted) in a typical data center vs. an optimized one. Standard, legacy data centers typically are poorly efficient (PUE > 2, or even 3), which means that up to 2/3 of the total energy entering the data center is wasted before reaching the IT equipment. What are the greatest offenders? Chiller plants, fan and pump losses, UPS/transformer losses. In contrast, the newest ENI datacenter has been designed with a 1.2 target PUE, which means that energy wasted in facilities is less than 20 percent.

What did ENI do to achieve such level of efficiency? The actions where: (a) use of free cooling for > 75% of the time (b) use of high-efficiency chillers (c) introduction of large chimneys for natural heat extraction (taller than a 10-floor building!) (d) use of offline, small (200 KW), efficient (>99.4%) UPS (e) cold air containment.

A key insight that ENI shared was why a pervasive, comprehensive, fine-grained monitoring system is paramount to understand and tune a complex plant such as a data center. ENI’s monitoring system is tracking 50,000 metrics with a 10-second collection interval – and no data aggregation or purging is planned! Such a vast amount of data enables ENI to identify anomalies, increase efficiency and prevent issues before they impact production operations, such as identifying fans rotating in opposite directions or uncalibrated energy meters reporting wrong values.

IMG_3184

I hope you enjoyed my summary. The main, positive message which I’d like to convey to IT and Facility managers struggling with tight budgets is: start looking closely to your datacenter efficiency and costs, chances are that you might save huge amounts of energy and money, decrease your company environmental footprint and increase your IT capacity, perhaps even avoid building unnecessarily new facilities. And be sure not to focus on facilities only, as the IT equipment is where most optimization potential can be realized.

If you’re looking for help, check out our Capacity Management offering!