Moviri blog

VMware vForum 2014 and the Future of the Data Center Operational Model

At the recent vForum conference in Milan, where several Moviri experts were in attendance, VMware unveiled the results of a survey of more than 1,800 IT executives. The key findings highlight the increasing gap between the needs of the business and what IT is actually able to deliver.

IT is slowing business down

Two-thirds of IT decision makers say that there is an average gap of about four months between what the business expects and what IT can provide. The exponential growth in business expectations is increasingly unsustainable for traditional IT management. The IT challenges in the Mobile-Cloud era, as defined by VMware, require for example real-time data analysis, continuous delivery or resource deployments in hours, if not in minutes. This is not achievable with old resource management defined by hardware-driven infrastructures.

The VMware’s answer

The answer, according to VMware, comes from the so called Software-Defined Data Center (SDDC). VMware’s vision for IT infrastructure extends virtualization concepts such as abstraction, pooling, and automation to all of the data center’s resources and services to achieve IT as a service (ITaaS). In a SDDC, all elements of the infrastructure (networking, storage, CPU and security) are virtualized and delivered as a service, in order to bring IT at the Speed of Business.

VMware IT at the Speed of Business

Nowadays, enterprises invest in average only 30% of their IT budget in innovation. The reasons include manual device management, slow and non-automated provisioning, production workloads handled via email and everything else IT needs to perform just to “keep the lights on”. According to VMware, SDDC could help enterprises save 30% of capex and up to 60% of opex, allowing the investment in innovation to reach 50% of the IT budget and thus increasing market competitiveness.

VMware NSX release

VMware has drawn inspiration from great players in infrastructure innovation like Amazon, Facebook, Netflix or Google and has developed products for each technologic silo: vSphere for x86 virtualization, VSAN for storage and the just recently released NSX for network virtualization, the big news of this year.

The VMware NSX network virtualization platform provides the critical third pillar of VMware’s SDDC architecture. NSX delivers for networking what VMware has already delivered for compute and storage using network virtualization concepts.

Network and Server Virtualization

In much the same way that server virtualization allows to manage virtual machines, network hypervisor enables virtual networks to be handled without requiring any reconfiguration of the physical network.

Network virtualization overview

With network virtualization, the functional equivalent of a network hypervisor reproduces the complete set of Layer 2 to Layer 7 networking services (e.g., switching, routing, access control, firewalling and load balancing) in software. The result fundamentally transforms the data center network operational model, reduces network provisioning and simplifies network operations.

Since networking is no more just connecting machines, but rather delivering services like enable balancing, manage firewall rules or route planning, the first impression is that network virtualization, thanks to the combination of OpenFlow capabilities and experienced companies like VMware or Cisco, will have a similar revolutionary impact on the network, as server virtualization has had on servers.

As for x86 hypervisors, network hypervisors do not replace but enhance and add features on top of physical layers. They do not make connectivity available, they provide services and improve datacenter network agility. Physical connectivity is still required, but complex connections are no longer a requirements because everything can be handled at the software level.

Network virtualization (NV) in a nutshell, it’s a tunnel. Rather than physically connecting two domains in a network, NV creates a connection through the existing network to connect two domains. NV is helpful because it saves the time required to physically wire up each new domain connection, especially for new virtual machines. This is valuable because companies don’t have to change what they have already done. They get a new instrument to virtualize their infrastructure and make changes on top of the existing infrastructure.

Network Virtualization

The key benefits of NV could be summarized in:

  • Ability to easily overcome VLAN limits to support scalability network requirements.
  • Each application can have its own network and security policy via NV traffic isolation improving multi-tenancy.
  • No need to touch Layer 1 for the majority of requests.
  • Improved performance for VM-to-VM traffic within the same server or rack due to the fact that traffic is handled by the virtual switch because all the hops to the physical layer are just skipped.
  • NV management tools represent a single point of configuration, monitoring and troubleshooting in large virtualized data centers.

However, there are some disadvantages:

  • The new workload coming with NV features is now handled by hypervisors’ kernel and not from dedicated hardware.
  • Performance degradation and network traffic increase by tunnel-header overhead.

Conclusion

Current adoption of NV technology is in its very early stages with a few organizations in production and more communications service providers and enterprises in pilot implementations. Running NV software like NSX as an overlay to existing network infrastructure provides a relatively easy way to manage VM networking challenges. As a result, we expect NV adoption to strongly increase during the next two years in order to close the gap with SDDC and speed up IT to meet business demands, as suggested by VMware.

(Images courtesy of VMware vForum 2014)

 

Insights from the DCIM leading edge at DatacenterDynamics Converged 2014

Claudio Bellia and I had the pleasure to attend the DatacenterDynamics Converged conference for the third year. DCD Converged is a one-day, peer-led data center conference that gathers IT and Facility professionals.

What I like the most about this conference are the case study sessions. During these sessions, organizations present real-life initiatives about how they managed to improve data center efficiency and save a good amount of energy – good for the environment and company cash. In the process, they share interesting internal details, sometime previously undisclosed, about the company data centers.

The DCD conference traditional audience are facility designers and operators. However, over the years I have noticed increasingly relevant IT sessions, which demonstrates a growing recognition that, in addition to facilities, also servers, storage and networks management offers large potential optimization opportunities.

Here’s some highlights from the two sessions I have found the most interesting.

Telecom Italia and CNR Case Study: Energy Consumption and Cost Reduction through Data Center Workload Consolidation

The case study highlighted how Telecom Italia (the largest italian telecommunication operator) is saving significant amounts of energy and costs thanks to initiatives specifically targeted at the “IT part” of the data center through servers, storage and workload consolidation.

The session started off by showing the global medium-term trends that are driving the enterprise IT evolution. Besides the usual suspects, such as cloud computing, big data and open source software, two lesser talked-about trends are the adoption of commodity hardware (no big news here) and IT efficiency, which can be summarized as proper server selection with energy efficiency in mind and, at the micro level, using knowledge of server workloads to perform consolidations and improve capacity planning (a relatively new concept). As IT optimization experts, in Moviri we wholeheartedly believe in IT Efficiency as as a major source of innovation, energy and cost savings, available to organizations todays and in the future.

The next key point was about technology refresh initiatives that Telecom Italia has performed to take advantage of the evolution in servers such as adoption of virtualization and more powerful and efficient CPUs and storage such as thin-provisioning and autotiering, to optimize the usage of the existing resources and to slash energy bills. Traditional capacity management approaches too often can be summarized as: “do not buy new hardware until the installed capacity (read: sunk investment) is fully used”. At Moviri we believe this mantra has become obsolete, as current data center cost structures are very different from 20 years ago. Energy costs are impacting data center TCO in important ways (20% and rising) and proper technology refresh and server selection are paramount to achieving energy and costs saving, while reducing footprint and increasing processing capacity too!

Another point is related to server processor selection. Despite CPUs increasing computing power and capacity over time (courtesy of Moore’s Law), what is often overlooked is which processor provides the best fit from a performance and cost perspective. Telecom Italia highlighted how newer Intel CPUs, if properly selected, can be a source of cost and energy savings. I can add that, in my experience, I have seen how different CPU performance and price greatly vary among different models on the market, so equipping the entire server fleet with the same, standard CPU will guarantee unused capacity, unnecessarily high acquisition (CAPEX) and energy (OPEX) costs. As performance is workload dependent, a proper characterization of datacenter workloads is paramount to really understand what are the requirements and consequently make the best investment.

Finally, the session focused on the adoption of an emerging paradigm called Intelligent Workload Management: managing the workloads in dynamic, virtualized environment, to achieve increased utilization levels, reduce stranded capacity and save costs. Telecom Italia adopted this concept by implementing two products: Intel Datacenter Manager and Eco4Cloud. The former product enables a fine-grained collection of power and thermal metrics directly from the servers. The latter is an automated workload consolidation solution, designed by a CNR (Italian National Research Council) spin-off company, that can pack virtual machines into fewer servers and hibernate the others. This resulted in at least 20% energy savings (gigawatt-hours!) and clearly highlights how a data center infrastructure management solution (DCIM) is important to optimize data center efficiency and capacity.

IMG_3198

Case Study: Eni Green Data Center – Why 360° Monitoring?

This case study, presented by Eni (the largest italian gas utility), highlighted the energy efficiency design and operation of the newest company data center.

The first interesting data point relates to facility efficiency (a.k.a. Power Usage Effectiveness or PUE) and to where the power goes (read: is wasted) in a typical data center vs. an optimized one. Standard, legacy data centers typically are poorly efficient (PUE > 2, or even 3), which means that up to 2/3 of the total energy entering the data center is wasted before reaching the IT equipment. What are the greatest offenders? Chiller plants, fan and pump losses, UPS/transformer losses. In contrast, the newest ENI datacenter has been designed with a 1.2 target PUE, which means that energy wasted in facilities is less than 20 percent.

What did ENI do to achieve such level of efficiency? The actions where: (a) use of free cooling for > 75% of the time (b) use of high-efficiency chillers (c) introduction of large chimneys for natural heat extraction (taller than a 10-floor building!) (d) use of offline, small (200 KW), efficient (>99.4%) UPS (e) cold air containment.

A key insight that ENI shared was why a pervasive, comprehensive, fine-grained monitoring system is paramount to understand and tune a complex plant such as a data center. ENI’s monitoring system is tracking 50,000 metrics with a 10-second collection interval – and no data aggregation or purging is planned! Such a vast amount of data enables ENI to identify anomalies, increase efficiency and prevent issues before they impact production operations, such as identifying fans rotating in opposite directions or uncalibrated energy meters reporting wrong values.

IMG_3184

I hope you enjoyed my summary. The main, positive message which I’d like to convey to IT and Facility managers struggling with tight budgets is: start looking closely to your datacenter efficiency and costs, chances are that you might save huge amounts of energy and money, decrease your company environmental footprint and increase your IT capacity, perhaps even avoid building unnecessarily new facilities. And be sure not to focus on facilities only, as the IT equipment is where most optimization potential can be realized.

If you’re looking for help, check out our Capacity Management offering!

Loadrunner in the cloud: testing everywhere you need

The new version 12 of Loadrunner and Performance Center, HP introduces several new features to the testing market-leading software and adds some enhancements that will leverage the time-to-test and extend the technical testing capabilities of the solution.

Testing everywhere

Cloud-based Load Generators

It is now possible to inject load from cloud services through Amazon Web Services: with this solution, customers with internet exposed application can execute load tests in an hybrid mode with a mix of load generators within their network (as for the preview version of LG) and load generators in the cloud, in order to simulate traffic from all over the world.

Enhanced support for Mobile Testing

It is now possible to test any mobile application, simply by using Android application (on rooted devices) and integrating Shunra Network Virtualization, that allows customers to discover and then virtualize real-world network conditions in the test environment, simulating different location and bandwidth situations.

New VuGen Testing Script Recording Features

It’s now possible to use the latest versions of most common browsers (not only IE 11, but also FireFox 23 and Chrome 30) for scripts recording. With the new integration with Wireshark and Fiddler, customers can now generate scripts in an easier way, avoiding use of custom requests for calls not recorded by the VuGen.

With several protocol enhancements, recording SPDY, HTML5, Flex, SilverLight (and much more) will no longer be a problem. The new TruClient to Web/HTTP converter utility allows you to reduce time to script, supporting simple Web as well as modern JavaScript-based applications and reducing scripting time.

Platforms Support Enhancements

Product installation is now possible also in Windows Server 2012 and without administrative accounts, with UAC and DEP enabled. This meets the needs of customers with strong security policies. The integration with Jenkins, and latest versions of Eclipse Juno, JUnit, and Selenium is now supported as well.

And more!

Many other enhancements on Continuous integration, new Protocol support are available with HP Loadrunner and Performance Center 12. You can find a full list at the following links:

 

Demystifying Oracle database capacity management with workload characterization

From simple resource consumption to business KPIs models, leveraging database workload

In a nutshell, Capacity Management is all about modeling an IT system (i.e. applications and infrastructure) so that we can answer business questions such as:

  • how much business growth can my current system support? (residual capacity)
  • where and how should I focus investments (savings) to increase (optimize) my capacity?

When facing typical three-tier enterprise applications, creating accurate models is increasingly difficult as we move from the front-end layer to the back-end.

Front-ends (i.e. web servers) and Mid-ends (i.e. application servers) can be modeled with relative ease: their resource consumption is typically driven by user activity in business-related terms (i.e. business transactions per hour,  user visits per hour etc). The chart below shows a typical linear regression model for such a system: the application servers CPU shows a good linear relationship with the volume of purchase orders passing through the application. The high correlation index (R-squared value next to 1) is a statistical indication that orders volume explains remarkably well the pattern of CPU utilization.

Capacity_1

The problem: how to model databases?

When you deal with the back-end (i.e. database) component of an application, things are not as straightforward. Databases are typically difficult to understand and model as they are inherently shared and their resources pulled in several directions:

  • by multiple functions within the same application: the same database is typically used for both online transactions, background batches, business reporting, etc.
  • by multiple applications accessing the same database: quite often a single database is accessed by multiple applications at the same time, for different purposes
  • by infrastructure consolidation: a single server might run multiple database instances

The next chart shows a typical relations among the total CPU consumption of a shared database server vs. the business volume of one application. This model is evidently quite poor: a quick visual test shows that the majority of the samples do not lie on or close to the regression line. From a statistical standpoint, the R-squared value of 0.52 is well below normally acceptable levels (commonly used thresholds are 0,8 or 0,9).

Capacity_2

The end result is: simple resource consumption vs. business KPIs models do not work for databases. This is unfortunate as databases are usually the most critical application components and the ones customers are most interested in properly and efficiently managing from a capacity standpoint.

The solution: “characterize” the database work

I will now show you a methodology that we have developed and used to solve challenging database capacity management projects for our customers. The solution lies in a better characterization of the workload the database is supporting. In other words, we need to identify the different sources of work placing a demand on the database (e.g. workload classes) and measure their impact in terms of resources consumption (e.g. their service demands).

If the database is accessed by different applications (or functions), we need to measure the resource consumption of each of them (e.g. CPU used by application A and B, IO requests issued by online and batch transactions and so on). Once we have that specific data, we are in a much better position to create capacity models correlating the specific workload source (i.e. online activity) and the corresponding workload intensity (i.e. purchased orders per hour).

That’s the theory, but how can we do it in practice? Keep reading and you will learn how we can take advantage of Oracle database metrics to accomplish this goal!

Characterizing Oracle databases work using AWR/ASH views

Starting from release 10, Oracle has added a considerable amount of diagnostic information into the DBMS engine. It turns out that this same information is very helpful also from the capacity management perspective.

It is possible to extract CPU and I/O resource consumption by instance, schema (database user), machine or even application module/action/client id (Module/action/client id information is available provided that the application developer has properly instrumented it. See for example: http://docs.oracle.com/cd/B19306_01/appdev.102/b14258/d_appinf.htm ).

Oracle tracks resource consumption on a variety of ways. The most important ones, from the capacity management perspectives are: per session and per SQL statement basis. The former metrics can be retrieved from V$ACTIVE_SESSION_HISTORY, the latter from V$SQLSTATS. The corresponding DBA_HIST_<VIEW> can be queried for more historical samples.

The idea is to:

  1.  identify how to isolate the application workload we are interested in (i.e. a specific user the application is using, or machine from which it is connecting)
  2. aggregate the session (SQL) based resource consumption to form the total resource consumption of the application.

How to do it in practice?

II’ll show you how Oracle work of an instance can be broken down by the application accessing it using session metrics.

In my experience per-session metrics proved to be more accurate than per-SQL statement metrics. Although the latter metrics measure actual SQL resource consumption (no sampling), the drawback is that they might not track all the relevant SQL statements that the database executed. Indeed, SQL statements might go out of library cache and therefore not being counted in AWR views and reports! See for example: http://jonathanlewis.wordpress.com/2013/03/29/missing-sql/

Step 1 is a function of how the application connects to the database. In this example, a single instance database was being accessed by different applications each using a specific user.

Step 2 is more interesting. How can we aggregate the application sessions to get the total number of CPU consumed? The Oracle Active Session History view samples session states every second. A possible approach is to:

  1. count the number of sessions that are ‘ON CPU’ and assign them a 1-second of CPU time
  2. sum all the CPU times within the desired time window (i.e. one hour), getting the total CPU busy time in seconds

It is important to note that this is a statistical measure of CPU consumption and therefore might not be 100% accurate. Always double check totals with resource consumption from other data sources known to be correct (i.e. monitoring agents or the operating system).

An example query that does this calculations can be found here:

SELECT TO_CHAR(sample_time, 'yyyy/mm/dd HH24') as sample_time, username,
   round(sum(DECODE(session_state,'ON CPU',
DECODE(session_type,'BACKGROUND',0,1),0))/60/60*10,2) AS cpu_secs
FROM DBA_HIST_ACTIVE_SESS_HISTORY a, dba_users b 
WHERE sample_time > sysdate - 7
   and a.user_id=b.user_id 
GROUP BY TO_CHAR(sample_time, 'yyyy/mm/dd HH24'), username 
   order by 1,2

Results

By using the above outlined methodology, I have isolated the database CPU consumption caused by the work placed by a specific application I was interested in (for example, Order Management). The next chart shows the relation among the physical CPU consumption of the database (caused by our selected application) vs the business volume of the selected application. The model now works great!

Capacity_3

I hope you are now agree with me that workload characterization is an essential step in capacity management. If properly conducted, it can provide you with remarkably good models when they are most needed!

Moviri at the Politecnico of Milan annual conference on the Internet of Things

The following post has been co-authored by Giorgio Adami.

Every year MIP, Politecnico di Milan’s school of management, organizes a conference to present the results of its research on the Internet of Things market. Moviri was in attendance and we are pleased to report some of the key themes, discussed throughout the event by a large spectrum of experts and influencers, academics, venture capitalists and managers of the most important companies in the industry.

First of all, just to establish common ground, Internet of Things (IoT) is an expression used to describe the phenomenon whereby popular and sometimes mundane objects are connected to, and reachable from, the Internet, becoming in fact nodes in the “Internet of Things”.  The evolution of IoT not only involves an efficient improvement of services or processes that already exist, but it primarily entails disruptive innovation.

Internet of Things today

Key data shared during the conference shows that the IoT market is growing. In 2013 in Italy alone devices connected to the cellular network have reached the number of 6 million, with an 20% increase compared to the previous year.

At the same time, the value of the IoT solutions based on a cellular connection is estimated at €900 millions, with an increase of 11% compared to the previous year. These numbers are remarkable, especially when they are considered within the context of the general trends in the economy and the negative growth of the local ICT market (-5% in 2013).

iot_moviri_01

Img credit: rework of Osservatori.net digital innovation (Politecnico di Milano, DIG)

It is not usual to evaluate the trends in technology innovation with an observation window of only one year, but last year has been an exception. In 2013 three main events concurred to accelerate the technology development:

  • Bluetooth Low Energy (BLE) is becoming a recognized standard. With its adoption in the Android OS and several other platforms (BLE is present in the Apple world since 2011), BLE aims to be the “official” technology platform for every IoT application in the Personal Area Network segment. BLE can overcome limits like the configuration of a mesh network, the compatibility between software and hardware of specific vendors and backward compatibility. These limits have slowed down the creation and the development of IoT applications.
  • Numerous platforms able to manage and develop application for multi-vendor devices were released, aiming to overcome the gap in the standardization.
  • GSMA has issued the first embedded SIM specs, i.e. specifications for a SIM integrated in the device while still in the factory. The bond between the physical SIM and the telco operator that owns and manages the SIM can be removed, so enabling the provisioning and administration of the SIM Over The Air.

iot_moviri_02

Img credit: rework of Osservatori.net digital innovation (Politecnico di Milano, DIG) 

Internet of Things means Smart Car

The automotive industry is the segment of the IoT market with the biggest opportunity for growth in the next few years. Under the European eCall regulation, starting in 2015 every new vehicle will have to be able to make emergency calls. As far as the rest of the world is concerned, Brazil and Russia have already released similar laws, while China and India are about to do that.

Some new models are already equipped with optional equipment that is capable of monitoring the health of the driver, to communicate with other vehicles (V2V) and with other infrastructure near the road (V2I). Today  95% of smart vehicles are equipped with GPS/GPRS sensors to trace the location and the actions of the driver for insurance reasons.

Internet of things means Smart City

Smart City is a concept related not only to technology, but IoT is becoming the standard technology layer for each Smart City implementation. The adoption of the IoT paradigm allows the multi-functionality of devices, promoting the development of projects shared with different actors, in a scenario where the allocation of the costs is a key factor for success.

Putting in place the so called Smart Urban Infrastructure allows the implementation of different “smart” applications that share the same technology infrastructure, with a saving of 50% and more in investment and operation costs. The main smart applications are dedicated to traffic and parking control, as well as to smart city lighting and waste collection. The implementation of a Smart Urban Infrastructure requires a model of cooperation between public and private actors and worldwide there are many successful cases of such a kind of cooperation, while in Italy there is still room for improvement.

Internet of Things means Smart Metering

Smart counters interconnected for real time measuring have been in use for years. In 2013 the Smart Metering segment has confirmed its positive growing trend.

Internet of Things means Smart Home & Building

Smart solutions for domotics and industrial automation are already in use; in 2013 significant improvements, focused on consumer benefits, have been introduced. BLE is the key factor for the development of IoT in the Smart Home & Building. More and more startups and as well as big actors like Google (as the recent acquisition of Nest Labs demonstrates) have invested in this segment.

 

In, sum the close relationship between IoT and other technological fields like IT Governance and Big Data has proved to be outstanding. Moviri will continue to closely follow the development of IoT around the world, helping its customers to maximize the value of the data they collect, supporting new business cases and leveraging the Big Data competencies of its consultants.

Active, or passive monitoring, that is the question —

How to make your user experience / SLA awake: active and passive monitoring combined

Service Level Management is  fundamental to ensure service quality at both the technical and business level. In order to be effective, not only internal services  but also third party ones (ex. ads) need to be monitored.

SLA monitoring should be carried out with both active and passive methods because they focus on two distinctive features:

  • Active monitoring, also referred to as synthetic monitoring, performs regular, scripted checks that simulate end-user behavior in controlled and predictable conditions

  • Passive monitoring concentrates on real end-user activities by “watching at” what they are doing right now

to be or not to be

Active (aka synthetic) Monitoring

What are the distinctive features of active monitoring and why is it useful ?

Synthetic script-based approach is the only feasible method for availability monitoring because:

  • it occurs from outside the datacenter and so it enables for monitoring of timing and errors specific to external web components not hosted locally

  • it determines if the site (or specific page) is up or down

  • it verifies the site availability when there are no end users hitting that particular page

  • it can test various protocols (not only HTTP/S) like Flash, Silverlight, Citrix ICA, FTP, etc.

  • it is able to execute scripts from specific locations and so it can determine region-specific issues

  • it is essential in order to establish a performance baseline before a new application release deployment

Actually, there are some drawbacks too:

  • a script tests the same navigational path over and over again

  • the test is not “real”: you can’t buy something every five minutes on a real web site

  • periodic sampling does not provide a good indication of what real users are experiencing

  • a script cannot test odd human combinations

Passive (aka sniffing) Monitoring

What are the distinctive features of passive monitoring and why is it useful ?

Passive “sniffing” approach is the only feasible way to look at real users conditions:

  • it looks at the traffic generated by real users of the website

  • it enables monitoring of real transactions, such as bank transfers, purchases, and so on

  • it measures all aspects of your users’ experiences: user location, browser, ISP, device, etc.

  • it detects all errors that can occur and is able to take screenshots of these situations

Actually, there are some drawbacks in this case too:

  • the data collected is limited to the traffic on the network the router is attached to

  • It requires real users traffic; if there are no users on the site it cannot collect any data

  • It offers no ability to monitor issues that occur outside of the network, such as DNS problems, network issues, and firewall and router outages

The combination

The combination of active and passive monitoring provides a more comprehensive view of performance, availability and end user experience. In addition, for companies that outsource their infrastructure, active monitoring offers a way to validate the SLAs provided by the outsourcer.

Using an approach that combines both passive and active monitoring methods offers the highest degree of quality assurance because issues can be detected before they occur or in near real time.

For additional information, pls contact me.

Hello Boston!

Last november Moviri reached another important milestone with the opening of its new US headquarters in Boston. This follows the relocation of a start-up team of five colleagues that will take charge to launch and scale the new venture.

I’d like to celebrate this milestone and go through the key reasons that led us to undertake this endeavor.

First, it’s a big business opportunity.

We’ll have access to the most mature market for IT services in the world, a market which appreciates and rewards companies built on skills, reliability and high-level professionalism, which is exactly how Moviri is and is perceived.

We’ve done this step by step, starting with the incorporation of a controlled legal entity and the progressive relocation of the core team during 2013. We’ve done this after we’ve tested the market, we’ve built solid business relationships with a core set of customers and partners, to the point where the US-based business already accounts for almost one sixth of our consulting services worldwide revenues.

The US market boasts some of the largest organizations in the world, where IT is a business driver (and no longer just an “enabler”) and where our IT Performance offering can provide the greatest value to customers.

Second, we are believers in internationalization.

To see and understand opportunities in the early stage; to create cultural diversity in what started as a “strongly typed” engineering organization; to have more interesting and varied work experiences for our team. It’s the unique blend of situations we aim to create and be a part of.

Last, we follow dreams and trust “gut feeling”.

If one walks into the MIT buildings, one cannot not notice the resemblance to the classrooms and feeling as sense of connection to Moviri’s engineering and research roots. It accounts for a lot. “Revenue” can be boring (even though “cash” is king…) and business  is also about passion. Choosing a place like Boston is part of it.

Our new office is located at One Boston Place, an iconic landmark in the Bostonian landscape. We’re going to write a new address on our email signatures:

Moviri Inc.
One Boston Place, Suite 2600
Boston, MA 02108

Why Boston? There are of course many very rational reasons for choosing Boston:

  • It’s a culturally rich city that features one of the most prominent global academic hubs with centers like Harvard, MIT, Boston University. In total 5 junior colleges and 18 colleges that primarily grant baccalaureate and master’s degrees, 9 research universities, and 26 special-focus institutions.
  • The “Greater Boston Area” has been home to many disruptive IT companies and it competes with Silicon Valley and lately the New York technology park.
  • It is a short flight away from major economic centers like New York, Philadelphia, Toronto and Washington DC and only few time zones away from Europe.
  • It is close to many of our banking and insurance customers.

I’d like to say “Good luck” to the Boston team and give an head-up to everybody: this is not an achievement that we celebrate, it’s just the beginning of a new journey… stay tuned.

Moviri Boston Office

HP Discover 2013 Highlights

HP discover 1

From December 9 to 12, the annual Hewlett Packard european convention, HP Discover, (@HPDiscover) took place in Barcelona. For the second time in a row, the Californian behemoth delivered the full firepower of its business units: Hardware & Networking, Printing & Personal Solutions, Software, Services, with more than 800 sessions throughout the event. The most discussed themes in software were Security, Big Data and Mobility.

Some highlights:

  • The annual HP-Capgemini report on “world testing trends” (built with data provided by 1500 clients), You can get the report here.
  • The Accenture customer case study on Security, for a global oil and gas enterprise customer, which featured clear evidence of how the impact and the effectiveness of cyber attacks often depend more on social context and the industry, rather than on technology.
  • The software product roadmaps and news did not deliver the “splash” we were used to expect from HP and Mercury, but rather focused mostly on the integration in the HP product ecosystem of the many solutions and products recently acquired, especially Vertica and Autonomy. As well as they paid particular attention to open source solutions that ara gaining traction in the market.
  • The demo of Autonomy implemented in inside a social media center, which included real time dashboards with sentiment analysis, tends and any message or post on the web that had to do with HP Discover. Not immediately applicable to Moviri’s business maybe, but visually impactful nonetheless.

HP discover 2

HP Discover was also a great opportunity to meet and converse with our customers, to identify new business opportunities and to plan jointly with HP for the upcoming year.

Finally, what better occasion to be in Barcelona and enjoy a tennis-like scoring soccer match at Camp Nou stadium between the Blaugrana e the beaten up Scots of Celtic football club?

HP discover 3

4 Tips in Data Presentation

The following post has been co-authored by Mirko Canovi.

Alberto Cairo is a journalist and infographic expert,  who teaches information graphics and visualization at the University of Miami’s School of Communication. In his book “The Functional Art: An introduction to information graphics and visualization” he says:

“The first and main goal of any graphic and visualization is to be a tool for your eyes and brain to perceive what lies beyond their natural reach.”

In my opinion Cairo got the point, and he summarizes well the central role of the Data Presentation. No matter what kind of analysis are you doing, if the reader can’t understand the results, it’s better do not perform the analysis at all.

In this article i’d like to show you some considerations about data presentation. My purpose is not give you the correct “always-good recipe”. What I want is to talk about some tricks for improve the outcomes of a well-done job.

When we design the data presentation of an analysis, we must try to be as much as possible Simple, Clear, Complete and Effective.

1. Be simple.

According to Cairo, we must put the reader on the center. He’s the judge of our work, so we must present the data in a way as much as possible simple for him. In other words, the interpretation of the results must require the minimum effort for the reader.

There are several ways to do this.

Putting too many information in a graph can create confusion, it’s better in this case to use two illustrations instead of one. In addition, if you know that two metrics represent the same thing, then use only one.

Others elements that can create confusion are outliers. Almost every data-set in the real world has outliers or less-important samples. If you can’t filter these samples in your presentation, try to aggregate them.

In the following example, we consider the average number of users per hour of a web server. The chart in figure 1 shows the statistic for every hour. If we look at the data-set, we can find that in the early morning and the late night there are not too many users, so maybe we can show only one aggregate value, like I do in figure 2.

data visualization 1

Figure 1: average number of users – 24 hours

data visualization2

Figure 2: average numbers of users – aggregated

Generally speaking, the best suggestion is to use commonly accepted practices. That means both using the old good graphical representation (histograms, curves, dispersion diagrams) and choosing the ones that the reader can understand better. So, if you are working for a specific customer, look at his document templates (if he has some, of course !).

2. Be clear.

When you present the data, the context of the analysis must be clear. Always be sure that the data represented are well understandable by the audience. Minimize or avoid ambiguity, giving the correct information (for example, using proper labels and axes in charts).

Choose always the right option between tables and charts. Charts are good for show differences between two metrics, but if the absolute values are important it would be better using a table. Plus, when you put data in a table, remember to correctly round off values when necessary. Data are results of calculation, and there are often round off errors: for example, don’t use sub-multiple of milliseconds for represent the Round trip Delay of a network packet.

We have also to keep in mind that there are different types of charts, and we must choose the best charts for different variables. Histograms are good for discrete-quantitative data (like the numbers of orders elaborated by an Order Management System), curves are good for continuous data (like the response time of a web server).

But it’s not only a matter of variables, it’s also important what we want to show to the audience. Bubble-charts are a good example. If you are using a bubble chart to show the difference between two variables, keep in mind that the circle’s Area is not proportional to the radius but to the square of the radius.

It’s a matter of perception. If there are two circles, and the radius of one is two times the other, often people say that also the area is two times, but actually is the square! So it’s difficult for some people in this case to “visualize” the correct information (see figure 3).

data visualization 3

Figure 3: The radius of the blue circle is two times the purple’s one

Last but not least, pay attention using symbols and colors, they must be absolutely unambiguous. Think about what happens if you show a failure of a server with the “green” color (i swear, i met this case in my life, and it wasn’t a good experience).

3. Be complete.

To be useful, the information we are showing must be complete.

When I was at the university, one of the most frequent mistakes was to forget the measurement units. I agree with my professors, without units a data-set doesn’t make any sense. It’s also a good practice to always put in our chart axes, the origin, the correct labels and legend. This is a way to maximize the information in our chart.

In some analysis, it’s also important to use the confidence intervals for random quantities, when the variance of results is high and the average values are not enough to compare two metrics. Average values aren’t always the best statistics to explain data, but this is another story.

4. Be effective.

To capture the audience’s interest, we need to design the data presentation with the goal clear in mind.

We can’t simply put the data in a chart following the practices i presented before, we also have to help the reader browsing the data to get the information he needs. Remember, “he” and not “we”, because we made the analysis for him.

To achieve this result, when you are designing a chart or a table, try to ask yourself questions like the followings:

  • Is this chart/table truly necessary ?

  • Did I put the information in the correct order, the most important one first ?

  • Did I use the correct scale to represent data and to underline the behavior that i found on the data ? (sometimes the wrong scale can hide some important information)

  • Did I put a spotlight on the data correlations that i found ?

  • Are the data correlations that i found correct/useful ?

Data presentation is a very important and actual topic and everyone talks about of “the art of Data Presentation” in almost every branches of science and media communications.

In fact, people often don’t know the effort needed to perform an analysis (and they don’t want to know); the only way to show them the value of your job is to design in a good way the presentation of the results.

Wait! What about tools?

Ok, I agree with you, we didn’t talk about the tools to build our presentations. But let me ask you a question: is it truly a matter of tools ?

Of course, in the real life we have to deal with different kinds of tools and products to manage an ever bigger amount of data. Tools are good, they are necessary, they sometimes are our best friends, because they can speed up our work.

In Moviri we help our customers in finding, designing and building software solutions to leverage the information “hidden” in their systems, increasing the efficiency and the performance in IT Service management.

But a software can’t be the “silver bullet” good for every situation. The most powerful tool is always located between the monitor and the chair.

Still skeptical? Look at this guy: http://www.ted.com/talks/hans_rosling_on_global_population_growth.html.

 

BCO Lookup Tables Explained

One of the strongest and distinctive BMC Capacity Optimization features is the ability to import capacity-relevant metrics from virtually any enterprise data source, being it a monitoring platform, a configuration management system, an asset inventory or even flat files or generic database tables.

Making diverse data flows from heterogeneous sources work together requires a high degree of flexibility. BCO Lookup Tables address exactly this need.

In this post we describe the capabilities enabled by Lookup Tables, together with best practices collected thanks to Moviri long-standing experience of BCO implementation and management projects.

Introducing Lookup Tables

BCO extracts data from sources using tasks. A task interfaces with a single source and has its specific settings. Each task has its own Lookup Table (LT for short) where it keeps tracks of the entities whose data it has previously extracted.

You can think of LT as the “Contact List” of each task. Every time a task has to load some data it goes through its contact list, if contact is already present the task simply adds more data to an existing BCO entity, otherwise a new contact is added and a new BCO entity is at the same time created.

A task may also “share” its LT with other tasks, so that measurements coming from different sources can be merged into the same BCO entity. Take the example of a CMDB source, managed by task C, and a servers’ performance monitoring source, managed by task M. In the contact list analogy and without sharing, the two tasks would build up over time separate contact lists. When C finds a new contact a new BCO entity is created, even if that same contact is also present in task M contact list. Whereas with sharing C is able to “peek” for known contacts into M contact list, therefore enriching existing BCO entity with more data, instead of duplicating entities. After peeking, C copies the contact into its own LT, for future use (e.g. if M is deleted).

bmc-lookup-1

For our purposes we won’t go into the mechanisms used to do the actual “contact” (i.e. the LT entry) matching, which may be addressed in a separate post.

What you can do with Lookup Tables

As anticipated, Shared Lookup Tables are useful when you want to load different information from separate tasks into same entities, avoiding duplications. Two main use cases:

  • Different measurements from distinct management systems enrich the same BCO entity. E.g. a server gets configuration data (HW Vendor, CPU Model…) from a CMDB while it gets performance data (CPU Utilization, Swap rate…) from a monitoring tool.
  • As measurement systems change over time, BCO can keep continuous data series for measured objects. E.g. if the enterprise monitoring platform is migrated from vendor X to vendor Y, the memory utilization pattern for servers is maintained without disruption in BCO.

On the other hand Private Lookup Tables are handy when testing specific integrations, because they provide isolation with respect to already existing entities.

This isolation is actually achieved also specifying clusters of tasks sharing LTs. Take the example of task C, managing CMDB, sharing LT with task M, managing the monitoring tool. Suppose now we need to integrate an asset inventory, e.g. to load rack position information for already managed servers. In order to test the integration of the three sources we can setup, on the same BCO instance, a second cluster of tasks Ct, Mt and At, managing respectively the CMDB, the monitoring tool and the asset inventory. This second cluster of tasks will create its own separate test entities. After verifying everything works correctly we’ll drop Ct, Mt and At, together with the associated entities and data, then we’ll create a new task A and safely share its LT with C and M for steady “production” usage.

bmc-lookup-2

Sharing direction and Master Lookup Table practice

As you may have noticed, there is an actual sharing direction between tasks that share LTs. This however does not affect the way entities are identified, i.e. “C shares LT with M” is completely equivalent to “M shares LT with C”. For sake of simplicity, BCO prevents the creation of sharing graphs or hierarchies, enforcing that in a single cluster of tasks sharing LTs only one task can “share”, while all others will “receive” the sharing. This entails that in a sharing group there is one task that has the special role of connecting all other tasks’ LTs.

As a best practice, it is advisable that this special task does not manage any data flow, and serves only as the point of connection for the tasks actually interacting with the data sources. The best practice provides two benefits:

bmc-lookup-3

  • It makes the connecting task more easily identifiable. It is usually named “Master lookup Table” task.
  • It makes the sharing group independent from the specific data sources, as any associated task can be removed or substituted without impacting the others.

Except for testing purposes (or other very specific cases which could be discussed separately), there is no valid reason not to include all “production” tasks in the Master Lookup sharing group: either data sources have some entities in common and therefore they do need to be merged, or sources do not have common entities and therefore sharing would do no harm, but will prepare in case some common entities appear in the future. The best practice can be thus extended to share the Master Lookup Table with any task actively bearing metrics into BCO.

Limit entities creation

In latest BCO versions a further functionality has been made available: a task can be prevented from creating new entities, and thus allowed only to load metrics to existing ones. Used in conjunction with sharing groups the feature enables to elect only certain tasks to define the scope of the entities imported into BCO.

Moviri faced this use case for a large IT service provider that adopted BCO. The request was to use BCO specifically for one of the provider end-customers. The monitoring platform though included the whole servers’ estate, and there were no means of configuring the task managing that source to restrict the extraction according to the needs. However, we were able to configure the task managing the CMDB data source only to import that specific end-customer servers’, and then flagged that task alone as allowed to create entities. After putting the two tasks in the same sharing group we transparently achieved the desired scope delimitation for the BCO instance.

Conclusions

Lookup Tables are a simple yet powerful tool that enables BCO to manage concurrently active data flow tasks, interacting with several sources. Using provided hints and best practices, collected on the field, you can effectively leverage Lookup Tables to keep isolated or merge entities loaded from distinct sources, to best support your capacity management activities.