Meet Moviri at the Career Day of the Politecnico of Milan! The event will take place on May 16th 2012 from 9 am to 5 pm at the Campus Bovisa-Broggi, in Via Lambruschini 4, 20156 Milano.

You can meet our team at the Moviri Stand, Nr. 40, to discover and learn more about our job opportunities, or attend our speech at 10.50 am in room BL 27.02.

During Career Day, you will have a chance to attend our Company Presentation and to hear from our Top Management. We will talk about our values and our approach to work but we will also have as testimonials junior colleagues who, thanks to last year’s Career Day, have joined Movìri. You can discover through their experience what it is really like to take on exciting technology challenges in a great working environment!

See you very soon at Career Day!

Moviri speech was at 10.30 am in room BL 27.1.8 (first floor); at our booth #18 we shown all job opportunities we are offering in our Milan and San Francisco offices.

View all the photos of the day on our Facebook Page

For further details visit the official event website: http://www.careerservice.polimi.it/go/99344

“The 2nd annual Splunk worldwide users’ conference”. The title itself explains better than anything else what Splunk cares about and which is its primary market. This summer we decided to pay a visit to .conf2011 and see if other users and customers really share the same enthusiasm we experience in Italy.

The conference spanned over three days: the first one consisting of optional training classes called Splunk University and the rest made of more than 50 speeches, packed in 5 tracks over 10 sessions.

The registration process was as smooth as it gets and we received some partner brochure, a beer sleeve and a useful Splunk Quick Reference Guide that will make our next new customer very happy…

We arrived at the conference main stage after the “university”, just in time to socialize and get a beer (that’s what the sleeve was for!) at the “Birds of a feather” session. This informal, bring-your-own-ideas session was built around some major topics, which had been voted by the attendee community, that the participants could freely dispute. We got to know some of the 600+ attendees and noticed Splunk users are not only of the technical type.

Next on the agenda was a visit to the Splunk Solutions Lab, where we could get our hands on the newly released applications and interview their creators. The typical motto was: you just can’t do all of this with anything else. Splunk already had a comprehensive ecosystem of extension applications and additions like Splunk App for VMware, Splunk App for Microsoft Exchange, Splunk App for Transaction Profiling and Splunk App for Web Intelligence will greatly help current and new customers monitoring these complex environments.

Day Two

The second day started with the keynote. CEO Godfrey Sullivan shared his vision with us: enable Splunk users dominate the three major forces that shape IT today: the Cloud, Big Data, and the Consumerization of IT consumption. Machine-generated data will be accessible, usable and valuable to everyone. Thanks to Splunk, of course. It was then the turn of Eric Swan, co-founder and CTO, who highlighted the novel ways Splunk has been employed this year and shared some insights on the forthcoming release. Paul Sanford, Director of Product Management, described the new developer platform being built at the brand new Seattle office. Christina Noren, SVP Solutions, detailed how Splunk will grow its application offering to provide more out of the box value to end-users. Rob Das, co-founder and chief architect, unveiled a new, cloud-based product offering: Splunk Storm. A pay-as-you-go version of Splunk suitable to analyse data being generated in the cloud.

After the keynote we headed to our first session of choice: Building a solutions business on Splunk. Splunk partners from Sideview, SPP and Aplura explained how they are currently doing business through Splunk and what are the major issues they face while either developing custom applications or providing consulting services. We totally agree with what they reported: as Splunk enables new use cases, it’s very difficult for a customer to go over the “that can’t be done” idea and embrace Splunk not as a product, but as a platform onto which build custom solutions.

We then attended the official presentation of Splunk Storm. This cloud-based offering is basically made of two components: Splunk as we know it, plus the infrastructure to automatic provisioning, scaling and backing up. Users will be able to stream their applications’ log data to specific endpoints and have the ability to analyze it with Splunk without the need of a local deployment. We think a cloud-based solution is a must-have for most modern software providers but the Italian market is not yet ready due to a somehow slow cloud adoption (maybe due to strict Data protection authority measures?).

During the following session, Achieving enterprise level security visibility using Splunk, General Electric shared their experience while using Splunk as a SIEM to analyze more than 100 billion monthly security-related events. They have gone far integrating an automated asset discovery tool with Splunk. Their lesson learnt has the flavour of a best practice for almost anyone:

• categorize events through event types;
• documentation, documentation, documentation of all the fields available for any type of data;
• training of all the internal customers.

After the afternoon break, a large american university described how they had grown Splunk from a point solution to a platform. Their story was perfectly aligned to the company philosophy of starting small then growing big. It is not always an easy path as Splunk can do so many things that sooner or later its owners will want it to do something someone else is already doing. It takes patience but in the end, as this university told the audience, we will all be able to use Splunk for the most diverse use cases, with great satisfaction of its end users.

Just before getting ready for dinner we got the chance of listening to Paul Sanford’s What’s new in Splunk APIs and SKDs. The room was completely full: there wasn’t even space enough for people standing! The new Seattle-based office is all about creating the basis for a developer platform: 3rd party or custom products will be able to use Splunk repository and functionalities. The current work involves releasing new Python, Java and Javascript APIs, polishing and documenting the REST web-services and describing a custom application development as a streamlined process. Having experienced the lack of such tools ourselves we think the Seattle team will greatly help both Splunk customers and partners in expanding the class of problems the product will be able to address.

If that wasn’t enough for the day, the .conf2011 staff had organized a party at the California Academy of Sciences. After a walk through the rain forest, having the opportunity to admire fishes, exotic snakes and butterflies we took the Tour of the universe at the digital planetarium, exploring the universe from the universe point of view. So, in three words: Python and Big Data! The party went on with cool music (the day after we discovered the DJ was Rachel, head of the Documentation team at Splunk), nice food and a flood of beer.

Day Three

To start the third and last day the best way, we enrolled in the Optimizing searches session. We already knew some of the contents due to more than two years of consulting on the topic, but we now discovered why things work like that. Just before the end of the speech, we learnt how to write custom search commands using Splunk’s map-reduce model. What a nice take away!

Under the title Visibility inside your virtual environments, Splunk App for VMware, one of the most awaited apps, has been unveiled. This App stands as a collection point of any information generated in the virtual infrastructure, being able to poll data from both the Virtual Center and the ESX hosts. Now administrators will be able to answer questions like “Onto which host was my VM executing one week ago?” and much more. The current release has a focus on getting data in, and provides basic correlation dashboards. Version 2.0 will include many more dashboards and out of the box analyses. We can’t wait to see this in action.

To take a break on all the hot stuff coming out, we enrolled in the User Interface testing labs to provide our own feedback on the upcoming version of Splunk. We wanted to contribute somehow. And the 32nd floor of the Westin St Francis offered a magnificent set up.

Last but not least was the Roadmap feedback session where each of the Core, Storm, Dev, Windows and Solutions teams detailed its own vision on future work. Long-term roadmaps are never a statement of actions, but the message seemed clear:

• improve the user experience, by automating as much as is possible and expanding the configuration capabilities through the administration interface
• improve resiliency by hardening the High Availability and Disaster Recovery features
• build a base platform able to manage both unstructured and structured data, providing services through the current, web interface or third party and custom portals.

Wrapping up

This year’s user conference has been a major celebration of the many use cases Splunk has helped with, presented by the users themselves.

It has been a broad success, too, due both to the number of participants (which roughly doubled since last year) and the audience involvement we experienced. The many technical sessions were mostly addressed to users with minimum-to-intermediate experience on the product, but we were surprised to see so many attendees interested into the advanced ones!

Although we collected some criticism to the product (the Flash charting interface, the complex dashboarding system, too many configuration files, …) these all have one common denominator: they are all about minor aspects of the interface and none of them is about missing functionalities!

If you feel there are important features missing or (reasonable…) things you cannot do with Splunk, voice your questions and we will do our best to help. We have learned a lot in our three days in San Francisco.

Paolo Prigione

How this will this new directive impact banks and financial services firms, given that the it is required to be deployed within 30 months, that is at the end of 2013?

Requirements

According to DPA, banks are required to comply with both technical and organizational requirements. First of all, we can split them between mandatory [M] versus appropriate [A] (i.e. not mandatory) requirements.

Technical Requirements

1. Operations Logging [M]: all applications which allow customer data access by bank employees must produce a log where a minimum set of information is collected;  identification code for the employee who accessed the customer information, operation timestamp, workstation/terminal identification code, customer identification code which the operation refers to, financial product the operation refers to.
2. Logs Retention  [M]: logs must be stored in a secure way for at least 24 months.
3. Alert implementation [M]: alerts must be configured to trigger alarms when anomalies are detected (i.e. too much balance inquiries on the same bank account).

Organizational Requirements

1. Outsourcer Designation [M]: when customer information is exchanged between different branches or third parties, they became responsible and the law requires that evidence and disclosure is provided via a responsible formal designation.
2. Audit  [M]: internal audits must be in place. Audits shall be focused on how customer data is managed, which alerts and alarms were triggered and, moreover, on logs availability and reliability. Periodic checks must be in place (once a year) and specific audits are carried out when anomalies are detected. The Audit organizational function must not be the same function that is in charge (i.e. responsible) of data management. Audit results must be formalized and communicated to the internal organization / to DPA if required
3. Privacy Policy Integration (“Informativa”) [A]: If applicable, customers should be notified that their information is exchanged between different branches,
4. Customer notification [A]: Institutions are required to promptly inform customers of every illicit event in data management
5. DPA notification [A]: Institutions are required to promptly inform DPA of any significant illicit event in data management

The Scenario

The figure below depicts a typical scenario. A current balance inquiry performed on a customer account using an internal application to access customer financial information should produce a minimum set of information (A). These information are collected in logs which must be stored for two years (B). If an anomaly is detected an alert should be  triggered; the customer should be informed (C) about the anomaly. Both periodic and on demand reports are in place (D) to ensure internal staffs reviews (i.e. a Compliance Function) and a fast response to alerts.

How Moviri can help customers to be compliant with the DPA Measure

Moviri managed similar projects in the past using Splunk.

The Moviri – Splunk Inc. partnership started in 2008 has been proved to be the optimal solution in addressing complex compliance requirements, like the one expressed in a previous DPA Measure (the DPA System Administrator Measure ) where, again, both Technical and Organizational requirements were in place.

In my experience, Splunk is one of the most innovative and effective log management solution. Originally designed for IT Operations Management (i.e. incident management, troubleshooting, etc) but it revealed indeed a high degree of flexibility in different fields, not  directly related to Operations. Taking into account Splunk to satisfy compliance requirements, some benefits I’ve immediately appreciated are:

• Data indexing, a paradigm to extract information from different data sources and load data into Splunk flat files. Since the only requirement to data indexing is that it must contain a timestamp, actually every kind of information can be indexed without additional development effort.
• Integrity, generally speaking, a compliance measure that involves logs requires to store them in a secure way (otherwise, think about audits reliability…). One of the most useful features in this case is Data Signing, since it allows  data changes detection, i.e. a log file tampering after it has been stored in the database.
• Reporting can be managed with interactive dashboards or email alerts. I’ve found this very useful. Since people dislike using several tools all the time, it is better for them to investigate in depth only when an alert is raised in the email raised.

Conclusions

Given the centrality of logs in the DPA Measure, I can envision two cases:

1. A log concentrator is in place; for those of them who implemented it with Splunk, a limited effort will be required on the log concentrator side – even if at the moment the deployment is dedicated to IT Operations Management.
2. A log concentrator is not in place or, more generally, existing log concentrators aren’t as flexible as they should be: they could use this as an opportunity to put in place a pilot project with Moviri and Splunk.

In the meanwhile DPA will be asked to provide more explanation on the requirements. We know for instance that home banking is not  included in the directive, but which kind of operations are actually required to be logged? Only branches employees ones? Or are application support teams, system administrators and DBAs included as well?

Ready for the third and last day at Velocity Conference (here our take on day 1 and day 2), starting with a very interesting keynote: how Facebook approaches optimization in datacenter consumption (including a benchmark about power consumption in cooling 1U vs 2U vs 4U server)

After that we attended one of the quickiest, funniest but coolest talks by by Artur Bergman, director of engineering at Wikia, on “use SSD (Solid State Drive) and save lifes”,  with a 12 second MacBook boot live demo (thanks to SSD obviously).

Going on, we were impressed by Google guys using the Google docs presentation tool, inserting code, webpage, JavaScript,… Inside the slide. One of their talks was about the Google Instant search feature, and Chrome instant pages (the browser preloads the pages that you are most likely to click on). We think it will certainly cause headaches to everyone working on web analytics tools, due to “false positive” hits. But we have to admit that is very cool getting the right page while you are still writing the URL in the nav bar.

In the “What’s new sessions” provided by browsers development teams (Chrome, Firefox, Internet Explorer, Opera), we appreciated how all of them are adding a lot of feature and plugins for performance analysis and diagnostics. Sounds good to all performance & optimization consultants, and surely to all end-users that will experience faster websites.

Artur Bergman also had a great speech showing the capability of Google Analytics in facing huge amounts of data inside a web browser: during the demo he analyzed some web site data managing plots with 4 Billion points in real-time: more amazing stuff from Google.

Our last session was about ad serving s in website and performance related issues. Julia Lee form Yahoo! showed a case study: “YMail turned off ads for 3 days and load time dropped 73%! “. Lesson learned: be careful relying on third party services in your page, or design your apps for multi-thread loading and rendering (i.e. use ad-hoc iFrames).

In the exhibition hall, we spent a lot of time with two interesting companies: AppDynamics changes the approach to Java diagnostics monitoring, avoiding the “instrumentation conf file” and focusing on the whole transaction crossing the Java layer. Apica gave us a complete demo on their SaaS solutions for end user monitoring and load testing, with multiple geo-distributed measurement points and shared with us some interesting considerations on performance testing and monitoring in the cloud.

So, what do we take away with us from Velocity?

It was our first time here and we are happy to be part of this conference. Innovative ideas were shared and new performance-related tools were shown that should be part of our toolkit to help our customer in improving their quality of service. And we scouted new technologies that we will have to look into in more detail… surely a lot of work to do when we come back to our office in Italy.

Aside from the content, we also want to share some funny feedback on Velocity… Just seconds after a new slide goes up including some key-concepts – and even before the speaker starts talking – Twitter and Facebook posts go off  instantaneously. Social media rules.

And also Mac rules here… some years ago, all “tech gurus” used to carry a Linux laptop… now everything here is branded Apple (iPhones, iPads, Macbooks everywhere)… Geek’s evolution.

At the end, Steve Souders announced that he will co-chair the first Velocity Europe event (planned in November in Berlin). A great opportunity to bring “Velocity Culture” in Europe.

Luca & Nicholas

Another day at the O’Reilly Velocity Conference 2011. While day 1 was dedicated to workshop sessions, day 2 starts with the keynote speeches, followed by different talks, grouped by four main performance-related topics.

Also, today the exhibition hall opens, and we hope to get some insights aboout innovative solutions by emerging companies and start-ups, as well as to get to discuss with technical engineers about the solutions that they presented yesterday, during the workshops. We talked with several people, technical engineers, sales guys, we introduced ourselves and discussed about performance and optimization related topics. From all useful meetings we have had, we want to highlight some of those.

After some quick talk with the Splunk guys (a Moviri partner whose solution addresses log management and troubleshooting), we moved across the exibitors booths and had a quick chat with the DynaTrace engineers. They provide an interesting Java/.NET diagnostics and end-user monitoring solution with a cool feature: it can extract data from the code in execution and can identify the username (or other useful information). In addition, they released a free tool, called Ajax Edition, that can analyze performance client side and give you some tricks on optimizing rendering, DOM execution, Jasvascript, etc.

Then, we discussed with Aptimize, which provides a web optimization tool focused on increasing the performances of web applications by reducing the number of requests-response , compressing and caching the static content, optimizing rendering and calls to external resoruces.

While wondering trought the hall, we also saw a new solution that addresses the need for alerting policies: it works by collecting events from standard third party monitoring tools (such as NAGIOS, Ganglia, etc…) that would need to alert somebody, and then apply preconfugured rules and workflows to actually send alarms; the alarm itself can be routed to the operating support team, and if it doesn’t respond in time it can be escalated to the backup team. It’s also possible to “load balance” the calls and define scheduling policies between several support engineers. Cool!

Last, but not least, we talked with the guys at ExtraHop. They have an analysis and reporting tool which visualizes statistics gathered from network flows, a solution which is closer to the “infrastructural layer” than other end user passive monitoring solutions (such as HP RUM); it can be useful if you whant to analyze and summarize performance data coming from different layers (web, database, app server, … ).

On to the conference sessions. We followed a very interesting speech from the Microsoft team about some use cases for optimizing MSN and Bing systems, and a quick -but very interesting- speech by Gene Kim. This talk, which had the highest words-per- seconds rate I’ve ever heard, focused on creating a Dev/Test/PM/Ops Supertribe, by highlighting all kinds of conflicts that can arise between different organizations inside a company and sharing common goals in order to move to an High Performing IT Organization, because “Higher Performing IT organization are more stable, nimble, compliant and secure.”

Now waiting for the velocity 2011 night party.

Luca & Nicholas

Here we are:  day 1 of Velocity 2011.

While we attended several events in the past as partners of HP, it’s our first time attending this O’Reilly conference, and we are quite curious in anticipation of what we will hear in this week about IT performance, operations management, code optimization, mobile web analysis, and more.

The first step, as usual, is registration, then getting some funny gadgets from sponsors and filling up our coffee mugs. We are required to personalize our badges with some ribbons and create our personal profile… “Performance Tuning and Benchmarks” plus “web performance” should be ok.

After having completed our profile, it’s time to review the schedule of the day, refill one more time our mugs with hot java (pun intended) and then attend the first workshop of the day: “Performance Tools”. In this talk, a panel of Google, DynaTrace, Blaze and TruTV guys showed us a “must-have” toolkit that helps to address problem quicker by developing little tests. While that’s the best way to start a performance-related conference, we can understand that is becoming more and more importanto to approach performance matters also from the developer point of view (starting analyzing performance in dev environments, on one’s own laptop) , and taking care also of the client performance (rendering and resource consumption) as an integration to the standard approach, which includes server-side performance analysis, user concurrency, back-end optimization.

The second conference session was very surprising: under the title “Decisions in the Face of Uncertainty” we found a tightly-packed overview about how to manage data from a statistical point of view, from pascal triangle to baysian models, passing through percentile, normal distribution, game theory and other statistical concept. John Rauser (Amazon) gave a quick recap of basic statistics concepts (that were lost somewhere in our mind) that must have had sound like a lullaby to some of the attendee – the yawning rate was exceptionally high by the end of the talk.

Then the day went on to another workshop focused on mobile web:  the speakers showed us some fresh open-source tools that help developers testing in real-time their code and analyzing performance (e.g. phantomjs) and reduce bandwidth usage (e.g. the “cache manifest” technique, applied to mobile world). The other talks were focused on operations management issues and tools that can help takle them, especially in cloud environments: provisioning, orchestration and monitoring the cloud pattern are quickly became one of the hottest topic at this time. Cool stuff.

The last workshop of the day was about automating tests by creating some phyton script that interacts with web browsers based on the SeleniumHQ framework. A low cost way to get started addressing performance analysis from the development stage. It’s yet another interesting point of view, but our opinion is that this kind of approach cannot replace the standard load testing one, but can be a valid complement to it.

Summary of the first day: performance analysis can starts from the development phase thanks to open source tools, but when you are facing intranet application with restricted areas, complex navigational path, correlations, parameters, data sets, concurrent users and scalability, you will probably have to rely on standard performance management tools, with a dedicated testing environment.

More info at http://www.ideatoproduct.it/

In our experience, the most affordable way to address this issue is to follow a maturity model based on a pragmatic, staged approach that mitigates the risks impacting process development and organizational systems.

Maturity Level 0 – Firefighting

At this stage, there is a complete lack of any process and management has not even recognized that there is an issue to be addressed. Day-by-day emergencies are the driver and firefighting is the only approach to solve problems.

Maturity Level 1 – Single Customer / Application

At this stage, the most critical customer or application is selected because management has evidence that an issue needs to be addressed. However, there is no standardized process yet and ad-hoc approaches tend to be applied on a case-by-case basis.

Some of the key indicators at this level are:

• No maturity goals are defined
• Reliance on the knowledge of individuals
• Testing is used just to demonstrate that the system works
• No effort is performed to track the progress

Maturity Level 2 – Monitoring & Testing

At this stage, more than one core-business application is monitored with different vertical tools used by each department (Networking, DBAs, Sysadmins, etc.) and some procedures are put in place by different people undertaking the same task. Management has started initiative(s) aimed to define the common foundation needed in the next level.

Some of the key indicators at this level are:

• Several tools could be used by different departments
• Lack of a centralized monitoring solution
• Infrastructure and/or end-to-end monitoring are in place
• Basic testing techniques and methods are institutionalized
• No formal training or communication of standardized procedures

Maturity Level 3 – Performance Optimizing

At this stage, some formal procedures have been defined and communicated and a software testing group is established. Management has clearly defined maturity goals and standard tools have been selected. A centralized monitoring solution has been adopted in order to integrate the previous “silo” approach. Typically, the organization is transforming by building up new departments or teams dedicated to testing and monitoring.

Some of the key indicators at this level are:

• Basic procedures are in place, but they are the formalization of existing practices
• It is left to the individual to follow these procedures
• A centralized monitoring solution has been adopted
• A training program is established

Maturity Level 4 – Business Optimization

At this stage, it is possible to monitor and measure compliance with procedures and to take corrective actions where the process is not working effectively. A process-oriented culture is emerging and processes are under constant improvement.

Some of the key indicators at this level are:

• Procedures are institutionalized
• An organization-wide review program is established
• The testing process is part of Software Development Lifecycle

Maturity Level 5 – Process Optimization

At this stage, processes have been refined to best practices, based on the results of continuous improvement. Information Technology is able to improve quality and effectiveness, making quick to adapt to business changes.

Some of the key indicators at this level are:

• Statistical quality control methods are used
• Processes for defect prevention are in place
• Organization is using Information Technology to achieve business goals

Visit us at booth #1.03 and discover all job opportunities we are offering in our Milan and San Francisco offices.

For further details visit the official event website: Career Day at Politecnico di Milano