A few weeks ago Moviri attended the Strata Conference 2013 in Santa Clara, California, one of the most relevant and well-attended exhibitions on Big Data. Today’s analytics and data visualization technologies are undergoing a phase of vibrant development, with fast-changing paradigms. The Strata Conference represented a great opportunity to catch up on key market trends and real-world applications and experiences from business use cases implemented by Silicon Valley’s top-tier players in the field.
Here’s a few highlights on the most interesting trends and topics we encountered during the conference.
The advent of Hadoop and his distributed filesystem HDFS, Hadoop Distributed File System, is getting a lot of attention. From our current perspective, this is the most promising technology; several companies (like Cloudera and MapR) are building enterprise solutions on it, and most of R&D efforts are focused on reducing latency and elapsed time to launch queries over MapReduce. HDFS, pillar of Hadoop, appears to be an established early standard and reliable stack currently widely supported by third party Big Data software and monitoring platforms.
Despite Hadoop/MapReduce dramatically reduced processing time for huge datasets, it’s not at the moment an appropriate technology suitable to process real time data while they are collected. In scenarios like these, Apache Hive (for Hadoop), Cloudera’s Impala (for Hadoop) and Shark (for Spark) are very promising technologies.
The keyword here is in-memory analysis. Platfora, Impala and Tableau are all leveraging this concept and have shown how they are implementing their solutions allowing to easily perform data discovery.
It’s a while we have been hearing about NoSQL capabilities and its advantage in being unchained from strict structured data, however it’s clear that the market is asking for SQL-compliant DBMS Database Management Systems, especially in technology replacement scenarios, but also to facilitate adoption of new technologies. Big entreprise customers have, up until today, suffered more from the impossibility to scale databases horizontally than destructuring data. Several column-oriented DBMSs are already allowing this, and a lot of community-based projects are working on this direction. Tim O’Brien‘s keynote on this topic was illuminating as it analyzed current market trends and Google’s approach (follow this link to access the keynote page with downloadable presentation, see the comments).
During the exhibition, big players focused their attention to Big Data hardware (e.g. Intel) and software (e.g. HP Vertica, Splunk, Cloudera, MapR, Platfora) vendors passing on in different ways the message that they support top Big Data technologies and have platforms processing petabytes of data with little upfront set-up effort.
At Moviri, we help our customers make sense of the plethora of emerging Big Data technologies. We look forward to your comments and to hear your impressions on these trends and the impact they are having on information technology and the business it supports and enables.