rr © Eckerson | a Group Altiscale Data Cloud Big Data Platform Evaluation Phil Bowermaster December 20152 Eckerson Big Data Platform Evaluation: Altiscale Co Ese About the Authors With more than two decades’ experience analyzing and writing about emerging technologies, Phil Bowermaster focuses on the convergence of information and society as reflected in current developments around big data and its supporting technologies. He is the founder and publisher of Speculist Media and is co-host ofthe popular Internet radio series The World Transfermed. Phil helps clients understand the divers behind a rapidly changing business and technological landscape, and guides them in developing and implementing strategies for making the most of new opportunities, About Eckerson Group Eckerson Group is a research and consulting firm that helps business and analytics leaders use data and ‘technology to drive better insights and actions. The firm helps companies develop strategies and road maps that maximize their investment in data and analytics Its consultants and researchers each have more than 20 years. ‘of experience in the field and are uniquely qualified to help business and technical leaders succeed with business intelligence and analytics, big data management, data governance, performance management, nd the Internet of things About This Research ‘This product profiles part of alarger research study on big data platforms. The base report, Big Data Platforms: Building a Foundation forthe Future, serves asa strategic planning tool for organizations looking to implement or upgrade their business intlligence/analytics environment atthe platform level. Itaddresses the primary strategic concerns of organizations who are making the move from conventional data warehousing and I technology to big data, or who are evolving or expanding their existing big data environments. The report also provides criteria for evaluating bg data platforms along \with key questions to ask vendors. Vendors profiled in this series are Actian ltscale, MapR Technologies, SAP, and Teradata eherson Group 205 oeckreoncom oc Big Data Platform Evaluation: Altiscale Cs In Brief The Aliscale Dato Cloud aflly managed big-data-as-a-service (BDo0S) platform that provides enterprise access to production-ready Hadoop and Spark via te loud. The platform fs designed to addres the demands ofa broad range ofdata analytics requirements comes preconfigured with core compute engines such as Spark, Tez, and MapReduce, os wellas services such as Hive, Ooze, Pig, ond Spark SQL. Altscole provides salable big data infrastructure including servers, networking, and sofware configured for performance, reliability, ond security. The platform's strength isthe dedicated operations team that manages the Hadoop and Spark environment forthe enterprise custome. Altiscale instal, canfigures, and manages all hardware and software, and keeps Hadoop and Spark components upto date. The service monitors jobs and provides proactive support to maintain defined service levels. eherson Group 205 oeckreoncom2 Eckerson ig Data Platform Evaluation: Altiscale ESE Company Profile CompanyName @ altiscale ‘Aiscale provides Apache Hadoap and Apache Spark asa service via Foal Sit the clad manne: te ig ia moan x cate is complete bigdata-asa-service (8492S) solution provides Feces Citic sed bg saa patton ult oh Hades: ‘on-demand infrastructure as well as the operational staff to monitor Employees 0 ie Headquarters Palo to, cA Offices Palo Alto, A;Champaign Il; and Chennai, India __Aliscale was founded in 2012 by Raymie Stata, the former CTO of Oa a Yahoo. The management team includes many top engineers from Yahoo, Google, and Linkedin; several members helped Yahoo bulld and manage an Apache Hadoop environment with 40,000 nodes. Rather than developing a suite of sofware products and going to market with a Hadoop distribution, Altiscale deployed Hadoop as an end-to-end cloud-based service. The company compares its offering to a utility, a critical service that is managed for you and always available. The strategy i an textension of cloud deployment in general, which relieves the customer ofthe burden of managing computing infrastructure onsite. Focusing on highly data-intensive businesses and complex analytical use cases, Altiscale's BDaa takes care of the Hadoop and Spark deployment, management, and scaling Altiscale Cloud supports enterprise business and technical requirements with accelerated deployment of operational Hadoop environments. It then provides ongoing, active, and inthe ease of troubleshooting, proactive management ofthe environment. Atiscale concluded a $30 milion series 8 financing round in November 2014, Investors in the company include Northgate, Sequola Capital, General Catalyst, and Accel Partners, ekerson Group 205 oveckrsoncom o2 Eckerson ig Data Platform Evaluation: Altiscale eB Ee Customer Profile Numberof customers 20 ‘Typical Altea customers manage very large data sets, usually multiple terabytes of data. The Key Markets Financial services, healthcare, manufacturing, gaming, and adtech a ae customer base includes. large enterprises in ERNST ite Ese Lee financial services, healthcare, and manufacturing, Key Partners Birst,AtScale, lation, H20, Innovaccer a5 well as smaller companies in data-centric Pricing Model ‘Subscription flat monthly fee based on average usage industries such as marketing, adtech, online broadcasting, and gaming. In particul focuses on enabling analysts to create reports and visualizations as well as helping data scientists to conduct complex analysis against large datasets. ‘altiscale Altiscale pursues thre primary customer use cases: + Rapid implementation and ongoing management of new big data environments + Rapid migration of on-premises big data environments to the cloud + Data science or data exploration environments For example, the provider of a mobile advertising platform that optimizes ad placement and performance (supporting some 150,000 mobile applications) needed to perform very high-speed analysis on large and rapidly growing data sets in order to determine the correct content and format for each ad and to anticipate each placements impact on performance. The service provider migrated to Altscale after running its own Hadoop cluster ‘00a public cloud. The company decided it didnot want to maintain an internal team to manage Hadoop issues, preferring a fully managed solution. Other companies choose Altiscale for cost reasons. Altiscae's service is provided via a subscription model, witha flat monthly fee based on average usage throughout the month. Other solution, including laaS (infrastructure-as-a-service) and the fullservice B0aaS providers who re-sell their platforms (and manage the resulting environments) use a fixed capacity pricing model based on the maximum capacity required—even ifthe higher capacity is needed only during brief spikes. ltscale's pricing model lets users pay only for the capacity they need. 4) Se eherson Group 205 oeckreoncom o2 Eckerson Big Data Platform Evaluation: Altiscale Co Ese Customer Quotes “With Altiscale, we have no more job failures. There's no need to waste time duplicating or restartingjobs, which translatesinto better use of internal resources and huge cost savings” Cedar Milazzo, Vice President of Engineering, Devicescape "Keeping our internal resources focused on the data science and analytics that drive our business was the most important critera in building our analytics environment) Altiscale enabled us to do this without having to worry about maintaining an entire Hadoop ops team” Chris Meisl, CTO, Visible Measures© Eckerson ig Data Platform Evaluation: Altiscale es Product Profile Product Name Altiscale Data Cloud Altiscale Data Cloud sheen ame: ‘The Altiscale Data Cloud is a fully managed big data platform Current Release and Date 4.0, October 2015 providing Spark, Hive, MapReduce on YARN, and HDFS as a Key New Features Apache Hadoop2.7.1, Apache Hive 1.2, serie. The platform provides servers, networking, and sofware acs SoTL ERAT configured for performance, scalability, and secur Competitors ‘razon EMR, Microsoft Azure HD, Qubole,Cazena Altscale stands out from other BDaaS providers by supporting the full solution stack for the Hadoop and Spark environment, from cloud infrastructure to end-to end support. Altiscale offers a more or less turn-key solution in which the big data environment is configured, Implemented, and updated on behalf ofthe customer with litle or no user input requied after the initial installation. atiscale offers proactive trouble ‘management and job optimization as well as elastic scalability provided by a combination of automated capacity management and lve operational support. Atiscale provides automatic Kerberos-driven authentication and security designed to meet the extensive and well-defined security requirements of financial services, healthcare, and others handling personally identifiable information. atiscale environments are dedicated for each customer but signed to be multi-tenant across users within the same customer. Job-levelisolation and resource limits are provided through acombination of ACLS ‘onjob queues, containers, and strict enforcement of authentication and authorization through Kerberos. ‘As noted above, because it provides its own cloud infrastructure, Altscale is abe to offer usage-based pricing that can produce a lower monthly cost than las providers (or providers relying on aS services) Inthe laaS model, customers must pay each month for capacity to suppor thelr highest usage level, even if that capacity is required only in bref spikes. All ofthe managed services provides offer (within thei tiered pricing structures) capacity planning and updating, which icritical to keeping customers’ capacity costs under contol. In contrast, Altisale'susage-based billing enables a refined approach to capacity planning that looks at average usage as well as maximum usage. Figure 1 maps the Altiscale big data platform to the reference architecture presented inthe base report, Big Data Platforms: Building a Foundation for the Future, ekerson Group 205 oveckrsoncom 08© Big Data Platform Evaluation: Altiscale Beeson Figure 1: Altiscale Big Data Platform @altiscale eve fe Analysis and si 20 ey i — 7129 koe ] “Act, Tens, i 7 2 Detangestand] ren ae ; reser rer comes i oss i Infrastructure = q ——— ‘The Appendix sections provide more detaled information about the Atiscale Data Cloud and its supporting technologies.ig Data Platform Evaluation: Altiscale Differentiators Fullstack solution. Altscale provides both the core infrastructure as well as managed services forthe Hadoop and Spark environment. Cen eS ee ee Fully managed. Altiscale implements, manages, maintains, and supports the system with ttle or no staffing requirement from the user Usage-based pricing. Altiscale bills monthly by average usage rather than maximum capacity. ne eee eee eee eee ee Ce eee es Cee a eae eee ae ee ee fees Built in security. Kerberos security is provided automatically. The service is SOC? certified, PCI compliant, and HIPAA compliant Cee ee ee eee eee ee Siete ea ee eenig Data Platform Evaluation: Altiscale 2 Eckerson Coco Isthe Altiscale Big Data Platform a Strategic Fit? [Strong Fit If You: 1. Mead full stack Hoop and Spark solution hat proviesboth core inkasrecture and manage big data ervces roma sine provide ..Need api implementation. The ical Data Coud provided ar acloud Need fllbig data environment vi the cloud Support or ARN and HOFSie includes, ae ella Spar, Hine Mapes, Pig, andOote |4.Need fly managed solution wit combinedanS Pa, snd 80as5 a well ‘operational supporto provide alton thats manage rm endo end (Canyousay er) ‘5. Ned te manage capital expenditures. Nolarg captalinsstmentisrequied, ‘ongoing managementol the enveonmentis an operational expense ‘need to reduce the cost of managing big ts platform internal ora ass sing aslution that reduces house stain equrementse doesnot aque paying for capacity at masimum usage levels Less of a Ftit You: 1. Weedon las and or justookng or coud inastuture, without requrement fora fll managed Hadoop environment. 2.ant a conventional data warehouse with tardardROBNSechnlogy + 2. constrained by security requirement tht willot pcm at tobe ranagingtig data, withno plato ads nen projects equing adational tetonee ora ditrent apron '5.Weed only Temporary Hadoop or Spar uses tobe spun up for ocasionl jobs ‘.uantto deploy just a cloud based latina data warehouse a datamart without Hadoop or Speth.Appendix 1: Product Detail Users log in to access the Altscale workbench, which provides immediate access to big data tools and production-ready Hadoop clusters. This approach eliminates the need to devote time and resources to procuring, And provisioning hardware and installing and configuring Hadoop, ‘The Altiscale Data Cloud is optimized for high performance, with hardware, networking, and software specifically tuned for accelerated performance on large datasets, Automated tools manage and monitor ecosystem operations to ensure high performance and scalablty as required, Altiscale offers elastic big data clusters so users can scale their environments up or down. For security, ltiscae offers Kerberos authentication and complies with SOC 2 Type | and, PCS Level 1, nd HIPAA. Altiscale Workbench and Portal The Altiscale Workbench is a Linux machine running CentOS, a gateway that provides access to the data and services in the cluster. Users can log in to the workbench to perform activities such as loading data and applications, executing applications, and viewing status and results. ‘The Altiscale Portal provides real-time job status and usage data via a dashboard with a graphical view of HDF sage, status of current obs, and alerts for problematic conditions. The portal also supports core admin functions such as adding and viewing users and setting or changing roles Atiscale also provides 2 builtin user interface for Apache Hive SQL queries into the Altiscale Data Cloud. This interface leverages the Alation SQL integrated development environment to enable SQL access to all data within the altiscale Data Cloud. Operations Support The Atiscale Operations Team manages the implementation, ongoing operation, maintenance, and updating of the Altiscale Data Cloud. With fewer requirements for in-house management ofthe environment, customers can devote more resources to data discovery and analysis.Big Data Platform Evaluati : Altiscale @egeson Proactive Help Desk Altiscale monitors operations to identify problems as they arise and potential problems in advance; the response team then takes appropriate preventative or remedial action. Atiscale provides high avalability by using data centers on both the east and west coasts, User organizations have the option of replicating their environment from one data center to the other to ensure continuous availablity Such replicas can also suppor disaster recovery. User organizations also have the option of archiving directly into As, Appendix 2: Supporting Technologies ‘Apache Hadoop and Spark Altiscale supports Apache Hadaop by providing HDFS for ile storage and YARN for resource management. altiscale storage services include both the Hadoop Distributed File Sytem (HDFS) and HCatalog. HOFS isa highly reliable, fault-tolerant, distributed storage system used fr storing and retrieving big data at high throughput. HCatalog isa storage management layer that provides a common interface to multiple compute services. YARN isa large-scale istributed operating system for big data In addtion to Spark, users can run multiple altemative data processing engines, including Tez, and MapReduce, on top of Hadoop, making it possible for multiple analytics frameworks to run simultaneously and take advantage ofthe data stored in HDFS. eherson Group 205 oeckreoncom© Big Data Platform Evaluation: Altiscale secon Compute Services Compute services sit on top of the compute engines to perform different types of processing, Thete are several compute engines that run an top of Altiscale Data Cloud + Altiscale supports the MapReduce batch-based processing engine. MapReduce is a programming model and associated implementation for processing and generating large datasets with a parallel, distributed algorithm on a cluster. + Altiscale supports Apache Spark n production. Sparks an open source cluster computing framework that provides mult-stage, in-memory primitives supporting high performance for analytics. + Altiscale supports Tez for accelerated performance on MapReduce jobs. Tez is an extensible framework for building high-performance batch and interactive data processing applications. + Altiscale supports Apache Hive for SQL support in Hadoop. Hive is @ data warehouse infrastructure built on top of Hadoop for providing data summarization, query, and analysis. Users can visualize their data using Tableau Software or any tool that connects to Hive using JDBC or ODEC. Appendix 3: Altiscale Big Data Ecosystem In addition to Spark, Hive, Tez, and MapReduce, the Altiscale Data Cloud comes with Impala, Oozie, Pig, and Katka builtin, Altscale maintains an ‘ecosystem of big data partners, including Alation, AtScale, Datameer, 20, Innovaccer, Pivotal, Manthan Systems, and Zaloni