Practical Guide to setup Hadoop and Spark Cluster using CDH

What you'll learn
Learn Hadoop and Spark Administration using CDH
Provision Cluster from GCP (Google Cloud Platform) to setup Hadoop and Spark Cluster using CDH
Setup Ansible for server automation to setup pre-requisites to setup Hadoop and Spark Cluster using CDH
Setup 8 node cluster from scratch using CDH
Understand Architecture of HDFS, YARN, Spark, Hive, Hue and many more
Requirements
Basic Linux Skills
A 64 bit computer with minimum of 4 GB RAM
Operating System - Windows 10 or Mac or Linux Flavor
Description
Cloudera is one of the leading vendor for distributions related to Hadoop and Spark. As part of this Practical Guide, you will learn step by step process of setting up Hadoop and Spark Cluster using CDH.Install - Demonstrate an understanding of the installation process for Cloudera Manager, CDH, and the ecosystem projects.Set up a local CDH repositoryPerform OS-level configuration for Hadoop installationInstall Cloudera Manager server and agentsInstall CDH using Cloudera ManagerAdd a new node to an existing clusterAdd a service using Cloudera ManagerConfigure - Perform basic and advanced configuration needed to effectively administer a Hadoop clusterConfigure a service using Cloudera ManagerCreate an HDFS user's home directoryConfigure NameNode HAConfigure ResourceManager HAConfigure proxy for Hiveserver2/ImpalaManage - Maintain and modify the cluster to support day-to-day operations in the enterpriseRebalance the clusterSet up alerting for excessive disk fillDefine and install a rack topology scriptInstall new type of I/O compression library in clusterRevise YARN resource assignment based on user feedbackCommission/decommission a nodeSecure - Enable relevant services and configure the cluster to meet goals defined by security policy; demonstrate knowledge of basic security practicesConfigure HDFS ACLsInstall and configure SentryConfigure Hue user authorization and authenticationEnable/configure log and query redactionCreate encrypted zones in HDFSTest - Benchmark the cluster operational metrics, test system configuration for operation and efficiencyExecute file system commands via HTTPFSEfficiently copy data within a cluster/between clustersCreate/restore a snapshot of an HDFS directoryGet/set ACLs for a file or directory structureBenchmark the cluster (I/O, CPU, network)Troubleshoot - Demonstrate ability to find the root cause of a problem, optimize inefficient execution, and resolve resource contention scenariosResolve errors/warnings in Cloudera ManagerResolve performance problems/errors in cluster operationDetermine reason for application failureConfigure the Fair Scheduler to resolve application delaysOur ApproachYou will start with creating Cloudera QuickStart VM (in case you have laptop with 16 GB RAM with Quad Core). This will facilitate you to get comfortable with Cloudera Manager.You will be able to sign up for GCP and avail credit up to $300 while offer lasts. Credits are valid up to year.You will then understand brief overview about GCP and provision 7 to 8 Virtual Machines using templates. You will also attaching external hard drive to configure for HDFS later.Once servers are provisioned, you will go ahead and set up Ansible for Server Automation.You will take care of local repository for Cloudera Manager and Cloudera Distribution of Hadoop using Packages.You will then setup Cloudera Manager with custom database and then Cloudera Distribution of Hadoop using Wizard that comes as part of Cloudera Manager.As part of setting up of Cloudera Distribution of Hadoop you will setup HDFS, learn HDFS Commands, Setup YARN, Configure HDFS and YARN High Availability, Understand about Schedulers, Setup Spark, Transition to Parcels, Setup Hive and Impala, Setup HBase and Kafka etc.
Who this course is for
System Administrators who want to understand Big Data eco system and setup clusters
Experienced Big Data Administrators who want to learn how to manage Hadoop and Spark Clusters setup using CDH
Entry level professionals who want to learn basics and Setup Big Data Clusters
Homepage