Please note: Hortonworks courses are delivered using electronic courseware. for delegates attending remotely (Virtual classes or Attend from Anywhere) you must ensure that you have dual monitors or a single monitor plus tablet device. Dual monitors are required in order to allow you to view labs and lab instructions on separate screens.

Technical pre-requisites

• Delegates must have experience developing Java applications and using a Java IDE. Labs are completed using the Eclipse IDE and Gradle.

• No prior Hadoop knowledge is required.

• Describe Hadoop 2 and the Hadoop Distributed File System
• Describe the YARN framework
• Develop and run a Java MapReduce application on YARN
• Use combiners and in-map aggregation
• Write a custom partitioner to avoid data skew on reducers
• Perform a secondary sort
• Recognize use cases for built-in input and output formats
• Write a custom MapReduce input and output format
• Optimize a MapReduce job
• Configure MapReduce to optimize mappers and reducers
• Develop a custom RawComparator class
• Distribute files as LocalResources
• Describe and perform join techniques in Hadoop
• Perform unit tests using the UnitMR API
• Describe the basic architecture of HBase
• Write an HBase MapReduce application
• List use cases for Pig and Hive
• Write a simple Pig script to explore and transform big data
• Write a Pig UDF (User-Defined Function) in Java
• Write a Hive UDF in Java
• Use JobControl class to create a MapReduce workflow
• Use Oozie to define and schedule workflows

Hands-On Labs

• Configuring a Hadoop Development Environment
• Putting data into HDFS using Java
• Write a distributed grep MapReduce application
• Write an inverted index MapReduce application
• Configure and use a combiner
• Writing custom combiners and partitioners
• Globally sort output using the TotalOrderPartitioner
• Writing a MapReduce job to sort data using a composite key
• Writing a custom InputFormat class
• Writing a custom OutputFormat class
• Compute a simple moving average of stock price data
• Use data compression
• Define a RawComparator
• Perform a map-side join
• Using a Bloom filter
• Unit testing a MapReduce job
• Importing data into HBase
• Writing an HBase MapReduce job
• Writing User-Defined Pig and Hive functions
• Defining an Oozie workflow