自定義 MR1 之 Mapper | 學習筆記

開發者學堂課程【分布式資料庫 HBase 快速入門：自定義 MR1 之 Mapper 】學習筆記，與課程緊密聯系，讓使用者快速學習知識。

課程位址：

https://developer.aliyun.com/learning/course/101/detail/1755

自定義 MR1 之 Mapper

内容介紹

一、自定義 MR1 之 Mapper

自定義 HBase-MapReduce1

目标：将 fruit 表中的一部分資料，通過 MR 遷入到 fruit_mr 表中。

分步實作:

1．建構 ReadFruitMapper 類，用于讀取 fruit 表中的資料

package com.atguigu . hbase_mr;  

import java.io.IOException;  

import org.apache.hadoop.hbase.cell;  

import org. apache.hadoop. hbase.cellutil;  

import org.apache.hadoop.hbase.client. Put;  

import org.apache.hadoop.hbase.client. Result;  

Import org.apache.hadoop.hbase.io. ImmutableByteswritable;

import org.apache.hadoop.hbase.mapreduce.TableMapper;

import org.apache.hadoop. hbase.util.Bytes;  

Public              class              ReadFruitMapper           extends

TableMapper {

@override

protected void map(ImmutableByteswritable key,Result value,Context context)  

throws IOException,InterruptedException {

Tota1 time spent by a17 maps in occupied slots (ms)=5222Tota1 time spent by a17 reduces in occupied slots (ms)=0Total time spent by al7 map tasks (ms)=5222

Tota1 vcore-mi11iseconds taken by a17 map tasks=5222

Total megabyte-mil7iseconds taken by a17 map tasks=5347328

Map-Reduce Framework

Map input records=3Map output records=3Input split bytes=96spi77ed Records=oFailed shuffles=oMerged Map outputs=o

Gc time elapsed (ms)=272CPu time spent (ms)=2510

Physical memory (bytes) snapshot=222109696virtua7 memory (bytes) snapshot=3001012224Tota1 committed heap usage' (bytes)=152043520

ImportTsv

Bad Lines=o

File Input Format counters

Bytes Read=54

File output Format counters

Bytes written=o

[atguigu@hadoop102 hbase]$

/**

*Extends the base

Mapper

class to add the required input Key  

*and value classes.

*

*@param The type of the key.

*@param The type of the value.

*@see org.apache. hadoop. mapreduce.Mapper

*/

@InterfaceAudience.Public

@InterfaceStability.Stable

public abstract class TableMapper

extends Mapper{

}

import org.apache.hadoop. hbase. mapreduce.TableMapper;

import org.apache. hadoop.hbase.uti1.Bytes;

import java.io.IOException;

public class FruitMapper extends TableMapper

@Override

protected void map(ImmutableBytesWritable key，Result value，Context context) throws

IOException,InterruptedException {

//建構Put對象

Put put = new Put(key.get()) ;

//周遊資料

Cell[] cells = value.rawCells();

cell[] cells = value.rawCells();

for (Cell cell : cells){

if ("name".equals(Bytes.toString(CellUtil.cloneQualifier(cell)))){

  Putadd(cell);

}

}