mapreduce counter為提供我們一個視窗:觀察mapreduce job運作期的各種細節資料。mapreduce自帶了許多預設counter。
counter有"組group"的概念,用于表示邏輯上相同範圍的所有數值。mapreduce job提供的預設counter分為三個組
map-reduce frameword
map input records,map skipped records,map input bytes,map output records,map output bytes,combine input records,combine output records,reduce input records,reduce input groups,reduce output records,reduce skipped groups,reduce skipped records,spilled records
file systems
filesystem bytes read,filesystem bytes written
job counters
launched map tasks,launched reduce tasks,failed map tasks,failed reduce tasks,data-local map tasks,rack-local map tasks,other local map tasks
這些 counters你在web ui中,或是job結束後在控制台生成的統計報告中都看得到。 見如下mr運作日志:
使用java enum自定義counter
一個counter可以是任意的enum類,見如下代碼示例: