版權聲明:本文為部落客原創文章,未經部落客允許不得轉載。 https://blog.csdn.net/qq1010885678/article/details/44657011
1.安裝Pig
将pig添加到環境變量當中
2.pig使用
首先将資料庫中的資料導入到HDFS上
sqoop import --connect jdbc:mysql://192.168.1.10:3306/cloud --username root --
password JChubby123 --table trade_detail --target-dir '/sqoop/td'
password JChubby123 --table user_info --target-dir '/sqoop/ui'
td = load '/sqoop/td' using PigStorage(',') as (id:long, account:chararray,
income:double, expenses:double, time:chararray);
ui = load '/sqoop/ui' using PigStorage(',') as (id:long, account:chararray,
name:chararray, age:int);
td1 = foreach td generate account, income, expenses, income-expenses as surplus;
td2 = group td1 by account;
td3 = foreach td2 generate group as account, SUM(td1.income) as income, SUM
(td1.expenses) as expenses, SUM(td1.surplus) as surplus;
tu = join td3 by account, ui by account;
result = foreach tu generate td3::account as account, ui::name, td3::income,
td3::expenses, td3::surplus;
store result into '/result' using PigStorage(',');