在windows平台上部署僞分布式系統開發hadoop程式

大資料開發架構一般在linux系統上操作，但是Linux作業系統需要安裝虛拟機等原因，很多人選擇在windows平台上通過cygwin來虛拟linux系統

下面講一下如何通過cygwin來虛拟linux系統

1：軟體工具：cygwin putty jdk eclipse hadoop

首先安裝jdk，到www.oracle.com網站下載下傳自己電腦平台的jdk安裝包。輕按兩下可執行檔案，安裝。

将jdk安裝路徑下的bin（可執行檔案，指令java,javac）檔案夾對應的 位置配置到環境 變量中，如果要執行某指令，可以将其bin檔案加載到path中。

2 安裝cygwin

安裝cygwin過程中，将openssl,openssh包下載下傳安裝。

管理者權限運作cygwin，

開啟ssh服務

[email protected] ~

$ ssh-host-config

* Info: Generating missing SSH host keys

* Query: Overwrite existing /etc/ssh_config file? (yes/no) no

* Query: Overwrite existing /etc/sshd_config file? (yes/no) no

* Info: Updating /etc/sshd_config file

* Query: Do you want to install sshd as a service?

* Query: (Say “no” if it is already installed as a service) (yes/no) yes

* Query: Enter the value of CYGWIN for the daemon: [ntsec] ntsec 填入ntsec

* Info: On Windows Server 2003, Windows Vista, and above, the

* Info: SYSTEM account cannot setuid to other users – a capability

* Info: sshd requires. You need to have or to create a privileged

* Info: account. This script will help you do so.

* Info: No privileged account could be found.

* Info: This script plans to use ‘cyg_server’.

* Info: ‘cyg_server’ will only be used by registered services.

* Query: Do you want to use a different name? (yes/no) no

* Query: Create new privileged user account ‘ADMIN-PC\cyg_server’ (Cygwin name: ‘cyg_server’)? (yes/no) yes

* Info: Please enter a password for new user cyg_server. Please be sure

* Info: Entering no password will exit the configuration.

* Query: Please enter the password: 輸入密碼

* Query: Reenter: 再次輸入密碼

* Info: User ‘cyg_server’ has been created with password ‘xxxx’.

* Info: If you change the password, please remember also to change the

* Info: password for the installed services which use (or will soon use)

* Info: the ‘cyg_server’ account. //建立使用者

* Info: Host configuration finished. Have fun! //安裝成功

3為cygwin配置免秘鑰登入

[email protected] /

$ ssh-keygen -t rsa -P ” -f ~/.ssh/id_rsa

Generating public/private rsa key pair. //生成公鑰私鑰對

Your identification has been saved in /home/admin/.ssh/id_rsa. 私鑰

Your public key has been saved in /home/admin/.ssh/id_rsa.pub. 公鑰

The key fingerprint is:

SHA256:8lYCTGO+RcxsUW3KayKmp3FbRYNb7iG1yGeqHNoO4mo [email protected]

The key’s randomart image is:

+—[RSA 2048]—-+

| ++oo.. |

| = o=. o |

| +.o.=o |

| = Boo |

| o S O. |

| oo.Xo. |

| . ooo.=o. |

| E ..B.* |

|o.. o+* |

+—-[SHA256]—–+

[email protected] /

$

[email protected] ~

重新開機：

[email protected] ~

$ ssh localhost

The authenticity of host ‘localhost (::1)’ can’t be established.

ECDSA key fingerprint is SHA256:7eE4P+QcQtOu0MHPn3Nw9nHVWyUuGf3GOTQynHwbnzc.

Are you sure you want to continue connecting (yes/no)? yes 第一次啟動需要确認yes

Warning: Permanently added ‘localhost’ (ECDSA) to the list of known hosts.

[email protected]’s password: 輸入密碼，登入ssh

4，安裝qutty,

輕按兩下可執行檔案，輸入主機名（localhost）或ip位址

login as ：輸入使用者名（admin）

[email protected] ~

$

用putty連接配接ssh

login as: admin

[email protected]’s password:

Last login: Mon Feb 20 09:35:44 2017 from ::1

5，安裝hadoop

我們需要改四個配置檔案hadoop-env.sh；core-site.xml；hdfs- site.xml；mapred-site.xml；

下載下傳hadoop，通過cygwin 進入到hadoop/bin目錄下，

輸入./start-all.sh 開啟節點。

6,安裝eclipse，

将hadoop部署到eclipse上。

将下載下傳的hadoop-eclipse-plugin-2.2.0.jar檔案放到Eclipse的plugins 目錄下，重新開機Eclipse即可看到該插件已生效。