【Docker】一鍵搭建大數據環境、帶可視化界面的呢~~~

在這個大數據時代、我們要麼成為別人的數據、要麼別人成為我們的數據。

大豬來教大家快速搭建一個大數據的環境,我們可以使用Docker來實現,文章會演示如何使用。

【Docker】一鍵搭建大數據環境、帶可視化界面的呢~~~

一、使用方法

這裡我會使用HDP的最新版本3.0.1作為演示,包含的組件版本如下

【Docker】一鍵搭建大數據環境、帶可視化界面的呢~~~

【Docker】一鍵搭建大數據環境、帶可視化界面的呢~~~

二、安裝步驟

先下載docker鏡像(打個預防針:大小佔 26G 慢慢下吧哈,但收穫很大)

docker pull hortonworks/sandbox-hdp:3.0.1
docker pull hortonworks/sandbox-proxy:1.0

下載啟動配置腳本

git clone https://github.com/dounine/sandbox-hdp-3.0.1.git

添加hosts映射

127.0.0.1 sandbox-hdp.hortonworks.com
# 或者是使用公網IP或局域網IP

啟動配置

./docker-deploy-hdp265.sh

成功結果

root@lake /s/d/sandbox-hdp-3.0.1# ./docker-deploy-hdp265.sh 
+ registry=hortonworks
+ name=sandbox-hdp
+ version=3.0.1
+ proxyName=sandbox-proxy
+ proxyVersion=1.0
+ flavor=hdp
+ echo hdp
+ mkdir -p sandbox/proxy/conf.d
+ mkdir -p sandbox/proxy/conf.stream.d
+ docker pull hortonworks/sandbox-hdp:3.0.1
3.0.1: Pulling from hortonworks/sandbox-hdp
Digest: sha256:7b767af7b42030fb1dd0f672b801199241e6bef1258e3ce57361edb779d95921
Status: Image is up to date for hortonworks/sandbox-hdp:3.0.1
+ docker pull hortonworks/sandbox-proxy:1.0
1.0: Pulling from hortonworks/sandbox-proxy
Digest: sha256:42e4cfbcbb76af07e5d8f47a183a0d4105e65a1e7ef39fe37ab746e8b2523e9e
Status: Image is up to date for hortonworks/sandbox-proxy:1.0
+ '[' hdp == hdf ']'
+ '[' hdp == hdp ']'
+ hostname=sandbox-hdp.hortonworks.com
++ docker images
++ grep hortonworks/sandbox-hdp
++ awk '{print $2}'
+ version=3.0.1
+ docker network create cda
+ docker run --privileged --name sandbox-hdp -h sandbox-hdp.hortonworks.com --network=cda --network-alias=sandbox-hdp.hortonworks.com -d hortonworks/sandbox-hdp:3.0.1
46bf6b414dd3c0fb36a3816eac129219d30d49ea9421898158800e0ab3576048
+ echo ' Remove existing postgres run files. Please wait'
Remove existing postgres run files. Please wait
+ sleep 2
+ docker exec -t sandbox-hdp sh -c 'rm -rf /var/run/postgresql/*; systemctl restart postgresql;'
Failed to restart postgresql.service: Unit not found.
+ sed s/sandbox-hdp-security/sandbox-hdp/g assets/generate-proxy-deploy-script.sh
+ mv -f assets/generate-proxy-deploy-script.sh.new assets/generate-proxy-deploy-script.sh
+ chmod +x assets/generate-proxy-deploy-script.sh
+ assets/generate-proxy-deploy-script.sh

+ uname
+ grep MINGW
+ chmod +x sandbox/proxy/proxy-deploy.sh
+ sandbox/proxy/proxy-deploy.sh
7fa5c4d0737a6b71796fe997baf397d4078907d83fcfaa2a8c0f241772547147

需要先重置一下ambari的密碼才能登錄

docker exec -ti sandbox-hdp bash
ambari-admin-password-reset #重置密碼登錄

結果

[root@sandbox-hdp /]# ambari-admin-password-reset 
Please set the password for admin:
Please retype the password for admin:

The admin password has been set.
Restarting ambari-server to make the password change effective...

Using python /usr/bin/python
Restarting ambari-server
Waiting for server stop...
Ambari Server stopped
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start...................
Server started listening on 8080

DB configs consistency check: no errors and warnings were found.

輸入帳號密碼登錄 ambar webUI http://localhost:8080

【Docker】一鍵搭建大數據環境、帶可視化界面的呢~~~

是不是很簡單、後續還有更多大數據深入的文章送給你們。

【Docker】一鍵搭建大數據環境、帶可視化界面的呢~~~

最後

每天發佈更多新鮮有含量的技術文章、總有一款適合你。

快關注我、把我打包帶走、每天提醒你學習一手技術。


分享到:


相關文章: