第6章. Hadoop HDFS命令介紹

書上6.3與6.5節命令有誤,以下已經更正

6.1 啟動Hadoop Multi-Node Cluster
start-all.sh
6.2 建立與查看HDFS目錄
Step1 建立HDFS目錄
hadoop fs -mkdir /user
hadoop fs -mkdir /user/hduser
hadoop fs -mkdir /user/hduser/test
Step2 查看使用者HDFS目錄
hadoop fs -ls
Step3 查看HDFS完整目錄
hadoop fs -ls /
hadoop fs -ls /user
hadoop fs -ls /user/hduser
Step4 查看所有HDFS子目錄
hadoop fs -ls -R /
Step5 一次建立所有HDFS子目錄
hadoop fs -mkdir -p /dir1/dir2/dir3
hadoop fs -ls -R /
6.3 從本機複製檔案到 HDFS
Step1 複製本機(local)檔案到 HDFS
hadoop fs -copyFromLocal /usr/local/hadoop/README.txt /user/hduser/test
書上的命令-copyFromLocal後面少了空白,以下才是正確
hadoop fs -copyFromLocal /usr/local/hadoop/README.txt /user/hduser/test/test1.txt
hadoop fs -ls /user/hduser/test
Step2 列出HDFS目錄下的檔案
hadoop fs -cat /user/hduser/test/README.txt
hadoop fs -cat /user/hduser/test/README.txt|more
Step3 複製重複檔案至HDFS目錄
hadoop fs -copyFromLocal /usr/local/hadoop/README.txt /user/hduser/test
Step4 強迫複製重複檔案至HDFS目錄
hadoop fs -copyFromLocal -f /usr/local/hadoop/README.txt /user/hduser/test
Step5 複製多個本機檔案至HDFS目錄
hadoop fs -copyFromLocal /usr/local/hadoop/NOTICE.txt /usr/local/hadoop/LICENSE.txt  /user/hduser/test
Step6 copyFromLocal複製目錄至HDFS目錄
hadoop fs -copyFromLocal /usr/local/hadoop/etc /user/hduser/test
hadoop fs -ls /user/hduser/test
Step7 查看目錄下所有的檔案
hadoop fs -ls -R /user/hduser/test/etc
Step8 使用put複製檔案至HDFS目錄
hadoop fs -put /usr/local/hadoop/README.txt /user/hduser/test/test2.txt
Step9 使用put命令接受stdin(標準輸入)
echo abc | hadoop fs -put -  /user/hduser/test/echoin.txt
hadoop fs -cat /user/hduser/test/echoin.txt
Step10 使用put命令將本機目錄的列表,儲存到HDFS檔案
ls /usr/local/hadoop | hadoop fs -put - /user/hduser/test/hadooplist.txt
hadoop fs -cat /user/hduser/test/hadooplist.txt
6.4 將HDFS上的檔案複製到本機
Step1 將HDFS上的檔案複製到本機(local)
mkdir test
cd test
hadoop fs -copyToLocal /user/hduser/test/hadooplist.txt
ll
Step3 hadoop fs -get複製到本機(local)
hadoop fs -get  /user/hduser/test/README.txt localREADME.txt
6.5 複製與刪除HDFS檔案
Step1 複製HDFS檔案
書上的命令有誤,以下才是正確
hadoop fs -mkdir /user/hduser/test/temp
hadoop fs -cp /user/hduser/test/README.txt /user/hduser/test/temp
hadoop fs -ls /user/hduser/test/temp
Step2 刪除HDFS檔案
hadoop fs -ls /user/hduser/test
hadoop fs -rm /user/hduser/test/test2.txt
Step3 刪除HDFS目錄
hadoop fs -ls  /user/hduser/test
hadoop fs -rm -R /user/hduser/test/etc
6.6 Hadoop HDFS Web UI 介面瀏覽HDFS
http://master:50070




 
Share on Google Plus

About kevin

This is a short description in the author block about the author. You edit it by entering text in the "Biographical Info" field in the user admin panel.
    Blogger Comment
    Facebook Comment

1 意見:

  1. hadoop fs -copyFromLocal /usr/local/hadoop/README.txt /user/hduser/test
    執行後
    copyFromLocal: File /user/hduser/test/test1.txt._COPYING_ could only be replicated to 0 nodes instead of minReplication (=1). There are 0 datanode(s) running and no node(s) are excluded in this operation.

    回覆刪除