1. 啟動metastore服務
<code> ./hive --service metastore &/<code>
2. 建表
創建一個行表，用於存儲foobar.txt文件中的每行句子。
<code>create table tbl_line(line string) row format delimited fields terminated by '\\n';/<code>
3. 加載數據
將文件數據加載到hive表中。
<code>echo "Hadoop Common\\nHadoop Distributed File System\\nHadoop YARN\\nHadoop MapReduce " > /tmp/foobar.txt/<code>
<code>hive> load data local inpath '/tmp/foobar.txt' into table tbl_line;/<code>
加載的數據會放到Hadoop中/data/hive/warehouse/test.db目錄下，/data/hive/warehouse是hive-site.xml配置的hive.metastore.warehouse.dir值, test是數據庫名稱, tbl_line是表名。
4. HQL
根據MapReduce方式我們需要將每行句子拆分成獨立的單詞，然後對單詞彙總。
- split(字符串，分割符) 函數：用於分割字符串, 返回一個數組
- explode(數組)函數：將數組中的每個元素展開成列
<code>hive> select split("hello world", " ") from tbl_line limit 1;OK["hello","world"]hive> select * from tbl_line;OKHadoop CommonHadoop Distributed File SystemHadoop YARNHadoop MapReduce# 將每行句子分割成每個單詞數組hive> select split(line, " ") from tbl_line;OK["Hadoop","Common"]["Hadoop","Distributed","File","System"]["Hadoop","YARN"]["Hadoop","MapReduce",""]hive> select explode(split(line, " ")) from tbl_line;OKHadoopCommonHadoopDistributedFileSystemHadoopYARNHadoopMapReduce/<code><code># 創建一個單詞表hive> create table tbl_word(word string);# 將每一行句子拆分成每個單詞插入到表中hive> insert into table tbl_word select explode(split(line, " ")) as word from tbl_line;hive> select * from tbl_word;OKHadoopCommonHadoopDistributedFileSystemHadoopYARNHadoopMapReducehive> select word, count(*) as count from tbl_word group by word order by count desc;/<code>
使用Hadoop MapReduce需要寫代碼，然後執行jar包。使用Hive只需要寫HQL就可以了。相比之下使用Hive SQL更簡便。
5. 異常
- FAILED: HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient./hive --service metastore &
- Exception in thread "main" java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$POSIX.stat(Ljava/lang/String;)Lorg/apache/hadoop/io/nativeio/NativeIO$POSIX$Stat; 此異常是最常見的異常，這裡列舉出現此異常的幾個原因：
- 有可能是hql語法有錯誤，確保語法是正確。
- hadoop 中lib/native有問題，把lib重命名成lib2，然後重啟hadoop，再執行hive sql就好了，相當於不使用原生庫了lib/native。不使用原生庫hadoop就會報警告 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable ，先不用管這個警告。
分享到:
閱讀更多 Java實用技術 的文章
