Skip to content

Commit 220e7e9

Browse files
author
jackylk
committed
improve example doc
1 parent 84c2c05 commit 220e7e9

File tree

3 files changed

+31
-28
lines changed

3 files changed

+31
-28
lines changed

examples/spark-hbase-demo1.md

Lines changed: 19 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,28 +1,34 @@
1-
## Create and query SparkSQL table map to HBase (support multiple columns mapping to hbase rowkey)
2-
(1)TableName :
1+
## Create and query SparkSQL table map to HBase table (multiple columns map to hbase table rowkey)
2+
In this example, we create a new SparkSQL table and map it to a new HBase table with multiple column in rowkey.
3+
4+
Steps:
5+
(1) Create table in hbase-sql shell:
6+
```
7+
$SPARK_HBASE_Home/bin/hbase-sql
8+
CREATE TABLE teacher1k(grade int, class int, subject string, teacher_name string, teacher_age int, PRIMARY KEY (grade, class, subject)) MAPPED BY (hbase1k, COLS=[teacher_name=teacher.name, teacher_age=teacher.age]);
9+
```
10+
11+
This command will create following tables:
12+
Tables :
313
spark : teacher1k
414
hbase : hbase1k
515

6-
(2)Fields :
16+
Fields :
717
[grade,int]
818
[class,int]
919
[subject,string]
1020
[teacher_name,string]
1121
[teacher_age,int]
1222

13-
keyCols : grade,class,subject
14-
15-
(3) Create table:
16-
```
17-
CREATE TABLE teacher1k(grade int, class int, subject string, teacher_name string, teacher_age int, PRIMARY KEY (grade, class, subject)) MAPPED BY (hbase1k, COLS=[teacher_name=teacher.name, teacher_age=teacher.age]);
18-
```
19-
20-
(4) Load data :
23+
key columns : grade,class,subject
24+
non-key colums: teacher_name, teacher_age
25+
26+
(2) Load data from a csv data file:
2127
```
2228
LOAD DATA INPATH './examples/teacher1k.csv' INTO TABLE teacher1k FIELDS TERMINATED BY "," ;
2329
```
2430

25-
(5) Query :
31+
(3) Query :
2632
```
2733
// test where
2834
(1) select teacher_name,teacher_age from teacher1k where teacher_age > 25;
@@ -33,7 +39,7 @@ LOAD DATA INPATH './examples/teacher1k.csv' INTO TABLE teacher1k FIELDS TERMINAT
3339
// test subquery
3440
(3) select t1.teacher_name,t1.teacher_age from (select * from teacher1k where teacher_name like 'teacher_2_3%') t1 where t1.teacher_age < 25
3541
36-
//test group
42+
//test group by
3743
(4) select teacher_name, sum(teacher_age) from teacher1k where grade=1 group by teacher_name
3844
3945
//test join

examples/spark-hbase-demo2.md

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,17 @@
1-
## Create spark sql table map to existing hbase (only single column mapping to hbase rowkey is supported)
2-
(1) Create table in hbase, populate data
1+
## Create spark sql table map to existing hbase
2+
In this example, we create SparkSQL table and map it to a existing HBase table. (a single column map to hbase rowkey)
3+
4+
Steps:
5+
(1) Create table and populate data in HBase shell
36
```
7+
$HBase_Home/bin/hbase shell
48
create 'hbase10k', 'f'
59
for i in '1'..'10000' do for j in '1'..'2' do put 'hbase10k', "row#{i}", "f:c#{j}", "#{i}#{j}" end end
610
```
711

8-
(2) Map hbase table with sparksql table
12+
(2) Map hbase table with sparksql table in hbase-sql shell
913
```
14+
$SPARK_HBASE_Home/bin/hbase-sql
1015
CREATE TABLE spark10k(rowkey STRING, a INTEGER, b INTEGER, PRIMARY KEY (rowkey)) MAPPED BY (hbase10k, COLS=[a=f.c1, b=f.c2]);
1116
```
1217

examples/spark-hbase-demo3.md

Lines changed: 4 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,27 +1,19 @@
1-
## Create spark sql table map to existing hbase (only single column mapping to hbase rowkey is supported)
2-
(1) Create table in hbase, populate data
3-
```
4-
1+
## Similar to demo 1, but with larger sample file
2+
In this example, we create a new SparkSQL table and map it to a new HBase table with multiple column in rowkey.
53

6-
(2) Map hbase table with sparksql table
4+
(2) Create table in SparkSQL and in HBase
75
```
6+
$SPARK_HBASE_HOME/bin/hbase-sql
87
CREATE TABLE sales1m(id STRING, product STRING, region STRING, sales INTEGER, quantity INTEGER, PRIMARY KEY (id, product, region)) MAPPED BY (hbase_sales1m, COLS=[sales=f.sales, quantity=f.quantity]);
98
CREATE TABLE sales1m_onekey(id STRING, product STRING, region STRING, sales INTEGER, quantity INTEGER, PRIMARY KEY (id)) MAPPED BY (hbase_sales1m_onekey, COLS=[product=f.product, region=f.region, sales=f.sales, quantity=f.quantity]);
10-
11-
CREATE TABLE sales10m(id STRING, product STRING, region STRING, sales INTEGER, quantity INTEGER, PRIMARY KEY (id, product, region)) MAPPED BY (hbase_sales10m, COLS=[sales=f.sales, quantity=f.quantity]);
12-
CREATE TABLE sales10m_onekey(id STRING, product STRING, region STRING, sales INTEGER, quantity INTEGER, PRIMARY KEY (id)) MAPPED BY (hbase_sales10m_onekey, COLS=[product=f.product, region=f.region, sales=f.sales, quantity=f.quantity]);
139
```
1410

1511
(4) Load data :
1612
```
1713
LOAD DATA INPATH './examples/sales1m.csv' INTO TABLE sales1m FIELDS TERMINATED BY "," ;
1814
LOAD DATA INPATH './examples/sales1m.csv' INTO TABLE sales1m_onekey FIELDS TERMINATED BY "," ;
19-
20-
LOAD DATA INPATH './examples/sales10m.csv' INTO TABLE sales10m FIELDS TERMINATED BY "," ;
21-
LOAD DATA INPATH './examples/sales10m.csv' INTO TABLE sales10m_onekey FIELDS TERMINATED BY "," ;
2215
```
2316

24-
2517
(3) Query:
2618
```
2719
// test count *

0 commit comments

Comments
 (0)