Skip to content

Commit af20aa0

Browse files
author
anair-it
committed
Initial commit
0 parents  commit af20aa0

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

79 files changed

+1345
-0
lines changed

‎README.md

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
hadoop docker lite
2+
======
3+
Build a lightweight container with below mentioned hadoop services. These versions match what Hortonworks HDP 2.3.2 distribution has.
4+
5+
6+
Versions
7+
----
8+
- Ubuntu: 14.04
9+
- Java: Oracle JDK 7
10+
- Zookeeper: 3.4.6
11+
- Hbase: 1.1.2
12+
- Kafka: 0.8.2
13+
- Storm: 0.10.0
14+
15+
16+
Building all images
17+
-----
18+
- ```./build.sh```
19+
20+
>This will individually build the above mentioned images. This will take sometime and is required to be done only once.
21+
22+
Verification
23+
---
24+
- Run ```docker images``` to see all images created with proper tag version
25+
26+
Running
27+
----
28+
Starting
29+
- ```./start.sh```
30+
31+
Restarting
32+
- ```./restart.sh```
33+
34+
Reference
35+
-----
36+
- Refer README.md files in each image folder.

‎README.md~

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
Build HDP 2.3.2 compliant services
2+
======
3+
4+
Versions
5+
----
6+
- Ubuntu: 14.04
7+
- Java: Oracle JDK 7
8+
- Zookeeper: 3.4.6
9+
- Hbase: 1.1.2
10+
- Kafka: 0.8.2
11+
- Storm: 0.10.0
12+
13+
14+
Building all images
15+
-----
16+
- ```./build.sh```
17+
18+
>This will individually build the above mentioned images. This will take sometime and is required to be done only once.
19+
20+
Verification
21+
---
22+
- Run ```docker images``` to see all images created with proper tag version
23+
24+
Running
25+
----
26+
Starting
27+
- ```./start.sh```
28+
29+
Restarting
30+
- ```./restart.sh```
31+
32+
Reference
33+
-----
34+
- Refer README.md files in each image folder.

‎base/Dockerfile

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
FROM ubuntu:14.04
2+
MAINTAINER anair
3+
4+
LABEL description="Build Ubuntu image with Oracle JDK7"
5+
6+
RUN apt-get -y update
7+
8+
# Install Oracle Java7
9+
RUN apt-get install -y software-properties-common && \
10+
add-apt-repository -y ppa:webupd8team/java && \
11+
apt-get update && \
12+
echo debconf shared/accepted-oracle-license-v1-1 select true | debconf-set-selections && \
13+
echo debconf shared/accepted-oracle-license-v1-1 seen true | debconf-set-selections && \
14+
apt-get install -y oracle-java7-installer
15+
16+
ENV JAVA_HOME /usr/lib/jvm/java-7-oracle/
17+

‎base/Dockerfile~

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
FROM ubuntu:14.04
2+
MAINTAINER anair
3+
4+
LABEL description="Build Ubuntu image with oracle JDK7"
5+
6+
RUN apt-get -y update
7+
8+
# Install Oracle Java7
9+
RUN apt-get install -y software-properties-common && \
10+
add-apt-repository -y ppa:webupd8team/java && \
11+
apt-get update && \
12+
echo debconf shared/accepted-oracle-license-v1-1 select true | debconf-set-selections && \
13+
echo debconf shared/accepted-oracle-license-v1-1 seen true | debconf-set-selections && \
14+
apt-get install -y oracle-java7-installer
15+
16+
ENV JAVA_HOME /usr/lib/jvm/java-7-oracle/
17+

‎base/README.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
Create base image with Ubuntu 14 and Oracle JDK 7. All hadoop images should extend this one.
2+
3+
Refer (build.sh) for building the image.
4+
5+
##Version
6+
- Ubuntu: 14.04
7+
- Java: Oracle JDK 7
8+

‎base/README.md~

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
Create base image with Ubuntu 14 and Oracle JDK 7. All hadoop images should extend this one.
2+
3+
Refer (build.sh) for building the image.

‎base/build.sh

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
#!/bin/sh
2+
docker build -t anair/hadoop_base .
3+

‎base/build.sh~

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
#!/bin/sh
2+
docker build -t anair/hadoop_base
3+

‎build.sh

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
#!/bin/sh
2+
cd base; ./build.sh
3+
cd ../zookeeper-3.4.6; ./build.sh
4+
cd ../hbase-1.1.2; ./build.sh
5+
cd ../kafka-0.8.2; ./build.sh
6+
cd ../storm-0.10.0; ./build.sh

‎docker-compose.yml

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
###zookeeper
2+
zookeeper:
3+
image: anair/zookeeper:3.4.6
4+
container_name: zookeeper
5+
hostname: zk
6+
ports:
7+
- "2181:2181"
8+
9+
###hbase
10+
hbase:
11+
image: anair/hbase:1.1.2
12+
container_name: hbase
13+
hostname: hbase
14+
ports:
15+
- "60000:60000"
16+
- "60010:60010"
17+
- "60020:60020"
18+
- "60030:60030"
19+
links:
20+
- zookeeper:zk
21+
22+
###kafka
23+
kafka:
24+
image: anair/kafka:0.8.2
25+
container_name: kafka
26+
hostname: kafka
27+
ports:
28+
- "9092:9092"
29+
links:
30+
- zookeeper:zk
31+
environment:
32+
KAFKA_ADVERTISED_HOST_NAME: 192.168.56.102
33+
KAFKA_ADVERTISED_PORT: 9092
34+
KAFKA_BROKER_ID: 0
35+
volumes:
36+
- /var/run/docker.sock:/var/run/docker.sock
37+
38+
###Storm
39+
nimbus:
40+
image: anair/storm-nimbus:0.10.0
41+
container_name: storm-nimbus
42+
hostname: nimbus
43+
ports:
44+
- "49773:3773"
45+
- "49772:3772"
46+
- "49627:6627"
47+
- "22"
48+
links:
49+
- zookeeper:zk
50+
51+
supervisor:
52+
image: anair/storm-supervisor:0.10.0
53+
container_name: storm-supervisor
54+
ports:
55+
- "8000"
56+
- "22"
57+
links:
58+
- nimbus:nimbus
59+
- zookeeper:zk
60+
61+
ui:
62+
image: anair/storm-ui:0.10.0
63+
container_name: storm-ui
64+
ports:
65+
- "49080:8080"
66+
- "22"
67+
links:
68+
- nimbus:nimbus
69+
- zookeeper:zk

‎hbase-1.1.2/Dockerfile

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
FROM anair/hadoop_base
2+
MAINTAINER anair
3+
4+
LABEL description="Build docker hbase 1.1.2 image"
5+
6+
ENV HBASE_VERSION 1.1.2
7+
8+
# Download and Install HBase
9+
10+
RUN wget -q http://archive.apache.org/dist/hbase/1.1.2/hbase-$HBASE_VERSION-bin.tar.gz && \
11+
tar xzf hbase-$HBASE_VERSION-bin.tar.gz -C /opt/ && \
12+
ln -s /opt/hbase-$HBASE_VERSION /opt/hbase && \
13+
rm hbase-$HBASE_VERSION-bin.tar.gz
14+
15+
ADD hbase-site.xml /opt/hbase/conf/hbase-site.xml
16+
ADD start-hbase.sh /opt/hbase/bin/start-hbase.sh
17+
RUN chmod a+x /opt/hbase/bin/start-hbase.sh
18+
19+
RUN mkdir /opt/hbase/data
20+
RUN mkdir /opt/hbase/logs
21+
22+
VOLUME /opt/hbase/data
23+
VOLUME /opt/hbase/logs
24+
25+
# HBase Master API port
26+
EXPOSE 60000
27+
# HBase Master Web UI
28+
EXPOSE 60010
29+
# Regionserver API port
30+
EXPOSE 60020
31+
# HBase Regionserver web UI
32+
EXPOSE 60030
33+
34+
WORKDIR /opt/hbase/bin
35+
36+
ENV PATH=$PATH:/opt/hbase/bin
37+
38+
CMD start-hbase.sh

‎hbase-1.1.2/Dockerfile~

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
FROM anair/hadoop_base
2+
MAINTAINER Anoop Nair
3+
4+
LABEL description="Build docker hbase 1.1.2 image"
5+
6+
ENV HBASE_VERSION 1.1.2
7+
8+
# Download and Install HBase
9+
10+
RUN wget -q http://archive.apache.org/dist/hbase/1.1.2/hbase-$HBASE_VERSION-bin.tar.gz && \
11+
tar xzf hbase-$HBASE_VERSION-bin.tar.gz -C /opt/ && \
12+
ln -s /opt/hbase-$HBASE_VERSION /opt/hbase && \
13+
rm hbase-$HBASE_VERSION-bin.tar.gz
14+
15+
ADD hbase-site.xml /opt/hbase/conf/hbase-site.xml
16+
ADD start-hbase.sh /opt/hbase/bin/start-hbase.sh
17+
RUN chmod a+x /opt/hbase/bin/start-hbase.sh
18+
19+
RUN mkdir /opt/hbase/data
20+
RUN mkdir /opt/hbase/logs
21+
22+
VOLUME /opt/hbase/data
23+
VOLUME /opt/hbase/logs
24+
25+
# HBase Master API port
26+
EXPOSE 60000
27+
# HBase Master Web UI
28+
EXPOSE 60010
29+
# Regionserver API port
30+
EXPOSE 60020
31+
# HBase Regionserver web UI
32+
EXPOSE 60030
33+
34+
WORKDIR /opt/hbase/bin
35+
36+
ENV PATH=$PATH:/opt/hbase/bin
37+
38+
CMD start-hbase.sh

‎hbase-1.1.2/README.md

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
# Hbase docker image
2+
This is a hbase docker image
3+
4+
##Version
5+
1.1.2
6+
7+
##Exposed ports
8+
- HBase Master API port: 60000
9+
- HBase Master Web UI: 60010
10+
- Regionserver API port: 60020
11+
- HBase Regionserver web UI: 60030
12+
13+
##Building the image
14+
```./build.sh```
15+
16+
##Usage
17+
Start a cluster with zookeeper and hbase:
18+
19+
- ```docker-compose up -d ```
20+
21+
Destroy a cluster:
22+
23+
- ```docker-compose stop```
24+
25+
26+
## Running hbase commands
27+
- ```docker exec -it hbase bash```
28+
- ```hbase shell```
29+
- Type the hbase commands
30+
31+
## Accessing Hbase UI
32+
- (http://localhost:60010)

‎hbase-1.1.2/README.md~

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
# Hbase docker image
2+
This is a hbase docker image
3+
4+
##Version
5+
1.1.2
6+
7+
##Exposed ports
8+
- HBase Master API port: 60000
9+
- HBase Master Web UI: 60010
10+
- Regionserver API port: 60020
11+
- HBase Regionserver web UI: 60030
12+
13+
##Building the image
14+
```./build.sh```
15+
16+
##Usage
17+
Start a cluster:
18+
19+
- ```docker-compose up -d ```
20+
21+
Destroy a cluster:
22+
23+
- ```docker-compose stop```
24+
25+
26+
## Running hbase commands
27+
- ```docker exec -it hbase bash```
28+
- ```hbase shell```
29+
- Type the hbase commands
30+
31+
## Accessing Hbase UI
32+
- (http://localhost:60010)

‎hbase-1.1.2/build.sh

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
#!/bin/sh
2+
docker build -t anair/hbase:1.1.2 .
3+

‎hbase-1.1.2/build.sh~

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
#!/bin/sh
2+
docker build -t anair/hadoop_base .
3+

‎hbase-1.1.2/docker-compose.yml

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
###zookeeper
2+
zookeeper:
3+
image: anair/zookeeper:3.4.6
4+
container_name: zookeeper
5+
hostname: zk
6+
ports:
7+
- "2181:2181"
8+
9+
###hbase
10+
hbase:
11+
image: anair/hbase:1.1.2
12+
container_name: hbase
13+
hostname: hbase
14+
ports:
15+
- "60000:60000"
16+
- "60010:60010"
17+
- "60020:60020"
18+
- "60030:60030"
19+
links:
20+
- zookeeper:zk
21+

0 commit comments

Comments
 (0)