Skip to content

Commit 4a5516d

Browse files
author
anair-it
committed
Fixed README formatting issues
1 parent ccfcb9f commit 4a5516d

File tree

8 files changed

+86
-122
lines changed

8 files changed

+86
-122
lines changed

‎README.md

Lines changed: 24 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,8 @@
1-
Hadoop cluster - lite
2-
======
1+
# Hadoop cluster - lite
32
Build a lightweight hadoop cluster with below mentioned hadoop services. These versions match what Hortonworks HDP 2.3.4 distribution has.
43

54

6-
Versions
7-
----
5+
## Versions
86
- Alpine Linux: latest
97
- Debian: 8.3
108
- Java: Open JDK 7
@@ -19,52 +17,51 @@ Versions
1917
Note: Hadoop and storm are built on Debian Linux and all others on Alpine Linux
2018

2119

22-
Prerequsities
23-
---
20+
## Prerequsities
2421
- Install [Docker](https://docs.docker.com/engine/installation/)
2522
- Install [Docker compose](https://docs.docker.com/compose/install/)
2623

2724
> The Docker project is tested on Ubuntu host. There is no guarantee that this will work on Windows.
2825
2926

30-
Building all images
31-
-----
32-
- ```./build.sh```
27+
## Building all images
28+
- ``./build.sh``
3329

3430
>This will individually build the above mentioned images. This will take sometime and is required to be done only once.
3531
36-
- Get zookeeper IP address using ```docker inspect --format='{{.NetworkSettings.IPAddress}}' zookeeper````
37-
- Get hbase IP address using ```docker inspect --format='{{.NetworkSettings.IPAddress}}' hbase````
32+
- Get zookeeper IP address using
33+
34+
35+
docker inspect --format='{{.NetworkSettings.IPAddress}}' zookeeper`
36+
- Get hbase IP address using
37+
38+
39+
docker inspect --format='{{.NetworkSettings.IPAddress}}' hbase`
3840
- Add zookeeper and hbase entries in the host /etc/hosts file
3941

40-
Verification
41-
---
42-
- ```docker images``` to see all images created with proper tag version
43-
- ```docker logs -f __CONTAINER_NAME__``` to follow logs for a specific container
42+
## Verification
43+
- ``docker images`` to see all images created with proper tag version
44+
- ``docker logs -f _CONTAINER_NAME_`` to follow logs for a specific container
4445

4546
If there is inconsistency in the image creation process, run the affected build script only. The build script is in each component directory.
4647

47-
Starting
48-
---
49-
- ```./start.sh``` (background process)
48+
## Starting
49+
- ``./start.sh`` (background process)
5050
OR
51-
- docker-compose up (foreground process)
51+
- ``docker-compose up`` (foreground process)
5252

53-
Stopping
54-
----
55-
- ```docker-compose stop```
53+
## Stopping
54+
- ``docker-compose stop``
5655

5756

58-
Web UI
59-
----
57+
## Web UI
6058
- [HBASE](http://localhost:60010)
6159
- [Storm](http://localhost:49080)
6260
- [Kafka manager](http://localhost:9000)
6361
- [Hadoop cluster](http://localhost:8088)
6462

6563

66-
READMEs
67-
-----
64+
## READMEs
6865
Please go through below READMEs to get detailed information about each image
6966

7067
- [Alpine base README](alpine-base/README.md)
@@ -75,8 +72,7 @@ Please go through below READMEs to get detailed information about each image
7572
- [Kafka README](kafka/README.md)
7673
- [Storm README](storm/README.md)
7774

78-
Reference
79-
----
75+
## Reference
8076
- https://hub.docker.com/r/wurstmeister/kafka/
8177
- https://hub.docker.com/r/sheepkiller/kafka-manager/
8278
- https://hub.docker.com/r/wurstmeister/storm/

‎alpine-base/README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,12 @@
1-
- Create base image with [Alpine Linux distro](http://www.alpinelinux.org/)
1+
- Create base image with [Alpine Linux distro](http://www.alpinelinux.org/)
22
- Install essential linux packages
33
- Install Open JDK 7. All hadoop images should extend this one.
44
- Prepare image with directories like /opt
55

66

77
Building the image
88
----
9-
- ```./build.sh```
9+
- ``./build.sh``
1010

1111
Version
1212
---

‎debian-base/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55

66
Building the image
77
----
8-
- ```./build.sh```.
8+
- ``./build.sh``
99

1010
Version
1111
---

‎hadoop/README.md

Lines changed: 7 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,16 @@
1-
hadoop docker image
2-
=====
1+
# Hadoop docker image
32
This is a hadoop docker build project on Debian Linux distro.
43

5-
Version
6-
----
4+
## Version
75
2.7.1
86

97

10-
Building the image
11-
----
12-
- ```./build.sh```
8+
## Building the image
9+
- ``./build.sh``
1310

14-
Running
15-
----
16-
- ```docker run --name hadoop -p 49707:49707 -p 50010:50010 -p 50020:50020 -p 50030:50030 -p 50070:50070 -p 50075:50075 -p 50090:50090 -p 8030:8030 -p 8031:8031 -p 8032:8032 -p 8033:8033 -p 8040:8040 -p 8042:8042 -p 8088:8088 -it anair/hadoop_debian:2.7.1```
11+
## Running
12+
- ``docker run --name hadoop -p 49707:49707 -p 50010:50010 -p 50020:50020 -p 50030:50030 -p 50070:50070 -p 50075:50075 -p 50090:50090 -p 8030:8030 -p 8031:8031 -p 8032:8032 -p 8033:8033 -p 8040:8040 -p 8042:8042 -p 8088:8088 -it anair/hadoop_debian:2.7.1``
1713

18-
UI
19-
---
14+
## UI
2015
- http://localhost:8088/
2116

‎hbase/README.md

Lines changed: 15 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,46 +1,38 @@
1-
Hbase docker image
2-
=======
1+
# Hbase docker image
32
Build a hbase docker image on Alpine Linux distro. Also install Apache Phoenix.
43

5-
Version
6-
------
4+
## Version
75
- Hbase: 1.1.2
86
- Phoenix: 4.7.0
97

10-
Exposed ports
11-
---------
8+
## Exposed ports
129
- HBase Master API port: 60000
1310
- HBase Master Web UI: 60010
1411
- Regionserver API port: 60020
1512
- HBase Regionserver web UI: 60030
1613

17-
Building the image
18-
-------
19-
```./build.sh```
14+
## Building the image
15+
``./build.sh``
2016

21-
Usage
22-
-----
17+
## Usage
2318
Start a cluster with zookeeper and hbase:
2419

25-
- ```docker-compose up -d ```
20+
- ``docker-compose up -d ``
2621

2722
Destroy a cluster:
2823

29-
- ```docker-compose stop```
24+
- ``docker-compose stop``
3025

3126

32-
Running hbase commands
33-
--------
34-
- ```docker exec -it hbase bash```
35-
- ```hbase shell```
27+
## Running hbase commands
28+
- ``docker exec -it hbase bash``
29+
- ``hbase shell``
3630
- Type the hbase commands
3731

38-
Accessing Hbase UI
39-
---------
40-
- (http://localhost:60010)
32+
## Hbase UI
33+
http://localhost:60010
4134

42-
Using SQL client to connect to Hbase
43-
----------
35+
## Using SQL client to connect to Hbase
4436
Apache Phoenix is the SQL interface to the Hbase world. You can use DBeaver or Squirrel to write SQL. SQL will be translated by the Phoenix server component in the hbase region server into Hbase command.
4537

4638
### Setup DBeaver/Squirrel
@@ -49,4 +41,4 @@ Apache Phoenix is the SQL interface to the Hbase world. You can use DBeaver or S
4941
- Add new driver through Driver manager
5042
- Set URL to jdbc:phoenix:zk. Type “org.apache.phoenix.jdbc.PhoenixDriver” into the Class Name textbox and click OK to close this dialog. Add the Phoenix client jar into DBeaver /Squirrel.
5143
- Ensure Zookeeper and Hbase is running locally with Zookeper host as 'zk'
52-
- Test connection
44+
- Test connection

‎kafka/README.md

Lines changed: 19 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1,63 +1,55 @@
1-
Kafka docker image
2-
=======
1+
# Kafka docker image
32
Build a kafka docker image on Alpine Linux distro.
43

5-
Version
6-
----
4+
## Version
75
- Kafka: 0.9.0.1
86
- Kafka manager: latest
97

10-
Building the image
11-
-----
12-
```./build.sh```
8+
## Building the image
9+
``./build.sh``
1310

1411
This script will also pull kafka manager docker image from docker hub.
1512

16-
Usage
17-
----
13+
## Usage
1814
Start a cluster with zookeeper and kafka:
1915

20-
- ```docker-compose up -d ```
16+
- ``docker-compose up -d ``
2117

2218
Add more brokers:
2319

24-
- ```docker-compose scale kafka=3```
20+
- ``docker-compose scale kafka=3``
2521

2622
Destroy a cluster:
2723

28-
- ```docker-compose stop```
24+
- ``docker-compose stop``
2925

3026

31-
Notes
32-
-----
33-
The default ```docker-compose.yml``` should be seen as a starting point. By default each broker will get a new port number and broker id on restart. Depending on your use case this might not be desirable. If you need to use specific ports and broker ids, modify the docker-compose configuration accordingly
27+
## Notes
28+
The default _docker-compose.yml_ should be seen as a starting point. By default each broker will get a new port number and broker id on restart. Depending on your use case this might not be desirable. If you need to use specific ports and broker ids, modify the docker-compose configuration accordingly
3429

3530

36-
Automatically create topics
37-
---------
38-
If you want to have kafka-docker automatically create topics in Kafka during creation, a ```KAFKA_CREATE_TOPICS``` environment variable can be
39-
added in ```docker-compose.yml```.
31+
### Automatically create topics
32+
If you want to have kafka-docker automatically create topics in Kafka during creation, a _KAFKA_CREATE_TOPICS_ environment variable can be
33+
added in _docker-compose.yml_.
4034

41-
Here is an example snippet from ```docker-compose.yml```:
35+
Here is an example snippet from _docker-compose.yml_:
4236

4337
environment:
4438
KAFKA_CREATE_TOPICS: "INBOUND:1:2,OUTBOUND:1:1"
4539

46-
```INBOUND``` topic will have 1 partition and 2 replicas, ```OUTBOUND``` topic will have 1 partition and 1 replica.
40+
_INBOUND_ topic will have 1 partition and 2 replicas, _OUTBOUND_ topic will have 1 partition and 1 replica.
4741

48-
Custom environments variables
49-
------
42+
## Custom environments variables
5043
1. Refer environment variables in [Kafka documentation](http://kafka.apache.org/documentation.html#configuration)
5144
2. Convert the variable to uppercase, replace '.' with '_'
5245
3. Prepend KAFKA_ to the variable
5346
4. Use the variable as an environment variable
5447

55-
- Advertised hostname: ```KAFKA_ADVERTISED_HOST_NAME```
56-
- Advertised port: ```KAFKA_ADVERTISED_PORT```
48+
- Advertised hostname: _KAFKA_ADVERTISED_HOST_NAME_
49+
- Advertised port: _KAFKA_ADVERTISED_PORT_
5750

5851

59-
Kafka manager UI
60-
------
52+
## Kafka manager UI
6153
- http://localhost:9090
6254
- Create a cluster with zookeeper connection as 'zk:2181'
6355
- Automatically detect broker and topics

‎storm/README.md

Lines changed: 12 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,38 +1,31 @@
1-
Storm docker image
2-
============
1+
# Storm docker image
32
Build a storm docker image on Debian Linux distro.
43

54

6-
Verison
7-
----
5+
## Verison
86
0.10.0
97

10-
Building the image
11-
----
12-
- ```./build.sh```
13-
14-
Usage
15-
---
8+
## Building the image
9+
- ``./build.sh``
1610

11+
## Usage
1712
Start a cluster with zookeeper and docker:
1813

19-
- ```docker-compose up```
14+
- ``docker-compose up``
2015

2116
Destroy a cluster:
2217

23-
- ```docker-compose stop```
18+
- ``docker-compose stop``
2419

2520
Add more supervisors:
2621

27-
- ```docker-compose scale supervisor=3```
22+
- ``docker-compose scale supervisor=3``
2823

2924

30-
Access Storm UI
31-
----
32-
- (http://localhost:49080)
25+
## Storm UI
26+
http://localhost:49080
3327

3428

35-
Deploy a topology
36-
-----
37-
- ```storm jar my-topology.jar com.package topology-name```
29+
## Deploy a topology
30+
- ``storm jar my-topology.jar com.package topology-name``
3831

‎zookeeper/README.md

Lines changed: 6 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,14 @@
1-
Zookeeper docker image
2-
========
1+
# Zookeeper docker image
32
Build a zookeeper docker image on Alpine Linux distro.
43
The ports 2181,2888,3888 are exposed.
54

6-
Version
7-
----
5+
## Version
86
3.4.6
97

108

11-
Building the image
12-
----
13-
- ```./build.sh```
9+
## Building the image
10+
- ``./build.sh``
1411

15-
Running zookeeper on a container
16-
----
17-
- ```./run.sh```
12+
## Running zookeeper on a container
13+
- ``./run.sh``
1814

0 commit comments

Comments
 (0)