|
1 | | -# docker-hadoop |
2 | | -Hadoop (Single Node) with Docker from scratch. |
| 1 | +# Hadoop (Single Node) with Docker from scratch. |
| 2 | + |
| 3 | +[](https://registry.hub.docker.com/u/angelcervera/docker-hadoop/) |
| 4 | +[](https://registry.hub.docker.com/u/angelcervera/docker-hadoop/) |
| 5 | + |
| 6 | +With [this image](https://hub.docker.com/r/angelcervera/docker-hadoop/), you will have a **Hadoop 2.7.1 Single Node** container ready for development / testing / debug porpuse. |
| 7 | + |
| 8 | +## For impatient people. |
| 9 | +The first time, will take few minutes downloading the image. |
| 10 | + |
| 11 | +### To start the container (very impatient people). |
| 12 | +``` |
| 13 | +docker run --name hadoop-2.7.1 -it -P angelcervera/docker-hadoop:2.7.1-single |
| 14 | +``` |
| 15 | +After this, you will be inside of the hadoop docker container terminal. |
| 16 | + |
| 17 | +Basically: |
| 18 | +- Hadoop installation folder : **/opt/hadoop** |
| 19 | +- Hadoop tmp folder (data folder) : **/var/lib/hadoop** |
| 20 | +- Hadoop config folder : **/opt/hadoop/etc/hadoop** |
| 21 | + |
| 22 | +If you want to execute a hadoop example, for example: |
| 23 | +``` |
| 24 | +/opt/hadoop/bin/hadoop jar /opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar pi 16 100000 |
| 25 | +``` |
| 26 | + |
| 27 | +### To start the container (no so much impatient). |
| 28 | +**Where "username", put your username :)** |
| 29 | +``` |
| 30 | +mkdir /home/username/hadoop |
| 31 | +mkdir /home/username/hadoop/installation |
| 32 | +mkdir /home/username/hadoop/data |
| 33 | +mkdir /home/username/hadoop/shared |
| 34 | +docker run --name my-new-hadoop-2.7.1 \ |
| 35 | + -v /home/username/hadoop/installation:/opt/hadoop \ |
| 36 | + -v /home/username/hadoop/data:/var/lib/hadoop \ |
| 37 | + -v /home/username/hadoop/shared:/root/shared \ |
| 38 | + -p 50070:50070 \ |
| 39 | + -p 50075:50075 \ |
| 40 | + -p 50060:50060 \ |
| 41 | + -p 50030:50030 \ |
| 42 | + -p 19888:19888 \ |
| 43 | + -p 10033:10033 \ |
| 44 | + -p 8032:8032 \ |
| 45 | + -p 8030:8030 \ |
| 46 | + -p 8088:8088 \ |
| 47 | + -p 8033:8033 \ |
| 48 | + -p 8042:8042 \ |
| 49 | + -p 8188:8188 \ |
| 50 | + -p 8047:8047 \ |
| 51 | + -p 8788:8788 \ |
| 52 | + -it angelcervera/docker-hadoop:2.7.1-single |
| 53 | +``` |
| 54 | +After this, you will be inside of the hadoop docker container terminal. |
| 55 | + |
| 56 | +Basically: |
| 57 | +- Hadoop installation folder : **/opt/hadoop** but you can access throw your mapped local folder **/home/username/hadoop/installation** |
| 58 | +- Hadoop tmp folder (data folder) : **/var/lib/hadoop** but you can access throw your mapped local folder **/home/username/hadoop/data** |
| 59 | +- Hadoop config folder : **/opt/hadoop/etc/hadoop** but you can access throw your mapped local folder **/home/username/hadoop/installation/etc/hadoop** |
| 60 | +- Usesful folder to share files: **/root/shared mapped** to your local folder **/home/username/hadoop/shared** |
| 61 | +- There is a big list of ports exposed, so you can map only ports that you are going to use or you can expose all replacing **-p local_port:exposed_port** by **-P** You can check the full list of ports in the [Dockfile](https://github.com/angelcervera/docker-hadoop/blob/master/Dockerfile) |
| 62 | + |
| 63 | +## For curious people |
| 64 | +Because the best idea is not to use this image directly and create your own. In my blog, you can read a step by step article about how I implemented this images: [Full article!](http://www.acervera.com/blog/2015/10/04/hadoop_singlenode_with_docker_from_scratch) |
| 65 | + |
| 66 | + |
| 67 | +Enjoy!!! |
0 commit comments