add new deployment task for property/namespace

lucperkins · lucperkins · commit d0ba5790b3b5 · 2017-12-18T18:44:58.000-08:00
diff --git a/driver-kafka/README.md b/driver-kafka/README.md
@@ -0,0 +1,3 @@
+# Apache Kafka benchmarks
+
+This folder houses all of the assets necessary to run benchmarks for Apache Kafka.
diff --git a/driver-pulsar/README.md b/driver-pulsar/README.md
@@ -0,0 +1,41 @@
+# Apache Pulsar benchmarks
+
+This folder houses all of the assets necessary to run benchmarks for Apache Pulsar.
+
+## Creating a Pulsar cluster on Amazon Web Services (AWS) using Terraform and Ansible
+
+In order to create an Apache Pulsar cluster on AWS, you'll need to have the following installed:
+
+* [Terraform](https://terraform.io)
+* [The `terraform-inventory` plugin for Terraform](https://github.com/adammck/terraform-inventory)
+* [Ansible](http://docs.ansible.com/ansible/latest/intro_installation.html)
+* The messaging benchmarks repository:
+
+  ```bash
+  $ git clone https://github.com/streamlio/messaging-benchmark
+  $ cd messaging-benchmark/driver-pulsar/deploy
+  ```
+
+In addition, you will need to:
+
+* [Create an AWS account](https://aws.amazon.com/account/) (or use an existing account)
+* [Install the `aws` CLI tool](https://aws.amazon.com/cli/)
+* [Configure the `aws` CLI tool](http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html)
+
+Once those conditions are in place, you can create the necessary AWS resources using a single Terraform command (from this directory):
+
+```bash
+$ terraform apply
+```
+
+That will install the following [EC2](https://aws.amazon.com/ec2) instances (plus some other resources, such as a [Virtual Private Cloud](https://aws.amazon.com/vpc/) (VPC)):
+
+Resource | Description | Count
+:--------|:------------|:-----
+Pulsar/BookKeeper instances | The VMs on which a Pulsar broker and BookKeeper bookie will run | 3
+ZooKeeper instances | The VMs on which a ZooKeeper node will run | 3
+Client instance | The VM from which the benchmarking suite itself will be run | 1
+
+When you run `terraform apply`, you will be prompted to type `yes`. Type `yes` to continue with the installation or anything else to quit.
+
+Once the installation is complete, you will see a confirmation message listing the resources that have been installed. 
diff --git a/driver-pulsar/deploy/deploy.yaml b/driver-pulsar/deploy/deploy.yaml
@@ -101,7 +101,6 @@
         daemon_reload: yes
         name: "zookeeper"
     - name: Initialize cluster metadata
-      run_once: true
       command: >
         bin/pulsar initialize-cluster-metadata --cluster local
         --zookeeper localhost:2181
@@ -189,6 +188,16 @@
          dest: /opt/benchmark/bin/benchmark
          regexp: '^JVM_MEM='
          line: 'JVM_MEM="-Xms24G -Xmx24G -XX:+UseG1GC -XX:MaxGCPauseMillis=10 -XX:+ParallelRefProcEnabled -XX:+UnlockExperimentalVMOptions -XX:+AggressiveOpts -XX:+DoEscapeAnalysis -XX:ParallelGCThreads=32 -XX:ConcGCThreads=32 -XX:G1NewSizePercent=50 -XX:+DisableExplicitGC -XX:-ResizePLAB -XX:+PerfDisableSharedMem -XX:+AlwaysPreTouch -XX:-UseBiasedLocking"'
+    - name: Pulsar property and namespace setup
+      connection: ssh
+      become: true
+      command: >
+        bin/pulsar-admin {{ item }}
+      with_items:
+      - properties create benchmark -r all -c local
+      - namespaces create benchmark/local/ns
+      args:
+        chdir: /opt/pulsar
 
 - name:  Hosts addresses
   hosts: localhost

Original file line number	Diff line number	Diff line change
`@@ -0,0 +1,3 @@`
	`1`	`+# Apache Kafka benchmarks`
	`2`	`+`
	`3`	`+This folder houses all of the assets necessary to run benchmarks for Apache Kafka.`