This blog post covers installing Monasca using Ansible and assumes you have Ansible installed. The core of the post will be the Ansible playbook used to install Monasca on Ubuntu Trusty (14.04) having everything on on machine.
First thing is to setup the inventory file for the installation, let’s call it inventory. The following one is a generic one and you can replace monasca with the appropiate hostname.
monasca ansible_ssh_user=ubuntu
Next we will have to install missing Ansible role dependencies. We will retrieve them from Ansible Galaxy.
ansible-galaxy install tkuhlman.zookeeper tkuhlman.kafka tkuhlman.Influxdb tkuhlman.percona tkuhlman.monasca-schema tkuhlman.storm tkuhlman.monasca-api tkuhlman.monasca-persister tkuhlman.monasca-thresh tkuhlman.monasca-notification tkuhlman.monasca-agent
Having installed all role dependencies, create a file called monasca.yml and fill it in with the following content. Please change the variables according to your preferences as well as any other predefined values I’ve set throughout the playbook.
---
- hosts: monasca
sudo: yes
vars:
mysql_root_password: pass
mysql_username: monasca
mysql_password: password
influxdb_username: monasca
influxdb_password: password
keystone_host: keystone.mycloud.com
keystone_username: admin
keystone_password: password
monasca_agent_username: "monasca-agent"
monasca_agent_password: password
monasca_agent_project: project
roles:
- { role: tkuhlman.zookeeper,
zookeeper_hosts: 'localhost:2181' }
- { role: tkuhlman.kafka,
kafka_hosts: 'localhost:9092',
zookeeper_hosts: 'localhost:2181' }
- { role: tkuhlman.Influxdb,
zookeeper_hosts: "localhost:2181",
influxdb_version: 0.9.0-rc28,
influxdb: {raft_port: 8095, replication_factor: 1, seed_servers: [ ]} }
- { role: tkuhlman.percona,
mysql_root_password: "{{ mysql_root_password }}" }
- { role: tkuhlman.monasca-schema,
zookeeper_hosts: "localhost:2181",
mysql_admin_password: "{{ mysql_root_password }}",
influxdb_version: 0.9.0-rc28,
influxdb_users: {"{{ influxdb_username }}": "{{ influxdb_password}}"},
influxdb_url: "http://localhost:8086",
mysql_users: {"{{ mysql_username }}": "{{ mysql_password }}"},
kafka_replicas: 1 }
- { role: tkuhlman.storm,
storm_nimbus_enabled: true,
storm_supervisor_enabled: true,
nimbus_host: localhost,
zookeeper_hosts: "localhost:2181" }
- { role: tkuhlman.monasca-api,
zookeeper_hosts: 'localhost:2181',
kafka_hosts: 'localhost:9092',
api_region: RegionOne,
influxdb_version: 0.9.0-rc28,
influxdb_url: "http://localhost:8086",
influxdb_user: "{{ influxdb_username }}",
influxdb_password: "{{ influxdb_password }}",
kafka_hosts: "localhost:9092",
keystone_host: "{{ keystone_host }}",
keystone_admin: "{{ keystone_username }}",
keystone_admin_password: "{{ keystone_password }}",
mysql_host: "localhost",
mysql_user: "{{ mysql_username }}",
mysql_password: "{{ mysql_password }}" }
- { role: tkuhlman.monasca-persister,
zookeeper_hosts: "localhost:2181",
kafka_hosts: "localhost:9092",
influxdb_version: 0.9.0-rc28,
influxdb_url: "http://localhost:8086",
influxdb_user: "{{ influxdb_username }}",
influxdb_password: "{{ influxdb_password }}" }
- { role: tkuhlman.monasca-thresh,
zookeeper_hosts: "localhost:2181",
kafka_hosts: "localhost:9092",
mysql_host: "localhost",
mysql_user: "{{ mysql_username }}",
mysql_password: "{{ mysql_password }}" }
- { role: tkuhlman.monasca-notification,
zookeeper_hosts: "localhost:2181",
kafka_hosts: "localhost:9092",
mysql_host: "localhost",
mysql_user: "{{ mysql_username }}",
mysql_password: "{{ mysql_password }}",
smtp_host: "localhost" }
- { role: tkuhlman.monasca-agent,
keystone_url: "http://{{ keystone_host }}:35357/v3",
monasca_agent_user: "{{ monasca_agent_username }}",
monasca_agent_password: "{{ monasca_agent_password }}",
monasca_agent_project: "{{ monasca_agent_project }}",
monasca_agent_service: monasca,
monasca_agent_dimensions: "service:monasca" }
In order to run the playbook do a
ansible-playbook -i inventory monasca.yml
That’s it, everything should work fine now but should you have any problems with the playbook or if you think I’ve missed something please let me know.
Thanks for the Monasca installation steps. I am very new to this ansible installation. I have followed the given steps to install monasca and facing the below error and in the monasca.yml file I am not sure which parameter to edit / change and which parameters to leave it as it is. do you have any other documents with exact steps with monasca.yml what are all the parameter needs to be edited / changed.
PLAY [monasca] ****************************************************************
GATHERING FACTS ***************************************************************
ok: [monasca]
TASK: [tkuhlman.zookeeper | Stop zookeeper] ***********************************
skipping: [monasca]
TASK: [tkuhlman.zookeeper | Install OpenJDK] **********************************
ok: [monasca]
TASK: [tkuhlman.zookeeper | Install Zookeeper] ********************************
ok: [monasca]
TASK: [tkuhlman.zookeeper | Install OpenJDK] **********************************
skipping: [monasca]
TASK: [tkuhlman.zookeeper | Change ownership on monasca download directory.] ***
skipping: [monasca]
TASK: [tkuhlman.zookeeper | Change ownership on zokeeper directory.] **********
skipping: [monasca]
TASK: [tkuhlman.zookeeper | Download Zookeeper version] ***********************
skipping: [monasca]
TASK: [tkuhlman.zookeeper | Unpack tarball] ***********************************
skipping: [monasca]
TASK: [tkuhlman.zookeeper | Setup Apache ZooKeeper service] *******************
skipping: [monasca]
TASK: [tkuhlman.zookeeper | reload systemd] ***********************************
skipping: [monasca]
TASK: [tkuhlman.zookeeper | Setup zoo.cfg] ************************************
ok: [monasca]
TASK: [tkuhlman.zookeeper | Setup myid] ***************************************
ok: [monasca]
TASK: [tkuhlman.zookeeper | Setup environment] ********************************
ok: [monasca]
TASK: [tkuhlman.zookeeper | Create log_dir] ***********************************
ok: [monasca]
TASK: [tkuhlman.zookeeper | Setup log4j] **************************************
ok: [monasca]
TASK: [tkuhlman.zookeeper | Enable and start zookeeper] ***********************
ok: [monasca]
TASK: [tkuhlman.zookeeper | wait for zookeeper port] **************************
ok: [monasca]
TASK: [tkuhlman.kafka | Stop Kafka] *******************************************
skipping: [monasca]
TASK: [tkuhlman.kafka | Install OpenJDK] **************************************
ok: [monasca]
TASK: [tkuhlman.kafka | Fetch kafka binary package] ***************************
ok: [monasca]
TASK: [tkuhlman.kafka | setup group] ******************************************
ok: [monasca]
TASK: [tkuhlman.kafka | Setup user] *******************************************
ok: [monasca]
TASK: [tkuhlman.kafka | Uncompress the kafka tar] *****************************
ok: [monasca]
TASK: [tkuhlman.kafka | Link /opt/kafka to the right version] *****************
ok: [monasca]
TASK: [tkuhlman.kafka | Overwrite the start script so the Java Opts can be changed if Kafka 0.8.1.1] ***
ok: [monasca]
TASK: [tkuhlman.kafka | Detect if this is a systemd based system] *************
changed: [monasca]
TASK: [tkuhlman.kafka | set_fact use_systemd=True] ****************************
skipping: [monasca]
TASK: [tkuhlman.kafka | set_fact use_systemd=False] ***************************
ok: [monasca]
TASK: [tkuhlman.kafka | create upstart config] ********************************
ok: [monasca]
TASK: [tkuhlman.kafka | create systemd config] ********************************
skipping: [monasca]
TASK: [tkuhlman.kafka | command systemctl daemon-reload] **********************
skipping: [monasca]
TASK: [tkuhlman.kafka | Create data_dir] **************************************
ok: [monasca]
TASK: [tkuhlman.kafka | Remove lost+found in the datadir] *********************
ok: [monasca]
TASK: [tkuhlman.kafka | Create log_dir] ***************************************
ok: [monasca]
TASK: [tkuhlman.kafka | link conf_dir to /opt/kafka/config] *******************
ok: [monasca]
TASK: [tkuhlman.kafka | create log4j.properties] ******************************
ok: [monasca]
TASK: [tkuhlman.kafka | create server.properties] *****************************
ok: [monasca]
TASK: [tkuhlman.kafka | Enable and Start Kafka] *******************************
ok: [monasca]
TASK: [tkuhlman.kafka | wait for kafka port] **********************************
failed: [monasca] => {“elapsed”: 10, “failed”: true}
msg: Timeout when waiting for localhost:9092
FATAL: all hosts have already failed — aborting
PLAY RECAP ********************************************************************
to retry, use: –limit @/root/monasca.retry
monasca : ok=27 changed=1 unreachable=0 failed=1
Could you please check that Kafka is running? Also look at its settings to see if the port is set to something else. If its not running, run it by hand and see the error message. I’ve had issues that when running ansible several times on the same machine Kafka would fail to start. What I did then was just run it step by step.
for Kafka to run, you need to add hostname in /etc/hosts.
Eg: 127.0.0.1 localhost hostname
Hi,
I have just tried the playbook and I’ve got the same error as Arun. “service kafka status” shows this error:
java.lang.IllegalArgumentException: requirement failed: Missing required property ‘broker.id’
Adding “broker.id=0” to the “/etc/kafka/server.properties” and starting the kafka service (via “service kafka start”) it fixed the error but once I run
“ansible-playbook -i inventory monasca.yml”
the configuration file is overridden and the error is thrown again.
Is there a way to set the “broker.id” in the given playbook?
Thanks,
Mihai