Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

Saltstack with Zabbix

•Download as PPTX, PDF•
•
C
Christian McHugh

Example of how you can leverage the salt event bus to support your infrastructure life-cycle for monitoring with Zabbix. Enable workflows like when adding salt states to a minion automatically apply associated monitoring templates. or when decommissioning hosts, automatically remove them from Zabbix.

Related slideshows

Infrastructure as Code for Azure: ARM or Terraform?
Infrastructure as Code for Azure: ARM or Terraform?Infrastructure as Code for Azure: ARM or Terraform?
Infrastructure as Code for Azure: ARM or Terraform?
 
Mastering OpenStack - Episode 01 - Simple Architectures
Mastering OpenStack - Episode 01 - Simple ArchitecturesMastering OpenStack - Episode 01 - Simple Architectures
Mastering OpenStack - Episode 01 - Simple Architectures
 
Install SSH Server on Windows 2008 R2
Install SSH Server on Windows 2008 R2Install SSH Server on Windows 2008 R2
Install SSH Server on Windows 2008 R2
 
1 of 25
Salt with Zabbix
Saltstack Configuration Management Salt can do much more than configure a single machine It is a state engine which allows for complicated automation workflows
Monitoring SDLC As capabilities are added to the environment they must be monitored Zabbix provides capabilities and APIs for monitoring component, architecture, and service level monitoring Concerned with complexity, scale, and maintainability. Create patterns allowing distributed teams to interface with monitoring, without needing to know all the technical details.
Zabbix Server Zabbix Introduction host1 Nginx Service Web Service: myapp Host Group Host: host1 Templates OS Linux Nginx myapp Item: CPU Item: Connection status Web Scenario: Site availability Trigger: High CPU Graph: Connections SLA: Availability Inventory Data
Salt States: myapp service base: 'salt-client1.station': ... - openssh.config - zabbix.agent.conf - nginx - example_app
Salt States: myapp service base: 'salt-client1.station': ... - openssh.config - zabbix.agent.conf - nginx - example_app Mapping Linux Template Zabbix Template Nginx Template Myapp Template
state.highstate Salt-client1.station: ... Name: /etc/ssh/sshd_config - Function: file.managed - Result: Clean Started: - 22:10:19.597471 Duration: 110.485 ms Name: zabbix-agent - Function: pkg.installed - Result: Clean Started: - 22:10:19.790945 Duration: 11.088 ms Name: nginx - Function: pkg.installed - Result: Clean Started: - 22:10:20.002695 Duration: 10.612 ms Name: /etc/nginx/nginx.conf - Function: file.managed - Result: Clean Started: - 22:10:20.013534 Duration: 32.175 ms Name: /etc/nginx/sites-available/mysite - Function: file.managed - Result: Clean Started: - 22:10:20.046610 Duration: 27.779 ms Name: /var/www/html/mysite/index.html - Function: file.managed - Result: Clean Started: - 22:10:20.119449 Duration: 17.067 ms Summary for salt-client1.station ------------- Succeeded: 34 Failed: 0 ------------- Total states run: 34 Total run time: 548.998 ms
state.highstate
state.highstate Yay! But what about monitoring?
Salt States: Notify Zabbix base: 'salt-client1.station': ... - openssh.config - zabbix.agent.conf - nginx - example_app - monitoring
Monitoring State Example create_host_group: zabbix_hostgroup.present: - name: {{ zab_hostgroup }} create_host_{{ zab_host }}: zabbix_host.present: - host: {{ zab_host }} - groups: - {{ zab_hostgroup }} - interfaces: - {{ zab_fqdn }}: - ip: {{ zab_host_ip }} - type: 'Agent' - port: 10050 - inventory: {{ zab_inv }} add_zabbix_templates_to_host: zabbix_host.assign_templates: - host: {{ zab_host }} - templates: {{ zab_templates }} Add Zabbix host group Add host to Zabbix Assign templates to host in Zabbix
Bonus: Zabbix Inventory
Monitoring State Example Sounds simple enough. But how do we associate salt states with Zabbix templates? Goal: “Create patterns allowing distributed teams to interface with monitoring, without needing to know all the technical details.” add_zabbix_templates_to_host: zabbix_host.assign_templates: - host: {{ zab_host }} - templates: {{ zab_templates }}
Monitoring State Example Sounds simple enough. But how do we associate salt states with Zabbix templates? Goal: “Create patterns allowing distributed teams to interface with monitoring, without needing to know all the technical details.” add_zabbix_templates_to_host: zabbix_host.assign_templates: - host: {{ zab_host }} - templates: {{ zab_templates }} sshd_banner: file.managed: - name: {{ openssh.banner }} ... enable_linux_template: grains.list_present: - name: monitoring_templates - value: 'Template OS Linux by Zabbix agent active'
Monitoring State Example # salt 'salt-client1.station' grains.get monitoring_templates salt-client1.station: - Template OS Linux by Zabbix agent active - Template App Nginx by Zabbix agent - myapp
Monitoring State Example # salt 'salt-client1.station' grains.get monitoring_templates salt-client1.station: - Template OS Linux by Zabbix agent active - Template App Nginx by Zabbix agent - myapp
Monitoring State Example Cool! But what about the life cycle? No, the machine life cycle: deleting/updating
Phase 2: Salt Event Bus Up until now, we kept things direct by having the salt minion add itself to Zabbix But this has drawbacks in that the minion needs to have the Zabbix password, and cannot notify Zabbix it has been deleted.
Phase 2: Salt Event Bus Salt Minion Salt Master Zabbix Server Event Bus Highstate update Highstate update Orch: Gather data Add/update host in Zabbix Decom Process Minion destroyed Minion destroyed Remove host from Zabbix
Notify Salt Event Bus Update_monitor_templates: event.send: - name: custom/monitoring/update - data: ip_address: {{ salt['network.ip_addrs']() | first | default([]) }} default_group: {{ salt['grains.get']('ec2_tags:SupportGroup', 'GenericSupport') }}/{{ salt['grains.get']('ec2_tags:Environment', 'Tagless') }} fqdn: {{ hostfqdn }} inventory: [ "os": {{ salt['grains.get']('osrelease', 'UnknownOS') }}, ] - onchanges: - grains: monitoring_templates mineupdate: module.wait: - mine.update: - name: mine.update - watch: - grains: monitoring_templates
Event Reactor zabbix_update_host_templates: runner.state.orchestrate: - args: - mods: orchestrations.monitoring_update_template - pillar: event_data: {{ data | json() }}
Event Orchestration {%- set data = salt.pillar.get('event_data') %} {# Queries salt mine for monitoring_groups values for specified host #} {%- set monitoring_templates = salt.saltutil.runner('mine.get', tgt=data['id'], fun='monitoring_templates', tgt_type='list')[data['id']] %} ensure_in_zabbix: salt.state: - tgt: "salt-master1*" - tgt_type: glob - concurrent: True - sls: monitoring.orch.ensure_zabbixed - pillar: groups: {{ data['data']['default_group'] }} host: {{ data['id'] }} ip: {{ data['data']['ip_address'] }} templates: {{ monitoring_templates }} inventory: {{ data['data']['inventory'] }} fqdn: {{ data['data']['fqdn'] }}
Salt Event Bus Summary By using the salt event bus, we can tie the salt master’s state engine into any part of our provisioning process. Also means that the Zabbix password only needs to be granted to the salt master itself (or wherever you run the orchestration job (Zabbix server?))
Workflow Integration Summary Salt states simply need to add the Zabbix template name to a grain list. The monitoring team owns the monitoring integration state and orchestration
Questions? Thanks for attending!

More Related Content

Saltstack with Zabbix

  • 2. Saltstack Configuration Management Salt can do much more than configure a single machine It is a state engine which allows for complicated automation workflows
  • 3. Monitoring SDLC As capabilities are added to the environment they must be monitored Zabbix provides capabilities and APIs for monitoring component, architecture, and service level monitoring Concerned with complexity, scale, and maintainability. Create patterns allowing distributed teams to interface with monitoring, without needing to know all the technical details.
  • 4. Zabbix Server Zabbix Introduction host1 Nginx Service Web Service: myapp Host Group Host: host1 Templates OS Linux Nginx myapp Item: CPU Item: Connection status Web Scenario: Site availability Trigger: High CPU Graph: Connections SLA: Availability Inventory Data
  • 5. Salt States: myapp service base: 'salt-client1.station': ... - openssh.config - zabbix.agent.conf - nginx - example_app
  • 6. Salt States: myapp service base: 'salt-client1.station': ... - openssh.config - zabbix.agent.conf - nginx - example_app Mapping Linux Template Zabbix Template Nginx Template Myapp Template
  • 7. state.highstate Salt-client1.station: ... Name: /etc/ssh/sshd_config - Function: file.managed - Result: Clean Started: - 22:10:19.597471 Duration: 110.485 ms Name: zabbix-agent - Function: pkg.installed - Result: Clean Started: - 22:10:19.790945 Duration: 11.088 ms Name: nginx - Function: pkg.installed - Result: Clean Started: - 22:10:20.002695 Duration: 10.612 ms Name: /etc/nginx/nginx.conf - Function: file.managed - Result: Clean Started: - 22:10:20.013534 Duration: 32.175 ms Name: /etc/nginx/sites-available/mysite - Function: file.managed - Result: Clean Started: - 22:10:20.046610 Duration: 27.779 ms Name: /var/www/html/mysite/index.html - Function: file.managed - Result: Clean Started: - 22:10:20.119449 Duration: 17.067 ms Summary for salt-client1.station ------------- Succeeded: 34 Failed: 0 ------------- Total states run: 34 Total run time: 548.998 ms
  • 10. Salt States: Notify Zabbix base: 'salt-client1.station': ... - openssh.config - zabbix.agent.conf - nginx - example_app - monitoring
  • 11. Monitoring State Example create_host_group: zabbix_hostgroup.present: - name: {{ zab_hostgroup }} create_host_{{ zab_host }}: zabbix_host.present: - host: {{ zab_host }} - groups: - {{ zab_hostgroup }} - interfaces: - {{ zab_fqdn }}: - ip: {{ zab_host_ip }} - type: 'Agent' - port: 10050 - inventory: {{ zab_inv }} add_zabbix_templates_to_host: zabbix_host.assign_templates: - host: {{ zab_host }} - templates: {{ zab_templates }} Add Zabbix host group Add host to Zabbix Assign templates to host in Zabbix
  • 13. Monitoring State Example Sounds simple enough. But how do we associate salt states with Zabbix templates? Goal: “Create patterns allowing distributed teams to interface with monitoring, without needing to know all the technical details.” add_zabbix_templates_to_host: zabbix_host.assign_templates: - host: {{ zab_host }} - templates: {{ zab_templates }}
  • 14. Monitoring State Example Sounds simple enough. But how do we associate salt states with Zabbix templates? Goal: “Create patterns allowing distributed teams to interface with monitoring, without needing to know all the technical details.” add_zabbix_templates_to_host: zabbix_host.assign_templates: - host: {{ zab_host }} - templates: {{ zab_templates }} sshd_banner: file.managed: - name: {{ openssh.banner }} ... enable_linux_template: grains.list_present: - name: monitoring_templates - value: 'Template OS Linux by Zabbix agent active'
  • 15. Monitoring State Example # salt 'salt-client1.station' grains.get monitoring_templates salt-client1.station: - Template OS Linux by Zabbix agent active - Template App Nginx by Zabbix agent - myapp
  • 16. Monitoring State Example # salt 'salt-client1.station' grains.get monitoring_templates salt-client1.station: - Template OS Linux by Zabbix agent active - Template App Nginx by Zabbix agent - myapp
  • 17. Monitoring State Example Cool! But what about the life cycle? No, the machine life cycle: deleting/updating
  • 18. Phase 2: Salt Event Bus Up until now, we kept things direct by having the salt minion add itself to Zabbix But this has drawbacks in that the minion needs to have the Zabbix password, and cannot notify Zabbix it has been deleted.
  • 19. Phase 2: Salt Event Bus Salt Minion Salt Master Zabbix Server Event Bus Highstate update Highstate update Orch: Gather data Add/update host in Zabbix Decom Process Minion destroyed Minion destroyed Remove host from Zabbix
  • 20. Notify Salt Event Bus Update_monitor_templates: event.send: - name: custom/monitoring/update - data: ip_address: {{ salt['network.ip_addrs']() | first | default([]) }} default_group: {{ salt['grains.get']('ec2_tags:SupportGroup', 'GenericSupport') }}/{{ salt['grains.get']('ec2_tags:Environment', 'Tagless') }} fqdn: {{ hostfqdn }} inventory: [ "os": {{ salt['grains.get']('osrelease', 'UnknownOS') }}, ] - onchanges: - grains: monitoring_templates mineupdate: module.wait: - mine.update: - name: mine.update - watch: - grains: monitoring_templates
  • 21. Event Reactor zabbix_update_host_templates: runner.state.orchestrate: - args: - mods: orchestrations.monitoring_update_template - pillar: event_data: {{ data | json() }}
  • 22. Event Orchestration {%- set data = salt.pillar.get('event_data') %} {# Queries salt mine for monitoring_groups values for specified host #} {%- set monitoring_templates = salt.saltutil.runner('mine.get', tgt=data['id'], fun='monitoring_templates', tgt_type='list')[data['id']] %} ensure_in_zabbix: salt.state: - tgt: "salt-master1*" - tgt_type: glob - concurrent: True - sls: monitoring.orch.ensure_zabbixed - pillar: groups: {{ data['data']['default_group'] }} host: {{ data['id'] }} ip: {{ data['data']['ip_address'] }} templates: {{ monitoring_templates }} inventory: {{ data['data']['inventory'] }} fqdn: {{ data['data']['fqdn'] }}
  • 23. Salt Event Bus Summary By using the salt event bus, we can tie the salt master’s state engine into any part of our provisioning process. Also means that the Zabbix password only needs to be granted to the salt master itself (or wherever you run the orchestration job (Zabbix server?))
  • 24. Workflow Integration Summary Salt states simply need to add the Zabbix template name to a grain list. The monitoring team owns the monitoring integration state and orchestration

Editor's Notes

  1. Let’s explore salt’s capabilities by using it to power monitoring a machine lifecycle
  2. Everything on the network or that can affect the customer experience needs to be monitored Tools need to cope with dynamic environments and be managed through code at scale Systems need to be designed for interaction from various teams
  3. You can manage Zabbix Templates through salt
  4. In the first part, we associated salt states with Zabbix templates and had our host run a zabbix state to add itself to monitoring along with the needed templates. But if the host is destroyed, what notifys Zabbix?
  5. Rather than have the monitor state connect to zabbix itself, instead just raise an event on the salt event bus