Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

RuSiem events collection and forwarding

•Download as PPTX, PDF•
•
Olesya Shelestova
Olesya Shelestova

This document discusses using RuSIEM software to collect and forward event logs between different server regions. It provides examples of configuring RuSIEM nodes to: 1. Forward all event logs or logs matching conditions from Server Region A to Server Region B using TCP/UDP or a message queue. 2. Collect logs from other servers in a DMZ and forward to Server Region B, with firewall rules only allowing connections to the DMZ. 3. Stream events from Region A to a load balanced cluster in Region B using a message queue. 4. Correlate events across Region A, B, and C by forwarding selected events to a central HQ using a message queue. It details the

1 of 19
Co-Founder, CEO Олеся Шелестова oshelestova@rusiem.com Events collection and forwarding. (RuSIEM/RvSIEM free) 2017
Mode • Single mode • Multi-node • Load balancing (MQ) • Emulate back-connection • Distributed correlation • Forward all events • Forward events by condition • Forwarding formats: • Plain syslog • TLS syslog • CEF • Json (normalized) 3
Example A 4 Balanced Servers Region B Server Region A • You need to send the entire stream or by condition • Formats: syslog plain, syslog TLS, JSON, CEF TCP or UDP
Example B 5 Balanced Servers Region B Server Region A • You need collect logs from other server • Connections allowed only to DMZ DMZ Example X X
Example C 6 Balanced Servers Region B Server Region A • You need stream events from A-Region to B • In B-Region you have some server in the balanced mode TCP or UDP MQ
Example D 7 HQ Region Cluster Region A • You need correlate events in Region A/B/C as single installation • Correlate A and C region events • Correlate A/B/C only in HQ without forward all events (!) MQ without all events MQRegion B Region C
Events path 8 LSINPUT FRS_SERVER LSFILTER LSELASTIC From agent From syslog RuSIEM MQ Correlation Analytics RAW JSON normalized JSON normalized + symptoms +category + weight
How to do it (technical information) for Example A/B 9
Example A. Agenda. 10 • You need to send the entire stream or by condition • Formats: syslog plain, syslog TLS, JSON, CEF • Server Region A: • will be receive all events from Region A source • Normalize events • Operate symptomatic and weight events • Correlate (if need) • Store events • Server Region B: • will be receive all events from Region A server and sources Region B • Normalize events only from sources Region B • Operate symptomatic and weight events • Correlate (if need) • Store events
Example A 11 • You need to send the entire stream or by condition • Formats: syslog plain, syslog TLS, JSON, CEF For all events forwarding: 1. For node “Server Region A” create custom config with suffix “_user.conf” for frs_server. Example, “/opt/rusiem/frs_server/etc/send_tcp_user.conf“
“Server Region A” – “send_tcp_user.conf” for frs_server 12 input { internal { # pickup events from rusiem-mq internal key => "classified" }} filter { } output { tcp { codec => json_lines host => "172.16.0.125“ # change this for “server region B” IP port => 5016 }}
13 In node Server Region B: 2. For node “Server Region B” create custom config with suffix “_user.conf” for lsfilter. Example, “/opt/rusiem/lsfilter/etc/receive_tcp_user.conf“ input { tcp { codec => json port => 5016 type => syslog add_field => [ "[rcvr][port]", “5016" ] add_field => [ "[rcvr][proto]", "tcp" ] queue_type => file } } output { tcp { port => 261 codec => json } }
• Allow out/input tcp connection in /etc/init.d/firewall.sh for A/B node • Disable FW update in /opt/rusiem/modules_user.dat • Run command “service frs_server restart” in Server A • Run command “service lsfilter restart” in Server B • Check receive events in Node B “tcpdump dst port 5016” • Profit 14
Example B. Agenda. 15 • Server Region A: • will be receive all events from Region A source • Normalize events • Operate symptomatic and weight events • Correlate (if need) • Store events • Provide events from Region A to B over MQ • Server Region B: • will be receive all events from Region A • Receive normalized events from Region B over MQ • Normalize events only from sources Region B • Operate symptomatic and weight events • Correlate (if need) • Store events • You need collect logs from other server • Connections allowed only to DMZ. From from DMZ to Local/Region B - denied.
Example B 16 1. Change /etc/redis/redis.conf on node Region A: • Comment row bind 127.0.0.1 with #. Example, #bind 127.0.0.1. Restart redis with command “service redis-server restart” • Save and exit from editor 2. Create new user config file for frs_server on node Region A: nano /opt/rusiem/frs_server/etc/provide_user.conf : • You need collect logs from other server • Connections allowed only to DMZ. From from DMZ to Local/Region B - denied.
Example B “Server Region A” – “/opt/rusiem/frs_server/etc/provide_user.conf ” for frs_server 17 input { internal { key => "classified" }} filter { } output { redis { host => "127.0.0.1" data_type => "list" codec => json key => "provider" workers => 1 }}
Example B 18 3. On node Region B change /etc/redis/redis.conf: • Add row: slaveof 172.15.0.1 # Set ip Node Region A • Change row: slave-read-only yes to slave-read-only no • Save and exit • Restart redis-server on node B. Attention! Slaveof row may be only one! We cannot use more that one master node :/ • You need collect logs from other server • Connections allowed only to DMZ. From from DMZ to Local/Region B - denied.
Example B 19 4. On node Region B and Region A change /etc/init.d/firewall.conf: • On node A add rows: iptables -A INPUT -p tcp –s you_IP_node_B -d $EXTIP --dport 6379 -j ACCEPT iptables -A OUTPUT -p tcp -s $EXTIP --sport 6379 –d you_IP_node_B -j ACCEPT • On node B add rows: iptables -A INPUT -p tcp –s you_IP_node_A -d $EXTIP --sport 6379 -j ACCEPT iptables -A OUTPUT -p tcp -s $EXTIP --dport 6379 –d you_IP_node_A -j ACCEPT 5. On node A and B restart firewall with command: /etc/init.d/firewall.sh restart 6. Check events from node A on node B RuSIEM interface. • You need collect logs from other server • Connections allowed only to DMZ. From from DMZ to Local/Region B - denied.
Our site: https://www.rusiem.com Facebook: https://www.facebook.com/rvsiem Telegram news: https://t.me/rusiem Mail: support@rusiem.com CEO, Olesya Shelestova oshelestova@rusiem.com Thank you 20

More Related Content

RuSiem events collection and forwarding

  • 1. Co-Founder, CEO Олеся Шелестова oshelestova@rusiem.com Events collection and forwarding. (RuSIEM/RvSIEM free) 2017
  • 2. Mode • Single mode • Multi-node • Load balancing (MQ) • Emulate back-connection • Distributed correlation • Forward all events • Forward events by condition • Forwarding formats: • Plain syslog • TLS syslog • CEF • Json (normalized) 3
  • 3. Example A 4 Balanced Servers Region B Server Region A • You need to send the entire stream or by condition • Formats: syslog plain, syslog TLS, JSON, CEF TCP or UDP
  • 4. Example B 5 Balanced Servers Region B Server Region A • You need collect logs from other server • Connections allowed only to DMZ DMZ Example X X
  • 5. Example C 6 Balanced Servers Region B Server Region A • You need stream events from A-Region to B • In B-Region you have some server in the balanced mode TCP or UDP MQ
  • 6. Example D 7 HQ Region Cluster Region A • You need correlate events in Region A/B/C as single installation • Correlate A and C region events • Correlate A/B/C only in HQ without forward all events (!) MQ without all events MQRegion B Region C
  • 7. Events path 8 LSINPUT FRS_SERVER LSFILTER LSELASTIC From agent From syslog RuSIEM MQ Correlation Analytics RAW JSON normalized JSON normalized + symptoms +category + weight
  • 8. How to do it (technical information) for Example A/B 9
  • 9. Example A. Agenda. 10 • You need to send the entire stream or by condition • Formats: syslog plain, syslog TLS, JSON, CEF • Server Region A: • will be receive all events from Region A source • Normalize events • Operate symptomatic and weight events • Correlate (if need) • Store events • Server Region B: • will be receive all events from Region A server and sources Region B • Normalize events only from sources Region B • Operate symptomatic and weight events • Correlate (if need) • Store events
  • 10. Example A 11 • You need to send the entire stream or by condition • Formats: syslog plain, syslog TLS, JSON, CEF For all events forwarding: 1. For node “Server Region A” create custom config with suffix “_user.conf” for frs_server. Example, “/opt/rusiem/frs_server/etc/send_tcp_user.conf“
  • 11. “Server Region A” – “send_tcp_user.conf” for frs_server 12 input { internal { # pickup events from rusiem-mq internal key => "classified" }} filter { } output { tcp { codec => json_lines host => "172.16.0.125“ # change this for “server region B” IP port => 5016 }}
  • 12. 13 In node Server Region B: 2. For node “Server Region B” create custom config with suffix “_user.conf” for lsfilter. Example, “/opt/rusiem/lsfilter/etc/receive_tcp_user.conf“ input { tcp { codec => json port => 5016 type => syslog add_field => [ "[rcvr][port]", “5016" ] add_field => [ "[rcvr][proto]", "tcp" ] queue_type => file } } output { tcp { port => 261 codec => json } }
  • 13. • Allow out/input tcp connection in /etc/init.d/firewall.sh for A/B node • Disable FW update in /opt/rusiem/modules_user.dat • Run command “service frs_server restart” in Server A • Run command “service lsfilter restart” in Server B • Check receive events in Node B “tcpdump dst port 5016” • Profit 14
  • 14. Example B. Agenda. 15 • Server Region A: • will be receive all events from Region A source • Normalize events • Operate symptomatic and weight events • Correlate (if need) • Store events • Provide events from Region A to B over MQ • Server Region B: • will be receive all events from Region A • Receive normalized events from Region B over MQ • Normalize events only from sources Region B • Operate symptomatic and weight events • Correlate (if need) • Store events • You need collect logs from other server • Connections allowed only to DMZ. From from DMZ to Local/Region B - denied.
  • 15. Example B 16 1. Change /etc/redis/redis.conf on node Region A: • Comment row bind 127.0.0.1 with #. Example, #bind 127.0.0.1. Restart redis with command “service redis-server restart” • Save and exit from editor 2. Create new user config file for frs_server on node Region A: nano /opt/rusiem/frs_server/etc/provide_user.conf : • You need collect logs from other server • Connections allowed only to DMZ. From from DMZ to Local/Region B - denied.
  • 16. Example B “Server Region A” – “/opt/rusiem/frs_server/etc/provide_user.conf ” for frs_server 17 input { internal { key => "classified" }} filter { } output { redis { host => "127.0.0.1" data_type => "list" codec => json key => "provider" workers => 1 }}
  • 17. Example B 18 3. On node Region B change /etc/redis/redis.conf: • Add row: slaveof 172.15.0.1 # Set ip Node Region A • Change row: slave-read-only yes to slave-read-only no • Save and exit • Restart redis-server on node B. Attention! Slaveof row may be only one! We cannot use more that one master node :/ • You need collect logs from other server • Connections allowed only to DMZ. From from DMZ to Local/Region B - denied.
  • 18. Example B 19 4. On node Region B and Region A change /etc/init.d/firewall.conf: • On node A add rows: iptables -A INPUT -p tcp –s you_IP_node_B -d $EXTIP --dport 6379 -j ACCEPT iptables -A OUTPUT -p tcp -s $EXTIP --sport 6379 –d you_IP_node_B -j ACCEPT • On node B add rows: iptables -A INPUT -p tcp –s you_IP_node_A -d $EXTIP --sport 6379 -j ACCEPT iptables -A OUTPUT -p tcp -s $EXTIP --dport 6379 –d you_IP_node_A -j ACCEPT 5. On node A and B restart firewall with command: /etc/init.d/firewall.sh restart 6. Check events from node A on node B RuSIEM interface. • You need collect logs from other server • Connections allowed only to DMZ. From from DMZ to Local/Region B - denied.
  • 19. Our site: https://www.rusiem.com Facebook: https://www.facebook.com/rvsiem Telegram news: https://t.me/rusiem Mail: support@rusiem.com CEO, Olesya Shelestova oshelestova@rusiem.com Thank you 20