Optimising Your Mikrotik Layer2 Configuration: March 2019 © Jono Thompson Birchenallhowden LTD
Optimising Your Mikrotik Layer2 Configuration: March 2019 © Jono Thompson Birchenallhowden LTD
Optimising Your Mikrotik Layer2 Configuration: March 2019 © Jono Thompson Birchenallhowden LTD
Layer2 configuration
2
BirchenallHowden Ltd
• Established in 2006
• 29 staff
• Based in Sheffield, UK and working throughout the UK and
Europe
• Currently providing IT support for over 75 companies and 2800
users
• Currently have 2 MikroTik consultants
3
BirchenallHowden Ltd
• Services Provided
– Wired and wireless network design and installation,
– Desktop and server installation, support and maintenance
– ISP Services, leased lines, connectivity
– Telephony
– Wireless installs
– MikroTik Consultancy
– MikroTik Training
• Visit www.birchenallhowden.co.uk 4
Presentation Objectives
• Since version 6.41 there have been some major changes to the
Bridge configuration
5
Presentation Objectives
• This presentation will show some of the most common
mistakes made with Layer 2 configurations.
• Most of these are taken from real setups we have had to fix
6
Presentation Objectives
7
Configurations
• In the download PDF version the incorrect configurations will be marked
like this
8
Meet Mike
• Last time we met Mike, he had just
installed his new MikroTik Wireless
9
Mike and Dave
• They sit in Mike’s office
Kitchen while Mike shows
Dave how fast the WiFi
network now is…
10
Mike’s Network
11
Dave’s Visit
12
New Switch
• Mike does some searches the internet…
• Mike sees that MikroTik do PoE switches
• As they are cheap he buys one
• CRS328-24P-4S-RM
13
Neat Install
14
Mike looks at the features
• Mike is so excited about his new switch he Tweets about his
new tidy install
15
Dave’s Visit
16
Mike’s Network
ether2
Link / Act 25
Pow er
Spd*
Fault
Locator
Link / Act 26
Spd*
AP2 - 13
17
Guest Network - Router Configuration
• Dave sets about configuring a new bridge on his hAP ac2
Router for his guest network
18
The Problem
• Mike notices that he now has really slow throughput from his
wireless clients to his wired clients
• He also notices that his hAP ac2 router has a high CPU load.
• He calls Dave…..
19
What’s wrong?
• Dave tells Mike to look for the check hardware offloading is
enabled and the H flag to show its being used.
20
Multiple Bridges on a Single Switch Chip
Analysis:
21
Multiple Bridges on a Single Switch Chip
Solution:
Preferred solution:
22
Bridge VLAN filtering
• Mike does some reading about VLANs and sees that since
version 6.41 MikroTik supports VLANs with a single bridge
23
Bridge VLAN filtering
/interface vlan
add interface=ether4 name=vlan11 vlan-id=11
add interface=ether4 name=vlan201 vlan-id=201
/ip dhcp-server
add address-pool=dhcp_pool-lan disabled=no interface=vlan11 name=dhcp-lan
add address-pool=dhcp_pool-guest disabled=no interface=vlan201 name=dhcp-guest
24
VLAN Interfaces
• Mike now notices that his clients don’t always get a DHCP
address, even though he has a DHCP server running on the
VLAN….
25
VLAN Interfaces
Analysis:
26
VLAN Interfaces
27
VLAN Interfaces
Solution:
28
Bridge VLAN filtering – Correct Configuration
/interface vlan
add interface=bridge-lan name=vlan11 vlan-id=11
add interface=bridge-lan name=vlan201 vlan-id=201
/ip dhcp-server
add address-pool=dhcp_pool-lan disabled=no interface=vlan11 name=dhcp-lan
add address-pool=dhcp_pool-guest disabled=no interface=vlan201 name=dhcp-guest
29
Bridge VLAN filtering
• Mike tests his network and now see that he has slow
throughput through both networks
30
Bridge VLAN filtering
• Mike looks at his config on his hAP ac2 and sees that even
though hw-offload is enabled that there is still no H flag
31
Bridge VLAN filtering
• Mike can’t understand why when he thinks he done everything
right
– He only has 1 bridge
– He has hw-offloading enabled
32
Bridge – HW offloading
• Dave tells Mike to look at the manual……
33
Bridge – HW offloading
Switch Chip Model (example units) STP/RSTP MSTP DHCP Snooping VLAN Filtering Bonding3
CRS3xx ✓ ✓ ✓ ✓ ✓
CRS1xx/2xx ✓ ✓1
QCA8337 hAP ac / hEX PoE / 3011 (1Gb) ✓ ✓2
AR8327 hAP ac2/2011(1Gb)/1100AHx2 ✓ ✓2
AR8227 hAP/hEX lite/2011 (100Mb) ✓ ✓2
AR8316 ✓ ✓2
AR7240 ✓ ✓2
MT7621 hEX (750Gr3) ✓ ✓
RTL8367 1100AHx4 ✓
ICPlus175D ✓
1. Feature will not work properly in VLAN switching setups, you must make sure that required packets
are sent out with the correct VLAN tag using ACL rules.
2. DCHP Snooping will not work properly with VLAN switching
3. Bridge hardware offloading only supported using 802.3ad bonding
Complete list https://wiki.mikrotik.com/wiki/Manual:Switch_Chip_Features#Bridge_Hardware_Offloading
34
Bridge – HW offloading
• Only a few devices are able to offload the traffic to the switch
chip when using VLAN filtering
• When traffic is not offloaded to the switch chip the CPU is used
to forward traffic. This can result in lower than expected
throughput
35
Bridge VLAN filtering
• Mike carries out a network redesign and no longer uses his
hAP ac2 as a switch in his network.
36
Mike’s Network
• New network topology
ether23
Link / Act: Off = N o Link, Flash = Activity, On = Link 10/ 100/ 1000Base-T Ports (1 - 24) - Ports are Auto-M DIX
HP PS1810-24G Switch
Use only supported transceivers
Link / Act: 1 Spd* 3 5 7 9 11 Link / Act: 13 Spd* 15 17 19 21 23
Link / Act 25
Pow er
Spd*
Fault
Locator
Link / Act 26
Spd*
AP2 - 13
37
CRS326 VLAN Configuration
• Mike now sets about setting up his new CRS328-24P-4S+.
• He creates a bridge
38
CRS326 VLAN Configuration
39
CRS326 VLAN Configuration
• Creates VLANs and adds the ports for the Wireless Access
Points as Tagged Ports
40
CRS326 VLAN Configuration
• And finally Mike turns on VLAN filtering
41
CRS VLAN Configuration 1
/interface bridge add name=bridge-lan vlan-filtering=yes
/interface bridge port
add bridge=bridge-lan interface=ether1
add bridge=bridge-lan interface=ether2
add bridge=bridge-lan interface=ether3
add bridge=bridge-lan interface=ether4
add bridge=bridge-lan interface=ether5
add bridge=bridge-lan interface=ether6
add bridge=bridge-lan interface=ether7
add bridge=bridge-lan interface=ether8
add bridge=bridge-lan interface=ether9
add bridge=bridge-lan interface=ether10
add bridge=bridge-lan interface=ether11
add bridge=bridge-lan interface=ether12
add bridge=bridge-lan interface=ether24
/interface bridge vlan
add bridge=bridge-lan
tagged="ether1,ether2,ether3,ether4,ether5,ether6,ether7,ether8,ether9,ether10,ether11,ether12,ether2
4" vlan-ids=201
add bridge=bridge-lan
tagged="ether1,ether2,ether3,ether4,ether5,ether6,ether7,ether8,ether9,ether10,ether11,ether12,ether2
4" vlan-ids=11
42
VLAN in bridge with a physical interface
• Mike now adds in ether23 to link his old switch to the new
switch.
• As his old switch has no VLANs and only needs traffic from the
NEW VLAN11 he creates a VLAN on ether23 and adds this to
the bridge
/interface vlan
add interface=ether23 name=vlan11-ether23 vlan-id=11
/interface bridge port
add bridge=bridge-lan interface=vlan11-ether23
/interface bridge vlan
add bridge=bridge-lan untagged=vlan11-ether23 vlan-ids=11
43
VLAN in bridge with a physical interface
44
VLAN in bridge with a physical interface
• Mike is still having problems.
• He is seeing ports flapping on his old switch
• Spanning tree is doing strange things on his old switch
• Traffic just randomly stops
45
VLAN in bridge with a physical interface
Analysis:
• Though this often works, this violates the 802.1w STP.
• The BPDU packets being sent on ether23 which is an untagged
switch interface.
• The VLAN tagging is being applied in the CPU causing all packets
being sent out as tagged traffic in VLAN11.
• Not all other vendors can understand tagged BPDU.
• In some more complex settings a switch may receive its own
BPDU packet and trigger a loop detection when there is not one.
47
VLAN in bridge with a physical interface
Solution:
Preferred solution:
48
VLAN in bridge with a physical interface
• Mike decides he is going to use the preferred solution and
makes changes to his bridge VLAN filtering
• Mike adds the physical port to the bridge and set the PVID and
add the port as a untagged in the VLAN
49
VLAN in bridge with a physical interface
50
Untagged Interfaces
• Setting a PVID on a bridge port once
VLAN filtering is enabled will also set
untagged interface in the VLAN in the
VLAN table
51
Management Interface
• Mike realises his new CRS328 switch needs to have an IP
Address on it so that he can manage it, and also it can then get
NTP time sync so logging is useful
• Mike doesn’t want any of the other VLANs to access his switch
52
Management Interface 1
• Mike connects his laptop to ether22 and sets it untagged in
VLAN11.
53
Management Interface 1
54
Management Interface 1
• Mike is still unable to access his new CRS328 switch by IP
Address
55
Management Interface 1
Analysis:
• When you add an interface to the bridge it becomes a slave
interface
56
Management Interface 1
Solution:
57
Management Interface 2
• Mike add VLAN11 to the bridge with PVID11 and sets the VLAN
interface as an untagged port in the bridge….
/interface vlan
add interface=bridge-lan name=vlan11 vlan-id=11
/interface bridge port
add bridge=bridge-lan interface=vlan11 pvid=11
/interface bridge vlan
add bridge=bridge-lan untagged=vlan11 vlan-ids=11
/ip address
add address=10.100.1.2/24 interface=vlan11 network=10.100.1.0
58
Management Interface 2
59
Management Interface 2
• Mike is still unable to access his new CRS328 switch by IP
Address
60
Management Interface 2
Analysis
• The only connection from the switch chip to the CPU is through
the bridge interface.
• The bridge works like a Switch and this should not be mixed up
with physical interfaces with VLAN interfaces on.
61
Management Interface 2
Solution:
• This will allow tagged packets to enter the bridge from the
switch chip and will be then passed through to the specific
VLAN interface on the bridge.
62
Management Interface 2
Solution cont.:
63
Management Interface 2
Solution cont.:
64
Management Interface - Correct
• Create a VLAN interface on the bridge interface
65
Management Interface - Correct
• Add bridge as a Tagged Port on the
VLAN11 – IMPORTANT
• Add an IP Address to the VLAN interface
66
Ingress Filtering
• Mike also configures Ingress Filtering.
/interface bridge
add frame-types=admit-only-vlan-tagged ingress-filtering=yes name=bridge-lan vlan-filtering=yes
67
Egress Filtering
• When Bridge VLAN-filtering is enabled….
• By default the switch ports and the bridge port (switch CPU),
filter on Egress based on the VLAN table Invalid VLANs are
dropped on Egress.
68
Ingress Filtering
• Ingress Filtering can be used along with frame type to limit
which packets are allowed to access the device, both from
physical ports and also from the CPU.
• Ingress filtering will check the VLAN table and only allow
VLANs. VLANs not specified in the VLAN table will be dropped
69
Ingress Filtering
70
Bridge VLAN Filtering – Mike’s Learnt
71
Bridge VLAN Filtering
72
Bridge VLAN Filtering
Router or Switch
73
Bridge VLAN Filtering
74
Bridge VLAN Filtering
75
Packet flow with hardware offloading
• Mike notices that a couple of his ports have high traffic
• He calls Dave….
76
Packet flow with hardware offloading
• Dave immediately knows that Mike needs to look at the block
diagram to understand how the hardware functions
• Dave spent lots of time last year looking at the block diagrams
for different MikroTik hardware…
• Dave looks at the block diagram for the CRS328 with Mike and
explains how it works….
77
Packet flow with hardware offloading
78
Packet flow with hardware offloading
• Each ethernet port is connected to the switch chip
• The switch chip is connected the CPU (sometimes called
switch-cpu port)
• Once hw-offloading has been enabled, the switch chip
forwards packets between ports
• For packets to view in sniffer tools, they need to be sent to the
CPU
79
Packet flow with hardware offloading
• If you know the traffic you are interested in then you can copy
traffic to CPU.
80
LAG / Bonding
81
LAG / Bonding
ether23
Link / Act: Off = N o Link, Flash = Activity, On = Link 10/ 100/ 1000Base-T Ports (1 - 24) - Ports are Auto-M DIX
HP PS1810-24G Switch
Use only supported transceivers
Link / Act: 1 Spd* 3 5 7 9 11 Link / Act: 13 Spd* 15 17 19 21 23
Link / Act 25
Pow er
Spd*
Fault
Locator
Link / Act 26
Spd*
AP2 - 13
82
LAG / Bonding
83
LAG / Bonding
Switch Chip Model (example units) STP/RSTP MSTP DHCP Snooping VLAN Filtering Bonding3
CRS3xx ✓ ✓ ✓ ✓ ✓
CRS1xx/2xx ✓ ✓1
QCA8337 hAP ac / hEX PoE / 3011 (1gb) ✓ ✓2
AR8327 hAP ac2/2011/1100AHx2 (1gb) ✓ ✓2
AR8227 hAP/hEX lite/2011 (100mb) ✓ ✓2
AR8316 ✓ ✓2
AR7240 ✓ ✓2
MT7621 hEX (750Gr3) ✓ ✓
RTL8367 1100AHx4 ✓
ICPlus175D ✓
1. Feature will not work properly in VLAN switching setups, you must make sure that required packets
are sent out with the correct VLAN tag using ACL rules.
2. DCHP snooping will not work properly with VLAN switching
3. Bridge hardware offloading only supported using 802.3ad bonding
Complete list https://wiki.mikrotik.com/wiki/Manual:Switch_Chip_Features#Bridge_Hardware_Offloading
84
LAG / Bonding
• Mike configures two bonding interfaces and sets them as
untagged on VLAN11
/interface bonding
add mode=802.3ad name=bonding1 slaves=ether13,ether14
add mode=802.3ad name=bonding2 slaves=ether15,ether16
/interface bridge port
add bridge=bridge-lan interface=bonding1 pvid=11
add bridge=bridge-lan interface=bonding2 pvid=11
/interface bridge vlan
add bridge=bridge-lan untagged=bonding1,bonding2
85
LAG / Bonding
•
86
LAG / Bonding
• Mike notices that he still only gets 1Gb between his servers
when testing with a well known network performance tool
(iperf)
87
LAG / Bonding
88
LAG / Bonding
• This time he remembers to check that the ports are hw-
offloaded and the bonded interfaces are
• He calls Dave
89
LAG / Bonding
Analysis:
91
DHCP Snooping
92
DHCP Snooping
• Mike sees that he can run DHCP Snooping to prevent his users
plugging rogue routers into his network
93
DHCP Snooping
• Mike turns on DHCP Snooping on
his bridge
/interface bridge
add dhcp-snooping=yes name=bridge-lan vlan-
filtering=yes
94
DHCP Snooping
• Mike now has a problem…
• He Calls Dave….
95
DHCP Snooping
• Dave tells Mike to check the logs on both his CRS328 and his
hAP ac2
• CRS328
• hAP ac2
96
DHCP Snooping
Analysis:
99
Thank you for Listening
100
References
• Visio Templates – Mikrotik Forum user FernandoSuperGG
https://forum.mikrotik.com/viewtopic.php?f=2&t=120957
• MikroTik Manual
https://wiki.mikrotik.com/wiki/Manual:CRS_Router#CRS3xx_series_switches
https://wiki.mikrotik.com/wiki/Manual:CRS3xx_series_switches
https://wiki.mikrotik.com/wiki/Manual:Layer2_misconfiguration
https://wiki.mikrotik.com/wiki/Manual:Interface/Bridge
https://wiki.mikrotik.com/wiki/Manual:Switch_Chip_Features#Bridge_Hardware_Offloading
https://wiki.mikrotik.com/wiki/Manual:Bridge_VLAN_Table
101