Cisco SDWAN Self Hosted Lab Part 2: 3rd June 2019

2/26/2020 codingpackets.
com
Cisco SDWAN Self Hosted Lab Part 2

3rd June 2019
Introduction
This is the second post in a two part series on building a self hosted Cisco
SDWAN lab using an enterprise CA for certificates. You can find the first part
here.
In this part I will install the controller stack:
vManage
vBond
vSmart
Along with the WAN edges:
vEdge
cEdge
I am hosting this lab on an Ubuntu server running the KVM hypervisor. I will also
use the vManage as the enterprise root CA server.
Pre Requisites
This post assumes that you have the KVM hypervisor already installed along with
the libvirt management application. Additionally you will need the virt-install
and cloud-localds CLI tools.
It is also assumed that you have downloaded the virtual disk images from
https://software.cisco.com and the serial file from the PnP Portal generated in part
one.
Lab Diagram
The following diagram depicts the topology for the lab.
Network | Code | Stuff | Things    

https://codingpackets.com/blog/cisco-sdwan-self-hosted-lab-part-2 1/31
2/26/2020 codingpackets.com
Organization: sdwan-testlab
Site: 1000
vManage vBond vSmart
.1 .2 .3
172.16.255.0/24
.11 .12
vEdge cEdge
Site: 1 Site: 2
vManage
Over on the build server, create a working directory and copy the VM image files
to the directory. I will be working from the ~/sdwan-testlab directory which
contains the following files.
cmd
tree sdwan-testlab
# output
sdwan-testlab
├── csr1000v-ucmk9.16.11.1a-serial.qcow2
├── sdwan-testlab.viptela
├── viptela-bond-19.1.0-genericx86-64.qcow2
├── viptela-edge-19.1.0-genericx86-64.qcow2
├── viptela-smart-19.1.0-genericx86-64.qcow2
└── viptela-vmanage-19.1.0-genericx86-64.qcow2
 Note
The vBond and the vEdge share the same image. I copied the vEdge image
twice renaming it for the vBond .
The vManage requires a second disk for the database. Create a thin provisioned
disk.
cmd
qemu-img create -f qcow2 vmanage-disk2.qcow2 100G
# output
Formatting 'vmanage-disk2.qcow2', fmt=qcow2 size=107374182400 cluster_
Start the vManage installation
cmd
virt-install \
--name vmanage \
--os-type linux \
--os-variant ubuntu14.04 \
--cpu host \
--vcpus=2 \
--hvm \
--arch=x86_64 \
--ram 8192 \
--disk path=viptela-vmanage-19.1.0-genericx86-64.qcow2,size=16,dev
--disk path=vmanage-disk2.qcow2,size=16,device=disk,bus=ide,format
--network=network:default,model=virtio \
--graphics none \
--import
This will start the VM and attach to a virtual console. Once you see the message
System Ready login with the username/password admin . You will then need to
follow the prompts to initialize the database.
cmd
Sun Jun 2 03:33:42 UTC 2019: System Ready
viptela 19.1.0
vmanage login: admin

Password:
Welcome to Viptela CLI
admin connected from 127.0.0.1 using console on vmanage
Available storage devices:
hdb 100GB
1) hdb
Select storage device to use: 1
Would you like to format hdb? (y/n): y
Once this is done the VM will reboot. Login again and apply the bootstrap
configuration.
cmd
config
system
system-ip 1.1.1.1
site-id 1000
organization-name "sdwan-testlab"
vbond 172.16.255.2
!
vpn 0
no interface eth0
interface eth1
ip address 172.16.255.1/24
tunnel-interface
!
no shutdown
!
vpn 512
interface eth0
ip dhcp-client
no shutdown
!
commit and-quit
!
While here, generate the Root CA certificate. First login to the vshell which is a
Unix like shell. Login with the vshell command.
cmd
vmanage# vshell
vmanage:~$
Generate a Root CA key.
cmd
openssl genrsa -out ROOTCA.key 2048
# output
Generating RSA private key, 2048 bit long modulus
..............
..
e is 65537 (0x10001)
Next generate the Root CA certificate. I will generate a certificate with 5 years of
validity.
cmd

openssl req -x509 -new -nodes -key ROOTCA.key -sha256 -days 2000 \
-subj "/C=AU/ST=NSW/L=NSW/O=sdwan-testlab/CN=vmanage.lab" \
-out ROOTCA.pem
This will create a Root CA cert named ROOTCA.pem cat the file contents so you
can copy and paste them in an upcoming step.
exit back to the cli shell and find the IP address of the vpn512 interface.
cmd
vmanage# show int | tab
# output
IF IF IF
AF ADMIN OPER TRACKER ENC
VPN INTERFACE TYPE IP ADDRESS STATUS STATUS STATUS TYP
----------------------------------------------------------------------
0 eth1 ipv4 172.16.255.1/24 Up Up - nul
0 system ipv4 1.1.1.1/32 Up Up - nul
512 eth0 ipv4 192.168.122.129/24 Up Up - nul
Once you have the IP address browse to the URL https://<ip-address> and login
with the username/password admin .
The dashboard looks like this.

Once logged in navigate to the settings page.
Administration > Settings
Enter the Organization Name and click Save. This field must match what you used
when creating the vBond profile.
Enter the vBond IP Address and click Save.
Change the Controller Certificate Authorization to use Enterprise Root

Certificate. Paste in the contents of the generated ROOTCA.pem file from the
previous step and click Import & Save.

Keep the WAN Edge Cloud Certificate Authorization method as Automated

(vManage - signed Certificate). This way the vManage will automatically signed
the cloud edge certs when they connect to the vManage.
Next we need to create a CSR for the vManage. Navigate to the certificates
section.
Configuration > Certificates > Controllers > vManage > ... > Generate CSR

A window will popup with the CSR text. This can be ignored for the vManage.
Back in the vshell there will be a file called vmanage_csr . Sign this file with the
ROOTCA.key and ROOTCA.pem .
cmd
openssl x509 -req -in vmanage_csr \

-CA ROOTCA.pem -CAkey ROOTCA.key -CAcreateserial \
-out vmanage.crt -days 2000 -sha256
# output
Signature ok
subject=/C=US/ST=California/L=San Jose/OU=sdwan-testlab/O=Viptela LLC/
Getting CA Private Key
This creates a file called vmanage.crt cat the file in order to copy and paste it
into the web interface in the next step.
Navigate to the certificates page and install the certificate by pasting the contents
of the vmanage.crt file and click Install.
Configuration > Certificates > Controllers > Install Certificate

You should see a success message similar to the below.
To exit the console use one of these key combinations (Assuming English
keyboard).
CTRL + ]
CTRL + 5
Press and hold CTRL and SHIFT while pressing 6 then ]
For convenience sake, I recommend SSHing to the vManage from another

terminal in order to sign the cert of the other devices.
vBond
Start the install of the vBond VM.
cmd

virt-install \
--name vbond \
--os-type linux \
--cpu host \
--vcpus=2 \
--hvm \
--arch=x86_64 \
--ram 2048 \
--disk path=viptela-bond-19.1.0-genericx86-64.qcow2,size=16,device
--graphics none \
--import
Once you see the message System Ready login with the username/password
admin and apply the following bootstrap config.
cmd
config
system
host-name vbond
system-ip 1.1.1.2
site-id 1000
vbond 172.16.255.2 local vbond-only
!
vpn 0
interface ge0/0
no tunnel-interface
ip address 172.16.255.2/24
no shutdown
!
!
vpn 512
interface eth0
ip dhcp-client
no shutdown
!
commit and-quit
!
Navigate to the devices page to add the vBond to the vManage.
Configuration > Devices > Controllers > Add Controller > vBond

Enter the vBond details add click Add.
Navigate to the certificates page to get the vBond CSR text.
Configuration > Certificates > Controllers > vBond > ... > View CSR
Copy the CSR text and go to the vManage vshell to generate the certificate.

Use vim to create a file called vbond.csr with the contents of the vBond CSR
from the previous step. Then sign the CSR with the Root CA certificate.
cmd
openssl x509 -req -in vbond.csr \

-out vbond.crt -days 2000 -sha256
# output
Signature ok
This will create a certificate file called vbond.crt . cat the contents of the
vbond.crt file in order to copy and past the contents to the vManage in the next
step.
Navigate to the certificates page and install the vBond certificate by pasting in the
contents of the vbond.crt file and click Install.

If it was successful you will see a success message similar to the below.
Finally activate the control plane tunnel on the ge0/0 interface.
cmd
config
vpn 0
interface ge0/0
tunnel-interface
encapsulation ipsec
!
no shutdown
!
!
commit and-quit
!
Confirm that control connections are active between the vBond and vManage.
cmd
show orchestrator connections
# output PEER
PEER PEER PEER SITE DOMAIN PE
INSTANCE TYPE PROTOCOL SYSTEM IP ID ID PR
----------------------------------------------------------------------
0 vmanage dtls 1.1.1.1 1000 0 17
0 vmanage dtls 1.1.1.1 1000 0 17
keyboard).
CTRL + ]
CTRL + 5
vSmart
Start the install of the vSmart VM.
cmd
virt-install \
--name vsmart \
--os-type linux \
--cpu host \
--vcpus=2 \
--hvm \
--arch=x86_64 \
--ram 2048 \
--disk path=viptela-smart-19.1.0-genericx86-64.qcow2,size=16,devic
--graphics none \
--import
cmd

config
system
system-ip 1.1.1.3
site-id 1000
vbond 172.16.255.2
!
vpn 0
no interface eth0
interface eth1
ip address 172.16.255.3/24
no shutdown
!
!
vpn 512
interface eth0
ip dhcp-client
no shutdown
!
commit and-quit
!
Navigate to the devices page to add the vSmart to the vManage.
Configuration > Devices > Controllers > Add Controller > vSmart
Enter the vSmart details and click Add.

Navigate to the certificates page to get the vSmart CSR text.
Configuration > Certificates > Controllers > vSmart > ... > View CSR
Copy the CSR text and go to the vManage vshell to generate the certificate.

Use vim to create a file called vsmart.csr with the contents of the vSmart CSR
from the previous step. Then sign the CSR with the Root CA certificate.
cmd
openssl x509 -req -in vsmart.csr \

-out vsmart.crt -days 2000 -sha256
# output
Signature ok
This will create a certificate file called vsmart.crt . cat the contents of the
vsmart.crt file in order to copy and past the contents to the vManage in the next
step.
Navigate to the certificates page and install the vSmart certificate by pasting in the
contents of the vsmart.crt file and click Install.

If it was successful you will see a success message similar to the below.
Finally activate the control plane tunnel on the eth1 interface.
cmd
config
vpn 0
interface eth1
tunnel-interface
!
no shutdown
!
!
commit and-quit
!
Confirm that control connections are active between the vSmart, vBond and
vManage.
cmd
show orchestrator connections
# output PEER
PEER
PEER PEER PEER SITE DOMAIN PEER
INDEX TYPE PROT SYSTEM IP ID ID PRIVATE IP
----------------------------------------------------------------------
0 vbond dtls 0.0.0.0 0 0 172.16.255.2
0 vmanage dtls 1.1.1.1 1000 0 172.16.255.1
1 vbond dtls 0.0.0.0 0 0 172.16.255.2
keyboard).
CTRL + ]
CTRL + 5
Now with the controllers out of the way, let move onto the WAN edge devices.
WAN Edge List

Do you remember in part one when I said to keep the serial file safe for a later
time? Now is that time! In order for WAN edges to join the fabric, you need to
upload the serial file generated in part one of this series.
Navigate to the devices page and upload the serial file from the location you save
it to previously. Keep the box ticked to validate the list and send to controllers.
Configuration > Devices > Upload WAN Edge List

A success message looks similar to the below.

vEdge
To add a virtual edge you need to generate a bootstrap file. Navigate to the
devices page.
Configuration > Devices > WAN Edge List > A vEdge Cloud > ...
> Generate Bootstrap Configuration
For KVM select Cloud-Init . (VMWare uses Encoded String). Then click OK.

You can either download the file and SCP it across to the host server, or copy and
paste the contents via a terminal to the server. Use the method you are most
comfortable with.
On the host server, create and ISO image from the cloud-init file that can be
mounted to the vEdge on boot. I named the file vedge.cfg and copied it to the
working directory.
cmd
cloud-localds config.iso vedge.cfg
Boot the vEdge with the config.iso disk attached. This will ensure that the image
boots with the correct chassis number.
cmd
virt-install \
--name vedge \
--os-type linux \
--cpu host \
--vcpus=2 \
--hvm \
--arch=x86_64 \
--ram 2048 \
--disk path=viptela-edge-19.1.0-genericx86-64.qcow2,size=16,device
--disk path=config.iso,device=cdrom \
--graphics none \
--import
cmd

config
system
system-ip 3.1.1.1
site-id 1
vbond 172.16.255.2
!
vpn 0
interface ge0/0
tunnel-interface
encapsulation ipsec
ip address 172.16.255.11/24
no shutdown
!
!
vpn 512
interface eth0
ip dhcp-client
no shutdown
!
commit and-quit
!
Confirm that the chassis number is the same as the one from the cloud-init config.
cmd
show certificate serial
# output
Certificate not yet installed ... giving up.
Chassis number: 2d65e5a0-ffe4-ee4e-XXXX-XXXXXXXXXX serial number:
Install the Root CA cert by SCPing it from the vManage via the VPN 512 interface.
cmd
request root-cert-chain install vpn 512 scp://admin@192.168.122.129:/h
# output
Uploading root-ca-cert-chain via VPN 512
Copying ... admin@192.168.122.129:/home/admin/ROOTCA.pem via VPN 512
Warning: Permanently added '192.168.122.129' (ECDSA) to the list of kn
viptela 19.1.0
admin@192.168.122.129's password:
ROOTCA.pem 100% 1277 2.0MB/s
Updating the root certificate chain..
Successfully installed the root certificate chain

Now activate the vEdge using the chassis number and the one time password
from the cloud-init file.
cmd
request vedge-cloud activate chassis-number 2d65e5a0-ffe4-ee4e-XXXX-XX
It will take a minute or two, but confirm that the control connections are active to
the vManage, vBond and vSmart.
cmd
show control connections
# output

TYPE PROT SYSTEM IP ID ID PRIVATE IP
----------------------------------------------------------------------
vsmart dtls 1.1.1.3 1000 1 172.16.255.3
vbond dtls 0.0.0.0 0 0 172.16.255.2
vmanage dtls 1.1.1.1 1000 0 172.16.255.1
cEdge
Navigate to the devices page to generate the bootstrap config for the cEdge.
Configuration > Devices > WAN Edge List > A cEdge Cloud > ...
> Generate Bootstrap Configuration
For KVM select Cloud-Init . (VMWare uses Encoded String). Then click OK.

You can either download the file and SCP it across to the host server, or copy and
paste the contents via a terminal to the server. Use the method you are most
comfortable with.
Unlike the vEdge it does not look like you can mount a cloud-init ISO to set the
initial boot parameters. The cloud-init config file will need to be named
ciscosdwan.cfg and copied via SCP to the cEdge after it boots up.
Boot up the cEdge VM.
cmd
virt-install \
--name=cedge \
--os-type=linux \
--os-variant=rhel4 \
--arch=x86_64 \
--cpu host \
--vcpus=2 \
--hvm \
--ram=4096 \
--disk path=csr1000v-ucmk9.16.11.1a-serial.qcow2,size=16,device=di
--graphics none \
--import
Once you see the VM is booted up login with the username/password admin .
Find the IP address of the GigabitEthernet1 so you can SCP the ciscosdwan.cfg
file to the device.
cmd
show ip int brie
# output
Interface IP-Address OK? Method Status
GigabitEthernet1 192.168.122.66 YES DHCP up
GigabitEthernet2 unassigned YES unset up
Loopback65528 192.168.1.1 YES other up
From the host SCP the ciscosdwan.cfg file to the cEdge.
cmd
scp ciscosdwan.cfg admin@192.168.122.66:ciscosdwan.cfg
# output
Password:
ciscosdwan.cfg
Reset the SDWAN software in order for the chassis serial number from the
ciscosdwan.cfg file to be applied to the device.
cmd
request platform software sdwan software reset

Once the router is back online, login and confirm that the chassis number is the
same as the one from the ciscosdwan.cfg config.
cmd
show sdwan certificate serial
# output
Certificate not yet installed ... giving up.
Chassis number: CSR-A198293A-50EA-A18F-XXXX-XXXXXXXXXX serial number:
Apply the bootstrap configuration.
cmd
config-transaction
!
system
system-ip 3.1.2.1
site-id 2
admin-tech-on-failure
organization-name sdwan-testlab
vbond 172.16.255.2
!
hostname cedge
interface GigabitEthernet2
no shutdown
ip address 172.16.255.12 255.255.255.0
exit
interface Tunnel0
no shutdown
ip unnumbered GigabitEthernet2
tunnel source GigabitEthernet2
tunnel mode sdwan
exit
sdwan
interface GigabitEthernet2
tunnel-interface
encapsulation ipsec
color default
exit
exit
!
!
commit
end
!
Install the Root CA cert by SCPing it from the vManage via the GigabitEthernet1
interface.
cmd
copy scp://admin@192.168.122.129 bootflash:
# output
Address or name of remote host [192.168.122.129]?
Source username [admin]?
Source filename [home/admin/ROOTCA.pem]? /home/admin/ROOTCA.pem
Destination filename [ROOTCA.pem]?
viptela 19.1.0
Password:
Sending file modes: C0644 1277 ROOTCA.pem
!
1277 bytes copied in 1.790 secs (713 bytes/sec)
Now install the Root CA certificate.
cmd
request platform software sdwan root-cert-chain install bootflash:ROOT
# output
Uploading root-ca-cert-chain via VPN 0
Copying ... /bootflash/ROOTCA.pem via VPN 0
Updating the root certificate chain..
Successfully installed the root certificate chain
The vEdge should be activated automatically, but if it is not you can do it manually
using the the chassis number and the one time password from the cloud-init file.
cmd
request platform software sdwan vedge_cloud activate chassis-number CS
It will take a minute or two, but confirm that the control connections are active to
the vManage, vBond and vSmart.
cmd

show sdwan control connections
# output

TYPE PROT SYSTEM IP ID ID PRIVATE IP
----------------------------------------------------------------------
vsmart dtls 1.1.1.3 1000 1 172.16.255.3
vbond dtls - 0 0 172.16.255.2
vmanage dtls 1.1.1.1 1000 0 172.16.255.1
And that's it. A functioning Cisco SDWAN lab with both vEdges and cEdges. Here
is a couple of images of the finished product.
Summary
This was a pretty loooooonnnnngggg post. If you got this far, thanks for hanging in
there. In this series of posts, we configured a Cisco SDWAN lab using an
Enterprise CA with the vManage, vBond and vSmart controllers and both the
cEdge and vEdge devices. If Cisco dCloud labs don't meet your needs, then
building your own lab might. Until next time!
Links
https://sdwan-
docs.cisco.com/Product_Documentation/Getting_Started/Hardware_and_Software
_Installation/Software_Installation_and_Upgrade_for_Cisco_IOS_XE_Routers
https://stafwag.github.io/blog/blog/2019/03/03/howto-use-centos-cloud-images-
with-cloud-init/
https://manpages.debian.org/testing/cloud-image-utils/cloud-localds.1.en.html
 sdwan cisco networking kvm libvirt


Cisco SDWAN Self Hosted Lab Part 2: 3rd June 2019

Uploaded by

Copyright:

Available Formats

Cisco SDWAN Self Hosted Lab Part 2: 3rd June 2019

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Cisco SDWAN Self Hosted Lab Part 2: 3rd June 2019

Uploaded by

Copyright:

Available Formats

2/26/2020 codingpackets.

Cisco SDWAN Self Hosted Lab Part 2

In this part I will install the controller stack:

Along with the WAN edges:

Network | Code | Stuff | Things    

qemu-img create -f qcow2 vmanage-disk2.qcow2 100G

Start the vManage installation

Sun Jun 2 03:33:42 UTC 2019: System Ready

vmanage login: admin

Generate a Root CA key.

openssl genrsa -out ROOTCA.key 2048

Network | Code | Stuff | Things    

vmanage# show int | tab

The dashboard looks like this.

Network | Code | Stuff | Things    

Once logged in navigate to the settings page.

Administration > Settings

Enter the vBond IP Address and click Save.

Change the Controller Certificate Authorization to use Enterprise Root

Network | Code | Stuff | Things    

Keep the WAN Edge Cloud Certificate Authorization method as Automated

Network | Code | Stuff | Things    

openssl x509 -req -in vmanage_csr \

Configuration > Certificates > Controllers > Install Certificate

Network | Code | Stuff | Things    

You should see a success message similar to the below.

For convenience sake, I recommend SSHing to the vManage from another

Network | Code | Stuff | Things    

Navigate to the devices page to add the vBond to the vManage.

Network | Code | Stuff | Things    

Enter the vBond details add click Add.

Navigate to the certificates page to get the vBond CSR text.

Network | Code | Stuff | Things    

openssl x509 -req -in vbond.csr \

Configuration > Certificates > Controllers > Install Certificate

Network | Code | Stuff | Things    

Finally activate the control plane tunnel on the ge0/0 interface.

show orchestrator connections

Network | Code | Stuff | Things    

Navigate to the devices page to add the vSmart to the vManage.

Enter the vSmart details and click Add.

Network | Code | Stuff | Things    

Navigate to the certificates page to get the vSmart CSR text.

Network | Code | Stuff | Things    

openssl x509 -req -in vsmart.csr \

Configuration > Certificates > Controllers > Install Certificate

Network | Code | Stuff | Things    

Finally activate the control plane tunnel on the eth1 interface.

show orchestrator connections

WAN Edge List

Configuration > Devices > Upload WAN Edge List

Network | Code | Stuff | Things    

A success message looks similar to the below.

Network | Code | Stuff | Things    

Network | Code | Stuff | Things    

cloud-localds config.iso vedge.cfg

Network | Code | Stuff | Things    

show certificate serial

request root-cert-chain install vpn 512 scp://admin@192.168.122.129:/h