@XRDOCS PROGRAMMABILITY BLOGS TUTORIALS SEARCH  TAGS

Antoine Orsoni
Programmability enthusiast and Systems Engineer, Cisco Follow

Save to PDF

pyATS series - Collecting many show commands

 6 minutes read

 TA B L E O F C O NT E NT S

INTRODUCTION

G E T T I N G YO U R H A N D S D I R T Y

CONCLUSION

RESOURCES

Introduction

Ever dreamed of a test framework that could be used across multiple platforms, OS and vendors, which could do
regression, sanity and feature testing; already used by thousands of engineers and developers worldwide? Guess
what, it exists, it’s free, and you can start using it right now!

pyATS (Python Automated Test Systems, to be pronounced “py A. T. S.”) was rst created as an internal project,
to ease the validation of two OS versions. It has been made public in 2017 through Cisco Devnet.

This is the 4th blog post of the pyATS series. Today, we will cover our rst use case: how to collect many show
commands on many devices?

Today’s code will be available here.

More use cases are going to be covered in the next posts.

Other pyATS episodes

You’ve missed the rst episode? You would like to read more? Below the list of published episodes:

Episode URL What’s covered

1 - Install and use pyATS Link What’s pyATS, Install pyATS, Collect a raw CLI output

2 - Parsing like a pro Link Explore pyATS libraries, Collect and parse a CLI output

3 - Be a model Link What a pyATS model and when to use it

4 - Collecting many show commands Link How to collect many show commands on many devices?

5 - Tips and Tricks Link pyATS Tips and Tricks

6 - Pushing and removing con guration Link How to push and remove con guration using pyATS?

7 - pyATS NETCONF module Link How to use pyATS NETCONF module?

pyats_series_article_recap.md hosted with ❤ by GitHub view raw

pyATS libraries in a nutshell

2800+ parsers accross 11 OS (as of April 2021),

32 supported models (more to come about them in a coming episode),

Multiple tools for Test Harness such as triggers or tra c,
Ansible and Robot libraries for easy integration with other tools.

You can nd supported parsers and models in the o cial documentation.

Getting your hands dirty

Enough talking, let’s code!

pyATS installation has been covered in the First episode. Check it out to learn how to install pyATS.
 Today’s code will be available here.

We will rst explain the code in a high level view. We will then explain each building block individually.

In order for everyone to be able to run the code, we will use the IOS XR always-on sandbox on Cisco Devnet.
Below the sandbox information.

Key Value

IOS XRv 9000 host sandbox-iosxr-1.cisco.com

SSH Port 22

Username admin

Password C1sco12345

Use case

I had this use case a couple of months ago. We had a thermal issue on a device. We wanted to know if the other
devices (400) also su ered the same root cause. To verify, the TAC Engineer asked us to collect 4 show
commands on 400 devices.

How long would it take to do it manually? 60 seconds per device?

Without optimizing the script (ex: using threads) it took us 20 minutes to collect such outputs. We were more
than ready when the Engineer asked for a couple more commands.

High level code

The below diagram presents the high level code. In a nutshell:

0) extract the IP address of each device

1) generate the testbed using Jinja2
2) extract the show commands
3) python logic to collect each show command and write the output

For each device, we will have an output le with the show commands collected.
Extract the IP address of each device

The list of IP addresses is stored in templates/list_ip.yaml. We can add extra IP addresses by adding a new item
in the yaml le, as below.

List example in YAML

- ip_1
- ip_2
- ip_n

To create a list out of a yaml le, we are using the PyYAML package.

Extract a list of IP from a YAML le

import yaml

with open("./templates/list_ip.yaml", "r") as file:

list_ip = yaml.load(file, Loader=yaml.FullLoader)

You can nd PyYAML documentation here.

Generate the testbed using Jinja2

Jinja2 is a templating engine. You can create a sample structure with keywords. Jinja2 will nd and replace these
keywords with your own values. For example, you could have this sample IOS XR interface template.

Jinja2 IOS XR interface template

1 interface {{name}}
2 ipv6 address {{ipv6}}/{{mask}}
3 !

interface.tpl hosted with ❤ by GitHub view raw

In the above example, you would give Jinja2 three arguments: name , ipv6 and mask .

What’s interesting with Jinja2 is that it can have its own logic such as condition, loops and blocks. Using a
Jinja2 loop, the above interface template could be reused multiple times. You could loop 50 times, to create 50
loopbacks with a unique ID (name) and a unique IPv6 address.

Here, we are using Jinja2 to create a template for our pyATS testbed. You can nd the pyATS testbed template in
templates/testbed.tpl.

The testbed construction has been covered in the First episode. Have a look to understand how to build a testbed from scratch.

The below outputs presents the Jinja2 logic used for our pyATS testbed. For brievity, we are not showing the full
template le.
 Jinja2 logic with pyATS testbed

1 devices:
2 {% for ip, id in list_ip_id %}

3 Node_{{id}}:
4 type: iosxr-devnet
5 os: iosxr
6 connections:

7 vty:

8 protocol: ssh

9 ip: {{ip}}

10 settings:
11 GRACEFUL_DISCONNECT_WAIT_SEC: 0
12 POST_DISCONNECT_WAIT_SEC: 0
13 arguments:

14 connection_timeout: 10

15 {% endfor %}

jinja2_testbed.txt hosted with ❤ by GitHub view raw

We are giving Jinja2 a list of lists: list_ip_id . Each sub-list contains the ip address of a device and a unique
id to identify the node’s name in the testbed. This value has to be unique. For each list in list_ip_id we will

create a new node id and populate its ip .

Now, we need to write Python logic to give this list_ip_id to Jinja2 template. That’s how you do it.

Python logic to populate Jinja2 template

# Where's the folder with my templates (or my folders, if multiple)

template_loader = jinja2.FileSystemLoader(searchpath="./templates")

# Instance of the Environment class. Gives the loader (above), optionally parameters
# block strings, variable strings etc.
template_env = jinja2.Environment(loader=template_loader)

# Which file is my template

template = template_env.get_template("testbed.tpl")

# We give the template two lists:

# - list_ip: the IP of our devices
# - range(len(list_ip)), the id (from 0 to the max device) that will be used in devi
testbed = load(template.render(list_ip_id = zip(list_ip, range(len(list_ip)))))

In Python, the zip() function takes iterables (can be zero or more), aggregates them in a tuple, and returns it.
More information here.

The testbed information is speci c to the Devnet sandbox: login, password, protocol, operating system… Feel free to change it.

Extract the list of show commands

The list of IP addresses is stored in templates/list_show.yaml. Same as before, we are using PyYAML to create a
list out of a yaml le.

Collect and write the outputs

Connect to each device

First, we need to connect to each device. In case we cannot connect to a device, Unicon will send a
ConnectionError . We are catching such error to print in the terminal if we cannot connect to a device. In the

below logic, if we cannot connect to a device, it doesn’t fail the script. We will still iterate through the other
devices, as long as we have devices in the testbed. Feel free to change this behavior if needed.

Python logic to connect to each device

for device in testbed:

try:
device.connect(learn_hostname=True,
init_exec_commands=[],
init_config_commands=[],
log_stdout=False)
except ConnectionError:
print("-- ERROR --")
print(f" Can't connect to {device.connections.vty.ip}")
continue

The connect() method has been covered in the First episode. Have a look to understand how it works.

Collecting CLI output

Last, we need to collect each CLI output and write it in a le. File name will be the device’s hostname (learned
with pyATS).

In case a command is invalid, Unicon will send a SubCommandFailure . We are cathing this error, to tell in the
terminal which show command failed. We will still iterate through the other show commands , as long as we have show

commands in the list_show . Feel free to change this behavior if needed.

with open(f'./outputs/{device.hostname}.txt', 'w') as file:

# Collect and write each output

for show in list_show:
file.write(f'--- {show} ---\n')
try:
file.write(device.execute(show))
file.write('\n\n')
except SubCommandFailure:
print(f' /!\ `{show}` invalid command. Skipping.')

Conclusion

In this fourth episode of the pyATS series, we learnt:

How to extract a list from a YAML le,

 How to use Jinja2 logic to generate a testbed,
How to catch exceptions to modify the default python behavior.

In the next post, we will learn more about pyATS Dq (dictionnary querry) and how pyATS can nd a pair of key:

value in a nested structure. Yes, it even supports regex!

Resources

Below a few useful pyATS resources.

List of supported pyATS parsers,

The o cial pyATS documentation,

List of Unicon supported platforms,

Devnet code exchange,
Join the Webex space with the pyATS community.

 Tags: cisco iosxr pyATS Python

 Updated: August 11, 2021

SHARE ON

   

Leave a Comment
 What do you think?
7 Responses

Upvote Funny Love Surprised

0 Comments 
1 Login

G Start the discussion…

LOG IN WITH OR SIGN UP WITH DISQUS ?

Name

 Share Best Newest Oldest

Be the rst to comment.

Subscribe Privacy Do Not Sell My Data

FOLLOW:  TWITTER  GITHUB  FEED

This site is maintained by Cisco Systems, Inc. employees. Powered by Jekyll & Minimal Mistakes.