Cheat Sheet: API's and Data Collection

Package/Method Description Code Example

Syntax:
Access the
value of a attribute = element[(attribute)]
Accessing specific
element attribute attribute of an Example:
HTML
element. href = link_element[(href)]

Parse the
HTML content Syntax:
of a web page
using soup = BeautifulSoup(html, (html.parser))
BeautifulSoup() BeautifulSoup.
Example:
The parser
type can vary html = (https://api.example.com/data) soup = BeautifulSoup(html, (html.parser))
based on the
project.

Send a
DELETE
request to
Syntax:
remove data or
a resource response = requests.delete(url)
from the
delete()
server. Example:
DELETE
requests delete response = requests.delete((https://api.example.com/delete))
a specified
resource on
the server.

Syntax:
Find the first
HTML element = soup.find(tag, attrs)
element that
find()
matches the Example:
specified tag
and attributes. first_link = soup.find((a), {(class): (link)})

Find all Syntax:

HTML elements = soup.find_all(tag, attrs)
elements that
find_all()
match the Example:
specified tag
and attributes. all_links = soup.find_all((a), {(class): (link)})</td>

Syntax:
Find all child children = element.findChildren()
elements of an
findChildren()
HTML Example:
element.
child_elements = parent_div.findChildren()

Perform a
GET request
to retrieve data
from a
specified
URL. GET
requests are Syntax:
typically used
for reading response = requests.get(url)
get() data from an
Example:
API. The
response response = requests.get((https://api.example.com/data))
variable will
contain the
server's
response,
which you can
process
further.
 Include
custom
headers in the
request. Syntax:
Headers can
provide headers = {(HeaderName): (Value)}
Headers additional
Example:
information to
the server, base_url = (https://api.example.com/data) headers = {(Authorization): (Bearer YOUR_TOKEN)} response = request
such as
authentication
tokens or
content types.

Import the
necessary Syntax:
Import Libraries Python from bs4 import BeautifulSoup
libraries for
web scraping.

Parse JSON
data from the
response. This
extracts and
works with the
data returned Syntax:
by the API.
The data = response.json()
response.json()
json() Example:
method
converts the response = requests.get((https://api.example.com/data))
JSON data = response.json()
response into a
Python data
structure
(usually a
dictionary or
list).

Syntax:
Find the next sibling = element.find_next_sibling()
sibling
next_sibling()
element in the Example:
DOM.
next_sibling = current_element.find_next_sibling()

Access the Syntax:

parent element parent = element.parent
in the
parent
Document Example:
Object Model
(DOM). parent_div = paragraph.parent

Send a POST
request to a
specified URL
with data.
Create or Syntax:
update POST
requests using response = requests.post(url, data)
post() resources on
Example:
the server. The
data parameter response = requests.post((https://api.example.com/submit), data={(key): (value)})
contains the
data to send to
the server,
often in JSON
format.
 Send a PUT
request to
update data on
the server.
PUT requests
Syntax:
are used to
update an response = requests.put(url, data)
existing
put()
resource on Example:
the server with
the data response = requests.put((https://api.example.com/update), data={(key): (value)})
provided in the
data
parameter,
typically in
JSON format.

Pass query
parameters in Syntax:
the URL to
filter or params = {(param_name): (value)}
customize the
Query parameters request. Query Example:
parameters
base_url = "https://api.example.com/data"
specify params = {"page": 1, "per_page": 10}
conditions or response = requests.get(base_url, params=params)
limits for the
requested data.

Syntax:
Select HTML
elements from element = soup.select(selector)
select() the parsed
Example:
HTML using a
CSS selector. titles = soup.select((h1))

Check the
HTTP status
code of the
response. The
HTTP status Syntax:
code indicates
the result of response.status_code
the request
status_code (success, error, Example:
redirection). url = "https://api.example.com/data"
Use the HTTP response = requests.get(url)
status codeIt status_code = response.status_code
can be used for
error handling
and decision-
making in
your code.

Specify any
valid HTML Tag Example:
tag as the tag
parameter to - (a): Find anchor () tags.
- (p): Find paragraph ((p)) tags.
search for - (h1), (h2), (h3), (h4), (h5), (h6): Find heading tags from level 1 to 6 ( (h1),n (h2)).
elements of - (table): Find table () tags.
tags for find()
that type. Here - (tr): Find table row () tags.
and find_all()
are some - (td): Find table cell ((td)) tags.
common - (th): Find table header cell ((td))tags.
- (img): Find image ((img)) tags.
HTML tags - (form): Find form ((form)) tags.
that you can - (button): Find button ((button)) tags.
use with the
tag parameter.

Syntax:
Retrieve the text = element.text
text content of
text
an HTML Example:
element.
title_text = title_element.text

© IBM Corporation. All rights reserved.