19/04/2023

Chapter 4
Internet Network and Protocol
(Mạng Internet và Giao thức)
Software Engineering Dept., CICT,
Can Tho University
Lecturer: Trương Minh Thái
email: tmthai@ctu.edu.vn

Outline

PART 3
• TCP/IP Internet Protocol ?
• HTTP Protocol
• MQTT Protocol
– A scenario of IoT Application
– Message Queuing Telemetry Transport (MQTT)
– MQTT Message Format

Truong Minh Thai 1

 19/04/2023

A scenario of IoT Application

• For example, consider low-cost temperature and relative humidity (RH) sensors sending
data over an LPWA LoRaWAN infrastructure. Temperature is represented as 2 bytes and
RH as another 2 bytes of data. This small data payload is directly transported on the
LoRaWAN MAC layer, without TCP/IP.
• The raw data for temperature and relative humidity can be decoded by the application.
While many constrained devices, such as sensors and actuators, have adopted
deployments with no application layer, this transportation method has not been
standardized. This lack of standardization makes it difficult for generic implementations of
this transport method to be successful from an interoperability perspective.
• A temperature value will always be present in the data transmitted by each sensor, but
decoding this data will be vendor specific. If you scale this scenario out across hundreds
or thousands of sensors, the problem of allowing various applications to receive and
interpret temperature values delivered in different formats becomes increasingly complex.
3

A scenario of IoT Application

• The solution to this problem is to use an IoT

data broker, as detailed in Figure 4.3.1. An IoT
data broker is a piece of middleware that
standardizes sensor output into a common
format that can then be retrieved by authorized
applications.
• Sensors X, Y, and Z are all temperature
sensors, but their output is encoded differently.
The IoT data broker understands the different
formats in which the temperature is encoded
and is, therefore, able to decode this data into a
common, standardized format. Applications A,
B, and C can access this temperature data
without having to deal with decoding multiple Figure 4.3.1 IoT Data Broker
4
temperature data formats. 4

Truong Minh Thai 2

 19/04/2023

IoT Application Layer Protocols

• When considering constrained networks and/or

large-scale deployment of constrained nodes,
verbose web-based and data model protocols,
that may be too heavy for IoT applications
• To address this problem, the IoT industry is
working on new lightweight protocols that are
better suited to large numbers of constrained
nodes and networks. Two of the most popular
protocols are CoAP and MQTT.
• CoAP and MQTT are naturally at the top of this
sample IoT stack, based on an IEEE 802.15.4
mesh network. While there are a few
exceptions, you will almost always find CoAP Figure 4.3.2 Example of a High-Level IoT Protocol
Stack for CoAP and MQTT
deployed
5
over UDP and MQTT running over
5
TCP.
5

Outline

PART 3
• TCP/IP Internet Protocol ?
• HTTP Protocol
• MQTT Protocol
– A scenario of IoT Application
– Message Queuing Telemetry Transport (MQTT)
– MQTT Message Format

Truong Minh Thai 3

 19/04/2023

Message Queuing Telemetry Transport (MQTT)

• At the end of the 1990s, engineers from IBM and Arcom were looking for a reliable,
lightweight, and cost-effective protocol to monitor and control a large number of sensors
and their data from a central server location, as typically used by the oil and gas
industries.
• Their research resulted in the development and implementation of the Message Queuing
Telemetry Transport (MQTT) protocol that is now standardized by the Organization for
the Advancement of Structured Information Standards (OASIS). (For more information on
OASIS, see www.oasis-open.org)

Message Queuing Telemetry Transport (MQTT)

• Protocol for minimal battery loss and

minimal bandwidth to connect with oil
pipelines via satellite.
• Enables a publish/subscribe messaging
model in an extremely lightweight way
• Core Features of MQTT
– Simple implementation – arbitrary
messages up to 256MB
– Quality of Service data delivery – in order
to deliver per publisher
– Lightweight and bandwidth efficient – little
client state, TCP/Websockets
Figure 4.3.3. Publish Subscribe Model
– Data agnostic
– Continuous session awareness. 8

Truong Minh Thai 4

 19/04/2023

Message Queuing Telemetry Transport (MQTT)

• An MQTT client can act as a publisher to send data (or resource information) to an
MQTT server acting as an MQTT message broker.

Figure 4.3.4 MQTT Publish/Subscribe Framework 9

Message Queuing Telemetry Transport (MQTT)

• The MQTT client on the left side publishes its data (Temperature and
Relative Humidity …)
• The MQTT server (or message broker) accepts the network
connection along with application messages, such as Temp/RH data,
from the publishers. It also handles the subscription and
unsubscription process and pushes the application data to MQTT
clients acting as subscribers.
• The application on the right side is an MQTT client that is a
subscriber to the Temp/RH data being generated by the publisher or
sensor on the left. This model, where subscribers express a desire to
receive information from publishers, is well known.
• With MQTT, clients can subscribe to all data (using a wildcard
character) or specific data from the information tree of a publisher.
The presence of a message broker in MQTT decouples the data
transmission between clients acting as publishers and subscribers. 10

10

Truong Minh Thai 5

 19/04/2023

Client, Broker and Connection Establishment

• Client:
– any device (from a microcontroller up to a full-fledged server) that runs an MQTT library
and connects to an MQTT broker over a network. Any device that speaks MQTT over a
TCP/IP stack can be called an MQTT client.
– MQTT client libraries are available for a huge variety of programming languages. For
example, Android, Arduino, C, C++, C#, Go, iOS, Java, JavaScript, and .NET.
• Broke :
– responsible for receiving all messages, filtering the messages, determining who is
subscribed to each message, and sending the message to these subscribed clients.
– holds the session data of all clients that have persistent sessions, including subscriptions
and missed messages.
– authentication and authorization of clients

11

MQTT Connection

• MQTT protocol is based on TCP/IP.

Both the client and the broker need to
have a TCP/IP stack.

• MQTT connection is always

between one client and the
broker. Clients never connect
to each other directly

12

Truong Minh Thai 6

 19/04/2023

MQTT Packet Components

• ClientId: identifies each MQTT client that connects to an MQTT

broker. The broker uses the ClientId to identify the client and the
current state of the client.Therefore, this Id should be unique per
client and broker.
• Clean Session: The clean session flag tells the broker whether
the client wants to establish a persistent session or not. In a
persistent session (CleanSession = false), the broker stores all
subscriptions for the client and all missed messages for the client.
• Username/Password : MQTT can send a user name and
password for client authentication and authorization.
• Will Message: notifies other clients when a client disconnects
ungracefully.
• KeepAlive: time interval in seconds that the client specifies and
communicates to the broker when the connection established. It
defines the longest period of time that the broker and client can
endure without sending a message.

13

Broker Response

• Session Present flag: The session present flag tells the client
whether the broker already has a persistent session available
from previous interactions with the client.
• Connect return code: tells the client whether the connection
attempt was successful or not. ( 0 = connection accepted )

14

Truong Minh Thai 7

 19/04/2023

Publish

• MQTT client can publish messages as soon as it connects to a

broker.
• MQTT utilizes topic-based filtering of the messages on the
broker
• Each message must contain a topic that the broker can use to
forward the message to interested clients. Typically, each
message has a payload that contains the data to transmit in byte
format.
• The use case of the client determines how the payload is
structured.
• The sending client (publisher) decides whether it wants to send
binary data, text data, or even full-fledged XML or JSON.

15

Publish Message Components

• Topic Name: a simple string that is hierarchically structured with forward slashes as delimiters. For
example, “myhome/livingroom/temperature”
• QoS: indicates the Quality of Service Level (QoS) of the message. There are three levels: 0, 1, and
2. The service level determines what kind of guarantee a message has for reaching the intended
recipient (client or broker)
• Retain Flag: defines whether the message is saved by the broker as the last known good value for
a specified topic. When a new client subscribes to a topic, they receive the last message that is
retained on that topic.
• Payload: actual content of the message. MQTT is data-agnostic. It is possible to send images, text
in any encoding, encrypted data, and virtually every data in binary.
• Packet Identifier: uniquely identifies a message as it flows between the client and broker. The
packet identifier is only relevant for QoS levels greater than zero. The client library and/or the broker
is responsible for setting this internal MQTT identifier.
• DUP flag: indicates that the message is a duplicate and was resent because the intended recipient
(client or broker) did not acknowledge the original message. This is only relevant for QoS greater
than 0

16

Truong Minh Thai 8

 19/04/2023

Subscribe
• Publishing a message doesn’t make sense if no one ever
receives it. In other words, if there are no clients to subscribe
to the topics of the messages. To receive messages on topics
of interest, the client sends a SUBSCRIBE message to the
MQTT broker. This subscribe message is very simple, it
contains a unique packet identifier and a list of subscriptions.
• Packet Identifier: uniquely identifies a message as it flows
between the client and broker. The client library and/or the
broker is responsible for setting this internal MQTT identifier.
• List of Subscriptions: A SUBSCRIBE message can contain
multiple subscriptions for a client. Each subscription is made
up of a topic and a QoS level. The topic in the subscribe
message can contain wildcards that make it possible to
subscribe to a topic pattern rather than a specific topic. If
there are overlapping subscriptions for one client, the broker
delivers the message that has the highest QoS level for that
topic
17

Subscribe Acknowledgement

• Packet Identifier: unique identifier used to identify

a message, same as in the SUBSCRIBE msg.
• Return Code: The broker sends one return code
for each topic/QoS-pair that it receives in the
SUBSCRIBE message.

18

Truong Minh Thai 9

 19/04/2023

Unsubscribe

19

Outline

PART 3
• TCP/IP Internet Protocol ?
• HTTP Protocol
• MQTT Protocol
– A scenario of IoT Application
– Message Queuing Telemetry Transport (MQTT)
– MQTT Message Format

20

20

Truong Minh Thai 10

 19/04/2023

MQTT Message Format

• MQTT control packets run over a TCP

transport using port 1883. TCP ensures an
ordered, lossless stream of bytes between
the MQTT client and the MQTT server.
Optionally, MQTT can be secured using
TLS on port 8883, and WebSocket (defined
in RFC 6455) can also be used.
• MQTT is a lightweight protocol because
each control packet consists of a 2-byte
fixed header with optional variable header
fields and an optional payload. A control
packet can contain a payload of up to 256
MB
Figure 4.3.5. MQTT Message Format
21

21

MQTT Message Format

• MQTT control packets run over a TCP transport

using port 1883. TCP ensures an ordered,
lossless stream of bytes between the MQTT client
and the MQTT server. Optionally, MQTT can be
secured using TLS on port 8883, and WebSocket
(defined in RFC 6455) can also be used.
• MQTT is a lightweight protocol because each
control packet consists of a 2-byte fixed header
with optional variable header fields and optional
payload. A control packet can contain a payload
up to 256 MB

Figure 4.3.5. MQTT Message Format

22

22

Truong Minh Thai 11

 19/04/2023

MQTT Message Format

- MQTT Control header

• The first MQTT field in the header is Message

Type, which identifies the kind of MQTT packet
within a message.
• Fourteen different types of control packets are
specified in MQTT version 3.1.1. Each of them
has a unique value that is coded into the
Message Type field. Note that values 0 and 15
MQTT Message Types 23
are reserved

23

MQTT Message Format

- MQTT Control header

• The next field in the MQTT header is DUP (Duplication Flag). This flag, when set, allows the client to
notate that the packet has been sent previously, but an acknowledgment was not received.
• The QoS header field allows for the selection of three different QoS levels. The MQTT protocol offers
three levels of quality of service (QoS).
• The next field is the Retain flag. Only found in a PUBLISH message, the Retain flag notifies the
server to hold onto the message data. This allows new subscribers to instantly receive the last
known value without having to wait for the next update from the publisher. 24

24

Truong Minh Thai 12

 19/04/2023

MQTT Message Format

- MQTT Control header

Control Flag bits

25

25

MQTT Message Format

- Remaining Length

• This field specifies the number of bytes in the

MQTT packet following this field.
• The remaining length is the number of bytes
following the length field, including variable
length header and payload as illustrated.
• This is of variable length between 1 and 4
bytes.
• Each byte uses 7 bits for the length with the
MSB used as a continuation flag.
• If the continuation flag bit of a byte is 1, it means
the next byte is also part of the remaining
length. And if the continuation flag bit is 0, it
means that the byte is the last one of the
remaining length.
26

26

Truong Minh Thai 13

 19/04/2023

MQTT Message Format

- Remaining Length
• A packet length of 321 bytes requires a 2-byte
• The remaining Packet length 64 bytes of remaining length field:
requires only 1 byte:

27

27

MQTT Message Format

- Variable header

• A variable header is not present in all the MQTT packets. Some MQTT commands or messages
use this field to provide additional information or flags and they vary depending on the packet type.
A packet identifier is common in most packet types.
28

28

Truong Minh Thai 14

 19/04/2023

MQTT Message Format

- Payload

• In the end, the packet may contain a payload. Even the payload is optional and varies with the type
of packet. This field usually contains the data which is being sent.

29

29

MQTT CONNECT packet

• The first byte of the connect packet will be 10.

Because the value of CONNECT command is 1, the
first 4 MSB will be 1 and there are no flags so the
next 4 bits will be 0.

• The second byte should be the remaining length. which is the length of the variable header and the
length of the payload. Let us decide this length after completing the variable header and payload.

https://openlabpro.com/guide/mqtt-packet-format/ 30

30

Truong Minh Thai 15

 19/04/2023

MQTT CONNECT packet

• The second byte should be the remaining length. which is the length of the variable heaIn the variable header, first, there
should be the protocol name. And for this, the first 2 bytes should mention the length of the protocol name followed by the
protocol name. In our case, the protocol name is MQTT which is of length 4.
• After the protocol name, there is the protocol level. This determines which version of MQTT it supports. For version 3.1.1, the
protocol is level 4.
• After that, there is the connect flag byte.
• The next two bytes are used to mention the keep-alive duration in seconds. For 60 seconds, the value will be 003C in hex.
• After the variable header, there will be the payload and which will contain; the client id, username, and password. In our case
there is no username and password, so only the client Id will be present. Same as we did for the protocol name, the first 2
bytes will denote the client id length. Let’s assume our client id is PQRST.
• der and the length of the payload. Let us decide this length after completing the variable header and payload. 31

31

MQTT CONNECT packet

To determine the ‘remaining length’ which is supposed to be here. If you count the total bytes used for the
variable header and payload. It is 17. Therefore the ‘remaining length will’ be 17.

32

32

Truong Minh Thai 16

 19/04/2023

MQTT PUBLISH packet

Publish the message “HELLO” to the topic OPENLABPRO.

• For the publish packet command value is 3. With QoS level 0 and without retaining the
message control flag will be 0. In the variable header section, the first 2 bytes will denote the
length of the topic and then be followed by the topic. In a similar way in payload, the section's
first 2 bytes will denote the length of the message which is followed by the message.

https://openlabpro.com/guide/mqtt-packet-format/ 33

33

MQTT SUBSCRIBE packet

• To subscribe to a topic the client has to send the SUBSCRIBE packet.

• The command value of Subscribe packet is 8 and the Control flag is reserved and should be 2.
• The variable header will contain a non-zero 16-bit packet ID. And as payload, there will be a topic
to subscribe to follow by the requested QOS level.
To subscribe to the topic OPENLABPRO with QOS 0:

34

34

Truong Minh Thai 17

 19/04/2023

MQTT Vs. HTTP

MQTT HTTP
Purpose Messaging Documents
Protocol Efficiency High Average
Power Efficiency Yes No
Client Languages Many Many

35

Mosquitto MQTT Broker

• Mosquitto : lightweight open source message broker Implements MQTT versions 3.1.1, 3.1,
and version 5.0. Main website : https://mosquitto.org/

Totorials:
• https://www.switchdoc.com/2018/02/tutorial-installing-and-testing-mosquitto-mqtt-on-
raspberry-pi/

36

Truong Minh Thai 18

 19/04/2023

Q&A

37

Truong Minh Thai 19