Checked Baggage Inspection
Checked Baggage Inspection
Checked Baggage Inspection
ABSTRACT
We present a novel system for enhancing the efficiency and accuracy of checked baggage screening process at
airports. The system requirements address the identification and retrieval of objects of interest that are prohibited
in a checked luggage. The automated testbed is comprised of a Baxter research robot designed by Rethink
Robotics for luggage and object manipulation, and a down-looking overhead RGB-D sensor for inspection and
detection. We discuss an overview of current system implementations, areas of opportunity for improvements,
robot system integration challenges, details of the proposed software architecture and experimental results from
a case study for identifying various kinds of lighters in checked bags.
1. INTRODUCTION
Created in November 2001, the Transportation Security Administration (TSA) bears the responsibility of se-
curing the civil aviation system in the United States. By law, TSA has a mandate to screen all commercial
airline passengers and baggage. Two recent reports by the Government Accountability Office acknowledge the
progress TSA has made in deploying screening and explosive detection systems for checked bags at the airports it
oversees.1, 2 However, a number of shortcomings of the current implementations have been pointed out: (1) 76%
of all airports have optimal systems, (2) only 36% of larger airports are equipped with systems that meet the
current needs for in-line and/or stand-alone baggage screening, (3) existing implementations fail to meet current
standards for detecting explosives in checked luggage (4) TSA estimates 60% of existing systems will reach the
end of their useful life within the next 5 years. These shortcomings are despite the TSA budget increasing to
$7.91 billion dollars for the 2013 fiscal year from $4.7 billion in fiscal year 2002. The costs for securing the civilian
aviation system are only expected to rise with the most recent Senate budget proposal including an increase of
the September 11 security fee.3 The fee is charged to every passenger for each leg of their itinerary up to a
maximum of twice and is expected to rise from its current level of $2.50 to $5.00 and up to $7.50 by 2017.
The Electronic Baggage Screening Program (EBSP) Strategic Planning Framework outlines the goals for the
TSA in the development of baggage screening systems in the airports. From the Planning Guidelines and Design
Standards for Checked Baggage Inspection Systems,4 the goals of the strategic planning framework are to:
1. Increase security through deploying explosives detection system (EDS) equipment to as many airports as
practicable and implementing more labor-intensive explosives trace detection (ETD) screening protocols
at those locations where ETD will continue to be used for primary screening.
2. Minimize EBSP life-cycle costs by deploying the best possible screening solutions at each airport, appro-
priately balancing capital investment and operating cost tradeoffs.
3. Minimize impacts to TSA and airport/airline operations through well designed and well-placed EDS solu-
tions.
4. Provide a flexible security infrastructure “platform” for accommodating growing airline traffic and other
industry changes over the next 20 years and for addressing potential new threats.
To contact the authors by email: {mpdedonato, vdimitrov, tpadir}@wpi.edu
Sensors, and Command, Control, Communications, and Intelligence (C3I) Technologies for Homeland Security
and Homeland Defense XIII, edited by Edward M. Carapezza, Proc. of SPIE Vol. 9074, 90740N
© 2014 SPIE · CCC code: 0277-786X/14/$18 · doi: 10.1117/12.2053866
Automated EDS
Images
Manual Baggage
Inspection
Police Inspection
Figure 1. A simplified flow chart, modified from the TSA planning guidelines,4 showing the hierarchical levels of the CBIS
as recommended by TSA. Bags that are cleared at any point through the screening process leave the CBIS. The baggage
progresses through the automated explosives detection system (EDS) and if needed, the images are viewed by a screener,
and if needed pass through explosives trace detection (ETD) and physical inspection. Finally, if the bag does not clear
ETD, the local police inspect the bag.
bag data (tag numbers, timestamps through each level/station, tracking ID, type, EDS machine serial number,
Level 1/2 screening status, and total time in system), faults in the baggage handling system, statistics on the
EDS machines, and overall system efficiency statistics. These metrics are then used to evaluate the effectiveness
both in terms of throughput and security of the whole screening operation at a particular airport.
The complexity of the system required by TSA is significant, but necessary to ensure safety of the civil aviation
industry. In addition to meeting all these requirements, manufacturers need to be able to scale their systems
to meet the needs to airports and operations of various scales. The TSA splits the systems into four different
categories: high-volume in-line systems, medium-volume in-line systems, mini in-line systems, and stand-alone
systems. The in-line systems are connected to the airport-wide baggage handling conveyor belts to handle large
3. AREAS OF OPPORTUNITY
Because of the complexity and scale of some of the CBISs implemented at airports, there are significant areas
of opportunity to improve the screening process in terms of the metrics outlined above and to minimize the
associated costs with implementing some of these systems. For example, the Oakland International Airport
case-study4 places the most cost-effective option’s life cycle cost at $23 million over 20 years and the least
cost-effective option at $41 million. Even small optimizations in any part of the system will result in dramatic
savings for airports the size of Oakland International, and even more savings for the largest US and international
airports.
For the purposes of this research, we consider the areas of opportunity that could be augmented with robots,
such as the Baxter described in Section 4, to provide a better screening process. An opportunity area is im-
mediately available as the bags enter the CBIS where they have to be split up and sent to the EDS machines.
The TSA has very strict performance metrics in terms of how many bags may jam since every jam slows the
throughput of the EDS machine. While static deflectors and conveyor belts are used to align bags with the
machines, errors still are possible and Baxter could be used to help prevent jams ahead of the EDS machines,
and if a jam is detected, the robot could be used to help clear the jam before a backlog occurs.
Further along the system, for bags that are unlabeled or mistracked, they are automatically diverted to Level
3 ETD screening. ETD screening is time-consuming and labor-intensive, so the number of bags sent to ETD
need to be kept to the smallest set that indicate anomalies in the EDS machines. An area of opportunity is
available to utilize the Baxter robots to make sure that a bag is truly unlabeled or mistracked before being sent
to the Level 3 ETD screening. Every bag entering the system is tagged with the traditional label that goes
around a handle on the bag, but is additionally tagged with a corresponding small barcode sticker. A Baxter
robot could be used to manipulate and scan the bag for these small stickers or remnants of a mangled large label
before sending the bag to Level 3. Every bag that can simply be retagged because a sticker or label was mangled
saves resources at Level 3.
Since Level 2 screening mostly involves the clearing of bags by an operator based on the images from the
EDS, there are few opportunities to utilize robots. On the other hand, Level 3 ETD screening provides many
opportunities to utilize robots, especially since Baxter is specifically designed to work in close proximity to
humans. In order to do ETD screening, bags need to be opened and physically inspected. The robot can assist
in the act of opening the bags for inspection, the actual inspection process, the placement of information cards in
the bags, the closing of the bags, and finally the re-entry of the bags into the BHS. Many of these tedious tasks
are currently performed by TSA agents, who could be focusing on more important and mentally stimulating
tasks. Some of the items the TSA agents are looking for that are not allowed in check bags are: flares, gun
powder, blasting caps, dynamite, fireworks, grenades, plastic explosives, replicas of explosives, aerosol, fuels, gas,
torches, lighter fluid, lighters, strike-anywhere matches, flammable paint, turpentine, replicas of incendiaries,
chlorine, compressed gas cylinders, bleach, spillable batteries, spray paint, and tear gas.5 These are all items
that Baxter could locate given the appropriate object recognition algorithms while inspecting a bag.
Because of Baxter’s flexibility in completing tasks and reinforcement learning, a collection of robots in a
facility could quickly transition between the identified areas of opportunity to help distribute load within the
system. The TSA goes through significant research collecting data to calculate the required metrics so that
the system can handle the increases in load such as on major holidays or even variations throughout the day.
Providing the TSA with more tools to be able to effectively balance the load, can reduce the amount of overhead
that needs to be designed in to the CBISs. In addition, many similar opportunities are present in the inspection
4. BAXTER ROBOT
Baxter, developed by Rethink Robotics, enables humans to intuitively interact with robots in close proximity.
Figure 2. A picture showing Baxter along with the test setup for the case study. Baxter can work in close proximity to
humans and utilize an intuitive user interface enabling flexible operations in a variety of scenarios. The robot can be
trained and retrained while in the field for performing different tasks.
Because one of Baxter’s target markets is production floors, the robot is designed with work on a conveyor
line in mind. Baxter is designed to:6
• work safely alongside people, without the need for protective cages;
These features and capabilities fit well within possible solutions to achieve the goals set by the TSA for
CBISs. The ability to work alongside TSA agents is especially appealing because the cost to integrate Baxter
robots in the CBIS will be relatively low. Because Baxter and other robots of similar capabilities and quality are
orders of magnitude cheaper than the millions that the ETD machines can cost, they could provide a compelling
cost-effective solution for taking over some tasks in baggage screening. If the efficiency gained by the introduction
of Baxter robots can eliminate the need for even one extra ETD machine, that can significantly reduce the life
cycle cost.
The Baxter research edition robot we utilized to implement the case study has a convenient Robot Operating
System (ROS) API allowing us to leverage the existing nodes, like MoveIt for motion planning and collision
avoidance. Baxter has two 7-DOF manipulators with two-fingered claw grippers and cameras integrated in the
5. CASE STUDY
With the identified areas of opportunity in mind and considering Baxter’s current capabilities, we decided to
focus on implementing a proof of concept system demonstrating the ability to search a bag for prohibited items.
By looking at the TSA prohibited items list, one item in particular is relatively common and likely to be left
in bags accidentally: the lighter. Lighters pose a risk to aircraft because they are pressurized, and if the fuel
disperses as a mist in the cargo hold, due to failure of the pressure vessel, it could cause a serious fire. In addition,
due to their generally small physical size, they can be difficult to detect reliably.
The aim of this case study is to create a software framework to demonstrate robotics enabled bag inspection
in which the robot performs a task that would reasonably be expected to occur in reality. The case study is a
first-step in an attempt to deploy robots in airport security and is by no means exhaustive as to the wide range of
threats facing the aviation industry daily. By simplifying the conditions of the case study, we aim to highlight the
utility of the software architecture rather than the performance of algorithms with respect to detection metrics.
In actual implementations of the system, fault-tolerance, safety, and optimization need to be emphasized.
Figure 2 shows the general setup and assumptions we made while conducting this work. We assume that a
bag starts in front of Baxter, open and ready to be inspected. We also assume we have access to a sensor that
can provide both image data and depth data. In our case, we utilize the Microsoft Kinect sensor because of its
good performance in indoor environments. To simplify the problem, we assume the mistakenly placed lighter is
on top of the clothes in bag. This is not a reasonable assumption in a real-world scenario, but extending the
implementation with a pick and place algorithm to search through the clothes in the bag is not within the scope
of this work. For the case study, Baxter needs to locate the lighter, pick it up, deposit it in a ”Prohibited Items”
bin, and finally place a placard in the bag indicating to the passenger that their bag has been searched.
6. SOFTWARE ARCHITECTURE
The general structure of the software architecture follows the pipeline for processing the data from the Kinect
sensor, shown in Figure 3. We use the point cloud library (PCL) to implement plane segmentation,8 allowing us
to remove the tabletop and other extra surfaces from the image data. We then create a masked image focused
on the detection area. The image data is then processed using OpenCV’s template matching algorithm9 which
has been trained apriori on synthetically generated distorted images of a lighter. When a lighter is detected, its
3D position with respect to the sensor is calculated. This is translated into robot-centric coordinates through
ROS’s TF tool.11 The path controller generates waypoints to the pickup of the lighter and this information is
forwarded into the MoveIt tool which handles motion planning using OMPL. Finally, the generated trajectory is
sent to Baxter through the API and Baxter picks up the lighter. The same process using MoveIt is then repeated
to place the placard in the bag.
MoveIt is a ROS library that can implement the OMPL motion planning library and integrated with ROS
visualization tools and ROS message passing structure.10 Instead of biasing MoveIt to plan a path out of the way
of the suitcase, we create a simple waypoint generator that forces MoveIt to do pickups always from above the
suitcase, avoiding the issue of pushing it off the table. Figure 4 shows a sample visualization of MoveIt motion
planning for both manipulators.
7. RESULTS
Using the test setup shown in Figure 2, we were able to successfully locate and pickup a lighter placed on top of
clothes inside a suitcase. Figure 5 shows a series of images as Baxter completes the task of finding and removing
Point Cloud
1
Image
Plane
Segmentation <- Used to remove table and extra unneeded surfaces
Segmented Cloud
1
Template Image
Masked Image
i f
Extract 3D
Pose Used to combine point cloud and image position info
3D position (x,y,z)
i
Waypoint
Generation Used to prevent robot from pushing bag off table
Path points
Figure 3. A flow chart showing the pipeline used to process the Kinect RGBD data and detect lighters using template
matching. Point cloud library’s plane segmentation8 is used to help create a masked image which is then put into the
template matching algorithm.9 Once the 3D position of the lighter is determined, the waypoint generation and MoveIt10
generate a trajectory to pick up the lighter without pushing the bag.
the lighter from the bag. The image sequence starts at the top left and ends at the bottom right. Robot first
locates the lighter using the template matching, and then calculates the position of the lighter in robot-centric
coordinates. MoveIt plans the path to the lighter and picks it up using the two finger gripper. Baxter then
deposits the lighter in the designated bin. In the last couple images, Baxter picks up a standard TSA form which
tells the owner that their bag was searched and places it in the bag. The entire process takes about 45 seconds
on average to complete.
Figure 6 shows the image data provided by the Kinect, and the result of the template matching algorithm.
The dark area indicates a good match, and thus the algorithm has successfully found the lighter. We place a
rectangular box around the lighter to indicate to the user where it was located. It should be noted that the
object detection algorithm used in this study is not dependent on color. Color and shape based detection can be
incorporated to enhance the robustness of the system. Other classifiers such as histogram of gradients cascade
classifier12 or part-based detection13 can also be added in conjunction to the presented template matching for
increased performance.
Overall, the effectiveness of the developed software architecture is demonstrated in a proof of concept system
showing how Baxter or a similar robot could be used in a security inspection scenario.
8. FUTURE WORK
Multiple directions exist as future work on using robots in airport security situations. The work presented here
provides the framework for a system to improve efficiency of Level 3 ETD screening process. If Baxter can be
programmed to open luggage, search for prohibited items other than just lighters, and close the bags, the TSA
agents will be able to focus on other important aspects of their jobs. It will also make the screening process
faster, and more resilient to changes in load conditions. Idling robots do not lose focus and concentration like
humans who may not have much to do during off-peak times. If Baxter is upgraded with a mobile base, the
robot could also load and unload bags from the conveyor bringing bags to ETD screening.
One area of particular research interest would be to enable Baxter to search for prohibited items that are
difficult to detect using x-rays. If Baxter can accurately detect these items, it could potentially be a significant
improvement in security. If all baggage can be physically searched instead of just scanned as the majority of
baggage is currently checked, the detection rates would likely increase significantly. In addition, concerns of
privacy advocates would be at least partially addressed by using robots instead of humans to search the luggage.
Enabling Baxter to detect these items though will require a concentrated effort to fuse various sensing capabilities
to detect a very wide range of possible threats.
Figure 5. A series of pictures (starting at the top left, ending at the bottom right) showing Baxter locate the lighter in
the bag, pick it up, place it in the designated bin, and place the informational. The template matching finds the lighter
despite the range of colors, textures, and bends in the fabric of the clothes in the suitcase.
We further identified possible areas of opportunity for Baxter to augment existing systems for aligning bags to
help eliminate jams at the entrance of the CBIS. Jams present serious issues and increase costs because airports
need to add excess capacity to the system in order to alleviate backlog when a jam occurs. If Baxter can be used
to minimize the occurrence of jams, the amount of excess capacity that needs to be added decreases with the
decreased risk. Future work could explore the possibility of enabling Baxter to sense misaligned bags and with
non-prehensile manipulation align bags on the conveyors.
Another area of possible future work could focus on giving Baxter the ability to search for barcode stickers.
In cases where the luggage tag has been torn off or destroyed, Baxter could be programmed to search the bag for
the secondary tag. Enabling this solution will reduce the rate of lost bags in the system and reduce the number
of bags that have to be manually screened with the ETD machine. Similar to above, this reduces the amount of
overhead capacity that needs to be factored into the system, reducing initial costs in addition to overall operating
9. CONCLUSION
We have presented the current state of CBIS implementation and TSA recommendations for new CBIS in airports.
We have identified several areas that could be improved using robots similar to Baxter. We also present a case
study with a proof of concept system to locate and remove prohibited lighters from checked baggage. Augmenting
checked baggage inspection processes with robots seems to be a feasible approach in near future.
REFERENCES
[1] Lord, S., [Checked Baggage Screening: TSA Has Deployed Optimal Systems at the Majority of TSA-Regulated
Airports, But Could Strengthen Cost Estimates], DIANE Publishing Company (2012).
[2] US Government, [Homeland Security: Protecting Federal Facilities Remains a Challenge for the Department
of Homeland Security’s Federal Protective Service ], General Books (2012).
[3] Department of Transportation, [Transportation Security Administration:49 CFR Chapter XII and Part 1510
Imposition and Collection of Passenger Civil Aviation Security Service Fees; Interim Final Rule ] (October
2001).
[4] Transportation Security Administration, [Planning Guidelines and Design Standards for Checked Baggage
Inspection Systems ] (October 2007).
[5] Transportation Security Administration, “Prepare for takeoff: Prohibited items list,” (March 2012).
[6] Fitzgerald, C., “Developing baxter,” in [Technologies for Practical Robot Applications (TePRA), 2013 IEEE
International Conference on ], 1–6 (2013).
[7] Rethink Robotics, “Baxter research robot,”
[8] Point Cloud Library, “Plane model segmentation.” http://pointclouds.org/.
[9] OpenCV, “Template matching.” http://docs.opencv.org/doc/.