The EXCITE (EXplainability Capability Information Testing Evaluation) approach assesses information fusion interpretability, explainability, and accountability for uncertainty analysis. Amongst many data and information fusion techniques... more
The EXCITE (EXplainability Capability Information Testing Evaluation) approach assesses information fusion interpretability, explainability, and accountability for uncertainty analysis. Amongst many data and information fusion techniques is the need to understand the information fusion system capability for the intended application. While many approaches for data fusion support uncertainty reduction from measured data, there are other contributing factors such as data source credibility, knowledge completeness, multiresolution, and problem alignment. To facilitate the alignment of the data fusion approach to the user’s intended action, there is a need towards a representation of the uncertainty. The paper highlights the approach to leverage recent research efforts in interpretability as methods of data handing in the Uncertainty Representation and Reasoning Evaluation Framework (URREF) while also proposing explainability and accountability as a representation criterion. Accountability is closely associated with the selected decision and the outcome which has these four attributes: amount of data towards the result, distance score of decision selection, accuracy/credibility/timeliness of results, and risk analysis. The risk analysis includes: verifiability, observability, liability, transparency, attributability, and remediabilty. Results are demonstrated on notional example.
Research Interests:
Research Interests:
Vehicle detection in wide area motion imagery (WAMI) has drawn increasing attention from the computer vision research community in recent decades. In this paper, we present a new architecture for vehicle detection on road using multi-task... more
Vehicle detection in wide area motion imagery (WAMI) has drawn increasing attention from the computer vision research community in recent decades. In this paper, we present a new architecture for vehicle detection on road using multi-task network, which is able to detect and segment vehicles, estimate their pose, and meanwhile yield road isolation for a given region. The multi-task network consists of three components: 1) vehicle detection, 2) vehicle and road segmentation, and 3) detection screening. Segmentation and detection components share the same backbone network and are trained jointly in an end-to-end way. Unlike background subtraction or frame differencing based methods, the proposed Multitask Assessment of Roads and Vehicles Network (MARVN) method can detect vehicles which are slowing down, stopped, and/or partially occluded in a single image. In addition, the method can eliminate the detections which are located at outside road using yielded road segmentation so as to decrease the false positive rate. As few WAMI datasets have road mask and vehicles bounding box anotations, we extract 512 frames from WPAFB 2009 dataset and carefully refine the original annotations. The resulting dataset is thus named as WAMI512. We extensively compare the proposed method with state-of-the-art methods on WAMI512 dataset, and demonstrate superior performance in terms of efficiency and accuracy.
Research Interests:
Autonomy proliferates air and space traffic management with the National Aeronautics and Space Administration (NASA) initiative on Unmanned Aircraft System Traffic Management (UTM) New endeavors such as electric vertical take-off and... more
Autonomy proliferates air and space traffic management with the National Aeronautics and Space Administration (NASA) initiative on Unmanned Aircraft System Traffic Management (UTM) New endeavors such as electric vertical take-off and landing (eVTOL) and COVID19 are challenging every aspect of the NextGEN rollout Hence, to develop a safe and secure UTM, there is need for knowledge management Knowledge management comes from awareness about the environment and awareness is based on assessment The information fusion community has long developed methods for data fusion (e g , statistical analysis), sensor fusion (e g , navigation and tracking), information fusion (e g , Notice to Airman and air tracks), as well intelligence fusion (e g , response to malicious attacks) for knowledge assessment Each of these techniques has opportunities to enable UTM autonomy, joint all-domain command and control, and surveillance This paper explores the various uses of fusion available to support the autonomy For example, three types of autonomy have been proposed: autonomy at rest (e g , flight plans, radar positions), autonomy in motion (e g , dynamic tracking with automatic dependent surveillance-broadcast (ADS-B) and weather), as well as autonomy in use (e g , getting the right data at the correct time) A use case is presented for UTM that utilizes data fusion from ADS-B, radar, LiDAR, and visual data to provide effective positioning in response to various cyber attacks on the ADS-B data © 2021, American Institute of Aeronautics and Astronautics Inc, AIAA All rights reserved
Research Interests:
Real-time information fusion based on WAMI (Wide-Area Motion Imagery), FMV (Full Motion Video), and Text data is highly desired for many mission critical emergency or security applications. Cloud Computing has been considered promising to... more
Real-time information fusion based on WAMI (Wide-Area Motion Imagery), FMV (Full Motion Video), and Text data is highly desired for many mission critical emergency or security applications. Cloud Computing has been considered promising to achieve big data integration from multi-modal sources. In many mission critical tasks, however, powerful Cloud technology cannot satisfy the tight latency tolerance as the servers are allocated far from the sensing platform, actually there is no guaranteed connection in the emergency situations. Therefore, data processing, information fusion, and decision making are required to be executed on-site (i.e., near the data collection). Fog Computing, a recently proposed extension and complement for Cloud Computing, enables computing on-site without outsourcing jobs to a remote Cloud. In this work, we have investigated the feasibility of processing streaming WAMI in the Fog for real-time, online, uninterrupted target tracking. Using a single target tracking algorithm, we studied the performance of a Fog Computing prototype. The experimental results are very encouraging that validated the effectiveness of our Fog approach to achieve real-time frame rates.
Research Interests:
Space superiority requires space protection and space situational awareness (SSA), which rely on rapid and accurate space object behavioral and operational intent discovery. The analytics required for space object detection, tracking, and... more
Space superiority requires space protection and space situational awareness (SSA), which rely on rapid and accurate space object behavioral and operational intent discovery. The analytics required for space object detection, tracking, and pattern classification include machine learning. The development of deep learning (DL) methods shows promise in many areas. In this paper, a convolutional neural network (CNN) classifies the behaviors of space objects for evasive satellite behaviors detection. Additionally, within the Adaptive Markov Inference Game Optimization (AMIGO) engine, a game theoretic approach describes the situation versus a control problem. Using data-level fusion, stochastic modeling/propagation, and DL pattern classification. Numerical simulations demonstrate the advantage of DL methods for space object pattern classification improving from 34% to 98% with training.
Research Interests:
Research Interests:
Research Interests:
Research Interests:
In a cognitive reasoning system, the four-stage Observe-Orient-Decision-Act (OODA) reasoning loop is of interest. The OODA loop is essential for the situational awareness especially in heterogeneous data fusion. Cognitive reasoning for... more
In a cognitive reasoning system, the four-stage Observe-Orient-Decision-Act (OODA) reasoning loop is of interest. The OODA loop is essential for the situational awareness especially in heterogeneous data fusion. Cognitive reasoning for making decisions can take advantage of different formats of information such as symbolic observations, various real-world sensor readings, or the relationship between intelligent modalities. Markov Logic Network (MLN) provides mathematically sound technique in presenting and fusing data at multiple levels of abstraction, and across multiple intelligent sensors to conduct complex decision-making tasks. In this paper, a scenario about vehicle interaction is investigated, in which uncertainty is taken into consideration as no systematic approaches can perfectly characterize the complex event scenario. MLNs are applied to the terrestrial domain where the dynamic features and relationships among vehicles are captured through multiple sensors and informatio...
Research Interests:
Image compression is an important component in modern imaging systems as the size of the raw data collected is getting larger. For example, a commercial Wide Area Motion Imagery (WAMI) image data size is over 144 Megapixels (12,000 ×... more
Image compression is an important component in modern imaging systems as the size of the raw data collected is getting larger. For example, a commercial Wide Area Motion Imagery (WAMI) image data size is over 144 Megapixels (12,000 × 12,000 pixels), and the next generation WAMI image data size will be at the level of 1.6 Giga-pixels (40,000 × 40,000 pixels). The large file size transmission is limiting performance without resorting to some compression. Lossless compression is able to preserve all the information, but has limited reduction power. On the other hand, lossy compression, which may result in very high compression ratios, suffers from information loss. In this paper, we model the compression induced information loss in terms of the National Imagery Interpretability Rating Scale or NIIRS. NIIRS is a subjective quantification of image collections widely adopted by the Geographic Information System (GIS) community. Specifically, we present the Compression Degradation Image Function Index (CoDIFI) framework that predicts the NIIRS degradation, (i.e., a decrease of NIIRS rating scale) for a given compression setting. The CoDIFI-NIIRS framework enables a user to broker the maximum compression setting while maintaining a specified NIIRS rating.
Research Interests:
A comprehensive approach is offered for avionics engineering topics and curricula for educational activities that better meet contemporary aerospace industry requirements. These topics include a growing demand for airspace mobility... more
A comprehensive approach is offered for avionics engineering topics and curricula for educational activities that better meet contemporary aerospace industry requirements. These topics include a growing demand for airspace mobility services, the proliferation of unmanned aircraft systems (UAS), and the required enhancements in avionics and air traffic management (ATM) design standards (hardware and software components); targeting the evolving safety, interoperability, and cybersecurity needs of the aeronautical and space sectors. Proposed topics and approaches stem from the Institute of Electrical and Electronics Engineers (IEEE) Aerospace and Electronics Systems Society (AESS) Avionics Systems Panel (ASP) discussions aimed at aligning educational approaches to relevant industry needs and technical advancements in the field of avionics engineering. Suitable curricular development approaches are proposed to address the career life cycle of avionics engineers, including undergraduate and graduate education. Collaborative exchange and discussion, including both industrial and academic perspectives, focus on informing curricular development approaches that bridge the gaps between higher education, industry practice, and public stakeholder needs toward maximizing educational outcomes and preparedness of the avionics engineering workforce to tackle some of the most important challenges and opportunities faced by the aerospace sector.
Research Interests:
This paper develops and evaluates an orbital emulator (OE) for space situational awareness (SSA). The OE can produce 3D satellite movements using capabilities generated from omni-wheeled robot and robotic arm motion methods. The 3D motion... more
This paper develops and evaluates an orbital emulator (OE) for space situational awareness (SSA). The OE can produce 3D satellite movements using capabilities generated from omni-wheeled robot and robotic arm motion methods. The 3D motion of a satellite is partitioned into the movements in the equatorial plane and the up-down motions in the vertical plane. The 3D actions are emulated by omni-wheeled robot models while the up-down motions are performed by a stepped-motor-controlled-ball along a rod (robotic arm), which is attached to the robot. For multiple satellites, a fast map-merging algorithm is integrated into the robot operating system (ROS) and simultaneous localization and mapping (SLAM) routines to locate the multiple robots in the scene. The OE is used to demonstrate a pursuit-evasion (PE) game theoretic sensor management algorithm, which models conflicts between a space-based-visible (SBV) satellite (as pursuer) and a geosynchronous (GEO) satellite (as evader). The cost function of the PE game is based on the informational entropy of the SBV-tracking-GEO scenario. GEO can maneuver using a continuous and low thruster. The hard-in-loop space emulator visually illustrates the SSA problem solution based PE game.
Research Interests:
For the short-arc angle only orbit initialization problem, the admissible area is often used. However, the accuracy using a single sensor is often limited. For high value space objects, it is desired to achieve more accurate results.... more
For the short-arc angle only orbit initialization problem, the admissible area is often used. However, the accuracy using a single sensor is often limited. For high value space objects, it is desired to achieve more accurate results. Fortunately, multiple sensors, which are dedicated to space situational awareness, are available. The work in this paper uses multiple sensors’ information to cooperatively initialize the orbit based on the fusion of multiple admissible areas. Both the centralized fusion and decentralized fusion are discussed. Simulation results verify the expectation that the orbit initialization accuracy is improved by using information from multiple sensors.
Research Interests:
This paper presents a novel approach for the uplink transmission power control for satellite communication (SATCOM) links that share a transponder's transmission power. The proposed approach provides explicit link margins (LM) for... more
This paper presents a novel approach for the uplink transmission power control for satellite communication (SATCOM) links that share a transponder's transmission power. The proposed approach provides explicit link margins (LM) for both uplinks and downlinks against losses from various sources that are uncertain in nature. The feature allows the proposed algorithm to work effectively with information from SATCOM radio frequency (RF) situation awareness to establish transponded SATCOM links that achieve desired Quality of Service (QoS) requirements. The algorithm is applicable for both traditional satellite communication systems and protected Frequency Hopping (FH) SATCOM systems with transparent transponders.
Research Interests:
This paper presents a real-time propagating and visualizing the uncertainty of multiple orbit satellites within the framework of graphics computing unit and multi-threading processing. The paper presents a system to predict the future... more
This paper presents a real-time propagating and visualizing the uncertainty of multiple orbit satellites within the framework of graphics computing unit and multi-threading processing. The paper presents a system to predict the future position and velocity of orbiting objects based on a Monte Carlo method with multi-threading (Central Processing Unit (CPU)) and multi-stream (Graphics Processing Unit (GPU)) processing for the application of multiple satellite orbits estimation, collision probability calculation and visualization. The introduced SATellite Uncertinty Processing with GpU and Multi-threading (SAT-UPGUM) approach is general purpose in the sense that it can extended to other types of Space Situation Awareness (SSA) applications such as Stochastic Collocation, Pursui-Evasion, and Patterns of Life. The GPU based computing leads to a real-time outcome of orbit satellites uncertainty propagation and visualization compared to the situations when the whole work flow is applied in CPU only. The obtained propagation results for the multiple satellite orbits indicate that our GPU and multi-threading based approach provides dramatically improved frame rate under realistic conditions.
Research Interests:
This paper develops and evaluates a satellite orbital testbed (SOT) for satellite communications (SATCOM). SOT can emulate the 3D satellite orbit using the omni-wheeled robots and a robotic arm. The 3D motion of satellite is partitioned... more
This paper develops and evaluates a satellite orbital testbed (SOT) for satellite communications (SATCOM). SOT can emulate the 3D satellite orbit using the omni-wheeled robots and a robotic arm. The 3D motion of satellite is partitioned into the movements in the equatorial plane and the up-down motions in the vertical plane. The former actions are emulated by omni-wheeled robots while the up-down motions are performed by a stepped-motor-controlled-ball along a rod (robotic arm), which is attached to the robot. The emulated satellite positions will go to the measure model, whose results will be used to perform multiple space object tracking. Then the tracking results will go to the maneuver detection and collision alert. The satellite maneuver commands will be translated to robots commands and robotic arm commands. In SATCOM, the effects of jamming depend on the range and angles of the positions of satellite transponder relative to the jamming satellite. We extend the SOT to include USRP transceivers. In the extended SOT, the relative ranges and angles are implemented using omni-wheeled robots and robotic arms.
Research Interests:
Multiple space object tracking is vital to space situational awareness. In this paper, the multiple hypothesis filter is used to track multiple space objects via a space-based optical sensor, which has many distinct advantages over... more
Multiple space object tracking is vital to space situational awareness. In this paper, the multiple hypothesis filter is used to track multiple space objects via a space-based optical sensor, which has many distinct advantages over ground-based sensors. Due to the limited observations obtained from the space-based sensor, the constrained admissible area is used to initialize the orbit. A semi-greedy track selection (SGTS) algorithm is used to solve the multidimensional assignment problem in the observation to track association. The Cubature Kalman filter (CKF) will be used to update the orbit for each space object. Because of the various geometric relationships between the Sun, space objects, and the Earth, many trackelets are generated by the same space object. To facilitate the space object classification, the least squares method associates tracklets to objects. A set of objects with geosynchronous equatorial orbit obtained from the space catalogue is used to test and demonstrate the effectiveness of the proposed algorithm.
Research Interests:
Humans have always had a keen interest in understanding activities and the surrounding environment for mobility, communication, and survival. Thanks to recent progress in photography and breakthroughs in aviation, we are now able to... more
Humans have always had a keen interest in understanding activities and the surrounding environment for mobility, communication, and survival. Thanks to recent progress in photography and breakthroughs in aviation, we are now able to capture tens of megapixels of ground imagery, namely Wide Area Motion Imagery (WAMI), at multiple frames per second from unmanned aerial vehicles (UAVs). WAMI serves as a great source for many applications, including security, urban planning and route planning. These applications require fast and accurate image understanding which is time consuming for humans, due to the large data volume and city-scale area coverage. Therefore, automatic processing and understanding of WAMI imagery has been gaining attention in both industry and the research community. This paper focuses on an essential step in WAMI imagery analysis, namely vehicle classification. That is, deciding whether a certain image patch contains a vehicle or not. We collect a set of positive and negative sample image patches, for training and testing the detector. Positive samples are 64 × 64 image patches centered on annotated vehicles. We generate two sets of negative images. The first set is generated from positive images with some location shift. The second set of negative patches is generated from randomly sampled patches. We also discard those patches if a vehicle accidentally locates at the center. Both positive and negative samples are randomly divided into 9000 training images and 3000 testing images. We propose to train a deep convolution network for classifying these patches. The classifier is based on a pre-trained AlexNet Model in the Caffe library, with an adapted loss function for vehicle classification. The performance of our classifier is compared to several traditional image classifier methods using Support Vector Machine (SVM) and Histogram of Oriented Gradient (HOG) features. While the SVM+HOG method achieves an accuracy of 91.2%, the accuracy of our deep network-based classifier reaches 97.9%.
Research Interests:
In this paper we consider a problem of estimating the signal-to-interference-plus-noise ratio (SINR) for satellite transmission system in the presence of jamming signals. Additive white Gaussian noise (AWGN) channels are considered for... more
In this paper we consider a problem of estimating the signal-to-interference-plus-noise ratio (SINR) for satellite transmission system in the presence of jamming signals. Additive white Gaussian noise (AWGN) channels are considered for baseband quadrature phase shift keying (QPSK) data transmission system. Two interference models are proposed with Gaussian or non-Gaussian interference signals in order to investigate the SINR for different satellite transmission jamming scenarios. Both non-data-aided moment-based and data-aided maxi-mum likelihood SINR estimators are derived for the systems. The normalized mean square errors of the SINR estimation algorithms are examined by means of computer simulations. The numerical results show the robust-ness of derived SINR estimators. The development of the SINR estimators are applicable to a large number of applications utilizing satellite communication systems.
Research Interests:
This paper investigates the constellation and mapping optimization for amplitude phase shift keying (APSK) modulation, which is deployed in Digital Video Broadcasting Satellite - Second Generation (DVB-S2) and Digital Video Broadcasting -... more
This paper investigates the constellation and mapping optimization for amplitude phase shift keying (APSK) modulation, which is deployed in Digital Video Broadcasting Satellite - Second Generation (DVB-S2) and Digital Video Broadcasting - Satellite services to Handhelds (DVB-SH) broadcasting standards due to its merits of power and spectral efficiency together with the robustness against nonlinear distortion. The mapping optimization is performed for 32-APSK according to combined cost functions related to Euclidean distance and mutual information. A Binary switching algorithm and its modified version are used to minimize the cost function and the estimated error between the original and received data. The optimized constellation mapping is tested by combining DVB-S2 standard Low-Density Parity-Check (LDPC) codes in both Bit-Interleaved Coded Modulation (BICM) and BICM with iterative decoding (BICM-ID) systems. The simulated results validate the proposed constellation labeling optimization scheme which yields better performance against conventional 32-APSK constellation defined in DVB-S2 standard.
Research Interests:
Spectral efficiency and energy efficiency are fundamental trade-off in wireless communications. Spectral efficiency (SE), defined as the average data rate per unit bandwidth, quantifies how efficiently the available spectrum is utilized.... more
Spectral efficiency and energy efficiency are fundamental trade-off in wireless communications. Spectral efficiency (SE), defined as the average data rate per unit bandwidth, quantifies how efficiently the available spectrum is utilized. Energy efficiency (EE), defined as the successful transmitted information bits per unit energy from transmitter to receiver, quantifies how efficiently the energy is utilized. Basically, with higher average energy per bit to noise power spectral density ratio at the receiver, the packet can be more successfully detected, thus utilizing the spectrum more efficiently, giving higher SE; however, in this case, it requires more energy, lowering EE, and vice versa. In this paper, we study the trade-off between SE and EE, specifically for the cognitive radio considering its configurability. We propose a general metric SEE (Spectral/Energy Efficiency) to facilitate the analysis which quantifies the preference of SE or EE. Closed-form solutions for symbol transmission energy and the length of information bits per frame are obtained for various combined modulation and channel coding schemes. The closed-form solutions further facilitate the adaptivity of cognitive radio considering both SE and EE in various scenarios. Using the proposed metric shows that our scheme is capable to perform balanced trade-off between SE and EE. Considering only maximizing SE, our scheme gains much larger EE while only sacrificing little SE; and comparing with maximizing EE, larger SE can be obtained while sacrificing a small amount EE.
Research Interests:
Communication link maintenance is crucial for aeronautical systems operations. To ensure the reliability, robustness, and security of aeronautical communication links; we investigate various types of interference in aeronautical... more
Communication link maintenance is crucial for aeronautical systems operations. To ensure the reliability, robustness, and security of aeronautical communication links; we investigate various types of interference in aeronautical communication systems, which can be categorized into unintentional and intentional interference. An interference model is built and incorporated into an aeronautical communication link design, where spread spectrum techniques are employed to mitigate the interference effects. To ensure comprehensive communication link quality-of-services, a direct-sequence spread spectrum (DSSS) and frequency-hopping spread spectrum (FHSS) are investigated and compared. Turbo coding is employed in conjunction with DSSS/FHSS for overall interference mitigation and performance evaluations. As a practical use case, the Rician fading channel is evaluated when analyzing the data link performances for three phases of air traffic surface management: taxing, takeoff/landing, and departure/approach scenarios. The results demonstrate consideration for future NextGen avionics designs for security and maintenance of communication links between the aircraft and air traffic control operations.
Research Interests:
ABSTRACT Network sensor-based defense (NSD) systems have been widely used to defend against cyber threats. Nonetheless, if the adversary finds ways to identify the location of monitor sensors, the effectiveness of NSD systems can be... more
ABSTRACT Network sensor-based defense (NSD) systems have been widely used to defend against cyber threats. Nonetheless, if the adversary finds ways to identify the location of monitor sensors, the effectiveness of NSD systems can be reduced. In this paper, we propose both temporal and spatial perturbation based defense mechanisms to secure NSD systems and make the monitor sensor invisible to the adversary. The temporal-perturbation based defense manipulates the timing information of published data so that the probability of successfully recognizing monitor sensors can be reduced. The spatial-perturbation based defense dynamically redeploys monitor sensors in the network so that the adversary cannot obtain the complete information to recognize all of the monitor sensors. We carried out experiments using real-world traffic traces to evaluate the effectiveness of our proposed defense mechanisms. Our data shows that our proposed defense mechanisms can reduce the attack accuracy of recognizing detection sensors.
Research Interests:
ABSTRACT
Research Interests:
Research Interests:
This text reviews the fundamental theory and latest methods for including contextual information in fusion process design and implementation. Chapters are contributed by the foremost international experts, spanning numerous developments... more
This text reviews the fundamental theory and latest methods for including contextual information in fusion process design and implementation. Chapters are contributed by the foremost international experts, spanning numerous developments and applications. The book highlights high- and low-level information fusion problems, performance evaluation under highly demanding conditions, and design principles. A particular focus is placed on approaches that integrate research from different communities, emphasizing the benefit of combining different techniques to overcome the limitations of a single perspective. Features: introduces the terminology and core elements in information fusion and context; presents key themes for context-enhanced information fusion; discusses design issues in developing context-aware fusion systems; provides mathematical grounds for modeling the contextual influences in representative fusion problems; describes the fusion of hard and soft data; reviews a diverse r...
Research Interests:
AI is related to information fusion (IF). Many methods in AI that use perception and reasoning align to the functionalities of high-level IF (HLIF) operations that estimate situational and impact states. To achieve HLIF sensor, user, and... more
AI is related to information fusion (IF). Many methods in AI that use perception and reasoning align to the functionalities of high-level IF (HLIF) operations that estimate situational and impact states. To achieve HLIF sensor, user, and mission management operations, AI elements of planning, control, and knowledge representation are needed. Both AI reasoning and IF inferencing and estimation exploit context as a basis for achieving deeper levels of understanding of complex world conditions. Open challenges for AI researchers include achieving concept generalization, response adaptation, and situation assessment. This article presents a brief survey of recent and current research on the exploitation of context in IF and discusses the interplay and similarities between IF, context exploitation, and AI. In addition, it highlights the role that contextual information can provide in the next generation of adaptive intelligent systems based on explainable AI. The article describes termi...
Research Interests:
This chapter attempts to cover two topics which themselves are complex and multidisciplinary: the concept of “Context” and the concept of “Information Fusion”, both of which have long histories of research publications. This chapter thus... more
This chapter attempts to cover two topics which themselves are complex and multidisciplinary: the concept of “Context” and the concept of “Information Fusion”, both of which have long histories of research publications. This chapter thus attempts to provide the reader concise introductions to these two topics by providing a review of an established framework for data and information fusion that derives from the well-known functional model of the fusion process called the Joint Directors of Laboratories or JDL model of fusion. The latter part of the chapter introduces two frameworks for how information fusion and contextual information can possibly be joined together that would allow for improved exploitation and inferencing in a variety of applications; these frameworks should be viewed as suggestions of notional processing concepts for these purposes. The chapter also provides numerous references for the reader to follow up and explore any of the ideas offered herein.
Research Interests:
The rapid technological and business advances of point-to-point space transport prompt the need for an integrated Air Traffic Management (ATM) and Space Traffic Management (STM) framework. The introduction of a more flexible airspace and... more
The rapid technological and business advances of point-to-point space transport prompt the need for an integrated Air Traffic Management (ATM) and Space Traffic Management (STM) framework. The introduction of a more flexible airspace and vehicle/trajectory management services aims to harmonize the requirements of multi-domain and multi-entity stakeholders. The ATM-STM integration problem involves related air-and-space transport issues such as separation assurance/collision avoidance, , airspace capacity management, atmospheric pollution and emissions, and cybersecurity. To meet the ATM-STM integration challenges, ontologies are an attractive approach to enact and enhance situational awareness in such an integrated ATM-STM domain. In fact, the Federal Administration Agency (FAA) NextGen (Next Generation Air Transport Management) and SESAR (Single European Sky ATM Research) programs, as well as NASA, have proposed the use of ontologies to represent knowledge in the rapidly evolving ATM context. This paper presents a discussion on considerations to develop an Ontological Situation Awareness (OSAW) approach, which could be applied to an integrated air-and-space operational domain. These considerations include (1) both operational/technical challenges and opportunities, (2) the adoption of Artificial Intelligence (AI) and the associated impacts Cyber-Physical Systems (CPS) certifiability, and (3) contributions to sustainability using the OSAW. Realistic scenarios are presented to demonstrate the possible uses of the OSAW approach and how a Space Avionics Analytics Ontology (SAAO) can contribute to the development of an OSAW system for multi-domain traffic management. The SAAO also considers aircraft and spacecraft which are manned/unmanned with various degrees of automation (categories defined in line with current aviation/aerospace industry standards) and trusted autonomy (as an attribute of the applicable automation categories).
Research Interests:
Recent examples driving an increased need for cyber awareness include unmanned aerial vehicles (DAVs) in the airspace, development of the Automatic dependent surveillance—broadcast (ADS-B), and the risk of cyber intrusion. The incident of... more
Recent examples driving an increased need for cyber awareness include unmanned aerial vehicles (DAVs) in the airspace, development of the Automatic dependent surveillance—broadcast (ADS-B), and the risk of cyber intrusion. The incident of a civilian UAV disrupting a major airport, is one example of many incidents raising questions on the future of airspace security. While a civilian hobbyist might be ignorant of the impending harm, the situation could pose a threat to the air operations. Using this incident and others like it, ideas on the future trends in cyber awareness applied to avionics are considered. Three general ideas are incorporated: (1) mandates for new rules for airspace restrictions (e.g., geofence), (2) advanced display analytics to support the airport staff and pilots with visual displays, such as collision avoidance, and (3) electronics to support the mitigation of electronic cyber effects posed by the obtrusive aircraft. With a variety of opinions posed, the summary is intended to foster a discussion on research, industry, and commercial directions for the avionics community to consider with respect to cyber awareness.
Research Interests: Aerospace Engineering, Computer Science, Artificial Intelligence, Mobile Robotics, Cybercrimes, and 15 moreAeronautical Engineering, Autonomous Robotics, Intelligent Transportation Systems, Intelligent Systems, Aviation, Computer Security, Aerospace, Air Traffic Management, Cyberspace, Air traffic control, Avionics, Cyber Physical Systems, Cyber Security, Avionics Engineering, and Harm
A surging interest in space launch operations and in Advanced Air Mobility (AAM) concepts is exacerbating the limitations of current practices, still heavily reliant on airspace segregation and not supporting the multimodal/intermodal... more
A surging interest in space launch operations and in Advanced Air Mobility (AAM) concepts is exacerbating the limitations of current practices, still heavily reliant on airspace segregation and not supporting the multimodal/intermodal evolution of air and space transport. For a successful integration of these new transport modes, it is critical that an acceptable level of safety is provided, requiring the development of novel digital tools (e.g., mission planning and decision support systems) that utilize advanced Cyber-Physical Systems (CPS) and Artificial Intelligence (AI) technologies to allow a seamless integration of space operations in the current ATM network. This tutorial addresses the role of Aerospace CPS (ACPS) and AI research to enable the safe, efficient and sustainable development of the air and space transport sector in the next decade. While the technical maturity of propulsive and vehicle technologies is relatively high, there are several opportunities and challenges associated with the adoption of CPS and AI to enable the integration of point-to-point suborbital spaceflight with conventional atmospheric air transport. Current research aims at developing robust and fault-tolerant CPS architectures that ensure trusted autonomous air/space transport operations with the given hardware constraints, despite the uncertainties in physical processes, the limited predictability of environmental conditions, the variability of mission requirements, and the possibility of both cyber and human errors. A key point in these advanced CPS is the control of physical processes from the monitoring of variables and the use of computational intelligence to obtain a deep knowledge of the monitored environment, thus providing timely and more accurate decisions and actions. The growing interconnection of physical and digital elements, and the introduction of highly sophisticated and efficient AI techniques, has led to a new generation of CPS, that is referred to as intelligent (or smart) CPS (iCPS). By equipping physical objects with interfaces to the virtual world, and incorporating intelligent mechanisms to leverage collaboration between these objects, the boundaries between the physical and virtual worlds become blurred. Interactions occurring in the physical world are capable of changing the processing behavior in the virtual world, in a causal relationship that can be exploited for the constant improvement of processes. Exploiting iCPS, intelligent, self-aware, self-managing and self-configuring systems can be built to improve the efficiency of air and space transport, and to build trusted autonomy. However, aviation safety certification is established upon verifying that all possible safety-critical conditions have been identified and verified. Whereas, in the case of AI real-time software evolution cannot be perfectly predicted and verified in advance, this is the real challenge to certification. One solution is to specify AI functional boundaries in correlation with real-time monitoring and validation of AI solution. Implementation can be sequential with practical ground-based AI for scheduling and routing being the starting point. Next in line will be simpler, non-flight critical functions and finally moving on to flight or safety critical systems. Building a certification case requires that the final product operates in all modes and performs consistently and successfully under all actual operational and environmental conditions founded on conformance to the applicable specifications. This is one of the greatest challenges currently faced by the avionics and Air Traffic Management (ATM) industry, which is clearly amplified in the context of future commercial space transport operations. Much attention is currently being devoted to the on-orbit phase, where the unique hazards of the space environment are being examined and the required iCPS evolutions for Resident Space Objects (RSO) de-confliction and collision avoidance are being addressed, including the synergies between existing ground-based tracking systems and rapidly evolving Space-Based Space Surveillance (SBSS) solutions. The advancement of regulatory frameworks supporting spacecraft operations is a conspicuous factor, which requires a holistic approach and extensive government support for the successful development and establishment of sustainable business models, including space debris mitigation strategies, operational risk assessment and liability issues. Within the atmospheric domain, extensions and alternatives to the conventional airspace segregation approaches must be identified including ATM and Air Traffic Flow Management (ATFM) techniques to facilitate the integration of new-entrant platforms. Lastly, adequate modelling approaches to meet on-orbit risk criteria must be developed and evolutionary requirements to improve current operational procedures (and associated regulatory frameworks) must be addressed in order to establish a fully-integrated Multi-Domain Traffic Management (MDTM) framework, including AI-driven situation awareness and decision support mechanisms for air and space traffic management.
Research Interests: Aerospace Engineering, Artificial Intelligence, Machine Learning, Spacecraft Operations, Unmanned Aircraft Systems, and 15 moreAviation, Aviation Safety, Aeronautics and Aerospace Operations, Aerospace, Flight Dynamics and Control, Air Traffic Management, Unmanned Aerial Vehicles, Artificial Neural Networks, Air traffic control, Avionics, Avionics Engineering, Aeronautics, Unmanned Aerial Vehicle (UAV), Aeronautics and Astronautics, and Avionics Navigation Systems
Quantum physics has a growing influence on sensor technology; particularly, in the areas of quantum computer science, quantum communications, and quantum sensing based on recent insights from atomic, molecular and optical physics. These... more
Quantum physics has a growing influence on sensor technology; particularly, in the areas of quantum computer science, quantum communications, and quantum sensing based on recent insights from atomic, molecular and optical physics. These quantum contributions have the potential to impact information fusion techniques. Quantum information technology (QIT) methods of interest suggest benefits for information fusion, so a panel was organized to articulate methods of importance for the community. The panel discussion presented many ideas from which the leading impact for information fusion is directly related to the sub-Rayleigh sensing that reduces uncertainty for object assessment through enhanced resolution. The second areas of importance is in the cyber security of data that supports data, sensor, and information fusion. Some elements of QIT that require further analysis is in quantum computing for which only a limited set of information fusion techniques can harness the methods associated with quantum computer architectures. The panel reviewed various aspects of QIT for information fusion which provides a foundation to identify future alignment between quantum and information fusion techniques.
Research Interests:
Signal acquisition in a GPS receiver aims at quickly obtaining coarse estimates of a GPS signal’s time and frequency parameters so as to initialize the code and carrier tracking loops for subsequent refined signal estimation. Conventional... more
Signal acquisition in a GPS receiver aims at quickly obtaining coarse estimates of a GPS signal’s time and frequency parameters so as to initialize the code and carrier tracking loops for subsequent refined signal estimation. Conventional methods divide the time and frequency uncertainty zone of the signal into a grid of search points and then test each and every search point by correlating the incoming signal samples with those of a local replica generated with the parameters thereof. If several grid points can be checked at the same time per correlation, the uncertainty zone can be swept over quickly, leading to a fast acquisition process.
In this paper, we present a fast acquisition search technique (FAST) via simultaneous search of allowable frequency errors (SAFE). FAST is based on judicious combining of a number of carrier replicas at selected frequency search points, leading to a combined carrier replica (CCR) and a multi-frequency modulated code replica (MMCR). As such, it can reduce the total test points of MN, where M is the number of frequency bins and N is the number of code lags, into M+N. That is, the method achieves fast acquisition using two linear time searches. Optimality criteria and practical methods (a randomized Dirichlet kernel) to reduce implementation loss of CCR and MMCR are described. Simulation and experimental data processing results are presented to demonstrate the functionality and performance of FAST.
In this paper, we present a fast acquisition search technique (FAST) via simultaneous search of allowable frequency errors (SAFE). FAST is based on judicious combining of a number of carrier replicas at selected frequency search points, leading to a combined carrier replica (CCR) and a multi-frequency modulated code replica (MMCR). As such, it can reduce the total test points of MN, where M is the number of frequency bins and N is the number of code lags, into M+N. That is, the method achieves fast acquisition using two linear time searches. Optimality criteria and practical methods (a randomized Dirichlet kernel) to reduce implementation loss of CCR and MMCR are described. Simulation and experimental data processing results are presented to demonstrate the functionality and performance of FAST.
Research Interests:
During the 2016 SPIE DSS conference, nine panelists were invited to highlight the trends and opportunities in cyber-physical systems (CPS) and Internet of Things (IoT) with information fusion. The world will be ubiquitously outfitted with... more
During the 2016 SPIE DSS conference, nine panelists were invited to highlight the trends and opportunities in cyber-physical systems (CPS) and Internet of Things (IoT) with information fusion. The world will be ubiquitously outfitted with many sensors to support our daily living thorough the Internet of Things (IoT), manage infrastructure developments with cyber-physical systems (CPS), as well as provide communication through networked information fusion technology over the internet (NIFTI). This paper summarizes the panel discussions on opportunities of information fusion to the growing trends in CPS and IoT. The summary includes the concepts and areas where information supports these CPS/IoT which includes situation awareness, transportation, and smart grids.
Research Interests:
Research Interests:
Artificial Intelligence/Deep Learning (AI/DL) techniques are based on learning a model using large available data sets. The data sets typically are from a single modality (e.g., imagery) and hence the model is based on a single modality.... more
Artificial Intelligence/Deep Learning (AI/DL) techniques are based on learning a model using large available data sets. The data sets typically are from a single modality (e.g., imagery) and hence the model is based on a single modality. Likewise, multiple models are each built for a common scenario (e.g., video and natural language processing of text describing the situation). There are issues of robustness, efficiency, and explainability that need to be addressed. A second modality can improve efficiency (e.g., cueing), robustness (e.g., results cannot be fooled such as adversary systems), and explainability from different sources. The challenge is how to organize the data needed for joint data training and model building. For example, what is needed is (1) structure for indexing data as an object file, (2) recording of metadata for effective correlation, and (3) supporting methods of analysis for model interpretability for users. The Panel presents a variety of questions and responses discussed, explored, and analyzed for data fusion-based AI data fusion tools.
Research Interests:
The dynamic data-driven applications systems (DDDAS) paradigm is meant to inject measurements into the execution model for enhanced systems performance. One area off interest in DDDAS is for space situation awareness (SSA). For SSA, data... more
The dynamic data-driven applications systems (DDDAS) paradigm is meant to inject measurements into the execution model for enhanced systems performance. One area off interest in DDDAS is for space situation awareness (SSA). For SSA, data is collected about the space environment to determine object motions, environments, and model updates. Dynamically coupling between the data and models enhances the capabilities of each system by complementing models with data for system control, execution, and sensor management. The paper overviews some of the recent developments in SSA made possible from DDDAS techniques which are for object detection, resident space object tracking, atmospheric models for enhanced sensing, cyber protection, and information management.
Research Interests:
In this paper, a feedback training approach for efficiently dealing with distribution shift in synthetic aperture radar target detection using a Bayesian convolutional neural network is proposed. After training the network on... more
In this paper, a feedback training approach for efficiently dealing with distribution shift in synthetic aperture radar target detection using a Bayesian convolutional neural network is proposed. After training the network on in-distribution data, it is tested on out-of-distribution data. Samples that are classified incorrectly with high certainty are fed back for a second round of training. This results in the reduction of false positives in the out-of-distribution dataset. False positive target detections challenge human attention, sensor resource management, and mission engagement. In these types of applications, a reduction in false positives thus often takes precedence over target detection and classification performance. The classifier is used to discriminate the targets from the clutter and to classify the target type in a single step as opposed to the traditional approach of having a sequential chain of functions for target detection and localisation before the machine learn...
Research Interests:
The Uncertainty Representation and Reasoning Evaluation Framework (URREF) ontology addresses various semantic ambiguities associated with uncertainty. One of the main objectives of the URREF ontology is to define and articulate the... more
The Uncertainty Representation and Reasoning Evaluation Framework (URREF) ontology addresses various semantic ambiguities associated with uncertainty. One of the main objectives of the URREF ontology is to define and articulate the criteria which enable the systematic reasoning about and evaluation of uncertainty representation and reasoning in information fusion systems. Towards assessing the merits of the URREF, information quality requires close attention for the measuring veracity and relevance. Veracity is measured from source sensitivity. Relevance is determined from precision and recall measures which are added to the URREF ontology. An example based on the Avionics Analytics Ontology (AAO) for Air Traffic Management with multiple Automatic Dependent Surveillance-Broadcast (ADS-B) radars is provided. The use or the precision-recall, F1 score, and sensitivity provide useful analysis for semantically important discussions on “relevance” as a complement of “recall” in information fusion uncertainty assessment.
Research Interests:
Research Interests:
The number of sensors for space situational awareness is limited, while a large number of space objects are in the catalogue. To efficiently and flexibly use a limited number of sensors, we propose to use the consensus-based distributed... more
The number of sensors for space situational awareness is limited, while a large number of space objects are in the catalogue. To efficiently and flexibly use a limited number of sensors, we propose to use the consensus-based distributed sensor management (CSM) algorithm for space object tracking. With CSM, information exchange is only required between neighbors, which avoids the limitations in centralized sensor management algorithms. The auction algorithm is used to select tracking tasks and the consensus algorithm is used to achieve agreements between different sensors. Typical scenarios, which use multiple Electro-Optical (EO) sensors to track multiple objects, are used to demonstrate the effectiveness of the proposed consensus-based auction algorithm (CBAA) algorithm.
Research Interests:
The question addressed in this paper is “what” is to be evaluated by the Uncertainty Representation and Reasoning Evaluation Framework (URREF) ontology. We thus identify the elements composing uncertainty representation and reasoning... more
The question addressed in this paper is “what” is to be evaluated by the Uncertainty Representation and Reasoning Evaluation Framework (URREF) ontology. We thus identify the elements composing uncertainty representation and reasoning approaches, which constitute various subjects being assessed. We distinguish between primary evaluation subjects (Uncertainty Representation and Reasoning components of the fusion algorithm), and secondary evaluation subjects (source of information, piece of information, fusion method and mathematical model). This paper proposes a list of source quality criteria to be added to the ontology and establishes formal links between the secondary and primary evaluation subjects. The key contribution of the paper is the update of the definitions of sub-criteria of the Expressiveness criterion together with suggestions for complementary concepts to be included in the ontology (type of scale, type of uncertainty expression). Conclusions are drawn to extend the work in using the expressiveness criterion for information fusion analysis.
Research Interests:
Research Interests:
Advancements in artificial intelligence, information communication, and systems design are potential for autonomous systems emerging for space situation awareness (SSA) architectures. Examples of architecture designs are autonomy in... more
Advancements in artificial intelligence, information communication, and systems design are potential for autonomous systems emerging for space situation awareness (SSA) architectures. Examples of architecture designs are autonomy in motion (AIM) for dynamic data assessment systems (e.g., robotics) and autonomy at rest (AAR) for static data collection systems (e.g., surveillance). However, there is a need for data architectures which are tailored to the SSA missions, which necessitates autonomy in use (AIU). AIU requires pragmatic use of message passing and data flow architectures, contextual and theoretic modeling, and user and information fusion. Information fusion provides methods for data aggregation, correlation, and temporal assessment and awareness. Together, AIU accesses the dynamic data for autonomy in change (AIC), information fusion from AAR in order to make AIM real-time decisions. The paper discusses issues for space situation awareness directions focusing on autonomy in use.
Research Interests:
Over the course of Dave Hall's career, he highlighted various concerns associated with the implementation of data fusion methods. Many of the issues included the role of uncertainty in data fusion, practical implementation of sensor... more
Over the course of Dave Hall's career, he highlighted various concerns associated with the implementation of data fusion methods. Many of the issues included the role of uncertainty in data fusion, practical implementation of sensor fusion systems, and incorporating new technology into information fusion designs. These thoughts were communicated through technical books and Handbook collections of articles from authors in the fusion community as comprehensive discussions of data collection and processing to knowledge acquisition and delivery. A summary of the uncertainty issues from Dave Hall, originating with the Joint Directors of the Laboratories (JDL) model, include these attributes across the JDL Levels which are: data (variance), object assessment (covariance), situation (representation), threat (possibility), sensor management (delay), and user (cognition). This paper explores the concepts of uncertainty addressed by Dave Hall from many of his publications that can be used...
Research Interests:
Synchronization plays an important role in wireless communication systems when tracking a phase-shift keying (PSK) signal, especially when the initial frequency error is comparable to the loop bandwidth. In order to improve frequency... more
Synchronization plays an important role in wireless communication systems when tracking a phase-shift keying (PSK) signal, especially when the initial frequency error is comparable to the loop bandwidth. In order to improve frequency acquisition, an automatic frequency control (AFC) augmentation is used. This paper presents a composite AFC/Costas loop by combining both the AFC loop with a phase-locked loop (PLL) Costas loop for carrier frequency recovery. Therefore, pull-in from both frequency and phase errors is feasible using the composite AFC/Costas loop. The AFC/Costas loop combination filter coefficient setting is evaluated by a theoretic analysis. Improved frequency and phase acquisition can be realized by changing the first order AFC/Costas loop to the second order. First, the structure of the composite AFC/Costas loop is shown. This structure makes use of phase detectors to obtain the phase differences between the received signal and reference signal, where the phase differences can be used to generate the phase and frequency control signals. Difference equations are proposed to describe the composite AFC/Costas loop. Then, the theoretic analysis for both frequency and phase control are derived in a linearized model of the composite loop. The phase error variance is simulated to show the performance of the composite AFC/Costas loop. Moreover, the frequency and phase synchronization performance for different signal to noise ratio (SNR) and loop filter bandwidth are shown to demonstrate the effectiveness of the composite AFC/Costas loop. Finally, the Quadrature Phase Shift Keying (QPSK) data is demodulated and decoded through the composite AFC/Costas loop. Extensive simulations are implemented to show that the demodulated data matches the transmitted data, which proves that differential QPSK can effectively reduce the phase ambiguity and increase frequency pull-in range, especially for the low SNR region (Eb/N0 < 3 dB). The proposed composite AFC/Costas loop sheds insights on the design of frequency and phase synchronization in wireless communication systems.
Research Interests:
Research Interests:
– A human presented with a variety of displays is expected to fuse data to obtain information. An effective presentation of information would assist the human in fusing data. This paper describes a multisensor-multisource information... more
– A human presented with a variety of displays is expected to fuse data to obtain information. An effective presentation of information would assist the human in fusing data. This paper describes a multisensor-multisource information decision making tool that was designed to augment human cognitive fusion.