Showing 1–50 of 64 results for author: Santos, E

Toward Enhancing Vehicle Color Recognition in Adverse Conditions: A Dataset and Benchmark

Authors: Gabriel E. Lima, Rayson Laroca, Eduardo Santos, Eduil Nascimento Jr., David Menotti

Abstract: Vehicle information recognition is crucial in various practical domains, particularly in criminal investigations. Vehicle Color Recognition (VCR) has garnered significant research interest because color is a visually distinguishable attribute of vehicles and is less affected by partial occlusion and changes in viewpoint. Despite the success of existing methods for this task, the relatively low com… ▽ More Vehicle information recognition is crucial in various practical domains, particularly in criminal investigations. Vehicle Color Recognition (VCR) has garnered significant research interest because color is a visually distinguishable attribute of vehicles and is less affected by partial occlusion and changes in viewpoint. Despite the success of existing methods for this task, the relatively low complexity of the datasets used in the literature has been largely overlooked. This research addresses this gap by compiling a new dataset representing a more challenging VCR scenario. The images - sourced from six license plate recognition datasets - are categorized into eleven colors, and their annotations were validated using official vehicle registration information. We evaluate the performance of four deep learning models on a widely adopted dataset and our proposed dataset to establish a benchmark. The results demonstrate that our dataset poses greater difficulty for the tested models and highlights scenarios that require further exploration in VCR. Remarkably, nighttime scenes account for a significant portion of the errors made by the best-performing model. This research provides a foundation for future studies on VCR, while also offering valuable insights for the field of fine-grained vehicle classification. △ Less

Submitted 21 August, 2024; originally announced August 2024.

Comments: Accepted for presentation at the Conference on Graphics, Patterns and Images (SIBGRAPI) 2024

Enhancing Code Translation in Language Models with Few-Shot Learning via Retrieval-Augmented Generation

Authors: Manish Bhattarai, Javier E. Santos, Shawn Jones, Ayan Biswas, Boian Alexandrov, Daniel O'Malley

Abstract: The advent of large language models (LLMs) has significantly advanced the field of code translation, enabling automated translation between programming languages. However, these models often struggle with complex translation tasks due to inadequate contextual understanding. This paper introduces a novel approach that enhances code translation through Few-Shot Learning, augmented with retrieval-bas… ▽ More The advent of large language models (LLMs) has significantly advanced the field of code translation, enabling automated translation between programming languages. However, these models often struggle with complex translation tasks due to inadequate contextual understanding. This paper introduces a novel approach that enhances code translation through Few-Shot Learning, augmented with retrieval-based techniques. By leveraging a repository of existing code translations, we dynamically retrieve the most relevant examples to guide the model in translating new code segments. Our method, based on Retrieval-Augmented Generation (RAG), substantially improves translation quality by providing contextual examples from which the model can learn in real-time. We selected RAG over traditional fine-tuning methods due to its ability to utilize existing codebases or a locally stored corpus of code, which allows for dynamic adaptation to diverse translation tasks without extensive retraining. Extensive experiments on diverse datasets with open LLM models such as Starcoder, Llama3-70B Instruct, CodeLlama-34B Instruct, Granite-34B Code Instruct, and Mixtral-8x22B, as well as commercial LLM models like GPT-3.5 Turbo and GPT-4o, demonstrate our approach's superiority over traditional zero-shot methods, especially in translating between Fortran and CPP. We also explored varying numbers of shots i.e. examples provided during inference, specifically 1, 2, and 3 shots and different embedding models for RAG, including Nomic-Embed, Starencoder, and CodeBERT, to assess the robustness and effectiveness of our approach. △ Less

Submitted 28 July, 2024; originally announced July 2024.

Comments: LLM for code translation

Digital Storytelling for Competence Development in Games

Authors: Edgar Santos, Claudia Ribeiro, Manuel Fradinho, João Pereira

Abstract: The acquisition of complex knowledge and competences raises difficult challenges for the supporting tools within the corporate environment, which digital storytelling presents a potential solution. Traditionally, a driving goal of digital storytelling is the generation of dramatic stories with human significance, but for learning purposes, the need for drama is complemented by the requirement of a… ▽ More The acquisition of complex knowledge and competences raises difficult challenges for the supporting tools within the corporate environment, which digital storytelling presents a potential solution. Traditionally, a driving goal of digital storytelling is the generation of dramatic stories with human significance, but for learning purposes, the need for drama is complemented by the requirement of achieving particular learning outcomes. This paper presents a narrative engine that supports emergent storytelling to support the development of complex competences in the learning domains of project management and innovation. The approach is based on the adaptation on the Fabula model combined with cases representing situated contexts associated to particular competences. These cases are then triggered to influence the unfolding of the story such that a learner encounters dramatic points in the narrative where the associated competences need to be used. In addition to the description of the approach and corresponding narrative engine, an illustration is presented of how the competence 'conflict management' influences a story. △ Less

Submitted 21 June, 2024; originally announced July 2024.

Real-Time Scene Graph Generation

Authors: Maëlic Neau, Paulo E. Santos, Karl Sammut, Anne-Gwenn Bosser, Cédric Buche

Abstract: Scene Graph Generation (SGG) can extract abstract semantic relations between entities in images as graph representations. This task holds strong promises for other downstream tasks such as the embodied cognition of an autonomous agent. However, to power such applications, SGG needs to solve the gap of real-time latency. In this work, we propose to investigate the bottlenecks of current approaches… ▽ More Scene Graph Generation (SGG) can extract abstract semantic relations between entities in images as graph representations. This task holds strong promises for other downstream tasks such as the embodied cognition of an autonomous agent. However, to power such applications, SGG needs to solve the gap of real-time latency. In this work, we propose to investigate the bottlenecks of current approaches for real-time constraint applications. Then, we propose a simple yet effective implementation of a real-time SGG approach using YOLOV8 as an object detection backbone. Our implementation is the first to obtain more than 48 FPS for the task with no loss of accuracy, successfully outperforming any other lightweight approaches. Our code is freely available at https://github.com/Maelic/SGG-Benchmark. △ Less

Submitted 25 May, 2024; originally announced May 2024.

Hybrid Navigation Acceptability and Safety

Authors: Benoit Clement, Marie Dubromel, Paulo E. Santos, Karl Sammut, Michelle Oppert, Feras Dayoub

Abstract: Autonomous vessels have emerged as a prominent and accepted solution, particularly in the naval defence sector. However, achieving full autonomy for marine vessels demands the development of robust and reliable control and guidance systems that can handle various encounters with manned and unmanned vessels while operating effectively under diverse weather and sea conditions. A significant challeng… ▽ More Autonomous vessels have emerged as a prominent and accepted solution, particularly in the naval defence sector. However, achieving full autonomy for marine vessels demands the development of robust and reliable control and guidance systems that can handle various encounters with manned and unmanned vessels while operating effectively under diverse weather and sea conditions. A significant challenge in this pursuit is ensuring the autonomous vessels' compliance with the International Regulations for Preventing Collisions at Sea (COLREGs). These regulations present a formidable hurdle for the human-level understanding by autonomous systems as they were originally designed from common navigation practices created since the mid-19th century. Their ambiguous language assumes experienced sailors' interpretation and execution, and therefore demands a high-level (cognitive) understanding of language and agent intentions. These capabilities surpass the current state-of-the-art in intelligent systems. This position paper highlights the critical requirements for a trustworthy control and guidance system, exploring the complexity of adapting COLREGs for safe vessel-on-vessel encounters considering autonomous maritime technology competing and/or cooperating with manned vessels. △ Less

Submitted 17 April, 2024; originally announced April 2024.

Detecting Hidden Triggers: Mapping Non-Markov Reward Functions to Markov

Authors: Gregory Hyde, Eugene Santos Jr

Abstract: Many Reinforcement Learning algorithms assume a Markov reward function to guarantee optimality. However, not all reward functions are Markov. This paper proposes a framework for mapping non-Markov reward functions into equivalent Markov ones by learning specialized reward automata, Reward Machines. Unlike the general practice of learning Reward Machines, we do not require a set of high-level propo… ▽ More Many Reinforcement Learning algorithms assume a Markov reward function to guarantee optimality. However, not all reward functions are Markov. This paper proposes a framework for mapping non-Markov reward functions into equivalent Markov ones by learning specialized reward automata, Reward Machines. Unlike the general practice of learning Reward Machines, we do not require a set of high-level propositional symbols from which to learn. Rather, we learn hidden triggers, directly from data, that construct them. We demonstrate the importance of learning Reward Machines over their Deterministic Finite-State Automata counterparts given their ability to model reward dependencies. We formalize this distinction in our learning objective. Our mapping process is constructed as an Integer Linear Programming problem. We prove that our mappings form a suitable proxy for maximizing reward expectations. We empirically validate our approach by learning black-box, non-Markov reward functions in the Officeworld domain. Additionally, we demonstrate the effectiveness of learning reward dependencies in a new domain, Breakfastworld. △ Less

Submitted 16 August, 2024; v1 submitted 20 January, 2024; originally announced January 2024.

Experimental Shake Gesture Detection API for Apple Watch

Authors: Ezequiel França dos Santos

Abstract: In this paper we present the WatchShaker project The project involves an experimental API that detects the Apple Watchs shake gesturea surprisingly absent natively feature Through a simple heuristic leveraging the Apple Watchs accelerometer data the API discerns not just the occurrence of shake gestures but also their direction enhancing the interactivity potential of the device Despite the projec… ▽ More In this paper we present the WatchShaker project The project involves an experimental API that detects the Apple Watchs shake gesturea surprisingly absent natively feature Through a simple heuristic leveraging the Apple Watchs accelerometer data the API discerns not just the occurrence of shake gestures but also their direction enhancing the interactivity potential of the device Despite the projects simplicity and lack of formal testing it has garnered significant attention indicating a genuine interest and need within the developer community for such functionality The WatchShaker project exemplifies how a minimalistic approach can yield a practical and impactful tool in wearable technology providing a springboard for further research and development in intuitive gesture recognition △ Less

Submitted 18 January, 2024; originally announced January 2024.

Apple Vision Pro: Comments in Healthcare

Authors: Ezequiel Santos, Vanessa Castillo

Abstract: This paper objectively analyzes the emerging discourse surrounding Apple Vision Pro's application in healthcare and medical education. Released in June 2023, Apple Vision Pro represents a significant advancement in spatial computing, combining augmented and virtual reality to create new possibilities in digital interaction. We aim to compile and present recent articles. We used PubMed, IEEE Xplore… ▽ More This paper objectively analyzes the emerging discourse surrounding Apple Vision Pro's application in healthcare and medical education. Released in June 2023, Apple Vision Pro represents a significant advancement in spatial computing, combining augmented and virtual reality to create new possibilities in digital interaction. We aim to compile and present recent articles. We used PubMed, IEEE Xplore, Google Scholar, and JSTOR. Non-academic publications were excluded. The results were six commentaries, one a pre-print. All were majorly optimistic, with one mentioning VR/AR sickness. For future research directions, we stress the need for continued exploration of Apple Vision Pro's capabilities and limitations and expect expert opinions to englobe this discussion. △ Less

Submitted 1 April, 2024; v1 submitted 12 January, 2024; originally announced January 2024.

A Survey of Designs for Combined 2D+3D Visual Representations

Authors: Jiayi Hong, Rostyslav Hnatyshyn, Ebrar A. D. Santos, Ross Maciejewski, Tobias Isenberg

Abstract: We examine visual representations of data that make use of combinations of both 2D and 3D data mappings. Combining 2D and 3D representations is a common technique that allows viewers to understand multiple facets of the data with which they are interacting. While 3D representations focus on the spatial character of the data or the dedicated 3D data mapping, 2D representations often show abstract d… ▽ More We examine visual representations of data that make use of combinations of both 2D and 3D data mappings. Combining 2D and 3D representations is a common technique that allows viewers to understand multiple facets of the data with which they are interacting. While 3D representations focus on the spatial character of the data or the dedicated 3D data mapping, 2D representations often show abstract data properties and take advantage of the unique benefits of mapping to a plane. Many systems have used unique combinations of both types of data mappings effectively. Yet there are no systematic reviews of the methods in linking 2D and 3D representations. We systematically survey the relationships between 2D and 3D visual representations in major visualization publications -- IEEE VIS, IEEE TVCG, and EuroVis -- from 2012 to 2022. We closely examined 105 papers where 2D and 3D representations are connected visually, interactively, or through animation. These approaches are designed based on their visual environment, the relationships between their visual representations, and their possible layouts. Through our analysis, we introduce a design space as well as provide design guidelines for effectively linking 2D and 3D visual representations. △ Less

Submitted 12 January, 2024; v1 submitted 8 January, 2024; originally announced January 2024.

Journal ref: IEEE Transactions on Visualization and Computer Graphics 2024

A Case Study on Test Case Construction with Large Language Models: Unveiling Practical Insights and Challenges

Authors: Roberto Francisco de Lima Junior, Luiz Fernando Paes de Barros Presta, Lucca Santos Borborema, Vanderson Nogueira da Silva, Marcio Leal de Melo Dahia, Anderson Carlos Sousa e Santos

Abstract: This paper presents a detailed case study examining the application of Large Language Models (LLMs) in the construction of test cases within the context of software engineering. LLMs, characterized by their advanced natural language processing capabilities, are increasingly garnering attention as tools to automate and enhance various aspects of the software development life cycle. Leveraging a cas… ▽ More This paper presents a detailed case study examining the application of Large Language Models (LLMs) in the construction of test cases within the context of software engineering. LLMs, characterized by their advanced natural language processing capabilities, are increasingly garnering attention as tools to automate and enhance various aspects of the software development life cycle. Leveraging a case study methodology, we systematically explore the integration of LLMs in the test case construction process, aiming to shed light on their practical efficacy, challenges encountered, and implications for software quality assurance. The study encompasses the selection of a representative software application, the formulation of test case construction methodologies employing LLMs, and the subsequent evaluation of outcomes. Through a blend of qualitative and quantitative analyses, this study assesses the impact of LLMs on test case comprehensiveness, accuracy, and efficiency. Additionally, delves into challenges such as model interpretability and adaptation to diverse software contexts. The findings from this case study contributes with nuanced insights into the practical utility of LLMs in the domain of test case construction, elucidating their potential benefits and limitations. By addressing real-world scenarios and complexities, this research aims to inform software practitioners and researchers alike about the tangible implications of incorporating LLMs into the software testing landscape, fostering a more comprehensive understanding of their role in optimizing the software development process. △ Less

Submitted 21 December, 2023; v1 submitted 19 December, 2023; originally announced December 2023.

Reconstruction of Fields from Sparse Sensing: Differentiable Sensor Placement Enhances Generalization

Authors: Agnese Marcato, Daniel O'Malley, Hari Viswanathan, Eric Guiltinan, Javier E. Santos

Abstract: Recreating complex, high-dimensional global fields from limited data points is a grand challenge across various scientific and industrial domains. Given the prohibitive costs of specialized sensors and the frequent inaccessibility of certain regions of the domain, achieving full field coverage is typically not feasible. Therefore, the development of algorithms that intelligently improve sensor pla… ▽ More Recreating complex, high-dimensional global fields from limited data points is a grand challenge across various scientific and industrial domains. Given the prohibitive costs of specialized sensors and the frequent inaccessibility of certain regions of the domain, achieving full field coverage is typically not feasible. Therefore, the development of algorithms that intelligently improve sensor placement is of significant value. In this study, we introduce a general approach that employs differentiable programming to exploit sensor placement within the training of a neural network model in order to improve field reconstruction. We evaluated our method using two distinct datasets; the results show that our approach improved test scores. Ultimately, our method of differentiable placement strategies has the potential to significantly increase data collection efficiency, enable more thorough area coverage, and reduce redundancy in sensor deployment. △ Less

Submitted 14 December, 2023; originally announced December 2023.

Multimodal Speech Emotion Recognition Using Modality-specific Self-Supervised Frameworks

Authors: Rutherford Agbeshi Patamia, Paulo E. Santos, Kingsley Nketia Acheampong, Favour Ekong, Kwabena Sarpong, She Kun

Abstract: Emotion recognition is a topic of significant interest in assistive robotics due to the need to equip robots with the ability to comprehend human behavior, facilitating their effective interaction in our society. Consequently, efficient and dependable emotion recognition systems supporting optimal human-machine communication are required. Multi-modality (including speech, audio, text, images, and… ▽ More Emotion recognition is a topic of significant interest in assistive robotics due to the need to equip robots with the ability to comprehend human behavior, facilitating their effective interaction in our society. Consequently, efficient and dependable emotion recognition systems supporting optimal human-machine communication are required. Multi-modality (including speech, audio, text, images, and videos) is typically exploited in emotion recognition tasks. Much relevant research is based on merging multiple data modalities and training deep learning models utilizing low-level data representations. However, most existing emotion databases are not large (or complex) enough to allow machine learning approaches to learn detailed representations. This paper explores modalityspecific pre-trained transformer frameworks for self-supervised learning of speech and text representations for data-efficient emotion recognition while achieving state-of-the-art performance in recognizing emotions. This model applies feature-level fusion using nonverbal cue data points from motion capture to provide multimodal speech emotion recognition. The model was trained using the publicly available IEMOCAP dataset, achieving an overall accuracy of 77.58% for four emotions, outperforming state-of-the-art approaches △ Less

Submitted 3 December, 2023; originally announced December 2023.

Using Ornstein-Uhlenbeck Process to understand Denoising Diffusion Probabilistic Model and its Noise Schedules

Authors: Javier E. Santos, Yen Ting Lin

Abstract: The aim of this short note is to show that Denoising Diffusion Probabilistic Model DDPM, a non-homogeneous discrete-time Markov process, can be represented by a time-homogeneous continuous-time Markov process observed at non-uniformly sampled discrete times. Surprisingly, this continuous-time Markov process is the well-known and well-studied Ornstein-Ohlenbeck (OU) process, which was developed in… ▽ More The aim of this short note is to show that Denoising Diffusion Probabilistic Model DDPM, a non-homogeneous discrete-time Markov process, can be represented by a time-homogeneous continuous-time Markov process observed at non-uniformly sampled discrete times. Surprisingly, this continuous-time Markov process is the well-known and well-studied Ornstein-Ohlenbeck (OU) process, which was developed in 1930's for studying Brownian particles in Harmonic potentials. We establish the formal equivalence between DDPM and the OU process using its analytical solution. We further demonstrate that the design problem of the noise scheduler for non-homogeneous DDPM is equivalent to designing observation times for the OU process. We present several heuristic designs for observation times based on principled quantities such as auto-variance and Fisher Information and connect them to ad hoc noise schedules for DDPM. Interestingly, we show that the Fisher-Information-motivated schedule corresponds exactly the cosine schedule, which was developed without any theoretical foundation but is the current state-of-the-art noise schedule. △ Less

Submitted 29 November, 2023; originally announced November 2023.

Análise e modelagem de jogos digitais: relato de uma experiência educacional utilizando metodologias ativas em um grupo multidisciplinar

Authors: David de Oliveira Lemes, Ezequiel França dos Santos, Eduardo Romanek, Celso Fujimoto, Adriano Felix Valente

Abstract: The traditional teaching of software engineering is focused on technical skills. Active strategies, where students experience content and interact with reality, are effective. The market demands new skills in the digital transformation, dealing with the complexity of modeling businesses and the interconnection between people, systems, and technologies. The transition to active methodologies, such… ▽ More The traditional teaching of software engineering is focused on technical skills. Active strategies, where students experience content and interact with reality, are effective. The market demands new skills in the digital transformation, dealing with the complexity of modeling businesses and the interconnection between people, systems, and technologies. The transition to active methodologies, such as Problem-Based Learning (PBL), brings real market scenarios into the classroom. This article reports on the experience in the course, presenting concepts and results. △ Less

Submitted 1 September, 2024; v1 submitted 11 November, 2023; originally announced November 2023.

Comments: in Portuguese language

Report number: ISSN: 2764-0582

Journal ref: V Congresso Brasileiro Interdisciplinar em Ciência e Tecnologia - COBICET 2024

AI for Open Science: A Multi-Agent Perspective for Ethically Translating Data to Knowledge

Authors: Chase Yakaboski, Gregory Hyde, Clement Nyanhongo, Eugene Santos Jr

Abstract: AI for Science (AI4Science), particularly in the form of self-driving labs, has the potential to sideline human involvement and hinder scientific discovery within the broader community. While prior research has focused on ensuring the responsible deployment of AI applications, enhancing security, and ensuring interpretability, we also propose that promoting openness in AI4Science discoveries shoul… ▽ More AI for Science (AI4Science), particularly in the form of self-driving labs, has the potential to sideline human involvement and hinder scientific discovery within the broader community. While prior research has focused on ensuring the responsible deployment of AI applications, enhancing security, and ensuring interpretability, we also propose that promoting openness in AI4Science discoveries should be carefully considered. In this paper, we introduce the concept of AI for Open Science (AI4OS) as a multi-agent extension of AI4Science with the core principle of maximizing open knowledge translation throughout the scientific enterprise rather than a single organizational unit. We use the established principles of Knowledge Discovery and Data Mining (KDD) to formalize a language around AI4OS. We then discuss three principle stages of knowledge translation embedded in AI4Science systems and detail specific points where openness can be applied to yield an AI4OS alternative. Lastly, we formulate a theoretical metric to assess AI4OS with a supporting ethical argument highlighting its importance. Our goal is that by drawing attention to AI4OS we can ensure the natural consequence of AI4Science (e.g., self-driving labs) is a benefit not only for its developers but for society as a whole. △ Less

Submitted 31 October, 2023; v1 submitted 28 October, 2023; originally announced October 2023.

Comments: NeurIPS AI For Science Workshop 2023. 11 pages, 2 figures

AnuraSet: A dataset for benchmarking Neotropical anuran calls identification in passive acoustic monitoring

Authors: Juan Sebastián Cañas, Maria Paula Toro-Gómez, Larissa Sayuri Moreira Sugai, Hernán Darío Benítez Restrepo, Jorge Rudas, Breyner Posso Bautista, Luís Felipe Toledo, Simone Dena, Adão Henrique Rosa Domingos, Franco Leandro de Souza, Selvino Neckel-Oliveira, Anderson da Rosa, Vítor Carvalho-Rocha, José Vinícius Bernardy, José Luiz Massao Moreira Sugai, Carolina Emília dos Santos, Rogério Pereira Bastos, Diego Llusia, Juan Sebastián Ulloa

Abstract: Global change is predicted to induce shifts in anuran acoustic behavior, which can be studied through passive acoustic monitoring (PAM). Understanding changes in calling behavior requires the identification of anuran species, which is challenging due to the particular characteristics of neotropical soundscapes. In this paper, we introduce a large-scale multi-species dataset of anuran amphibians ca… ▽ More Global change is predicted to induce shifts in anuran acoustic behavior, which can be studied through passive acoustic monitoring (PAM). Understanding changes in calling behavior requires the identification of anuran species, which is challenging due to the particular characteristics of neotropical soundscapes. In this paper, we introduce a large-scale multi-species dataset of anuran amphibians calls recorded by PAM, that comprises 27 hours of expert annotations for 42 different species from two Brazilian biomes. We provide open access to the dataset, including the raw recordings, experimental setup code, and a benchmark with a baseline model of the fine-grained categorization problem. Additionally, we highlight the challenges of the dataset to encourage machine learning researchers to solve the problem of anuran call identification towards conservation policy. All our experiments and resources can be found on our GitHub repository https://github.com/soundclim/anuraset. △ Less

Submitted 11 July, 2023; originally announced July 2023.

Computing Education in the Era of Generative AI

Authors: Paul Denny, James Prather, Brett A. Becker, James Finnie-Ansley, Arto Hellas, Juho Leinonen, Andrew Luxton-Reilly, Brent N. Reeves, Eddie Antonio Santos, Sami Sarsa

Abstract: The computing education community has a rich history of pedagogical innovation designed to support students in introductory courses, and to support teachers in facilitating student learning. Very recent advances in artificial intelligence have resulted in code generation models that can produce source code from natural language problem descriptions -- with impressive accuracy in many cases. The wi… ▽ More The computing education community has a rich history of pedagogical innovation designed to support students in introductory courses, and to support teachers in facilitating student learning. Very recent advances in artificial intelligence have resulted in code generation models that can produce source code from natural language problem descriptions -- with impressive accuracy in many cases. The wide availability of these models and their ease of use has raised concerns about potential impacts on many aspects of society, including the future of computing education. In this paper, we discuss the challenges and opportunities such models present to computing educators, with a focus on introductory programming classrooms. We summarize the results of two recent articles, the first evaluating the performance of code generation models on typical introductory-level programming problems, and the second exploring the quality and novelty of learning resources generated by these models. We consider likely impacts of such models upon pedagogical practice in the context of the most recent advances at the time of writing. △ Less

Submitted 5 June, 2023; originally announced June 2023.

Comments: Accepted for publication as a Contributed Article in Communications of the ACM (CACM)

Fine-Grained is Too Coarse: A Novel Data-Centric Approach for Efficient Scene Graph Generation

Authors: Neau Maëlic, Paulo E. Santos, Anne-Gwenn Bosser, Cédric Buche

Abstract: Learning to compose visual relationships from raw images in the form of scene graphs is a highly challenging task due to contextual dependencies, but it is essential in computer vision applications that depend on scene understanding. However, no current approaches in Scene Graph Generation (SGG) aim at providing useful graphs for downstream tasks. Instead, the main focus has primarily been on the… ▽ More Learning to compose visual relationships from raw images in the form of scene graphs is a highly challenging task due to contextual dependencies, but it is essential in computer vision applications that depend on scene understanding. However, no current approaches in Scene Graph Generation (SGG) aim at providing useful graphs for downstream tasks. Instead, the main focus has primarily been on the task of unbiasing the data distribution for predicting more fine-grained relations. That being said, all fine-grained relations are not equally relevant and at least a part of them are of no use for real-world applications. In this work, we introduce the task of Efficient SGG that prioritizes the generation of relevant relations, facilitating the use of Scene Graphs in downstream tasks such as Image Generation. To support further approaches, we present a new dataset, VG150-curated, based on the annotations of the popular Visual Genome dataset. We show through a set of experiments that this dataset contains more high-quality and diverse annotations than the one usually use in SGG. Finally, we show the efficiency of this dataset in the task of Image Generation from Scene Graphs. △ Less

Submitted 25 September, 2023; v1 submitted 29 May, 2023; originally announced May 2023.

Blackout Diffusion: Generative Diffusion Models in Discrete-State Spaces

Authors: Javier E Santos, Zachary R. Fox, Nicholas Lubbers, Yen Ting Lin

Abstract: Typical generative diffusion models rely on a Gaussian diffusion process for training the backward transformations, which can then be used to generate samples from Gaussian noise. However, real world data often takes place in discrete-state spaces, including many scientific applications. Here, we develop a theoretical formulation for arbitrary discrete-state Markov processes in the forward diffusi… ▽ More Typical generative diffusion models rely on a Gaussian diffusion process for training the backward transformations, which can then be used to generate samples from Gaussian noise. However, real world data often takes place in discrete-state spaces, including many scientific applications. Here, we develop a theoretical formulation for arbitrary discrete-state Markov processes in the forward diffusion process using exact (as opposed to variational) analysis. We relate the theory to the existing continuous-state Gaussian diffusion as well as other approaches to discrete diffusion, and identify the corresponding reverse-time stochastic process and score function in the continuous-time setting, and the reverse-time mapping in the discrete-time setting. As an example of this framework, we introduce ``Blackout Diffusion'', which learns to produce samples from an empty image instead of from noise. Numerical experiments on the CIFAR-10, Binarized MNIST, and CelebA datasets confirm the feasibility of our approach. Generalizing from specific (Gaussian) forward processes to discrete-state processes without a variational approximation sheds light on how to interpret diffusion models, which we discuss. △ Less

Submitted 18 May, 2023; originally announced May 2023.

Comments: 29 pages, 13 figures, 2 tables. Accepted by the 40th International Conference on Machine Learning, Honolulu, Hawaii, USA

"It's Weird That it Knows What I Want": Usability and Interactions with Copilot for Novice Programmers

Authors: James Prather, Brent N. Reeves, Paul Denny, Brett A. Becker, Juho Leinonen, Andrew Luxton-Reilly, Garrett Powell, James Finnie-Ansley, Eddie Antonio Santos

Abstract: Recent developments in deep learning have resulted in code-generation models that produce source code from natural language and code-based prompts with high accuracy. This is likely to have profound effects in the classroom, where novices learning to code can now use free tools to automatically suggest solutions to programming exercises and assignments. However, little is currently known about how… ▽ More Recent developments in deep learning have resulted in code-generation models that produce source code from natural language and code-based prompts with high accuracy. This is likely to have profound effects in the classroom, where novices learning to code can now use free tools to automatically suggest solutions to programming exercises and assignments. However, little is currently known about how novices interact with these tools in practice. We present the first study that observes students at the introductory level using one such code auto-generating tool, Github Copilot, on a typical introductory programming (CS1) assignment. Through observations and interviews we explore student perceptions of the benefits and pitfalls of this technology for learning, present new observed interaction patterns, and discuss cognitive and metacognitive difficulties faced by students. We consider design implications of these findings, specifically in terms of how tools like Copilot can better support and scaffold the novice programming experience. △ Less

Submitted 5 April, 2023; originally announced April 2023.

Comments: 26 pages, 2 figures, TOCHI

Learning the Finer Things: Bayesian Structure Learning at the Instantiation Level

Authors: Chase Yakaboski, Eugene Santos Jr

Abstract: Successful machine learning methods require a trade-off between memorization and generalization. Too much memorization and the model cannot generalize to unobserved examples. Too much over-generalization and we risk under-fitting the data. While we commonly measure their performance through cross validation and accuracy metrics, how should these algorithms cope in domains that are extremely under-… ▽ More Successful machine learning methods require a trade-off between memorization and generalization. Too much memorization and the model cannot generalize to unobserved examples. Too much over-generalization and we risk under-fitting the data. While we commonly measure their performance through cross validation and accuracy metrics, how should these algorithms cope in domains that are extremely under-determined where accuracy is always unsatisfactory? We present a novel probabilistic graphical model structure learning approach that can learn, generalize and explain in these elusive domains by operating at the random variable instantiation level. Using Minimum Description Length (MDL) analysis, we propose a new decomposition of the learning problem over all training exemplars, fusing together minimal entropy inferences to construct a final knowledge base. By leveraging Bayesian Knowledge Bases (BKBs), a framework that operates at the instantiation level and inherently subsumes Bayesian Networks (BNs), we develop both a theoretical MDL score and associated structure learning algorithm that demonstrates significant improvements over learned BNs on 40 benchmark datasets. Further, our algorithm incorporates recent off-the-shelf DAG learning techniques enabling tractable results even on large problems. We then demonstrate the utility of our approach in a significantly under-determined domain by learning gene regulatory networks on breast cancer gene mutational data available from The Cancer Genome Atlas (TCGA). △ Less

Submitted 7 March, 2023; originally announced March 2023.

Mitigation of Spatial Nonstationarity with Vision Transformers

Authors: Lei Liu, Javier E. Santos, Maša Prodanović, Michael J. Pyrcz

Abstract: Spatial nonstationarity, the location variance of features' statistical distributions, is ubiquitous in many natural settings. For example, in geological reservoirs rock matrix porosity varies vertically due to geomechanical compaction trends, in mineral deposits grades vary due to sedimentation and concentration processes, in hydrology rainfall varies due to the atmosphere and topography interact… ▽ More Spatial nonstationarity, the location variance of features' statistical distributions, is ubiquitous in many natural settings. For example, in geological reservoirs rock matrix porosity varies vertically due to geomechanical compaction trends, in mineral deposits grades vary due to sedimentation and concentration processes, in hydrology rainfall varies due to the atmosphere and topography interactions, and in metallurgy crystalline structures vary due to differential cooling. Conventional geostatistical modeling workflows rely on the assumption of stationarity to be able to model spatial features for the geostatistical inference. Nevertheless, this is often not a realistic assumption when dealing with nonstationary spatial data and this has motivated a variety of nonstationary spatial modeling workflows such as trend and residual decomposition, cosimulation with secondary features, and spatial segmentation and independent modeling over stationary subdomains. The advent of deep learning technologies has enabled new workflows for modeling spatial relationships. However, there is a paucity of demonstrated best practice and general guidance on mitigation of spatial nonstationarity with deep learning in the geospatial context. We demonstrate the impact of two common types of geostatistical spatial nonstationarity on deep learning model prediction performance and propose the mitigation of such impacts using self-attention (vision transformer) models. We demonstrate the utility of vision transformers for the mitigation of nonstationarity with relative errors as low as 10%, exceeding the performance of alternative deep learning methods such as convolutional neural networks. We establish best practice by demonstrating the ability of self-attention networks for modeling large-scale spatial relationships in the presence of commonly observed geospatial nonstationarity. △ Less

Submitted 8 December, 2022; originally announced December 2022.

Programming Is Hard -- Or at Least It Used to Be: Educational Opportunities And Challenges of AI Code Generation

Authors: Brett A. Becker, Paul Denny, James Finnie-Ansley, Andrew Luxton-Reilly, James Prather, Eddie Antonio Santos

Abstract: The introductory programming sequence has been the focus of much research in computing education. The recent advent of several viable and freely-available AI-driven code generation tools present several immediate opportunities and challenges in this domain. In this position paper we argue that the community needs to act quickly in deciding what possible opportunities can and should be leveraged an… ▽ More The introductory programming sequence has been the focus of much research in computing education. The recent advent of several viable and freely-available AI-driven code generation tools present several immediate opportunities and challenges in this domain. In this position paper we argue that the community needs to act quickly in deciding what possible opportunities can and should be leveraged and how, while also working on how to overcome or otherwise mitigate the possible challenges. Assuming that the effectiveness and proliferation of these tools will continue to progress rapidly, without quick, deliberate, and concerted efforts, educators will lose advantage in helping shape what opportunities come to be, and what challenges will endure. With this paper we aim to seed this discussion within the computing education community. △ Less

Submitted 2 December, 2022; originally announced December 2022.

Comments: 7 pages

ACM Class: K.3.2; I.2; H.5.2

An agent-based approach to procedural city generation incorporating Land Use and Transport Interaction models

Authors: Luiz Fernando Silva Eugênio dos Santos, Claus Aranha, André Ponce de Leon F de Carvalho

Abstract: We apply the knowledge of urban settings established with the study of Land Use and Transport Interaction (LUTI) models to develop reward functions for an agent-based system capable of planning realistic artificial cities. The system aims to replicate in the micro scale the main components of real settlements, such as zoning and accessibility in a road network. Moreover, we propose a novel represe… ▽ More We apply the knowledge of urban settings established with the study of Land Use and Transport Interaction (LUTI) models to develop reward functions for an agent-based system capable of planning realistic artificial cities. The system aims to replicate in the micro scale the main components of real settlements, such as zoning and accessibility in a road network. Moreover, we propose a novel representation for the agent's environment that efficiently combines the road graph with a discrete model for the land. Our system starts from an empty map consisting only of the road network graph, and the agent incrementally expands it by building new sites while distinguishing land uses between residential, commercial, industrial, and recreational. △ Less

Submitted 21 October, 2022; originally announced November 2022.

Comments: 12 pages, 6 figures, XIX Encontro Nacional de Inteligência Artificial e Computacional (ENIAC 2022)

Predictive Scale-Bridging Simulations through Active Learning

Authors: Satish Karra, Mohamed Mehana, Nicholas Lubbers, Yu Chen, Abdourahmane Diaw, Javier E. Santos, Aleksandra Pachalieva, Robert S. Pavel, Jeffrey R. Haack, Michael McKerns, Christoph Junghans, Qinjun Kang, Daniel Livescu, Timothy C. Germann, Hari S. Viswanathan

Abstract: Throughout computational science, there is a growing need to utilize the continual improvements in raw computational horsepower to achieve greater physical fidelity through scale-bridging over brute-force increases in the number of mesh elements. For instance, quantitative predictions of transport in nanoporous media, critical to hydrocarbon extraction from tight shale formations, are impossible w… ▽ More Throughout computational science, there is a growing need to utilize the continual improvements in raw computational horsepower to achieve greater physical fidelity through scale-bridging over brute-force increases in the number of mesh elements. For instance, quantitative predictions of transport in nanoporous media, critical to hydrocarbon extraction from tight shale formations, are impossible without accounting for molecular-level interactions. Similarly, inertial confinement fusion simulations rely on numerical diffusion to simulate molecular effects such as non-local transport and mixing without truly accounting for molecular interactions. With these two disparate applications in mind, we develop a novel capability which uses an active learning approach to optimize the use of local fine-scale simulations for informing coarse-scale hydrodynamics. Our approach addresses three challenges: forecasting continuum coarse-scale trajectory to speculatively execute new fine-scale molecular dynamics calculations, dynamically updating coarse-scale from fine-scale calculations, and quantifying uncertainty in neural network models. △ Less

Submitted 20 September, 2022; originally announced September 2022.

Journal ref: Sci. Rep. 13, 16262 (2023)

Bag of Tricks for Long-Tail Visual Recognition of Animal Species in Camera-Trap Images

Authors: Fagner Cunha, Eulanda M. dos Santos, Juan G. Colonna

Abstract: Camera traps are a method for monitoring wildlife and they collect a large number of pictures. The number of images collected of each species usually follows a long-tail distribution, i.e., a few classes have a large number of instances, while a lot of species have just a small percentage. Although in most cases these rare species are the ones of interest to ecologists, they are often neglected wh… ▽ More Camera traps are a method for monitoring wildlife and they collect a large number of pictures. The number of images collected of each species usually follows a long-tail distribution, i.e., a few classes have a large number of instances, while a lot of species have just a small percentage. Although in most cases these rare species are the ones of interest to ecologists, they are often neglected when using deep-learning models because these models require a large number of images for the training. In this work, a simple and effective framework called Square-Root Sampling Branch (SSB) is proposed, which combines two classification branches that are trained using square-root sampling and instance sampling to improve long-tail visual recognition, and this is compared to state-of-the-art methods for handling this task: square-root sampling, class-balanced focal loss, and balanced group softmax. To achieve a more general conclusion, the methods for handling long-tail visual recognition were systematically evaluated in four families of computer vision models (ResNet, MobileNetV3, EfficientNetV2, and Swin Transformer) and four camera-trap datasets with different characteristics. Initially, a robust baseline with the most recent training tricks was prepared and, then, the methods for improving long-tail recognition were applied. Our experiments show that square-root sampling was the method that most improved the performance for minority classes by around 15%; however, this was at the cost of reducing the majority classes' accuracy by at least 3%. Our proposed framework (SSB) demonstrated itself to be competitive with the other methods and achieved the best or the second-best results for most of the cases for the tail classes; but, unlike the square-root sampling, the loss in the performance of the head classes was minimal, thus achieving the best trade-off among all the evaluated methods. △ Less

Submitted 6 March, 2023; v1 submitted 24 June, 2022; originally announced June 2022.

Roadblocks to Attracting Students to Software Testing Careers: Comparisons of Replicated Studies

Authors: Rodrigo E. C. Souza, Ronnie E. de Souza Santos, Luiz Fernando Capretz, Marlon A. S. de Sousa, Cleyton V. C. de Magalhaes

Abstract: Context. Recently, a family of studies highlighted the unpopularity of software testing careers among undergraduate students in software engineering and computer science courses. The original study and its replications explored the perception of students in universities in four countries (Cana-da, China, India, and Malaysia), and indicated that most students do not consider a career in software te… ▽ More Context. Recently, a family of studies highlighted the unpopularity of software testing careers among undergraduate students in software engineering and computer science courses. The original study and its replications explored the perception of students in universities in four countries (Cana-da, China, India, and Malaysia), and indicated that most students do not consider a career in software testing as an option after graduation. This scenario represents a problem for the software industry since the lack of skilled testing professionals might decrease the quality of software projects and increase the number of unsuccessful projects. Goal. The present study aims to replicate, in Brazil, the studies conducted in the other four countries to establish comparisons and support the development of strategies to improve the visibility and importance of software testing among undergraduate students across the globe. Method. We followed the same protocol in the original study to collect data using a questionnaire and analyzed the answers using descriptive statistics and qualitative data analysis. Results. Our findings indicate similarities among the results obtained in Brazil in comparison to those obtained from other countries. We observed that students are not motivated to follow a testing career in the software industry based on a belief that testing activities lack challenges and opportunities for continuous learning. Conclusions. In summary, students seem to be interested in learning more about software testing. However, the lack of discussions about the theme in software development courses, as well as the limited offer of courses focused on software quality at the university level reduce the visibility of this area, which causes a decrease in the interest in this career. △ Less

Submitted 15 June, 2022; originally announced June 2022.

The way we cite: common metadata used across disciplines for defining bibliographic references

Authors: Erika Alves dos Santos, Silvio Peroni, Marcos Luiz Mucheroni

Abstract: Current citation practices observed in articles are very noisy, confusing, and not standardised, making identifying the cited works problematic for hu-mans and any reference extraction software. In this work, we want to investigate such citation practices for referencing different types of entities and, in particular, to understand the most used metadata in bibliographic refer-ences. We identified… ▽ More Current citation practices observed in articles are very noisy, confusing, and not standardised, making identifying the cited works problematic for hu-mans and any reference extraction software. In this work, we want to investigate such citation practices for referencing different types of entities and, in particular, to understand the most used metadata in bibliographic refer-ences. We identified 36 types of cited entities (the most cited ones were articles, books, and proceeding papers) within the 34,140 bibliographic references extracted from a vast set of journal articles on 27 different subject ar-eas. The analysis of such bibliographic references, grouped by the particular type of cited entities, enabled us to highlight the most used metadata for de-fining bibliographic references across the subject areas. However, we also noticed that, in some cases, bibliographic references did not provide the essential elements to identify the work they refer to easily. △ Less

Submitted 21 July, 2022; v1 submitted 26 May, 2022; originally announced May 2022.

Practices to Improve Teamwork in Software Development During the COVID-19 Pandemic: An Ethnographic Study

Authors: Ronnie E. de Souza Santos, Paul Ralph

Abstract: Context. Due to the COVID-19 pandemic, software professionals had to abruptly shift to ostensibly temporary home offices, which affected teamwork in several ways. Goal. This study aims to explore how these professionals coped with remote work during the pandemic and to identify practices that supported the team activities. Method. Ethnographic methods, including participant observation and qualita… ▽ More Context. Due to the COVID-19 pandemic, software professionals had to abruptly shift to ostensibly temporary home offices, which affected teamwork in several ways. Goal. This study aims to explore how these professionals coped with remote work during the pandemic and to identify practices that supported the team activities. Method. Ethnographic methods, including participant observation and qualitative data analysis, were used. Results. Three practices were created by the observed team to improve their engagement: costume meeting, second-language day, and project happy hour. These practices appear to increase individual involvement, improve team cohesion, reduce monotony, and create opportunities for knowledge acquisition. Conclusions. The three observed practices may help remote software teams cope with adversity. More research is needed to determine if these practices work in other settings, including remote-first and hybrid remote / on-site teams, post-pandemic. △ Less

Submitted 17 March, 2022; originally announced March 2022.

A Grounded Theory of Coordination in Remote-First and Hybrid Software Teams

Authors: Ronnie E. de Souza Santos, Paul Ralph

Abstract: While the long-term effects of the COVID-19 pandemic on software professionals and organizations are difficult to predict, it seems likely that working from home, remote-first teams, distributed teams, and hybrid (part-remote/part-office) teams will be more common. It is therefore important to investigate the challenges that software teams and organizations face with new remote and hybrid work. Co… ▽ More While the long-term effects of the COVID-19 pandemic on software professionals and organizations are difficult to predict, it seems likely that working from home, remote-first teams, distributed teams, and hybrid (part-remote/part-office) teams will be more common. It is therefore important to investigate the challenges that software teams and organizations face with new remote and hybrid work. Consequently, this paper reports a year-long, participant-observation, constructivist grounded theory study investigating the impact of working from home on software development. This study resulted in a theory of software team coordination. Briefly, shifting from in-office to at-home work fundamentally altered coordination within software teams. While group cohesion and more effective communication appear protective, coordination is undermined by distrust, parenting and communication bricolage. Poor coordination leads to numerous problems including misunderstandings, help requests, lower job satisfaction among team members, and more ill-defined tasks. These problems, in turn, reduce overall project success and prompt professionals to alter their software development processes (in this case, from Scrum to Kanban). Our findings suggest that software organizations with many remote employees can improve performance by encouraging greater engagement within teams and supporting employees with family and childcare responsibilities. △ Less

Submitted 23 February, 2022; v1 submitted 21 February, 2022; originally announced February 2022.

An analysis of citing and referencing habits across all scholarly disciplines: approaches and trends in bibliographic referencing and citing practices

Authors: Erika Alves dos Santos, Silvio Peroni, Marcos Luiz Mucheroni

Abstract: Purpose. In this study, we want to identify current possible causes for citing and referencing errors in scholarly literature to compare if something changed from the snapshot provided Sweetland in his 1989 paper. Design/methodology/approach. We analysed reference elements, i.e. bibliographic references, mentions, quotations, and respective in-text reference pointers, from 729 articles published i… ▽ More Purpose. In this study, we want to identify current possible causes for citing and referencing errors in scholarly literature to compare if something changed from the snapshot provided Sweetland in his 1989 paper. Design/methodology/approach. We analysed reference elements, i.e. bibliographic references, mentions, quotations, and respective in-text reference pointers, from 729 articles published in 147 journals across the 27 subject areas. Findings. The outcomes of our analysis pointed out that bibliographic errors have been perpetuated for decades and that their possible causes have increased, despite the encouraged use of technological facilities, i.e., the reference managers. Originality. As far as we know, our study is the best recent available analysis of errors in referencing and citing practices in the literature since Sweetland (1989). △ Less

Submitted 10 June, 2023; v1 submitted 17 February, 2022; originally announced February 2022.

Bucket-brigade inspired power line network protocol for sensed quantity profile acquisition with smart sensors deployed as a queue in harsh environment

Authors: Edval J. P. Santos

Abstract: Pressure and temperature profile are key data for safe production in oil and gas wells. In this paper, a bucket-brigade inspired sensor network protocol is proposed which can be used to extract sensed data profile from the nanoscale up to kilometer long structures. The PHY/MAC layers are discussed. This protocol is best suited for low data rate exchanges in small fixed-size packets, named buckets,… ▽ More Pressure and temperature profile are key data for safe production in oil and gas wells. In this paper, a bucket-brigade inspired sensor network protocol is proposed which can be used to extract sensed data profile from the nanoscale up to kilometer long structures. The PHY/MAC layers are discussed. This protocol is best suited for low data rate exchanges in small fixed-size packets, named buckets, transmitted as time-domain bursts among high-precision smart sensors deployed as a queue. There is only one coordinator, which is not directly accessible by most of the sensor nodes. The coordinator is responsible for collecting the measurement profile and send it to a supervisory node. There is no need for complex routing mechanism, as the network topology is determined during deployment. There are many applications which require sensors to be deployed as a long queue and sensed data could be transmitted at low data rates. Examples of such monitoring applications are: neural connected artificial skin, oil/gas/water pipeline integrity, power transmission line tower integrity, (rail)road/highway lighting and integrity, individualized monitoring in vineyard or re-foresting or plantation, underwater telecommunications cable integrity, oil/gas riser integrity, oil/gas well temperature and pressure profile, among others. For robustness and reduced electromagnetic interference, wired network is preferred. Besides in some harsh environment wireless is not feasible. To reduce wiring, communications can be carried out in the same cable used to supply electrical power. △ Less

Submitted 11 June, 2021; originally announced June 2021.

Comments: 19 pages, 13 Figures

Uma implementação do jogo Pedra, Papel e Tesoura utilizando Visao Computacional

Authors: Ezequiel França dos Santos, Gabriel Fontenelle

Abstract: This paper presents a game, controlled by computer vision, in identification of hand gestures (hand-tracking). The proposed work is based on image segmentation and construction of a convex hull with Jarvis Algorithm , and determination of the pattern based on the extraction of area characteristics in the convex hull. This paper presents a game, controlled by computer vision, in identification of hand gestures (hand-tracking). The proposed work is based on image segmentation and construction of a convex hull with Jarvis Algorithm , and determination of the pattern based on the extraction of area characteristics in the convex hull. △ Less

Submitted 20 May, 2021; originally announced May 2021.

Comments: 14 pages, in Portuguese

Monitoring electrical systems data-network equipment by means of Fuzzy and Paraconsistent Annotated Logic

Authors: Hyghor Miranda Cortes, Paulo Eduardo Santos, Joao Inacio da Silva Filho

Abstract: The constant increase in the amount and complexity of information obtained from IT data networkelements, for its correct monitoring and management, is a reality. The same happens to data net-works in electrical systems that provide effective supervision and control of substations and hydro-electric plants. Contributing to this fact is the growing number of installations and new environmentsmonitor… ▽ More The constant increase in the amount and complexity of information obtained from IT data networkelements, for its correct monitoring and management, is a reality. The same happens to data net-works in electrical systems that provide effective supervision and control of substations and hydro-electric plants. Contributing to this fact is the growing number of installations and new environmentsmonitored by such data networks and the constant evolution of the technologies involved. This sit-uation potentially leads to incomplete and/or contradictory data, issues that must be addressed inorder to maintain a good level of monitoring and, consequently, management of these systems. Inthis paper, a prototype of an expert system is developed to monitor the status of equipment of datanetworks in electrical systems, which deals with inconsistencies without trivialising the inferences.This is accomplished in the context of the remote control of hydroelectric plants and substationsby a Regional Operation Centre (ROC). The expert system is developed with algorithms definedupon a combination of Fuzzy logic and Paraconsistent Annotated Logic with Annotation of TwoValues (PAL2v) in order to analyse uncertain signals and generate the operating conditions (faulty,normal, unstable or inconsistent / indeterminate) of the equipment that are identified as importantfor the remote control of hydroelectric plants and substations. A prototype of this expert systemwas installed on a virtualised server with CLP500 software (from the EFACEC manufacturer) thatwas applied to investigate scenarios consisting of a Regional (Brazilian) Operation Centre, with aGeneric Substation and a Generic Hydroelectric Plant, representing a remote control environment. △ Less

Submitted 23 May, 2021; v1 submitted 16 May, 2021; originally announced May 2021.

Comments: 38 pages; 14 figures; Under submission

Filtering Empty Camera Trap Images in Embedded Systems

Authors: Fagner Cunha, Eulanda M. dos Santos, Raimundo Barreto, Juan G. Colonna

Abstract: Monitoring wildlife through camera traps produces a massive amount of images, whose a significant portion does not contain animals, being later discarded. Embedding deep learning models to identify animals and filter these images directly in those devices brings advantages such as savings in the storage and transmission of data, usually resource-constrained in this type of equipment. In this work,… ▽ More Monitoring wildlife through camera traps produces a massive amount of images, whose a significant portion does not contain animals, being later discarded. Embedding deep learning models to identify animals and filter these images directly in those devices brings advantages such as savings in the storage and transmission of data, usually resource-constrained in this type of equipment. In this work, we present a comparative study on animal recognition models to analyze the trade-off between precision and inference latency on edge devices. To accomplish this objective, we investigate classifiers and object detectors of various input resolutions and optimize them using quantization and reducing the number of model filters. The confidence threshold of each model was adjusted to obtain 96% recall for the nonempty class, since instances from the empty class are expected to be discarded. The experiments show that, when using the same set of images for training, detectors achieve superior performance, eliminating at least 10% more empty images than classifiers with comparable latencies. Considering the high cost of generating labels for the detection problem, when there is a massive number of images labeled for classification (about one million instances, ten times more than those available for detection), classifiers are able to reach results comparable to detectors but with half latency. △ Less

Submitted 18 April, 2021; originally announced April 2021.

Comments: Accepted to CVPR 2021 (Mobile AI workshop and challenges)

Discriminative Singular Spectrum Classifier with Applications on Bioacoustic Signal Recognition

Authors: Bernardo B. Gatto, Juan G. Colonna, Eulanda M. dos Santos, Alessandro L. Koerich, Kazuhiro Fukui

Abstract: Automatic analysis of bioacoustic signals is a fundamental tool to evaluate the vitality of our planet. Frogs and bees, for instance, may act like biological sensors providing information about environmental changes. This task is fundamental for ecological monitoring still includes many challenges such as nonuniform signal length processing, degraded target signal due to environmental noise, and t… ▽ More Automatic analysis of bioacoustic signals is a fundamental tool to evaluate the vitality of our planet. Frogs and bees, for instance, may act like biological sensors providing information about environmental changes. This task is fundamental for ecological monitoring still includes many challenges such as nonuniform signal length processing, degraded target signal due to environmental noise, and the scarcity of the labeled samples for training machine learning. To tackle these challenges, we present a bioacoustic signal classifier equipped with a discriminative mechanism to extract useful features for analysis and classification efficiently. The proposed classifier does not require a large amount of training data and handles nonuniform signal length natively. Unlike current bioacoustic recognition methods, which are task-oriented, the proposed model relies on transforming the input signals into vector subspaces generated by applying Singular Spectrum Analysis (SSA). Then, a subspace is designed to expose discriminative features. The proposed model shares end-to-end capabilities, which is desirable in modern machine learning systems. This formulation provides a segmentation-free and noise-tolerant approach to represent and classify bioacoustic signals and a highly compact signal descriptor inherited from SSA. The validity of the proposed method is verified using three challenging bioacoustic datasets containing anuran, bee, and mosquito species. Experimental results on three bioacoustic datasets have shown the competitive performance of the proposed method compared to commonly employed methods for bioacoustics signal classification in terms of accuracy. △ Less

Submitted 18 March, 2021; originally announced March 2021.

Comments: 15 pages

MudrockNet: Semantic Segmentation of Mudrock SEM Images through Deep Learning

Authors: Abhishek Bihani, Hugh Daigle, Javier E. Santos, Christopher Landry, Masa Prodanovic, Kitty Milliken

Abstract: Segmentation and analysis of individual pores and grains of mudrocks from scanning electron microscope images is non-trivial because of noise, imaging artifacts, variation in pixel grayscale values across images, and overlaps in grayscale values among different physical features such as silt grains, clay grains, and pores in an image, which make their identification difficult. Moreover, because gr… ▽ More Segmentation and analysis of individual pores and grains of mudrocks from scanning electron microscope images is non-trivial because of noise, imaging artifacts, variation in pixel grayscale values across images, and overlaps in grayscale values among different physical features such as silt grains, clay grains, and pores in an image, which make their identification difficult. Moreover, because grains and pores often have overlapping grayscale values, direct application of threshold-based segmentation techniques is not sufficient. Recent advances in the field of computer vision have made it easier and faster to segment images and identify multiple occurrences of such features in an image, provided that ground-truth data for training the algorithm is available. Here, we propose a deep learning SEM image segmentation model, MudrockNet based on Google's DeepLab-v3+ architecture implemented with the TensorFlow library. The ground-truth data was obtained from an image-processing workflow applied to scanning electron microscope images of uncemented muds from the Kumano Basin offshore Japan at depths < 1.1 km. The trained deep learning model obtained a pixel-accuracy about 90%, and predictions for the test data obtained a mean intersection over union (IoU) of 0.6591 for silt grains and 0.6642 for pores. We also compared our model with the random forest classifier using trainable Weka segmentation in ImageJ, and it was observed that MudrockNet gave better predictions for both silt grains and pores. The size, concentration, and spatial arrangement of the silt and clay grains can affect the petrophysical properties of a mudrock, and an automated method to accurately identify the different grains and pores in mudrocks can help improve reservoir and seal characterization for petroleum exploration and anthropogenic waste sequestration. △ Less

Submitted 5 February, 2021; originally announced February 2021.

Comments: 24 pages, 8 figures, submitted to Computers and Geosciences

ACM Class: I.4.6; I.4.3

Guided Navigation from Multiple Viewpoints using Qualitative Spatial Reasoning

Authors: Danilo Perico, Paulo E. Santos, Reinaldo Bianchi

Abstract: Navigation is an essential ability for mobile agents to be completely autonomous and able to perform complex actions. However, the problem of navigation for agents with limited (or no) perception of the world, or devoid of a fully defined motion model, has received little attention from research in AI and Robotics. One way to tackle this problem is to use guided navigation, in which other autonomo… ▽ More Navigation is an essential ability for mobile agents to be completely autonomous and able to perform complex actions. However, the problem of navigation for agents with limited (or no) perception of the world, or devoid of a fully defined motion model, has received little attention from research in AI and Robotics. One way to tackle this problem is to use guided navigation, in which other autonomous agents, endowed with perception, can combine their distinct viewpoints to infer the localisation and the appropriate commands to guide a sensory deprived agent through a particular path. Due to the limited knowledge about the physical and perceptual characteristics of the guided agent, this task should be conducted on a level of abstraction allowing the use of a generic motion model, and high-level commands, that can be applied by any type of autonomous agents, including humans. The main task considered in this work is, given a group of autonomous agents perceiving their common environment with their independent, egocentric and local vision sensors, the development and evaluation of algorithms capable of producing a set of high-level commands (involving qualitative directions: e.g. move left, go straight ahead) capable of guiding a sensory deprived robot to a goal location. △ Less

Submitted 2 November, 2020; originally announced November 2020.

Comments: 26 pages

Journal ref: Spatial Cognition and Computation - 2020

Deep Learning for Distinguishing Normal versus Abnormal Chest Radiographs and Generalization to Unseen Diseases

Authors: Zaid Nabulsi, Andrew Sellergren, Shahar Jamshy, Charles Lau, Edward Santos, Atilla P. Kiraly, Wenxing Ye, Jie Yang, Rory Pilgrim, Sahar Kazemzadeh, Jin Yu, Sreenivasa Raju Kalidindi, Mozziyar Etemadi, Florencia Garcia-Vicente, David Melnick, Greg S. Corrado, Lily Peng, Krish Eswaran, Daniel Tse, Neeral Beladia, Yun Liu, Po-Hsuan Cameron Chen, Shravya Shetty

Abstract: Chest radiography (CXR) is the most widely-used thoracic clinical imaging modality and is crucial for guiding the management of cardiothoracic conditions. The detection of specific CXR findings has been the main focus of several artificial intelligence (AI) systems. However, the wide range of possible CXR abnormalities makes it impractical to build specific systems to detect every possible conditi… ▽ More Chest radiography (CXR) is the most widely-used thoracic clinical imaging modality and is crucial for guiding the management of cardiothoracic conditions. The detection of specific CXR findings has been the main focus of several artificial intelligence (AI) systems. However, the wide range of possible CXR abnormalities makes it impractical to build specific systems to detect every possible condition. In this work, we developed and evaluated an AI system to classify CXRs as normal or abnormal. For development, we used a de-identified dataset of 248,445 patients from a multi-city hospital network in India. To assess generalizability, we evaluated our system using 6 international datasets from India, China, and the United States. Of these datasets, 4 focused on diseases that the AI was not trained to detect: 2 datasets with tuberculosis and 2 datasets with coronavirus disease 2019. Our results suggest that the AI system generalizes to new patient populations and abnormalities. In a simulated workflow where the AI system prioritized abnormal cases, the turnaround time for abnormal cases reduced by 7-28%. These results represent an important step towards evaluating whether AI can be safely used to flag cases in a general setting where previously unseen abnormalities exist. △ Less

Submitted 29 October, 2021; v1 submitted 21 October, 2020; originally announced October 2020.

Journal ref: Nature Scientific Reports (2021)

CAPTION: Correction by Analyses, POS-Tagging and Interpretation of Objects using only Nouns

Authors: Leonardo Anjoletto Ferreira, Douglas De Rizzo Meneghetti, Paulo Eduardo Santos

Abstract: Recently, Deep Learning (DL) methods have shown an excellent performance in image captioning and visual question answering. However, despite their performance, DL methods do not learn the semantics of the words that are being used to describe a scene, making it difficult to spot incorrect words used in captions or to interchange words that have similar meanings. This work proposes a combination of… ▽ More Recently, Deep Learning (DL) methods have shown an excellent performance in image captioning and visual question answering. However, despite their performance, DL methods do not learn the semantics of the words that are being used to describe a scene, making it difficult to spot incorrect words used in captions or to interchange words that have similar meanings. This work proposes a combination of DL methods for object detection and natural language processing to validate image's captions. We test our method in the FOIL-COCO data set, since it provides correct and incorrect captions for various images using only objects represented in the MS-COCO image data set. Results show that our method has a good overall performance, in some cases similar to the human performance. △ Less

Submitted 2 October, 2020; originally announced October 2020.

Comments: Published at the First Annual International Workshop on Interpretability: Methodologies and algorithms (IMA 2019)

Citing and referencing habits in Medicine and Social Sciences journals in 2019

Authors: Erika Alves dos Santos, Silvio Peroni, Marcos Luiz Mucheroni

Abstract: This article explores citing and referencing systems in Social Sciences and Medicine articles from different theoretical and practical perspectives, considering bibliographic references as a facet of descriptive representation. The analysis of citing and referencing elements (i.e. bibliographic references, mentions, quotations, and respective in-text reference pointers) identified citing and refer… ▽ More This article explores citing and referencing systems in Social Sciences and Medicine articles from different theoretical and practical perspectives, considering bibliographic references as a facet of descriptive representation. The analysis of citing and referencing elements (i.e. bibliographic references, mentions, quotations, and respective in-text reference pointers) identified citing and referencing habits within disciplines under consideration and errors occurring over the long term as stated by previous studies now expanded. Future expected trends of information retrieval from bibliographic metadata was gathered by approaching these referencing elements from the FRBR Entities concepts. Reference styles do not fully accomplish with their role of guiding authors and publishers on providing concise and well-structured bibliographic metadata within bibliographic references. Trends on representative description revision suggest a predicted distancing on the ways information is approached by bibliographic references and bibliographic catalogs adopting FRBR concepts, including the description levels adopted by each of them under the perspective of the FRBR Entities concept. This study was based on a subset of Medicine and Social Sciences articles published in 2019 and, therefore, it may not be taken as a final and broad coverage. Future studies expanding these approaches to other disciplines and chronological periods are encouraged. By approaching citing and referencing issues as descriptive representation's facets, findings on this study may encourage further studies that will support Information Science and Computer Science on providing tools to become bibliographic metadata description simpler, better structured and more efficient facing the revision of descriptive representation actually in progress. △ Less

Submitted 20 January, 2021; v1 submitted 11 September, 2020; originally announced September 2020.

Comments: Accepted for publication on 18 January 2021 in Journal of Documentation

Modeling nanoconfinement effects using active learning

Authors: Javier E. Santos, Mohammed Mehana, Hao Wu, Masa Prodanovic, Michael J. Pyrcz, Qinjun Kang, Nicholas Lubbers, Hari Viswanathan

Abstract: Predicting the spatial configuration of gas molecules in nanopores of shale formations is crucial for fluid flow forecasting and hydrocarbon reserves estimation. The key challenge in these tight formations is that the majority of the pore sizes are less than 50 nm. At this scale, the fluid properties are affected by nanoconfinement effects due to the increased fluid-solid interactions. For instanc… ▽ More Predicting the spatial configuration of gas molecules in nanopores of shale formations is crucial for fluid flow forecasting and hydrocarbon reserves estimation. The key challenge in these tight formations is that the majority of the pore sizes are less than 50 nm. At this scale, the fluid properties are affected by nanoconfinement effects due to the increased fluid-solid interactions. For instance, gas adsorption to the pore walls could account for up to 85% of the total hydrocarbon volume in a tight reservoir. Although there are analytical solutions that describe this phenomenon for simple geometries, they are not suitable for describing realistic pores, where surface roughness and geometric anisotropy play important roles. To describe these, molecular dynamics (MD) simulations are used since they consider fluid-solid and fluid-fluid interactions at the molecular level. However, MD simulations are computationally expensive, and are not able to simulate scales larger than a few connected nanopores. We present a method for building and training physics-based deep learning surrogate models to carry out fast and accurate predictions of molecular configurations of gas inside nanopores. Since training deep learning models requires extensive databases that are computationally expensive to create, we employ active learning (AL). AL reduces the overhead of creating comprehensive sets of high-fidelity data by determining where the model uncertainty is greatest, and running simulations on the fly to minimize it. The proposed workflow enables nanoconfinement effects to be rigorously considered at the mesoscale where complex connected sets of nanopores control key applications such as hydrocarbon recovery and CO2 sequestration. △ Less

Submitted 6 May, 2020; v1 submitted 6 May, 2020; originally announced May 2020.

Comments: Full paper

Journal ref: J. Phys. Chem. C 2020

Hardware Architecture Proposal for TEDA algorithm to Data Streaming Anomaly Detection

Authors: Lucileide M. D. da Silva, Maria G. F. Coutinho, Carlos E. B. Santos, Mailson R. Santos, Luiz Affonso Guedes, M. Dolores Ruiz, Marcelo A. C. Fernandes

Abstract: The amount of data in real-time, such as time series and streaming data, available today continues to grow. Being able to analyze this data the moment it arrives can bring an immense added value. However, it also requires a lot of computational effort and new acceleration techniques. As a possible solution to this problem, this paper proposes a hardware architecture for Typicality and Eccentricity… ▽ More The amount of data in real-time, such as time series and streaming data, available today continues to grow. Being able to analyze this data the moment it arrives can bring an immense added value. However, it also requires a lot of computational effort and new acceleration techniques. As a possible solution to this problem, this paper proposes a hardware architecture for Typicality and Eccentricity Data Analytic (TEDA) algorithm implemented on Field Programmable Gate Arrays (FPGA) for use in data streaming anomaly detection. TEDA is based on a new approach to outlier detection in the data stream context. In order to validate the proposals, results of the occupation and throughput of the proposed hardware are presented. Besides, the bit accurate simulation results are also presented. The project aims to Xilinx Virtex-6 xc6vlx240t-1ff1156 as the target FPGA. △ Less

Submitted 8 March, 2020; originally announced March 2020.

Decentralized Validation for Non-malicious Arbitrary Fault Tolerance in Paxos

Authors: Rodrigo R. Barbieri, Enrique S. dos Santos, Gustavo M. D. Vieira

Abstract: Fault-tolerant distributed systems offer high reliability because even if faults in their components occur, they do not exhibit erroneous behavior. Depending on the fault model adopted, hardware and software errors that do not result in a process crashing are usually not tolerated. To tolerate these rather common failures the usual solution is to adopt a stronger fault model, such as the arbitrary… ▽ More Fault-tolerant distributed systems offer high reliability because even if faults in their components occur, they do not exhibit erroneous behavior. Depending on the fault model adopted, hardware and software errors that do not result in a process crashing are usually not tolerated. To tolerate these rather common failures the usual solution is to adopt a stronger fault model, such as the arbitrary or Byzantine fault model. Algorithms created for this fault model, however, are considerably more complex and require more system resources than the ones developed for less strict fault models. One approach to reach a middle ground is the non-malicious arbitrary fault model. This model assumes it is possible to detect and filter faults with a given probability, if these faults are not created with malicious intent, allowing the isolation and mapping of these faults to benign faults. In this paper we describe how we incremented an implementation of active replication in the non-malicious fault model with a basic type of distributed validation, where a deviation from the expected algorithm behavior will make a process crash. We experimentally evaluate this implementation using a fault injection framework showing that it is feasible to extend the concept of non-malicious failures beyond hardware failures. △ Less

Submitted 18 February, 2020; originally announced February 2020.

Comments: 14 pages

Journal ref: WTF '19: Proceedings of the XX Fault Tolerance Workshop, SBC, 2019, 34-47

A Decision-Based Dynamic Ensemble Selection Method for Concept Drift

Authors: Regis Antonio Saraiva Albuquerque, Albert Franca Josua Costa, Eulanda Miranda dos Santos, Robert Sabourin, Rafael Giusti

Abstract: We propose an online method for concept driftdetection based on dynamic classifier ensemble selection. Theproposed method generates a pool of ensembles by promotingdiversity among classifier members and chooses expert ensemblesaccording to global prequential accuracy values. Unlike currentdynamic ensemble selection approaches that use only local knowl-edge to select the most competent ensemble for… ▽ More We propose an online method for concept driftdetection based on dynamic classifier ensemble selection. Theproposed method generates a pool of ensembles by promotingdiversity among classifier members and chooses expert ensemblesaccording to global prequential accuracy values. Unlike currentdynamic ensemble selection approaches that use only local knowl-edge to select the most competent ensemble for each instance,our method focuses on selection taking into account the decisionspace. Consequently, it is well adapted to the context of driftdetection in data stream problems. The results of the experimentsshow that the proposed method attained the highest detection pre-cision and the lowest number of false alarms, besides competitiveclassification accuracy rates, in artificial datasets representingdifferent types of drifts. Moreover, it outperformed baselines indifferent real-problem datasets in terms of classification accuracy. △ Less

Submitted 26 September, 2019; originally announced September 2019.

Tensor Analysis with n-Mode Generalized Difference Subspace

Authors: Bernardo B. Gatto, Eulanda M. dos Santos, Alessandro L. Koerich, Kazuhiro Fukui, Waldir S. S. Junior

Abstract: The increasing use of multiple sensors, which produce a large amount of multi-dimensional data, requires efficient representation and classification methods. In this paper, we present a new method for multi-dimensional data classification that relies on two premises: 1) multi-dimensional data are usually represented by tensors, since this brings benefits from multilinear algebra and established te… ▽ More The increasing use of multiple sensors, which produce a large amount of multi-dimensional data, requires efficient representation and classification methods. In this paper, we present a new method for multi-dimensional data classification that relies on two premises: 1) multi-dimensional data are usually represented by tensors, since this brings benefits from multilinear algebra and established tensor factorization methods; and 2) multilinear data can be described by a subspace of a vector space. The subspace representation has been employed for pattern-set recognition, and its tensor representation counterpart is also available in the literature. However, traditional methods do not use discriminative information of the tensors, degrading the classification accuracy. In this case, generalized difference subspace (GDS) provides an enhanced subspace representation by reducing data redundancy and revealing discriminative structures. Since GDS does not handle tensor data, we propose a new projection called n-mode GDS, which efficiently handles tensor data. We also introduce the n-mode Fisher score as a class separability index and an improved metric based on the geodesic distance for tensor data similarity. The experimental results on gesture and action recognition show that the proposed method outperforms methods commonly used in the literature without relying on pre-trained models or transfer learning. △ Less

Submitted 29 November, 2020; v1 submitted 4 September, 2019; originally announced September 2019.

Comments: Submitted to Expert Systems with Applications

Heuristics, Answer Set Programming and Markov Decision Process for Solving a Set of Spatial Puzzles

Authors: Thiago Freitas dos Santos, Paulo E. Santos, Leonardo A. Ferreira, Reinaldo A. C. Bianchi, Pedro Cabalar

Abstract: Spatial puzzles composed of rigid objects, flexible strings and holes offer interesting domains for reasoning about spatial entities that are common in the human daily-life's activities. The goal of this work is to investigate the automated solution of this kind of puzzles adapting an algorithm that combines Answer Set Programming (ASP) with Markov Decision Process (MDP), algorithm oASP(MDP), to u… ▽ More Spatial puzzles composed of rigid objects, flexible strings and holes offer interesting domains for reasoning about spatial entities that are common in the human daily-life's activities. The goal of this work is to investigate the automated solution of this kind of puzzles adapting an algorithm that combines Answer Set Programming (ASP) with Markov Decision Process (MDP), algorithm oASP(MDP), to use heuristics accelerating the learning process. ASP is applied to represent the domain as an MDP, while a Reinforcement Learning algorithm (Q-Learning) is used to find the optimal policies. In this work, the heuristics were obtained from the solution of relaxed versions of the puzzles. Experiments were performed on deterministic, non-deterministic and non-stationary versions of the puzzles. Results show that the proposed approach can accelerate the learning process, presenting an advantage when compared to the non-heuristic versions of oASP(MDP) and Q-Learning. △ Less

Submitted 15 February, 2019; originally announced March 2019.

Comments: Submitted to Journal of Heuristics

Lessons from Four Years of PHONELAB Experimentation

Authors: Jinghao Shi, Edwin Santos, Geoffrey Challen

Abstract: Over the last four years we have operated a public smartphone platform testbed called PHONELAB. PHONELAB consists of up to several-hundred participants who run an experimental platform image on their primary smartphone. The experimental platform consists of both instrumentation and experimental changes to platform components, including core Android services and Linux. This paper describes the desi… ▽ More Over the last four years we have operated a public smartphone platform testbed called PHONELAB. PHONELAB consists of up to several-hundred participants who run an experimental platform image on their primary smartphone. The experimental platform consists of both instrumentation and experimental changes to platform components, including core Android services and Linux. This paper describes the design of the testbed, the process of conducting PHONELAB experiments, and some of the research the testbed has supported. We also offer many lessons learned along the way, almost all of which have been learned the hard way--through trial and a lot of error. We expect our experiences will help those contemplating operating large user-facing testbeds, anyone conducting experiments on smartphones, and many mobile systems researchers. △ Less

Submitted 20 January, 2019; originally announced February 2019.

Distribution Power Network Reconfiguration in the Smart Grid

Authors: Eonassis O. Santos, Joberto S. B. Martins

Abstract: The power network reconfiguration algorithm with an "R" modeling approach evaluates its behavior in computing new reconfiguration topologies for the power grid in the context of the Smart Grid. The power distribution network modelling with the R language is used to represent the network and support computation of different algorithm configurations for the evaluation of new reconfiguration topologi… ▽ More The power network reconfiguration algorithm with an "R" modeling approach evaluates its behavior in computing new reconfiguration topologies for the power grid in the context of the Smart Grid. The power distribution network modelling with the R language is used to represent the network and support computation of different algorithm configurations for the evaluation of new reconfiguration topologies. This work presents a reconfiguration solution of distribution networks, with a construction of an algorithm that receiving the network configuration data and the nodal measurements and from these data build a radial network, after this and using a branch exchange algorithm And verifying the best configuration of the network through artificial intelligence, so that there are no unnecessary changes during the operation, and applied an algorithm that analyses the load levels, to suggest changes in the network. △ Less

Submitted 20 June, 2018; originally announced June 2018.

Comments: 13 pages

Journal ref: International Seminar on Politics, Incentives, Technology and Regulation of Smart Grids, Rio de Janeiro, 2017

A Formal Model to Facilitate Security Testing in Modern Automotive Systems

Authors: Eduardo dos Santos, Andrew Simpson, Dominik Schoop

Abstract: Ensuring a car's internal systems are free from security vulnerabilities is of utmost importance, especially due to the relationship between security and other properties, such as safety and reliability. We provide the starting point for a model-based framework designed to support the security testing of modern cars. We use Communicating Sequential Processes (CSP) to create architectural models of… ▽ More Ensuring a car's internal systems are free from security vulnerabilities is of utmost importance, especially due to the relationship between security and other properties, such as safety and reliability. We provide the starting point for a model-based framework designed to support the security testing of modern cars. We use Communicating Sequential Processes (CSP) to create architectural models of the vehicle bus systems, as well as an initial set of attacks against these systems. While this contribution represents initial steps, we are mindful of the ultimate objective of generating test code to exercise the security of vehicle bus systems. We present the way forward from the models created and consider their potential integration with commercial engineering tools △ Less

Submitted 14 May, 2018; originally announced May 2018.

Comments: In Proceedings IMPEX 2017 and FM&MDD 2017, arXiv:1805.04636

ACM Class: D.2.4; D.4.6

Journal ref: EPTCS 271, 2018, pp. 95-104