-
MLEM: Generative and Contrastive Learning as Distinct Modalities for Event Sequences
Authors:
Viktor Moskvoretskii,
Dmitry Osin,
Egor Shvetsov,
Igor Udovichenko,
Maxim Zhelnin,
Andrey Dukhovny,
Anna Zhimerikina,
Evgeny Burnaev
Abstract:
This study explores the application of self-supervised learning techniques for event sequences. It is a key modality in various applications such as banking, e-commerce, and healthcare. However, there is limited research on self-supervised learning for event sequences, and methods from other domains like images, texts, and speech may not easily transfer. To determine the most suitable approach, we…
▽ More
This study explores the application of self-supervised learning techniques for event sequences. It is a key modality in various applications such as banking, e-commerce, and healthcare. However, there is limited research on self-supervised learning for event sequences, and methods from other domains like images, texts, and speech may not easily transfer. To determine the most suitable approach, we conduct a detailed comparative analysis of previously identified best-performing methods. We find that neither the contrastive nor generative method is superior. Our assessment includes classifying event sequences, predicting the next event, and evaluating embedding quality. These results further highlight the potential benefits of combining both methods. Given the lack of research on hybrid models in this domain, we initially adapt the baseline model from another domain. However, upon observing its underperformance, we develop a novel method called the Multimodal-Learning Event Model (MLEM). MLEM treats contrastive learning and generative modeling as distinct yet complementary modalities, aligning their embeddings. The results of our study demonstrate that combining contrastive and generative approaches into one procedure with MLEM achieves superior performance across multiple metrics.
△ Less
Submitted 3 July, 2024; v1 submitted 29 January, 2024;
originally announced January 2024.
-
Maximum Relative Divergence Principle for Grading Functions on Direct Products of Chains
Authors:
Alexander Dukhovny
Abstract:
The concept of Shannon Entropy for probability distributions and associated Maximum Entropy Principle are extended here to the concepts of Relative Divergence of one Grading Function from another and Maximum Relative Divergence Principle for grading functions on direct products of totally ordered chains (chain bundles). Several Operations Research applications are analyzed.
The concept of Shannon Entropy for probability distributions and associated Maximum Entropy Principle are extended here to the concepts of Relative Divergence of one Grading Function from another and Maximum Relative Divergence Principle for grading functions on direct products of totally ordered chains (chain bundles). Several Operations Research applications are analyzed.
△ Less
Submitted 24 March, 2023;
originally announced March 2023.
-
Maximum Relative Divergence Principle for Grading Functions on Power Sets
Authors:
Alexander Dukhovny
Abstract:
The concept of Relative Divergence of one Grading Function from another is extended from totally ordered chains to power sets of finite event spaces. Shannon Entropy concept is extended to normalized grading functions on such power sets. Maximum Relative Divergence Principle is introduced as a generalization of the Maximum Entropy Principle as a tool for determining the "most reasonable" grading f…
▽ More
The concept of Relative Divergence of one Grading Function from another is extended from totally ordered chains to power sets of finite event spaces. Shannon Entropy concept is extended to normalized grading functions on such power sets. Maximum Relative Divergence Principle is introduced as a generalization of the Maximum Entropy Principle as a tool for determining the "most reasonable" grading function and used in some Operations Research applications where that function is supposed to be "element-additive" or "cardinality-dependent" under application-specific linear constraints.
△ Less
Submitted 14 July, 2022;
originally announced July 2022.
-
Axiomatic Origins of Mathematical Entropy: Grading Ordered Sets
Authors:
Alexander Dukhovny
Abstract:
Shannon's entropy and other entropy-based concepts are derived from the new, more general concept of relative divergence of one "grading' function on a linearly ordered set from another such function. The definition of relative divergence is derived based on "common sense' assumptions about comparing grading functions. Shannon's entropy formulas emerge from the respective relative divergence ones,…
▽ More
Shannon's entropy and other entropy-based concepts are derived from the new, more general concept of relative divergence of one "grading' function on a linearly ordered set from another such function. The definition of relative divergence is derived based on "common sense' assumptions about comparing grading functions. Shannon's entropy formulas emerge from the respective relative divergence ones, entropy based methods are extended to more general cases and some new applications.
△ Less
Submitted 12 March, 2019;
originally announced March 2019.
-
Reliability of systems with dependent components based on lattice polynomial description
Authors:
Alexander Dukhovny,
Jean-Luc Marichal
Abstract:
Reliability of a system is considered where the components' random lifetimes may be dependent. The structure of the system is described by an associated "lattice polynomial" function. Based on that descriptor, general framework formulas are developed and used to obtain direct results for the cases where a) the lifetimes are "Bayes-dependent", that is, their interdependence is due to external facto…
▽ More
Reliability of a system is considered where the components' random lifetimes may be dependent. The structure of the system is described by an associated "lattice polynomial" function. Based on that descriptor, general framework formulas are developed and used to obtain direct results for the cases where a) the lifetimes are "Bayes-dependent", that is, their interdependence is due to external factors (in particular, where the factor is the "preliminary phase" duration) and b) where the lifetimes' dependence is implied by upper or lower bounds on lifetimes of components in some subsets of the system. (The bounds may be imposed externally based, say, on the connections environment.) Several special cases are investigated in detail.
△ Less
Submitted 14 April, 2011;
originally announced April 2011.
-
Reliability analysis of semicoherent systems through their lattice polynomial descriptions
Authors:
Alexander Dukhovny,
Jean-Luc Marichal
Abstract:
A semicoherent system can be described by its structure function or, equivalently, by a lattice polynomial function expressing the system lifetime in terms of the component lifetimes. In this paper we point out the parallelism between the two descriptions and use the natural connection of lattice polynomial functions and relevant random events to collect exact formulas for the system reliability…
▽ More
A semicoherent system can be described by its structure function or, equivalently, by a lattice polynomial function expressing the system lifetime in terms of the component lifetimes. In this paper we point out the parallelism between the two descriptions and use the natural connection of lattice polynomial functions and relevant random events to collect exact formulas for the system reliability. We also discuss the equivalence between calculating the reliability of semicoherent systems and calculating the distribution function of a lattice polynomial function of random variables.
△ Less
Submitted 8 September, 2008;
originally announced September 2008.
-
System reliability and weighted lattice polynomials
Authors:
Alexander Dukhovny,
Jean-Luc Marichal
Abstract:
The lifetime of a system of connected units under some natural assumptions can be represented as a random variable Y defined as a weighted lattice polynomial of random lifetimes of its components. As such, the concept of a random variable Y defined by a weighted lattice polynomial of (lattice-valued) random variables is considered in general and in some special cases. The central object of inter…
▽ More
The lifetime of a system of connected units under some natural assumptions can be represented as a random variable Y defined as a weighted lattice polynomial of random lifetimes of its components. As such, the concept of a random variable Y defined by a weighted lattice polynomial of (lattice-valued) random variables is considered in general and in some special cases. The central object of interest is the cumulative distribution function of Y. In particular, numerous results are obtained for lattice polynomials and weighted lattice polynomials in case of independent arguments and in general. For the general case, the technique consists in considering the joint probability generating function of "indicator" variables. A connection is studied between Y and order statistics of the set of arguments.
△ Less
Submitted 28 May, 2008; v1 submitted 5 December, 2007;
originally announced December 2007.
