*\AtPageUpperLeft
A Demonstration of ARCANE Reweighting: Reducing the Sign Problem in the MC@NLO Generation of Events
Prasanth Shyamsundar1
1 Fermi National Accelerator Laboratory, Batavia, Illinois 60510, USA
Abstract
Negatively weighted events, which appear in the simulation of particle collisions, significantly increase the computational requirements of collider experiments. A new technique called ARCANE reweighting has been introduced in a companion paper to tackle this problem. This paper demonstrates the technique for the next-to-leading-order generation of events. By redistributing the contributions of “standard” and “hard remainder” pathways in the generator that lead to the same final event, ARCANE reweighting almost completely eliminates the negative weights problem for this process. Some thoughts on implementing the technique in other scenarios are provided.
Copyright attribution to authors.
This work is a submission to SciPost Physics.
License information to appear upon publication.
Publication information to appear upon publication.
Received Date
Accepted Date
Published Date
Contents
- 1 Introduction
-
2 Base Event Generator, Groundwork for ARCANE Reweighting
- 2.1 General Overview of the Generation Pipeline
- 2.2 Choosing a “Stopping Point” for Performing ARCANE Reweighting
- 2.3 Unresolved Event Kinematics
- 2.4 Emission Kinematics
- 2.5 Overview of the Parton Showering Procedure
- 2.6 Single Parton Shower Emission With the Veto Algorithm
- 2.7 Generation Pipeline Until the Chosen Stopping Point
- 2.8 List of Relevant Event-Attributes
- 3 Walkthrough of the ARCANE Reweighting Implementation
- 4 Experiments and Results
- 5 An Alternative Implementation: “Hard Remainder Spreading”
- 6 Summary and Conclusions, and Outlook
- Code and Data Availability
- A Some Extensions of the ARCANE Implementation in This Paper
1 Introduction
Negatively weighted events, which appear in the generation of collider events at next-to-leading-order (NLO) accuracy, e.g. using the MC@NLO formalism, significantly reduce the efficiency of collider simulations [1, 2]. The presence of negatively weighted events increases the total number of simulated events needed to attain specific target precisions in the Monte Carlo predictions made using the simulated data [1, 2]. Since negative event weights cannot be eliminated using the standard rejection reweighting and unweighting techniques, the inefficiency is reflected not just in the event generation stage but also in the subsequent stages of the simulation pipeline, including detector simulation, electronics simulation, etc. This problem has received some attention in the last few years, leading to the invention of several new theoretical and statistical/Monte Carlo solutions to ameliorate the problem [3, 4, 5, 6, 7, 8, 9, 10, 11]. However, the negative weights problem continues to be a major challenge for the collider physics community [1, 2].
In Ref. [12], a new technique dubbed ARCANE reweighting has been introduced in order to reduce the negative weights problem in collider event generation. The present paper serves as a companion to Ref. [12] and provides a demonstration of the technique for the NLO generation of events using the MC@NLO formalism [13, 14, 15]. Admittedly, the negative weights problem in the generation of this process using existing techniques is not particularly severe, when compared to processes like and . However, the chosen example captures the essence of the negative weights problem in the more important cases while being simpler to implement, because it avoids a few complications associated with hadron collisions (like parton distribution functions and initial state radiation).
The ARCANE reweighting technique can be briefly summarized as follows.111The presentation here has some minor changes in notation compared to Ref. [12]. For example, is used here for hidden information instead of to avoid confusion with the notation for hard remainder events in the MC@NLO formalism. Consider a generation pipeline that produces events parameterized as , where
-
•
represents all the attributes of an event that are “visible” to (i.e., will be used by) the subsequent stages of the simulation and analysis pipeline,
-
•
represents all the latent attributes of an event that are “hidden” from the subsequent stages of the simulation pipeline, and
-
•
is a special weight attribute used to modify the distribution represented by the simulated data.
Here, and are assumed to capture all sources of randomness in the generation of the event. So, is fully determined by and , and will be interchangeably written as . ARCANE reweighting involves additively modifying the event weights as
(1) |
where is the sampling probability density of under the give simulation pipeline and is a special function called the ARCANE redistribution function that satisfies the following condition:
(2) |
where and are the domains of and , respectively, and the integration with respect to is performed using an appropriate reference measure. The reweighting in (1) can be thought of as redistributing the contributions from different Monte Carlo “histories” or pathways, denoted by , in the event generator that lead to the exact same value of . The weighted densities of , also referred to as the quasi densities here and in Ref. [12], under the “ORIG” and “ARCANE” weighting schemes are given by
(3) |
Likewise the quasi densities of , under the different weighting schemes, are given by
(4) |
It can be seen that the quasi densities are related as
(5) | ||||
(6) |
In this way, ARCANE reweighting modifies the joint quasi density of without affecting the quasi density of . The original event-weights could be negative if is negative for certain values of . In such situations, redistributing the contributions of the different pathways, as in (5), can reduce the negative weights problem. In addition to (2), the ARCANE redistribution function needs to satisfy certain conditions to ensure proper coverage and finiteness of weights; this will be reviewed later, as needed, in this paper.
From the simple description above of ARCANE reweighting, it may not be clear how or if the technique can be used to tackle the negative weights problem in realistic event generation scenarios; this paper is intended to bridge this gap. The rest of the paper is organized as follows. Section 2 provides a detailed overview of the event generator pipeline, constructed using previously existing techniques, for generating events under the MC@NLO formalism. Some groundwork for the subsequent implementation of ARCANE reweighting is also described in this section. Section 3 describes the implementation of ARCANE reweighting in detail, for the example at hand. Some results are described and discussed in Section 4. Some concluding remarks, including thoughts on implementing the technique for other processes of interest, are provided in Section 6.
All functions used in this paper are assumed to (or will be ensured to) have finite values in the relevant domains—the physics formalisms used to tackle divergences, like subtraction schemes, are assumed to already be incorporated into the event generation pipeline prior to modifying the pipeline with ARCANE reweighting. All functions that are used as integrands in this paper are assumed to satisfy appropriate notions of integrability. All integrals in this paper are assumed to be performed with respect to appropriate reference measures. For simplicity, throughout this paper, the phrase “ is a function” will mean “ is a function of the variable ,” unless explicitly stated otherwise.
2 Base Event Generator, Groundwork for ARCANE Reweighting
The base (i.e., prior to applying ARCANE reweighting) event-generation pipeline used is this paper is taken directly from Ref. [16], which is a tutorial (with an accompanying codebase) on matching NLO calculations with parton showers and a few other related topics. In this section, the mechanics of the event generation pipeline will be described in sufficient detail so as to understand the subsequent implementation of ARCANE reweighting. Monte Carlo collider-event-generation experts may be able to skip Section 2.1 and Sections 2.3-2.6 and still follow the rest of the paper. Additional details needed to reproduce the present work can be found in Ref. [16] and/or the codebase associated with the present work. The physics motivations and formalisms behind the generation pipeline can be found in Refs. [17, 16].
2.1 General Overview of the Generation Pipeline
The events of interest in this paper contain in the initial state and contain either or in the final state. The event-attributes of interest, ultimately, are the momenta, flavors, and colors of the final state particles, as well as the starting scale for the subsequent parton showering, which will be discussed later. For each event, the flavor of the quark will be one of , , , , and . The generator uses the large-number-of-colors approximation. Under this, without loss of generality, the colors of , , and can be set to and , and , respectively, in all events with the final state.222In a color tuple , is the color charge and is the anticolor charge. The different colors are indexed as , and a value of indicates no color charge. Likewise, the colors of and can be set to and , respectively, in all events with the final state. All initial and final state particles are taken to be massless in the event kinematics. The flowchart in Figure 1 provides a general overview of the pipeline for generating a single event, which roughly proceeds as follows:
-
•
First the “event-type” attribute is chosen randomly from the set . Here, -events and -events stand for “hard remainder” and “standard” events, respectively.
-
•
If the event is chosen to be of the -type, a “resolved event” is sampled. On the other hand, if the event is chosen to be of the -type, an “unresolved event” is sampled. Here, sampling an event means to sample the particles’ momenta as well as the quark flavor.
-
•
Subsequently, the events undergo parton showers, which produce additional quarks and/or gluons, until a pre-chosen infra-red cutoff scale, specified by the parameter , is reached. The parton shower will begin from an event-specific starting scale . The outcome of this sampling procedure is one parton-level event.

The events are sampled and weighted by the generator in such a way that they correspond to a specific theoretical/phenomenological/heuristic model (with specific values for the model parameters). The ingredients of this model include the Standard Model of particle physics and various aspects of perturbative Quantum Chromodynamics (QCD) [18], including the Catani–Seymour dipole factorization [19], the MC@NLO formalism for next-to-leading-order event generation, running of coupling constants, etc. The exact formulas (involving matrix elements, subtraction schemes, dipole splitting functions, etc.) used to compute the weight of an event (with a completely-specified MC-history) are mostly irrelevant to the present work, and will only be discussed as needed. Due to the nature of the NLO matching performed by MC@NLO, some -type events have negative weights for this collision process; this is the problem being tackled in this paper.
The event-type attribute is not relevant beyond the parton-level event-generator, i.e., will not be used in subsequent processing of the event, e.g., in fragmentation and hadronization, detector and electronics simulations, object reconstructions, and physics analyses. One can have two functionally identical parton-level events (ignoring differences in weights) be of different event-types. The idea in this paper is to redistribute the “contributions” of the - and -pathways in the event generator to a given parton-level event, in order to reduce the negative weights problem.
2.2 Choosing a “Stopping Point” for Performing ARCANE Reweighting
In order to do the redistribution of contributions correctly, one needs to pick a stopping point (or criterion), for all possible pathways within the event generator and identify the visible and hidden event-attributes, denoted by and respectively, at this point of the generator. While a convenient choice of stopping point for the example under consideration may be obvious to the reader, it could be instructive to discuss a few bad stopping points first. Consider the following stopping point: immediately after sampling a resolved event for the -pathway and immediately after sampling an unresolved event for -pathway. The event-type can be treated as hidden at this point, so moving contributions from - to -pathways is acceptable. However, there is no overlap in the distributions of - and -type events at this point, since their final states are different. So it is not possible to perform the desired redistribution, making this an ineffective choice of stopping point.
Another possible stopping point is immediately after the complete parton showering process. Beyond this, the event-type can be treated as hidden. Furthermore, there is significant overlap between the distributions of - and -type events at this point. So this is a viable point in the generator for performing ARCANE reweighting. However, there could be a lot of emissions in the parton showering stage. This increases the complexity of the Monte Carlo event-history to be tracked and handled; it is preferable to reduce this complexity if possible.
The stopping point used in this paper is the following: a) immediately after sampling a resolved event for the -pathway and b) immediately after a single parton-shower emission step, following the sampling of an unresolved event, for the -pathway. The emission step may or may not result in a successful emission; this will be discussed later. A successful emission leads to a final state in the -pathway (with equivalent color structure as in the resolved -type events). As before, the event-type can be treated as hidden and there is an overlap between the distributions of - and -events. So this is a viable (and a relatively simple) point in the generation pipeline for redistributing the contributions of - and -pathways. The different components of the generation pipeline, up to the chosen stopping point, will be discussed next.
2.3 Unresolved Event Kinematics
This subsection describes the kinematics of an unresolved (or leading-order) event . The four-momenta of the incident colliding particles and are given by
(7) |
respectively, where is the speed of light and is the total energy of colliding particles in their center-of-momentum frame, which is also the lab frame; is a fixed parameter of the event generator. The four momenta of the daughter particles and are given by
(8) | ||||
(9) |
respectively, where and are angles parameterizing the collision kinematics. Note that given and , the angles and are uniquely determined.
Phase-space element.
The following fact is noted here for later use. For two massless particles and constrained to satisfy , where is a timelike four-momentum, the Lorentz-invariant phase space elements (for two different parametrizations) are related by333Using an equality sign in (10) is a convenient abuse of notation, albeit a benign one.
(10) |
where and are the spherical angles (in some orientation of axes) of the three-momentum of, say, particle in the center-of-momentum frame of . Here, (i) and are the energy and three-momentum components, respectively, of and (ii) is the four-dimensional Dirac delta function.
2.4 Emission Kinematics
Let us a consider a generic emission process . Here , , , and represent the emitter particle before the emission, the emitter particle after the emission, the emitted “daughter” particle, and the “spectator” particle, respectively; all these particles are taken to be massless. Let and be the four-momenta of the particles and , respectively, before the emission. Let , , and be the four-momenta of the particles , , and , respectively, after the emission. Let be the total energy of the emission system in its center-of-momentum frame:
(11) |
The emission process is parameterized by the variables , , and . The final momenta are given, in terms of the initial momenta and the emission-parameters, by
(12) | ||||
(13) | ||||
(14) |
Here is a four-momentum determined by the initial momenta and emission-parameters, and satisfies the following conditions:
(15) |
These conditions ensure that . In the center-of-momentum frame of the emission process, is given by
(16) |
where is a three-vector that lies in the plane perpendicular to the emitter’s direction of travel (a) before emission (b) in the center-of-momentum frame of the emission process.444Equivalently, lies in the plane perpendicular to the spectator’s direction of travel (a) before or after the emission (b) in the center-of-momentum frame of the emission process. The magnitude and direction of are given, respectively, by
(17) |
where and are unit-vectors that (a) lie in , (b) are perpendicular to each other, and (c) are completely determined by and . Note that given a valid choice of , the value of is uniquely determined, and can be computed, e.g., as follows:555If needed, can be computed using , by computing as an intermediate step. However, typically, and in this paper, the relevant conditional quasi (i.e., weighted) densities and sampling densities are all independent of , so it will not need to be computed for the purposes of this paper.
(18a) | |||||
(18b) | |||||
(18c) |
Some other quantities of interest are the energy scale parameters and defined as
(19a) | ||||