Showing 1–50 of 86 results for author: Herrmann, C

Direct finiteness of representable regular rings with involution: A counterexample

Authors: Christian Herrmann

Abstract: Bruns and Roddy constructed a $3$-generated modular ortholattice $L$ which cannot be embedded into any complete modular ortholattice. Motivated by their approach, we use shift operators to construct a $*$-regular $*$-ring $R$ of endomorphisms of an inner product space (which can be chosen as the Hilbert space $\ell^2$) such that direct finiteness fails for $R$. Bruns and Roddy constructed a $3$-generated modular ortholattice $L$ which cannot be embedded into any complete modular ortholattice. Motivated by their approach, we use shift operators to construct a $*$-regular $*$-ring $R$ of endomorphisms of an inner product space (which can be chosen as the Hilbert space $\ell^2$) such that direct finiteness fails for $R$. △ Less

Submitted 31 August, 2024; v1 submitted 29 August, 2024; originally announced August 2024.

Comments: As observed by Wehrung, the shift has no quasi-inverse in the ring of row and column finite matrices. Thus, the claimed example does not work

MSC Class: 16E50

Singlet-Triplet Kondo Effect in Blatter Radical Molecular Junctions: Zero-bias Anomalies and Magnetoresistance

Authors: Gautam Mitra, Jueting Zheng, Karen Schaefer, Michael Deffner, Jonathan Z. Low, Luis M. Campos, Carmen Herrmann, Theo A. Costi, Elke Scheer

Abstract: The Blatter radical has been suggested as a building block in future molecular spintronic devices due to its radical character and expected long-spin lifetime. However, whether and how the radical character manifests itself in the charge transport and magnetotransport properties seems to depend on the environment or has not yet been studied. Here, we investigate single-molecule junctions of the Bl… ▽ More The Blatter radical has been suggested as a building block in future molecular spintronic devices due to its radical character and expected long-spin lifetime. However, whether and how the radical character manifests itself in the charge transport and magnetotransport properties seems to depend on the environment or has not yet been studied. Here, we investigate single-molecule junctions of the Blatter radical molecule in a mechanically controlled break junction device at low temperature. Differential conductance spectroscopy on individual junctions shows two types of zero-bias anomalies attributed to Kondo resonances revealing the ability to retain the open-shell nature of the radical molecule in a two-terminal device. Additionally, a high negative magnetoresistance is also observed in junctions without showing a zero-bias peak. We posit that the high negative magnetoresistance is due to the effect of a singlet-triplet Kondo effect under magnetic field originating from a double-quantum-dot system consisting of a Blatter radical molecule with a strong correlation to a second side-coupled molecule. Our findings not only provide the possibility of using the Blatter radical in a two-terminal system under cryogenic conditions but also reveal the magnetotransport properties emerging from different configurations of the molecule inside a junction. △ Less

Submitted 1 August, 2024; originally announced August 2024.

Comments: 17 pages, 5 figures

The XOC X-ray Beamline: Probing Colder, Quieter, and Softer

Authors: Haley R. Stueber, Tanmoy Chattopadhyay, Sven C. Herrmann, Peter Orel, Tsion Gebre, Aanand Joshi, Steven W. Allen, Glenn Morris, Artem Poliszczuk

Abstract: Future strategic X-ray satellite telescopes, such as the probe-class Advanced X-ray Imaging Satellite (AXIS), will require excellent soft energy response in their imaging detectors to enable maximum discovery potential. In order to characterize Charge-Coupled Device (CCD) and Single Electron Sensitive Read Output (SiSeRO) detectors in the soft X-ray region, the X-ray Astronomy and Observational Co… ▽ More Future strategic X-ray satellite telescopes, such as the probe-class Advanced X-ray Imaging Satellite (AXIS), will require excellent soft energy response in their imaging detectors to enable maximum discovery potential. In order to characterize Charge-Coupled Device (CCD) and Single Electron Sensitive Read Output (SiSeRO) detectors in the soft X-ray region, the X-ray Astronomy and Observational Cosmology (XOC) group at Stanford has developed, assembled, and commissioned a 2.5-meter-long X-ray beamline test system. The beamline is designed to efficiently produce monoenergetic X-ray fluorescence lines in the 0.3-10 keV energy range and achieve detector temperatures as low as 173 K. We present design and simulation details of the beamline, and discuss the vacuum, cooling, and X-ray fluorescence performance achieved. As a workhorse for future detector characterization at Stanford, the XOC beamline will support detector development for a broad range of X-ray astronomy instruments. △ Less

Submitted 23 July, 2024; originally announced July 2024.

Comments: To appear in SPIE Astronomical Telescopes and Instrumentation 2024 proceedings. 17 pages, 13 figures

WonderWorld: Interactive 3D Scene Generation from a Single Image

Authors: Hong-Xing Yu, Haoyi Duan, Charles Herrmann, William T. Freeman, Jiajun Wu

Abstract: We present WonderWorld, a novel framework for interactive 3D scene extrapolation that enables users to explore and shape virtual environments based on a single input image and user-specified text. While significant improvements have been made to the visual quality of scene generation, existing methods are run offline, taking tens of minutes to hours to generate a scene. By leveraging Fast Gaussian… ▽ More We present WonderWorld, a novel framework for interactive 3D scene extrapolation that enables users to explore and shape virtual environments based on a single input image and user-specified text. While significant improvements have been made to the visual quality of scene generation, existing methods are run offline, taking tens of minutes to hours to generate a scene. By leveraging Fast Gaussian Surfels and a guided diffusion-based depth estimation method, WonderWorld generates geometrically consistent extrapolation while significantly reducing computational time. Our framework generates connected and diverse 3D scenes in less than 10 seconds on a single A6000 GPU, enabling real-time user interaction and exploration. We demonstrate the potential of WonderWorld for applications in virtual reality, gaming, and creative design, where users can quickly generate and navigate immersive, potentially infinite virtual worlds from a single image. Our approach represents a significant advancement in interactive 3D scene generation, opening up new possibilities for user-driven content creation and exploration in virtual environments. We will release full code and software for reproducibility. Project website: https://WonderWorld-2024.github.io/ △ Less

Submitted 14 June, 2024; v1 submitted 13 June, 2024; originally announced June 2024.

Comments: Project website: https://WonderWorld-2024.github.io/

On the equational theory of finite modular lattices

Authors: Christian Herrmann

Abstract: It is shown that there is $N$ such that there is no algorithm to decide for identities in at most $N$ variables validity in the class of finite modular lattices. This is based on Slobodskoi's result that the Restricted Word Problem is unsolvable for the class of finite groups and relies on Freese's technique of capturing group presentations within free modular lattices. It is shown that there is $N$ such that there is no algorithm to decide for identities in at most $N$ variables validity in the class of finite modular lattices. This is based on Slobodskoi's result that the Restricted Word Problem is unsolvable for the class of finite groups and relies on Freese's technique of capturing group presentations within free modular lattices. △ Less

Submitted 8 July, 2024; v1 submitted 6 June, 2024; originally announced June 2024.

MSC Class: 06C05; 03D40; 06B25

DreamWalk: Style Space Exploration using Diffusion Guidance

Authors: Michelle Shu, Charles Herrmann, Richard Strong Bowen, Forrester Cole, Ramin Zabih

Abstract: Text-conditioned diffusion models can generate impressive images, but fall short when it comes to fine-grained control. Unlike direct-editing tools like Photoshop, text conditioned models require the artist to perform "prompt engineering," constructing special text sentences to control the style or amount of a particular subject present in the output image. Our goal is to provide fine-grained cont… ▽ More Text-conditioned diffusion models can generate impressive images, but fall short when it comes to fine-grained control. Unlike direct-editing tools like Photoshop, text conditioned models require the artist to perform "prompt engineering," constructing special text sentences to control the style or amount of a particular subject present in the output image. Our goal is to provide fine-grained control over the style and substance specified by the prompt, for example to adjust the intensity of styles in different regions of the image (Figure 1). Our approach is to decompose the text prompt into conceptual elements, and apply a separate guidance term for each element in a single diffusion process. We introduce guidance scale functions to control when in the diffusion process and \emph{where} in the image to intervene. Since the method is based solely on adjusting diffusion guidance, it does not require fine-tuning or manipulating the internal layers of the diffusion model's neural network, and can be used in conjunction with LoRA- or DreamBooth-trained models (Figure2). Project page: https://mshu1.github.io/dreamwalk.github.io/ △ Less

Submitted 3 April, 2024; originally announced April 2024.

Lumiere: A Space-Time Diffusion Model for Video Generation

Authors: Omer Bar-Tal, Hila Chefer, Omer Tov, Charles Herrmann, Roni Paiss, Shiran Zada, Ariel Ephrat, Junhwa Hur, Guanghui Liu, Amit Raj, Yuanzhen Li, Michael Rubinstein, Tomer Michaeli, Oliver Wang, Deqing Sun, Tali Dekel, Inbar Mosseri

Abstract: We introduce Lumiere -- a text-to-video diffusion model designed for synthesizing videos that portray realistic, diverse and coherent motion -- a pivotal challenge in video synthesis. To this end, we introduce a Space-Time U-Net architecture that generates the entire temporal duration of the video at once, through a single pass in the model. This is in contrast to existing video models which synth… ▽ More We introduce Lumiere -- a text-to-video diffusion model designed for synthesizing videos that portray realistic, diverse and coherent motion -- a pivotal challenge in video synthesis. To this end, we introduce a Space-Time U-Net architecture that generates the entire temporal duration of the video at once, through a single pass in the model. This is in contrast to existing video models which synthesize distant keyframes followed by temporal super-resolution -- an approach that inherently makes global temporal consistency difficult to achieve. By deploying both spatial and (importantly) temporal down- and up-sampling and leveraging a pre-trained text-to-image diffusion model, our model learns to directly generate a full-frame-rate, low-resolution video by processing it in multiple space-time scales. We demonstrate state-of-the-art text-to-video generation results, and show that our design easily facilitates a wide range of content creation tasks and video editing applications, including image-to-video, video inpainting, and stylized generation. △ Less

Submitted 5 February, 2024; v1 submitted 23 January, 2024; originally announced January 2024.

Comments: Webpage: https://lumiere-video.github.io/ | Video: https://www.youtube.com/watch?v=wxLr02Dz2Sc

Efficient Hybrid Zoom using Camera Fusion on Mobile Phones

Authors: Xiaotong Wu, Wei-Sheng Lai, YiChang Shih, Charles Herrmann, Michael Krainin, Deqing Sun, Chia-Kai Liang

Abstract: DSLR cameras can achieve multiple zoom levels via shifting lens distances or swapping lens types. However, these techniques are not possible on smartphone devices due to space constraints. Most smartphone manufacturers adopt a hybrid zoom system: commonly a Wide (W) camera at a low zoom level and a Telephoto (T) camera at a high zoom level. To simulate zoom levels between W and T, these systems cr… ▽ More DSLR cameras can achieve multiple zoom levels via shifting lens distances or swapping lens types. However, these techniques are not possible on smartphone devices due to space constraints. Most smartphone manufacturers adopt a hybrid zoom system: commonly a Wide (W) camera at a low zoom level and a Telephoto (T) camera at a high zoom level. To simulate zoom levels between W and T, these systems crop and digitally upsample images from W, leading to significant detail loss. In this paper, we propose an efficient system for hybrid zoom super-resolution on mobile devices, which captures a synchronous pair of W and T shots and leverages machine learning models to align and transfer details from T to W. We further develop an adaptive blending method that accounts for depth-of-field mismatches, scene occlusion, flow uncertainty, and alignment errors. To minimize the domain gap, we design a dual-phone camera rig to capture real-world inputs and ground-truths for supervised training. Our method generates a 12-megapixel image in 500ms on a mobile platform and compares favorably against state-of-the-art methods under extensive evaluation on real-world scenarios. △ Less

Submitted 2 January, 2024; originally announced January 2024.

Comments: Accepted to SIGGRAPH Asia 2023 (ACM TOG). Project website: https://www.wslai.net/publications/fusion_zoom

Boundary Attention: Learning to Localize Boundaries under High Noise

Authors: Mia Gaia Polansky, Charles Herrmann, Junhwa Hur, Deqing Sun, Dor Verbin, Todd Zickler

Abstract: We present a differentiable model that infers explicit boundaries, including curves, corners and junctions, using a mechanism that we call boundary attention. Boundary attention is a boundary-aware local attention operation that, when applied densely and repeatedly, progressively refines a field of variables that specify an unrasterized description of the local boundary structure in every overlapp… ▽ More We present a differentiable model that infers explicit boundaries, including curves, corners and junctions, using a mechanism that we call boundary attention. Boundary attention is a boundary-aware local attention operation that, when applied densely and repeatedly, progressively refines a field of variables that specify an unrasterized description of the local boundary structure in every overlapping patch within an image. It operates in a bottom-up fashion, similar to classical methods for sub-pixel edge localization and edge-linking, but with a higher-dimensional description of local boundary structure, a notion of spatial consistency that is learned instead of designed, and a sequence of operations that is end-to-end differentiable. We train our model using simple synthetic data and then evaluate it using photographs that were captured under low-light conditions with variable amounts of noise. We find that our method generalizes to natural images corrupted by real sensor noise, and predicts consistent boundaries under increasingly noisy conditions where other state-of-the-art methods fail. △ Less

Submitted 18 March, 2024; v1 submitted 1 January, 2024; originally announced January 2024.

Comments: Project website at boundaryattention.github.io: http://boundaryattention.github.io

Zero-Shot Metric Depth with a Field-of-View Conditioned Diffusion Model

Authors: Saurabh Saxena, Junhwa Hur, Charles Herrmann, Deqing Sun, David J. Fleet

Abstract: While methods for monocular depth estimation have made significant strides on standard benchmarks, zero-shot metric depth estimation remains unsolved. Challenges include the joint modeling of indoor and outdoor scenes, which often exhibit significantly different distributions of RGB and depth, and the depth-scale ambiguity due to unknown camera intrinsics. Recent work has proposed specialized mult… ▽ More While methods for monocular depth estimation have made significant strides on standard benchmarks, zero-shot metric depth estimation remains unsolved. Challenges include the joint modeling of indoor and outdoor scenes, which often exhibit significantly different distributions of RGB and depth, and the depth-scale ambiguity due to unknown camera intrinsics. Recent work has proposed specialized multi-head architectures for jointly modeling indoor and outdoor scenes. In contrast, we advocate a generic, task-agnostic diffusion model, with several advancements such as log-scale depth parameterization to enable joint modeling of indoor and outdoor scenes, conditioning on the field-of-view (FOV) to handle scale ambiguity and synthetically augmenting FOV during training to generalize beyond the limited camera intrinsics in training datasets. Furthermore, by employing a more diverse training mixture than is common, and an efficient diffusion parameterization, our method, DMD (Diffusion for Metric Depth) achieves a 25\% reduction in relative error (REL) on zero-shot indoor and 33\% reduction on zero-shot outdoor datasets over the current SOTA using only a small number of denoising steps. For an overview see https://diffusion-vision.github.io/dmd △ Less

Submitted 20 December, 2023; originally announced December 2023.

WonderJourney: Going from Anywhere to Everywhere

Authors: Hong-Xing Yu, Haoyi Duan, Junhwa Hur, Kyle Sargent, Michael Rubinstein, William T. Freeman, Forrester Cole, Deqing Sun, Noah Snavely, Jiajun Wu, Charles Herrmann

Abstract: We introduce WonderJourney, a modularized framework for perpetual 3D scene generation. Unlike prior work on view generation that focuses on a single type of scenes, we start at any user-provided location (by a text description or an image) and generate a journey through a long sequence of diverse yet coherently connected 3D scenes. We leverage an LLM to generate textual descriptions of the scenes… ▽ More We introduce WonderJourney, a modularized framework for perpetual 3D scene generation. Unlike prior work on view generation that focuses on a single type of scenes, we start at any user-provided location (by a text description or an image) and generate a journey through a long sequence of diverse yet coherently connected 3D scenes. We leverage an LLM to generate textual descriptions of the scenes in this journey, a text-driven point cloud generation pipeline to make a compelling and coherent sequence of 3D scenes, and a large VLM to verify the generated scenes. We show compelling, diverse visual results across various scene types and styles, forming imaginary "wonderjourneys". Project website: https://kovenyu.com/WonderJourney/ △ Less

Submitted 12 April, 2024; v1 submitted 6 December, 2023; originally announced December 2023.

Comments: Project website with video results: https://kovenyu.com/WonderJourney/

DreamSync: Aligning Text-to-Image Generation with Image Understanding Feedback

Authors: Jiao Sun, Deqing Fu, Yushi Hu, Su Wang, Royi Rassin, Da-Cheng Juan, Dana Alon, Charles Herrmann, Sjoerd van Steenkiste, Ranjay Krishna, Cyrus Rashtchian

Abstract: Despite their wide-spread success, Text-to-Image models (T2I) still struggle to produce images that are both aesthetically pleasing and faithful to the user's input text. We introduce DreamSync, a model-agnostic training algorithm by design that improves T2I models to be faithful to the text input. DreamSync builds off a recent insight from TIFA's evaluation framework -- that large vision-language… ▽ More Despite their wide-spread success, Text-to-Image models (T2I) still struggle to produce images that are both aesthetically pleasing and faithful to the user's input text. We introduce DreamSync, a model-agnostic training algorithm by design that improves T2I models to be faithful to the text input. DreamSync builds off a recent insight from TIFA's evaluation framework -- that large vision-language models (VLMs) can effectively identify the fine-grained discrepancies between generated images and the text inputs. DreamSync uses this insight to train T2I models without any labeled data; it improves T2I models using its own generations. First, it prompts the model to generate several candidate images for a given input text. Then, it uses two VLMs to select the best generation: a Visual Question Answering model that measures the alignment of generated images to the text, and another that measures the generation's aesthetic quality. After selection, we use LoRA to iteratively finetune the T2I model to guide its generation towards the selected best generations. DreamSync does not need any additional human annotation. model architecture changes, or reinforcement learning. Despite its simplicity, DreamSync improves both the semantic alignment and aesthetic appeal of two diffusion-based T2I models, evidenced by multiple benchmarks (+1.7% on TIFA, +2.9% on DSG1K, +3.4% on VILA aesthetic) and human evaluation. △ Less

Submitted 28 November, 2023; originally announced November 2023.

Telling Left from Right: Identifying Geometry-Aware Semantic Correspondence

Authors: Junyi Zhang, Charles Herrmann, Junhwa Hur, Eric Chen, Varun Jampani, Deqing Sun, Ming-Hsuan Yang

Abstract: While pre-trained large-scale vision models have shown significant promise for semantic correspondence, their features often struggle to grasp the geometry and orientation of instances. This paper identifies the importance of being geometry-aware for semantic correspondence and reveals a limitation of the features of current foundation models under simple post-processing. We show that incorporatin… ▽ More While pre-trained large-scale vision models have shown significant promise for semantic correspondence, their features often struggle to grasp the geometry and orientation of instances. This paper identifies the importance of being geometry-aware for semantic correspondence and reveals a limitation of the features of current foundation models under simple post-processing. We show that incorporating this information can markedly enhance semantic correspondence performance with simple but effective solutions in both zero-shot and supervised settings. We also construct a new challenging benchmark for semantic correspondence built from an existing animal pose estimation dataset, for both pre-training validating models. Our method achieves a PCK@0.10 score of 65.4 (zero-shot) and 85.6 (supervised) on the challenging SPair-71k dataset, outperforming the state of the art by 5.5p and 11.0p absolute gains, respectively. Our code and datasets are publicly available at: https://telling-left-from-right.github.io/. △ Less

Submitted 24 March, 2024; v1 submitted 28 November, 2023; originally announced November 2023.

Comments: Accepted by CVPR 24, project page: https://telling-left-from-right.github.io/

Towards Standardized Grid Emission Factors: Methodological Insights and Best Practices

Authors: Malte Schäfer, Felipe Cerdas, Christoph Herrmann

Abstract: Inconsistent calculation of grid emission factors (EF) can result in widely divergent corporate greenhouse gas (GHG) emissions reports. We dissect this issue through a comprehensive literature review, identifying nine key aspects - each with two to six methodological choices - that substantially influence the reported EF. These choices lead to relative effect variations ranging from 1.2% to 69%. U… ▽ More Inconsistent calculation of grid emission factors (EF) can result in widely divergent corporate greenhouse gas (GHG) emissions reports. We dissect this issue through a comprehensive literature review, identifying nine key aspects - each with two to six methodological choices - that substantially influence the reported EF. These choices lead to relative effect variations ranging from 1.2% to 69%. Using Germany's 2019-2022 data as a case study, our method yields results that largely align with prior studies, yet reveal relative effects from 0.4% to 34.6%. This study is the first to methodically unpack the key determinants of grid EF, quantify their impacts, and offer clear guidelines for their application in corporate GHG accounting. Our findings hold implications for practitioners, data publishers, researchers, and guideline-making organizations. By openly sharing our data and calculations, we invite replication, scrutiny, and further research. △ Less

Submitted 14 March, 2024; v1 submitted 2 November, 2023; originally announced November 2023.

Comments: 11 pages main paper, 26 pages supplementary information, graphical abstract on the very last page. Code: https://doi.org/10.24355/dbbs.084-202309131139-0, Data: https://doi.org/10.24355/dbbs.084-202309111514-0

Journal ref: RSC Energy & Environmental Science, 2024

ZeroNVS: Zero-Shot 360-Degree View Synthesis from a Single Image

Authors: Kyle Sargent, Zizhang Li, Tanmay Shah, Charles Herrmann, Hong-Xing Yu, Yunzhi Zhang, Eric Ryan Chan, Dmitry Lagun, Li Fei-Fei, Deqing Sun, Jiajun Wu

Abstract: We introduce a 3D-aware diffusion model, ZeroNVS, for single-image novel view synthesis for in-the-wild scenes. While existing methods are designed for single objects with masked backgrounds, we propose new techniques to address challenges introduced by in-the-wild multi-object scenes with complex backgrounds. Specifically, we train a generative prior on a mixture of data sources that capture obje… ▽ More We introduce a 3D-aware diffusion model, ZeroNVS, for single-image novel view synthesis for in-the-wild scenes. While existing methods are designed for single objects with masked backgrounds, we propose new techniques to address challenges introduced by in-the-wild multi-object scenes with complex backgrounds. Specifically, we train a generative prior on a mixture of data sources that capture object-centric, indoor, and outdoor scenes. To address issues from data mixture such as depth-scale ambiguity, we propose a novel camera conditioning parameterization and normalization scheme. Further, we observe that Score Distillation Sampling (SDS) tends to truncate the distribution of complex backgrounds during distillation of 360-degree scenes, and propose "SDS anchoring" to improve the diversity of synthesized novel views. Our model sets a new state-of-the-art result in LPIPS on the DTU dataset in the zero-shot setting, even outperforming methods specifically trained on DTU. We further adapt the challenging Mip-NeRF 360 dataset as a new benchmark for single-image novel view synthesis, and demonstrate strong performance in this setting. Our code and data are at http://kylesargent.github.io/zeronvs/ △ Less

Submitted 23 April, 2024; v1 submitted 27 October, 2023; originally announced October 2023.

Comments: Accepted to CVPR 2024. 12 pages

Perspectivity in complemented modular lattices and regular rings

Authors: Christian Herrmann

Abstract: Based on an analogue for systems of partial isomorphisms between lower sections in a complemented modular lattice we prove that principal right ideals $aR \cong bR$ in a (von Neumann) regular ring $R$ are perspective if $aR \cap bR$ is of finite height in $L(R)$. This is applied to derive, for existence-varieties $\mathcal{V}$ of regular rings, equivalence of unit-regularity and direct finitenes… ▽ More Based on an analogue for systems of partial isomorphisms between lower sections in a complemented modular lattice we prove that principal right ideals $aR \cong bR$ in a (von Neumann) regular ring $R$ are perspective if $aR \cap bR$ is of finite height in $L(R)$. This is applied to derive, for existence-varieties $\mathcal{V}$ of regular rings, equivalence of unit-regularity and direct finiteness, both conceived as a property shared by all members of $\mathcal{V}$. △ Less

Submitted 26 October, 2023; originally announced October 2023.

MSC Class: 06C20; 16E50

Substance or Style: What Does Your Image Embedding Know?

Authors: Cyrus Rashtchian, Charles Herrmann, Chun-Sung Ferng, Ayan Chakrabarti, Dilip Krishnan, Deqing Sun, Da-Cheng Juan, Andrew Tomkins

Abstract: Probes are small networks that predict properties of underlying data from embeddings, and they provide a targeted, effective way to illuminate the information contained in embeddings. While analysis through the use of probes has become standard in NLP, there has been much less exploration in vision. Image foundation models have primarily been evaluated for semantic content. Better understanding th… ▽ More Probes are small networks that predict properties of underlying data from embeddings, and they provide a targeted, effective way to illuminate the information contained in embeddings. While analysis through the use of probes has become standard in NLP, there has been much less exploration in vision. Image foundation models have primarily been evaluated for semantic content. Better understanding the non-semantic information in popular embeddings (e.g., MAE, SimCLR, or CLIP) will shed new light both on the training algorithms and on the uses for these foundation models. We design a systematic transformation prediction task and measure the visual content of embeddings along many axes, including image style, quality, and a range of natural and artificial transformations. Surprisingly, six embeddings (including SimCLR) encode enough non-semantic information to identify dozens of transformations. We also consider a generalization task, where we group similar transformations and hold out several for testing. We find that image-text models (CLIP and ALIGN) are better at recognizing new examples of style transfer than masking-based models (CAN and MAE). Overall, our results suggest that the choice of pre-training algorithm impacts the types of information in the embedding, and certain models are better than others for non-semantic downstream tasks. △ Less

Submitted 10 July, 2023; originally announced July 2023.

Comments: 27 pages, 9 figures

The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimation

Authors: Saurabh Saxena, Charles Herrmann, Junhwa Hur, Abhishek Kar, Mohammad Norouzi, Deqing Sun, David J. Fleet

Abstract: Denoising diffusion probabilistic models have transformed image generation with their impressive fidelity and diversity. We show that they also excel in estimating optical flow and monocular depth, surprisingly, without task-specific architectures and loss functions that are predominant for these tasks. Compared to the point estimates of conventional regression-based methods, diffusion models also… ▽ More Denoising diffusion probabilistic models have transformed image generation with their impressive fidelity and diversity. We show that they also excel in estimating optical flow and monocular depth, surprisingly, without task-specific architectures and loss functions that are predominant for these tasks. Compared to the point estimates of conventional regression-based methods, diffusion models also enable Monte Carlo inference, e.g., capturing uncertainty and ambiguity in flow and depth. With self-supervised pre-training, the combined use of synthetic and real data for supervised training, and technical innovations (infilling and step-unrolled denoising diffusion training) to handle noisy-incomplete training data, and a simple form of coarse-to-fine refinement, one can train state-of-the-art diffusion models for depth and optical flow estimation. Extensive experiments focus on quantitative performance against benchmarks, ablations, and the model's ability to capture uncertainty and multimodality, and impute missing values. Our model, DDVM (Denoising Diffusion Vision Model), obtains a state-of-the-art relative depth error of 0.074 on the indoor NYU benchmark and an Fl-all outlier rate of 3.26\% on the KITTI optical flow benchmark, about 25\% better than the best published method. For an overview see https://diffusion-vision.github.io. △ Less

Submitted 5 December, 2023; v1 submitted 2 June, 2023; originally announced June 2023.

Comments: NeurIPS 2023 (Oral)

A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence

Authors: Junyi Zhang, Charles Herrmann, Junhwa Hur, Luisa Polania Cabrera, Varun Jampani, Deqing Sun, Ming-Hsuan Yang

Abstract: Text-to-image diffusion models have made significant advances in generating and editing high-quality images. As a result, numerous approaches have explored the ability of diffusion model features to understand and process single images for downstream tasks, e.g., classification, semantic segmentation, and stylization. However, significantly less is known about what these features reveal across mul… ▽ More Text-to-image diffusion models have made significant advances in generating and editing high-quality images. As a result, numerous approaches have explored the ability of diffusion model features to understand and process single images for downstream tasks, e.g., classification, semantic segmentation, and stylization. However, significantly less is known about what these features reveal across multiple, different images and objects. In this work, we exploit Stable Diffusion (SD) features for semantic and dense correspondence and discover that with simple post-processing, SD features can perform quantitatively similar to SOTA representations. Interestingly, the qualitative analysis reveals that SD features have very different properties compared to existing representation learning features, such as the recently released DINOv2: while DINOv2 provides sparse but accurate matches, SD features provide high-quality spatial information but sometimes inaccurate semantic matches. We demonstrate that a simple fusion of these two features works surprisingly well, and a zero-shot evaluation using nearest neighbors on these fused features provides a significant performance gain over state-of-the-art methods on benchmark datasets, e.g., SPair-71k, PF-Pascal, and TSS. We also show that these correspondences can enable interesting applications such as instance swapping in two images. △ Less

Submitted 28 November, 2023; v1 submitted 24 May, 2023; originally announced May 2023.

Comments: Accepted by NeurIPS 23, project page: https://sd-complements-dino.github.io/

VQ3D: Learning a 3D-Aware Generative Model on ImageNet

Authors: Kyle Sargent, Jing Yu Koh, Han Zhang, Huiwen Chang, Charles Herrmann, Pratul Srinivasan, Jiajun Wu, Deqing Sun

Abstract: Recent work has shown the possibility of training generative models of 3D content from 2D image collections on small datasets corresponding to a single object class, such as human faces, animal faces, or cars. However, these models struggle on larger, more complex datasets. To model diverse and unconstrained image collections such as ImageNet, we present VQ3D, which introduces a NeRF-based decoder… ▽ More Recent work has shown the possibility of training generative models of 3D content from 2D image collections on small datasets corresponding to a single object class, such as human faces, animal faces, or cars. However, these models struggle on larger, more complex datasets. To model diverse and unconstrained image collections such as ImageNet, we present VQ3D, which introduces a NeRF-based decoder into a two-stage vector-quantized autoencoder. Our Stage 1 allows for the reconstruction of an input image and the ability to change the camera position around the image, and our Stage 2 allows for the generation of new 3D scenes. VQ3D is capable of generating and reconstructing 3D-aware images from the 1000-class ImageNet dataset of 1.2 million training images. We achieve an ImageNet generation FID score of 16.8, compared to 69.8 for the next best baseline method. △ Less

Submitted 14 February, 2023; originally announced February 2023.

Comments: 15 pages. For visual results, please visit the project webpage at http://kylesargent.github.io/vq3d

Accidental Light Probes

Authors: Hong-Xing Yu, Samir Agarwala, Charles Herrmann, Richard Szeliski, Noah Snavely, Jiajun Wu, Deqing Sun

Abstract: Recovering lighting in a scene from a single image is a fundamental problem in computer vision. While a mirror ball light probe can capture omnidirectional lighting, light probes are generally unavailable in everyday images. In this work, we study recovering lighting from accidental light probes (ALPs) -- common, shiny objects like Coke cans, which often accidentally appear in daily scenes. We pro… ▽ More Recovering lighting in a scene from a single image is a fundamental problem in computer vision. While a mirror ball light probe can capture omnidirectional lighting, light probes are generally unavailable in everyday images. In this work, we study recovering lighting from accidental light probes (ALPs) -- common, shiny objects like Coke cans, which often accidentally appear in daily scenes. We propose a physically-based approach to model ALPs and estimate lighting from their appearances in single images. The main idea is to model the appearance of ALPs by photogrammetrically principled shading and to invert this process via differentiable rendering to recover incidental illumination. We demonstrate that we can put an ALP into a scene to allow high-fidelity lighting estimation. Our model can also recover lighting for existing images that happen to contain an ALP. △ Less

Submitted 10 June, 2023; v1 submitted 12 January, 2023; originally announced January 2023.

Comments: CVPR2023. Project website: https://kovenyu.com/ALP/

Self-supervised AutoFlow

Authors: Hsin-Ping Huang, Charles Herrmann, Junhwa Hur, Erika Lu, Kyle Sargent, Austin Stone, Ming-Hsuan Yang, Deqing Sun

Abstract: Recently, AutoFlow has shown promising results on learning a training set for optical flow, but requires ground truth labels in the target domain to compute its search metric. Observing a strong correlation between the ground truth search metric and self-supervised losses, we introduce self-supervised AutoFlow to handle real-world videos without ground truth labels. Using self-supervised loss as t… ▽ More Recently, AutoFlow has shown promising results on learning a training set for optical flow, but requires ground truth labels in the target domain to compute its search metric. Observing a strong correlation between the ground truth search metric and self-supervised losses, we introduce self-supervised AutoFlow to handle real-world videos without ground truth labels. Using self-supervised loss as the search metric, our self-supervised AutoFlow performs on par with AutoFlow on Sintel and KITTI where ground truth is available, and performs better on the real-world DAVIS dataset. We further explore using self-supervised AutoFlow in the (semi-)supervised setting and obtain competitive results against the state of the art. △ Less

Submitted 22 May, 2023; v1 submitted 4 December, 2022; originally announced December 2022.

Disentangling Architecture and Training for Optical Flow

Authors: Deqing Sun, Charles Herrmann, Fitsum Reda, Michael Rubinstein, David Fleet, William T. Freeman

Abstract: How important are training details and datasets to recent optical flow models like RAFT? And do they generalize? To explore these questions, rather than develop a new model, we revisit three prominent models, PWC-Net, IRR-PWC and RAFT, with a common set of modern training techniques and datasets, and observe significant performance gains, demonstrating the importance and generality of these traini… ▽ More How important are training details and datasets to recent optical flow models like RAFT? And do they generalize? To explore these questions, rather than develop a new model, we revisit three prominent models, PWC-Net, IRR-PWC and RAFT, with a common set of modern training techniques and datasets, and observe significant performance gains, demonstrating the importance and generality of these training details. Our newly trained PWC-Net and IRR-PWC models show surprisingly large improvements, up to 30% versus original published results on Sintel and KITTI 2015 benchmarks. They outperform the more recent Flow1D on KITTI 2015 while being 3x faster during inference. Our newly trained RAFT achieves an Fl-all score of 4.31% on KITTI 2015, more accurate than all published optical flow methods at the time of writing. Our results demonstrate the benefits of separating the contributions of models, training techniques and datasets when analyzing performance gains of optical flow methods. Our source code will be publicly available. △ Less

Submitted 19 September, 2022; v1 submitted 20 March, 2022; originally announced March 2022.

Comments: Accepted to ECCV22. 33 pages, including supplementals. Website at: https://autoflow-google.github.io/

Kubric: A scalable dataset generator

Authors: Klaus Greff, Francois Belletti, Lucas Beyer, Carl Doersch, Yilun Du, Daniel Duckworth, David J. Fleet, Dan Gnanapragasam, Florian Golemo, Charles Herrmann, Thomas Kipf, Abhijit Kundu, Dmitry Lagun, Issam Laradji, Hsueh-Ti, Liu, Henning Meyer, Yishu Miao, Derek Nowrouzezahrai, Cengiz Oztireli, Etienne Pot, Noha Radwan, Daniel Rebain, Sara Sabour, Mehdi S. M. Sajjadi , et al. (10 additional authors not shown)

Abstract: Data is the driving force of machine learning, with the amount and quality of training data often being more important for the performance of a system than architecture and training details. But collecting, processing and annotating real data at scale is difficult, expensive, and frequently raises additional privacy, fairness and legal concerns. Synthetic data is a powerful tool with the potential… ▽ More Data is the driving force of machine learning, with the amount and quality of training data often being more important for the performance of a system than architecture and training details. But collecting, processing and annotating real data at scale is difficult, expensive, and frequently raises additional privacy, fairness and legal concerns. Synthetic data is a powerful tool with the potential to address these shortcomings: 1) it is cheap 2) supports rich ground-truth annotations 3) offers full control over data and 4) can circumvent or mitigate problems regarding bias, privacy and licensing. Unfortunately, software tools for effective data generation are less mature than those for architecture design and training, which leads to fragmented generation efforts. To address these problems we introduce Kubric, an open-source Python framework that interfaces with PyBullet and Blender to generate photo-realistic scenes, with rich annotations, and seamlessly scales to large jobs distributed over thousands of machines, and generating TBs of data. We demonstrate the effectiveness of Kubric by presenting a series of 13 different generated datasets for tasks ranging from studying 3D NeRF models to optical flow estimation. We release Kubric, the used assets, all of the generation code, as well as the rendered datasets for reuse and modification. △ Less

Submitted 7 March, 2022; originally announced March 2022.

Comments: 21 pages, CVPR2022

Pyramid Adversarial Training Improves ViT Performance

Authors: Charles Herrmann, Kyle Sargent, Lu Jiang, Ramin Zabih, Huiwen Chang, Ce Liu, Dilip Krishnan, Deqing Sun

Abstract: Aggressive data augmentation is a key component of the strong generalization capabilities of Vision Transformer (ViT). One such data augmentation technique is adversarial training (AT); however, many prior works have shown that this often results in poor clean accuracy. In this work, we present pyramid adversarial training (PyramidAT), a simple and effective technique to improve ViT's overall perf… ▽ More Aggressive data augmentation is a key component of the strong generalization capabilities of Vision Transformer (ViT). One such data augmentation technique is adversarial training (AT); however, many prior works have shown that this often results in poor clean accuracy. In this work, we present pyramid adversarial training (PyramidAT), a simple and effective technique to improve ViT's overall performance. We pair it with a "matched" Dropout and stochastic depth regularization, which adopts the same Dropout and stochastic depth configuration for the clean and adversarial samples. Similar to the improvements on CNNs by AdvProp (not directly applicable to ViT), our pyramid adversarial training breaks the trade-off between in-distribution accuracy and out-of-distribution robustness for ViT and related architectures. It leads to 1.82% absolute improvement on ImageNet clean accuracy for the ViT-B model when trained only on ImageNet-1K data, while simultaneously boosting performance on 7 ImageNet robustness metrics, by absolute numbers ranging from 1.76% to 15.68%. We set a new state-of-the-art for ImageNet-C (41.42 mCE), ImageNet-R (53.92%), and ImageNet-Sketch (41.04%) without extra data, using only the ViT-B/16 backbone and our pyramid adversarial training. Our code is publicly available at pyramidat.github.io. △ Less

Submitted 2 September, 2022; v1 submitted 29 November, 2021; originally announced November 2021.

Comments: Accepted to CVPR22 (oral, best paper finalist). 33 pages, including references & supplementary material

Experiments conducted in the burning plasma regime with inertial fusion implosions

Authors: J. S. Ross, J. E. Ralph, A. B. Zylstra, A. L. Kritcher, H. F. Robey, C. V. Young, O. A. Hurricane, D. A. Callahan, K. L. Baker, D. T. Casey, T. Doeppner, L. Divol, M. Hohenberger, S. Le Pape, A. Pak, P. K. Patel, R. Tommasini, S. J. Ali, P. A. Amendt, L. J. Atherton, B. Bachmann, D. Bailey, L. R. Benedetti, L. Berzak Hopkins, R. Betti , et al. (127 additional authors not shown)

Abstract: An experimental program is currently underway at the National Ignition Facility (NIF) to compress deuterium and tritium (DT) fuel to densities and temperatures sufficient to achieve fusion and energy gain. The primary approach being investigated is indirect drive inertial confinement fusion (ICF), where a high-Z radiation cavity (a hohlraum) is heated by lasers, converting the incident energy into… ▽ More An experimental program is currently underway at the National Ignition Facility (NIF) to compress deuterium and tritium (DT) fuel to densities and temperatures sufficient to achieve fusion and energy gain. The primary approach being investigated is indirect drive inertial confinement fusion (ICF), where a high-Z radiation cavity (a hohlraum) is heated by lasers, converting the incident energy into x-ray radiation which in turn drives the DT fuel filled capsule causing it to implode. Previous experiments reported DT fuel gain exceeding unity [O.A. Hurricane et al., Nature 506, 343 (2014)] and then exceeding the kinetic energy of the imploding fuel [S. Le Pape et al., Phys. Rev. Lett. 120, 245003 (2018)]. We report on recent experiments that have achieved record fusion neutron yields on NIF, greater than 100 kJ with momentary fusion powers exceeding 1PW, and have for the first time entered the burning plasma regime where fusion alpha-heating of the fuel exceeds the energy delivered to the fuel via compression. This was accomplished by increasing the size of the high-density carbon (HDC) capsule, increasing energy coupling, while controlling symmetry and implosion design parameters. Two tactics were successful in controlling the radiation flux symmetry and therefore the implosion symmetry: transferring energy between laser cones via plasma waves, and changing the shape of the hohlraum. In conducting these experiments, we controlled for known sources of degradation. Herein we show how these experiments were performed to produce record performance, and demonstrate the data fidelity leading us to conclude that these shots have entered the burning plasma regime. △ Less

Submitted 8 November, 2021; originally announced November 2021.

Deep survival analysis with longitudinal X-rays for COVID-19

Authors: Michelle Shu, Richard Strong Bowen, Charles Herrmann, Gengmo Qi, Michele Santacatterina, Ramin Zabih

Abstract: Time-to-event analysis is an important statistical tool for allocating clinical resources such as ICU beds. However, classical techniques like the Cox model cannot directly incorporate images due to their high dimensionality. We propose a deep learning approach that naturally incorporates multiple, time-dependent imaging studies as well as non-imaging data into time-to-event analysis. Our techniqu… ▽ More Time-to-event analysis is an important statistical tool for allocating clinical resources such as ICU beds. However, classical techniques like the Cox model cannot directly incorporate images due to their high dimensionality. We propose a deep learning approach that naturally incorporates multiple, time-dependent imaging studies as well as non-imaging data into time-to-event analysis. Our techniques are benchmarked on a clinical dataset of 1,894 COVID-19 patients, and show that image sequences significantly improve predictions. For example, classical time-to-event methods produce a concordance error of around 30-40% for predicting hospital admission, while our error is 25% without images and 20% with multiple X-rays included. Ablation studies suggest that our models are not learning spurious features such as scanner artifacts. While our focus and evaluation is on COVID-19, the methods we develop are broadly applicable. △ Less

Submitted 22 August, 2021; originally announced August 2021.

FRET nanoscopy enables seamless imaging of molecular assemblies with sub-nanometer resolution

Authors: Jan-Hendrik Budde, Nicolaas van der Voort, Suren Felekyan, Julian Folz, Ralf Kühnemuth, Paul Lauterjung, Markus Köhler, Andreas Schönle, Julian Sindram, Marius Otten, Matthias Karg, Christian Herrmann, Anders Barth, Claus A. M. Seidel

Abstract: By circumventing the optical diffraction limit, super-resolved fluorescence microscopies enable the study of larger cellular structures and molecular assemblies. However, fluorescence nanoscopy currently lacks the spatiotemporal resolution to resolve distances on the size of individual molecules and reveal the conformational fine structure and dynamics of molecular complexes. Here we establish FRE… ▽ More By circumventing the optical diffraction limit, super-resolved fluorescence microscopies enable the study of larger cellular structures and molecular assemblies. However, fluorescence nanoscopy currently lacks the spatiotemporal resolution to resolve distances on the size of individual molecules and reveal the conformational fine structure and dynamics of molecular complexes. Here we establish FRET nanoscopy by combining colocalization STED microscopy with multiparameter FRET spectroscopy. We simultaneously localize donor and acceptor dyes of single FRET pairs with nanometer resolution and quantitatively measure intramolecular distances with sub-nanometer precision over a large dynamic range. While FRET provides isotropic 3D distance information, colocalization measures the projected distance onto the image plane. The combined information allows us to directly determine its 3D orientation using Pythagoras's theorem. Studying two DNA model systems and the human guanylate binding protein hGBP1, we demonstrate that FRET nanoscopy unravels the interplay between their spatial organization and local molecular conformation in a complex environment. △ Less

Submitted 21 January, 2022; v1 submitted 30 July, 2021; originally announced August 2021.

Comments: Main Text (34 pages, 6 figures) with Supporting Information (90 pages, 29 figures, 20 tables)

AutoFlow: Learning a Better Training Set for Optical Flow

Authors: Deqing Sun, Daniel Vlasic, Charles Herrmann, Varun Jampani, Michael Krainin, Huiwen Chang, Ramin Zabih, William T. Freeman, Ce Liu

Abstract: Synthetic datasets play a critical role in pre-training CNN models for optical flow, but they are painstaking to generate and hard to adapt to new applications. To automate the process, we present AutoFlow, a simple and effective method to render training data for optical flow that optimizes the performance of a model on a target dataset. AutoFlow takes a layered approach to render synthetic data,… ▽ More Synthetic datasets play a critical role in pre-training CNN models for optical flow, but they are painstaking to generate and hard to adapt to new applications. To automate the process, we present AutoFlow, a simple and effective method to render training data for optical flow that optimizes the performance of a model on a target dataset. AutoFlow takes a layered approach to render synthetic data, where the motion, shape, and appearance of each layer are controlled by learnable hyperparameters. Experimental results show that AutoFlow achieves state-of-the-art accuracy in pre-training both PWC-Net and RAFT. Our code and data are available at https://autoflow-google.github.io . △ Less

Submitted 29 April, 2021; originally announced April 2021.

Comments: CVPR 2021

Scaling Beyond Bandwidth Limitations: Wireless Control With Stability Guarantees Under Overload

Authors: Fabian Mager, Dominik Baumann, Carsten Herrmann, Sebastian Trimpe, Marco Zimmerling

Abstract: An important class of cyber-physical systems relies on multiple agents that jointly perform a task by coordinating their actions over a wireless network. Examples include self-driving cars in intelligent transportation and production robots in smart manufacturing. However, the scalability of existing control-over-wireless solutions is limited as they cannot resolve overload situations in which the… ▽ More An important class of cyber-physical systems relies on multiple agents that jointly perform a task by coordinating their actions over a wireless network. Examples include self-driving cars in intelligent transportation and production robots in smart manufacturing. However, the scalability of existing control-over-wireless solutions is limited as they cannot resolve overload situations in which the communication demand exceeds the available bandwidth. This paper presents a novel co-design of distributed control and wireless communication that overcomes this limitation by dynamically allocating the available bandwidth to agents with the greatest need to communicate. Experiments on a real cyber-physical testbed with 20 agents, each consisting of a low-power wireless embedded device and a cart-pole system, demonstrate that our solution achieves significantly better control performance under overload than the state of the art. We further prove that our co-design guarantees closed-loop stability for physical systems with stochastic linear time-invariant dynamics. △ Less

Submitted 25 January, 2022; v1 submitted 16 April, 2021; originally announced April 2021.

Comments: Accepted article to appear in ACM Transactions on Cyber-Physical Systems

Hunting for Dark Matter and New Physics with GECCO

Authors: Adam Coogan, Alexander Moiseev, Logan Morrison, Stefano Profumo, Matthew G. Baring, Aleksey Bolotnikov, Gabriella A. Carini, Sven C. Herrmann, Francesco Longo, Floyd W. Stecker, Alessandro Armando Vigliano, Richard S. Woolf

Abstract: We outline the science opportunities in the areas of searches for dark matter and new physics offered by a proposed future MeV gamma-ray telescope, the Galactic Explorer with a Coded Aperture Mask Compton Telescope (GECCO). We point out that such an instrument would play a critical role in opening up a discovery window for particle dark matter with mass in the MeV or sub-MeV range, in disentanglin… ▽ More We outline the science opportunities in the areas of searches for dark matter and new physics offered by a proposed future MeV gamma-ray telescope, the Galactic Explorer with a Coded Aperture Mask Compton Telescope (GECCO). We point out that such an instrument would play a critical role in opening up a discovery window for particle dark matter with mass in the MeV or sub-MeV range, in disentangling the origin of the mysterious 511 keV line emission in the Galactic Center region, and in potentially discovering Hawking evaporation from light primordial black holes. △ Less

Submitted 2 May, 2023; v1 submitted 25 January, 2021; originally announced January 2021.

Comments: 27 pages, 10 figures, ~matches published version

Journal ref: Phys. Rev. D 107, 023022 (2023)

Selection, recombination, and the ancestral initiation graph

Authors: Frederic Alberti, Carolin Herrmann, Ellen Baake

Abstract: Recently, the selection-recombination equation with a single selected site and an arbitrary number of neutral sites was solved by means of the ancestral selection-recombination graph. Here, we introduce a more accessible approach, namely the ancestral initiation graph. The construction is based on a discretisation of the selection-recombination equation. We apply our method to systematically expla… ▽ More Recently, the selection-recombination equation with a single selected site and an arbitrary number of neutral sites was solved by means of the ancestral selection-recombination graph. Here, we introduce a more accessible approach, namely the ancestral initiation graph. The construction is based on a discretisation of the selection-recombination equation. We apply our method to systematically explain a long-standing observation concerning the dynamics of linkage disequilibrium between two neutral loci hitchhiking along with a selected one. In particular, this clarifies the nontrivial dependence on the position of the selected site. △ Less

Submitted 7 September, 2021; v1 submitted 25 January, 2021; originally announced January 2021.

Journal ref: Theor. Popul. Biol. 142 (2021) 46-56

Computationally Assessing Diamond as an Ultrafast Pulse Shaper for High Power Ultrawide Band Radar

Authors: Christopher C. Herrmann, Joseph Croman, Sergey V. Baryshev

Abstract: Diamond holds promise to reshape ultrafast and high power electronics. One such solid-state device is the diode avalanche shaper (DAS), which functions as an ultrafast closing switch where closing is caused by the formation of the streamer traversing the diode much faster than 10$^7$ cm/s. One of the most prominent applications of DAS is in ultrawide band (UWB) radio/radar. Here we simulate a diam… ▽ More Diamond holds promise to reshape ultrafast and high power electronics. One such solid-state device is the diode avalanche shaper (DAS), which functions as an ultrafast closing switch where closing is caused by the formation of the streamer traversing the diode much faster than 10$^7$ cm/s. One of the most prominent applications of DAS is in ultrawide band (UWB) radio/radar. Here we simulate a diamond-based DAS and compare the results to a silicon-based DAS. All DAS were simulated in mixed mode as ideal devices using the drift-diffusion model. The simulations show that diamond DAS promises to outperform Si DAS when sharpening kilovolt nanosecond input pulse. The breakdown field and streamer velocity ($\sim$10 times larger in diamond as compared to those in Si) are likely to be the major reasons enabling kV sub-50 ps switching using diamond DAS. △ Less

Submitted 14 July, 2023; v1 submitted 25 November, 2020; originally announced November 2020.

Object-centered image stitching

Authors: Charles Herrmann, Chen Wang, Richard Strong Bowen, Emil Keyder, Ramin Zabih

Abstract: Image stitching is typically decomposed into three phases: registration, which aligns the source images with a common target image; seam finding, which determines for each target pixel the source image it should come from; and blending, which smooths transitions over the seams. As described in [1], the seam finding phase attempts to place seams between pixels where the transition between source im… ▽ More Image stitching is typically decomposed into three phases: registration, which aligns the source images with a common target image; seam finding, which determines for each target pixel the source image it should come from; and blending, which smooths transitions over the seams. As described in [1], the seam finding phase attempts to place seams between pixels where the transition between source images is not noticeable. Here, we observe that the most problematic failures of this approach occur when objects are cropped, omitted, or duplicated. We therefore take an object-centered approach to the problem, leveraging recent advances in object detection [2,3,4]. We penalize candidate solutions with this class of error by modifying the energy function used in the seam finding stage. This produces substantially more realistic stitching results on challenging imagery. In addition, these methods can be used to determine when there is non-recoverable occlusion in the input data, and also suggest a simple evaluation metric that can be used to evaluate the output of stitching algorithms. △ Less

Submitted 23 November, 2020; originally announced November 2020.

Comments: ECCV 2018

Robust image stitching with multiple registrations

Authors: Charles Herrmann, Chen Wang, Richard Strong Bowen, Emil Keyder, Michael Krainin, Ce Liu, Ramin Zabih

Abstract: Panorama creation is one of the most widely deployed techniques in computer vision. In addition to industry applications such as Google Street View, it is also used by millions of consumers in smartphones and other cameras. Traditionally, the problem is decomposed into three phases: registration, which picks a single transformation of each source image to align it to the other inputs, seam finding… ▽ More Panorama creation is one of the most widely deployed techniques in computer vision. In addition to industry applications such as Google Street View, it is also used by millions of consumers in smartphones and other cameras. Traditionally, the problem is decomposed into three phases: registration, which picks a single transformation of each source image to align it to the other inputs, seam finding, which selects a source image for each pixel in the final result, and blending, which fixes minor visual artifacts. Here, we observe that the use of a single registration often leads to errors, especially in scenes with significant depth variation or object motion. We propose instead the use of multiple registrations, permitting regions of the image at different depths to be captured with greater accuracy. MRF inference techniques naturally extend to seam finding over multiple registrations, and we show here that their energy functions can be readily modified with new terms that discourage duplication and tearing, common problems that are exacerbated by the use of multiple registrations. Our techniques are closely related to layer-based stereo, and move image stitching closer to explicit scene modeling. Experimental evidence demonstrates that our techniques often generate significantly better panoramas when there is substantial motion or parallax. △ Less

Submitted 23 November, 2020; originally announced November 2020.

Comments: ECCV 2018

A Chirality-Based Quantum Leap

Authors: Clarice D. Aiello, Muneer Abbas, John M. Abendroth, Andrei Afanasev, Shivang Agarwal, Amartya S. Banerjee, David N. Beratan, Jason N. Belling, Bertrand Berche, Antia Botana, Justin R. Caram, Giuseppe Luca Celardo, Gianaurelio Cuniberti, Aitzol Garcia-Etxarri, Arezoo Dianat, Ismael Diez-Perez, Yuqi Guo, Rafael Gutierrez, Carmen Herrmann, Joshua Hihath, Suneet Kale, Philip Kurian, Ying-Cheng Lai, Alexander Lopez, Ernesto Medina , et al. (19 additional authors not shown)

Abstract: Chiral degrees of freedom occur in matter and in electromagnetic fields and constitute an area of research that is experiencing renewed interest driven by recent observations of the chiral-induced spin selectivity (CISS) effect in chiral molecules and engineered nanomaterials. The CISS effect underpins the fact that charge transport through nanoscopic chiral structures favors a particular electron… ▽ More Chiral degrees of freedom occur in matter and in electromagnetic fields and constitute an area of research that is experiencing renewed interest driven by recent observations of the chiral-induced spin selectivity (CISS) effect in chiral molecules and engineered nanomaterials. The CISS effect underpins the fact that charge transport through nanoscopic chiral structures favors a particular electronic spin orientation, resulting in large room-temperature spin polarizations. Observations of the CISS effect suggest opportunities for spin control and for the design and fabrication of room-temperature quantum devices from the bottom up, with atomic-scale precision. Any technology that relies on optimal charge transport, including quantum devices for logic, sensing, and storage, may benefit from chiral quantum properties. These properties can be theoretically and experimentally investigated from a quantum information perspective, which is presently lacking. There are uncharted implications for the quantum sciences once chiral couplings can be engineered to control the storage, transduction, and manipulation of quantum information. This forward-looking perspective provides a survey of the experimental and theoretical fundamentals of chiral-influenced quantum effects, and presents a vision for their future roles in enabling room-temperature quantum technologies. △ Less

Submitted 11 November, 2021; v1 submitted 31 August, 2020; originally announced September 2020.

Root Cause Analysis in Lithium-Ion Battery Production with FMEA-Based Large-Scale Bayesian Network

Authors: Michael Kirchhof, Klaus Haas, Thomas Kornas, Sebastian Thiede, Mario Hirz, Christoph Herrmann

Abstract: The production of lithium-ion battery cells is characterized by a high degree of complexity due to numerous cause-effect relationships between process characteristics. Knowledge about the multi-stage production is spread among several experts, rendering tasks as failure analysis challenging. In this paper, a new method is presented that includes expert knowledge acquisition in production ramp-up b… ▽ More The production of lithium-ion battery cells is characterized by a high degree of complexity due to numerous cause-effect relationships between process characteristics. Knowledge about the multi-stage production is spread among several experts, rendering tasks as failure analysis challenging. In this paper, a new method is presented that includes expert knowledge acquisition in production ramp-up by combining Failure Mode and Effects Analysis (FMEA) with a Bayesian Network. Special algorithms are presented that help detect and resolve inconsistencies between the expert-provided parameters which are bound to occur when collecting knowledge from several process experts. We show the effectiveness of this holistic method by building up a large scale, cross-process Bayesian Failure Network in lithium-ion battery production and its application for root cause analysis. △ Less

Submitted 15 June, 2020; v1 submitted 5 June, 2020; originally announced June 2020.

Comments: Submitted to CIRP Journal of Manufacturing Science and Technology (01.2020)

MSC Class: 62P30 ACM Class: I.2.1

Learning to Autofocus

Authors: Charles Herrmann, Richard Strong Bowen, Neal Wadhwa, Rahul Garg, Qiurui He, Jonathan T. Barron, Ramin Zabih

Abstract: Autofocus is an important task for digital cameras, yet current approaches often exhibit poor performance. We propose a learning-based approach to this problem, and provide a realistic dataset of sufficient size for effective learning. Our dataset is labeled with per-pixel depths obtained from multi-view stereo, following "Learning single camera depth estimation using dual-pixels". Using this data… ▽ More Autofocus is an important task for digital cameras, yet current approaches often exhibit poor performance. We propose a learning-based approach to this problem, and provide a realistic dataset of sufficient size for effective learning. Our dataset is labeled with per-pixel depths obtained from multi-view stereo, following "Learning single camera depth estimation using dual-pixels". Using this dataset, we apply modern deep classification models and an ordinal regression loss to obtain an efficient learning-based autofocus technique. We demonstrate that our approach provides a significant improvement compared with previous learned and non-learned methods: our model reduces the mean absolute error by a factor of 3.6 over the best comparable baseline algorithm. Our dataset and code are publicly available. △ Less

Submitted 2 May, 2020; v1 submitted 25 April, 2020; originally announced April 2020.

Comments: CVPR 2020

Kondo screening in Co adatoms with full Coulomb interaction

Authors: Angelo Valli, Marc Philipp Bahlke, Alexander Kowalski, Michael Karolak, Carmen Herrmann, Giorgio Sangiovanni

Abstract: Using a numerically exact first-principles many-body approach, we revisit the "prototypical" Kondo case of a cobalt impurity on copper. Even though this is considered a well understood example of the Kondo effect, we reveal an unexpectedly strong dependence of the screening properties on the parametrization of the local Coulomb tensor. As a consequence, the Kondo temperature can vary by orders of… ▽ More Using a numerically exact first-principles many-body approach, we revisit the "prototypical" Kondo case of a cobalt impurity on copper. Even though this is considered a well understood example of the Kondo effect, we reveal an unexpectedly strong dependence of the screening properties on the parametrization of the local Coulomb tensor. As a consequence, the Kondo temperature can vary by orders of magnitude depending on the complexity of the parametrization of the electron-electron interaction. Further, we challenge the established picture of a spin-$1$ moment involving two cobalt $d$-orbitals only, as orbital-mixing interaction terms boost the contribution of the remainder of the $d$-shell. △ Less

Submitted 5 October, 2020; v1 submitted 14 April, 2020; originally announced April 2020.

Comments: 15 pages, 14 figures; final version with extended bibliography and appendix

Journal ref: Phys. Rev. Research 2, 033432 (2020)

Integrative dynamic structural biology unveils conformers essential for the oligomerization of a large GTPase

Authors: Thomas-Otavio Peulen, Carola S. Hengstenberg, Ralf Biehl, Mykola Dimura, Charlotte Lorenz, Alessandro Valeri, Semra Ince, Tobias Vöpel, Bela Faragó, Holger Gohlke, Johann P. Klare, Andreas M. Stadler, Claus A. M. Seidel, Christian Herrmann

Abstract: Guanylate binding proteins (GBPs) are soluble dynamin-like proteins with structured domains that undergo a conformational transition for GTP-controlled oligomerization to exert their function as part of the innate immune system of mammalian cells - attacking intra-cellular parasites by disrupting their membranes. The structural basis and mechanism of this process is unknown. Therefore, we apply ne… ▽ More Guanylate binding proteins (GBPs) are soluble dynamin-like proteins with structured domains that undergo a conformational transition for GTP-controlled oligomerization to exert their function as part of the innate immune system of mammalian cells - attacking intra-cellular parasites by disrupting their membranes. The structural basis and mechanism of this process is unknown. Therefore, we apply neutron spin echo, X-ray scattering, fluorescence, and EPR spectroscopy as techniques for integrative dynamic structural biology to human GBP1 (hGBP1). We mapped hGBP1's essential dynamics from nanoseconds to milliseconds by motional spectra of sub-domains. We find a GTP-independent flexibility of the C-terminal effector domain in the $μ$s-regime and structurally characterize conformers being essential that hGBP1 can open like a pocketknife for oligomerization. This unveils the intrinsic flexibility, a GTP-triggered association of the GTPase-domains and assembly-dependent GTP-hydrolysis as functional design principles of hGBP1 that control its reversible oligomerization in polar assemblies and the subsequent formation of condensates. △ Less

Submitted 8 April, 2020; originally announced April 2020.

Comments: 83 pages

Test sets for tautologies in modular quantum logic

Authors: Christian Herrmann

Abstract: As defined by Dunn, Moss, and Wang, an universal test set in an ortholattice $L$ is a subset $T$ such that each term takes value $1$, only, if it does so under all substitutions from $T$. Generalizing their result for ortholattices of subspaces of finite dimensional Hilbert spaces, we show that no infinite modular ortholattice of finite dimension admits a finite universal test set. On the other ha… ▽ More As defined by Dunn, Moss, and Wang, an universal test set in an ortholattice $L$ is a subset $T$ such that each term takes value $1$, only, if it does so under all substitutions from $T$. Generalizing their result for ortholattices of subspaces of finite dimensional Hilbert spaces, we show that no infinite modular ortholattice of finite dimension admits a finite universal test set. On the other hand, answering a question of the same authors, we provide a countable universal test set for the ortholattice of projections of any type II$_1$ von Neumann algebra factor as well as for von Neumann's algebraic construction of a continuous geometry. These universal test sets consist of elements having rational normalized dimension with denominator a power of $2$. △ Less

Submitted 16 February, 2020; originally announced February 2020.

MSC Class: 06C20; 81P10

Current vortices in aromatic carbon molecules

Authors: Thomas Stegmann, John A. Franco-Villafañe, Yenni P. Ortiz, Michael Deffner, Carmen Herrmann, Ulrich Kuhl, Fabrice Mortessagne, Francois Leyvraz, Thomas H. Seligman

Abstract: The local current flow through three small aromatic carbon molecules, namely benzene, naphthalene and anthracene, is studied. Applying density functional theory and the non-equilibrium Green's function method for transport, we demonstrate that pronounced current vortices exist at certain electron energies for these molecules. The intensity of these circular currents, which appear not only at the a… ▽ More The local current flow through three small aromatic carbon molecules, namely benzene, naphthalene and anthracene, is studied. Applying density functional theory and the non-equilibrium Green's function method for transport, we demonstrate that pronounced current vortices exist at certain electron energies for these molecules. The intensity of these circular currents, which appear not only at the anti-resonances of the transmission but also in vicinity of its maxima, can exceed the total current flowing through the molecular junction and generate considerable magnetic fields. The $π$ electron system of the molecular junctions is emulated experimentally by a network of macroscopic microwave resonators. The local current flows in these experiments confirm the existence of current vortices as a robust property of ring structures. The circular currents can be understood in terms of a simple nearest-neighbor tight-binding Hückel model. Current vortices are caused by the interplay of the complex eigenstates of the open system which have energies close-by the considered electron energy. Degeneracies, as observed in benzene and anthracene, can thus generate strong circular currents, but also non-degenerate systems like naphthalene exhibit current vortices. Small imperfections and perturbations can couple otherwise uncoupled states and induce circular currents. △ Less

Submitted 21 January, 2020; originally announced January 2020.

Journal ref: Phys. Rev. B 102, 075405 (2020)

A Note on the "Third Life of Quantum Logic"

Authors: Christian Herrmann

Abstract: The purpose of this note is to discuss some of the questions raised by Dunn, J. Michael; Moss, Lawrence S.; Wang, Zhenghan in Editors' introduction: the third life of quantum logic: quantum logic inspired by quantum computing. The purpose of this note is to discuss some of the questions raised by Dunn, J. Michael; Moss, Lawrence S.; Wang, Zhenghan in Editors' introduction: the third life of quantum logic: quantum logic inspired by quantum computing. △ Less

Submitted 22 September, 2019; v1 submitted 7 August, 2019; originally announced August 2019.

MSC Class: 06C20; 81P10

Representations of *-regular rings and their ortholattices of projections

Authors: Christian Herrmann, Niklas Niemann

Abstract: We show that a subdirectly irreducible *-regular ring admits a representation within some inner product space provided so does its ortholattice of projections. We show that a subdirectly irreducible *-regular ring admits a representation within some inner product space provided so does its ortholattice of projections. △ Less

Submitted 16 February, 2020; v1 submitted 1 August, 2019; originally announced August 2019.

MSC Class: 06C20; 16E50; 16W10

Unit-regularity and representability for semiartinian *-regular rings

Authors: Christian Herrmann

Abstract: We show that any semiartinian *-regular ring R is unit-regular; if, in addition, R is subdirectly irreducible then it admits a representation within some inner product space. There is no proof of the result since a quoted result is incorrect We show that any semiartinian *-regular ring R is unit-regular; if, in addition, R is subdirectly irreducible then it admits a representation within some inner product space. There is no proof of the result since a quoted result is incorrect △ Less

Submitted 29 August, 2024; v1 submitted 31 July, 2019; originally announced July 2019.

MSC Class: 16E50; 16W10

(Co)isotropic triples and poset representations

Authors: Christian Herrmann, Jonathan Lorand, Alan Weinstein

Abstract: We study triples of coisotropic or isotropic subspaces in symplectic vector spaces; in particular, we classify indecomposable structures of this kind. The classification depends on the ground field, which we only assume to be perfect and not of characteristic 2. Our work uses the theory of representations of partially ordered sets with (order reversing) involution; for (co)isotropic triples, the r… ▽ More We study triples of coisotropic or isotropic subspaces in symplectic vector spaces; in particular, we classify indecomposable structures of this kind. The classification depends on the ground field, which we only assume to be perfect and not of characteristic 2. Our work uses the theory of representations of partially ordered sets with (order reversing) involution; for (co)isotropic triples, the relevant poset is "$2 + 2 + 2$" consisting of three independent ordered pairs, with the involution exchanging the members of each pair. A key feature of the classification is that any indecomposable (co)isotropic triple is either "split" or "non-split". The latter is the case when the poset representation underlying an indecomposable (co)isotropic triple is itself indecomposable. Otherwise, in the "split" case, the underlying representation is decomposable and necessarily the direct sum of a dual pair of indecomposable poset representations; the (co)isotropic triple is a "symplectification". In the course of the paper we develop the framework of "symplectic poset representations", which can be applied to a range of problems of symplectic linear algebra. The classification of linear Hamiltonian vector fields, up to conjugation, is an example; we briefly explain the connection between these and (co)isotropic triples. The framework lends itself equally well to studying poset representations on spaces carrying a non-degenerate symmetric bilinear form; we mainly keep our focus, however, on the symplectic side. △ Less

Submitted 11 June, 2019; originally announced June 2019.

Comments: 91 pages

MSC Class: 15A21; 53D99; 16G20

Varieties of *-regular rings

Authors: Christian Herrmann

Abstract: Given a subdirectly irreducible *-regular ring R, we show that R is a homomorphic image of a regular *-subring of an ultraproduct of the (simple) eRe, e in the minimal ideal of R; moreover, R (with unit) is directly finite if all eRe are unit-regular. Finally, unit-regularity is shown for every member of the variety generated by artinian *-regular rings (endowed with unit and pseudo-inversion). Le… ▽ More Given a subdirectly irreducible *-regular ring R, we show that R is a homomorphic image of a regular *-subring of an ultraproduct of the (simple) eRe, e in the minimal ideal of R; moreover, R (with unit) is directly finite if all eRe are unit-regular. Finally, unit-regularity is shown for every member of the variety generated by artinian *-regular rings (endowed with unit and pseudo-inversion). Lemma 5 is incorrect, whence so is Thm 4. There is a counterexample to be publish3d, elsewhere △ Less

Submitted 29 August, 2024; v1 submitted 9 April, 2019; originally announced April 2019.

MSC Class: 16E50; 16W10

Direct finiteness of representable regular rings with involution

Authors: Christian Herrmann

Abstract: We show that a von Neumann regular ring with involution is directly finite provided that it admits a representation as a ring of endomorphisms (the involution given by taking adjoints) of a vector space endowed with a non-degenerate orthosymmetric sesquilinear form. Actually, we hint to an error in the claimed proof and to a counterexample We show that a von Neumann regular ring with involution is directly finite provided that it admits a representation as a ring of endomorphisms (the involution given by taking adjoints) of a vector space endowed with a non-degenerate orthosymmetric sesquilinear form. Actually, we hint to an error in the claimed proof and to a counterexample △ Less

Submitted 26 August, 2024; v1 submitted 11 January, 2019; originally announced January 2019.

MSC Class: 16E50; 16W10

On the Finiteness Problem for classes of modular lattices

Authors: Christian Herrmann

Abstract: The Finiteness Problem is shown to be unsolvable for any sufficiently large class of modular lattices. The Finiteness Problem is shown to be unsolvable for any sufficiently large class of modular lattices. △ Less

Submitted 10 January, 2019; originally announced January 2019.

MSC Class: 06C05; 03D35

Channel selection using Gumbel Softmax

Authors: Charles Herrmann, Richard Strong Bowen, Ramin Zabih

Abstract: Important applications such as mobile computing require reducing the computational costs of neural network inference. Ideally, applications would specify their preferred tradeoff between accuracy and speed, and the network would optimize this end-to-end, using classification error to remove parts of the network. Increasing speed can be done either during training - e.g., pruning filters - or durin… ▽ More Important applications such as mobile computing require reducing the computational costs of neural network inference. Ideally, applications would specify their preferred tradeoff between accuracy and speed, and the network would optimize this end-to-end, using classification error to remove parts of the network. Increasing speed can be done either during training - e.g., pruning filters - or during inference - e.g., conditionally executing a subset of the layers. We propose a single end-to-end framework that can improve inference efficiency in both settings. We use a combination of batch activation loss and classification loss, and Gumbel reparameterization to learn network structure. We train end-to-end, and the same technique supports pruning as well as conditional computation. We obtain promising experimental results for ImageNet classification with ResNet (45-52% less computation). △ Less

Submitted 23 November, 2020; v1 submitted 10 December, 2018; originally announced December 2018.

Comments: ECCV 2020