¹¹institutetext: Computational Brain Science Lab, Division of Computational Science and Technology, KTH Royal Institute of Technology, SE-100 44 Stockholm, Sweden. ¹¹email: tony@kth.se

Do the receptive fields in the primary visual cortex span a variability over the degree of elongation of the receptive fields?^†^†thanks: The support from the Swedish Research Council (contract 2022-02969) is gratefully acknowledged.

Tony Lindeberg

Abstract

This paper presents the results of combining (i) theoretical analysis regarding connections between the orientation selectivity and the elongation of receptive fields for the affine Gaussian derivative model with (ii) biological measurements of orientation selectivity in the primary visual cortex, to investigate if (iii) the receptive fields can be regarded as spanning a variability in the degree of elongation.

From an in-depth theoretical analysis of idealized models for the receptive fields of simple and complex cells in the primary visual cortex, we have established that the orientation selectivity becomes more narrow with increasing elongation of the receptive fields. Combined with previously established biological results, concerning broad vs. sharp orientation tuning of visual neurons in the primary visual cortex, as well as previous experimental results concerning distributions of the resultant of the orientation selectivity curves for simple and complex cells, we show that these results are consistent with the receptive fields spanning a variability over the degree of elongation of the receptive fields. We also show that our principled theoretical model for visual receptive fields leads to qualitatively similar types of deviations from a uniform histogram of the resultant descriptor of the orientation selectivity curves for simple cells, as can be observed in the results from biological experiments.

To firmly determine if the underlying working hypothesis, about the receptive fields spanning a variability in the degree of elongation, would truly hold for the receptive fields in the primary visual cortex of higher mammals, we formulate a set of testable predictions, that can be used for investigating this property experimentally, and, if applicable, then also characterize if such a variability would, in a structured way, be related to the pinwheel structure in the visual cortex.

Keywords:

Receptive field Elongation Affine covariance Orientation selectivity Gaussian derivative Quasi quadrature Simple cell Complex cell Pinwheel Vision Theoretical neuroscience

^†^†journal: arXiv preprint

Refer to caption — Figure 1: Variabilities in image structures generated by viewing the same surface patterns from different viewing directions. Observe how the resulting perspective transformations lead to strong foreshortening effects, in that the image structures in one direction in the 2-D image space are compressed more than the image structures in the orthogonal direction. Here, where the viewpoint of the observer is moved horizontally in the world, the foreshortening effect is mainly along the horizontal direction, although complemented also with small rotations for non-central image points, because of using a planar image plane as opposed to a spherical retina. To first order of approximation of the projective mappings between pairwise views, these resulting image deformations can be modelled in terms of local affine transformations.

$\kappa=1$	$\kappa=2\sqrt{2}$	$\kappa=2$	$\kappa=2\sqrt{2}$	$\kappa=4$	$\kappa=4\sqrt{2}$

Figure 2: Variability in the elongation of affine Gaussian derivative receptive fields (for image orientation

\varphi=0

), with the scale parameter ratio

\kappa=\sigma_{2}/\sigma_{1}

increasing from

1

4\sqrt{2}

according to a logarithmic distribution, from left to right, with the vertical scale parameter kept constant

\sigma_{2}=4

and with the horizontal scale parameter being the smaller

\sigma_{1}\leq\sigma_{2}

. (first row) First-order directional derivatives of affine Gaussian kernels according to (5) for

m=1

. (second row) Second-order directional derivatives of affine Gaussian kernels according to (5) for

m=2

. (third row) Second-order directional derivatives of affine Gaussian kernels according to (5) for

m=3

. (fourth row) Second-order directional derivatives of affine Gaussian kernels according to (5) for

m=4

. When the image structures are subject to foreshortening transformations, because of varying slant angles between the local surface normals and the viewing direction, the shapes of the spatial components of the receptive fields need to be adapted accordingly, to achieve affine covariance in the sense that the receptive field responses should be possibly to be appropriately matched under different viewing conditions. (Horizontal axes: image coordinate

x_{1}\in[-16,16]

. Vertical axes: image coordinate

x_{2}\in[-16,16]

1 Introduction

When observing objects and events in our natural environment, the image structures in the visual stimuli will be subject to substantial variabilities caused by the natural image transformations. Specifically, if observing a smooth local surface patch from different viewing directions and viewing distances, this variability can, to first order of approximation, be approximated by local affine transformations (the derivative of the projective mappings between the different views). Within the 4-D variability of general centered 2-D affine transformations, a 1-D variability in the slant angle of the surface normal relative to the viewing direction does, in terms of covariance properties, correspond to a variability in the elongation of the receptive fields, if we would like the responses to be possible to perfectly match under such variabilities (see Figures 1 and 2 for illustrations and Section 2.1 for a complementary theoretical background).

In Lindeberg (2021; 2023; 2024a), we have outlined a framework for how covariance properties with respect to geometric image transformations may constitute a fundamental constraint for the receptive fields in the primary visual cortex of higher mammals, to enable the visual computations to be robust under the variabilities in the image structures generated by the natural image transformations. According to the presented theory, based on axiomatically determined receptive field shapes derived from symmetry properties that reflect structural properties of the environment, in combination with additional constraints to guarantee consistency between image representations over multiple spatial and temporal scales, the population of receptive fields in the primary visual cortex ought to, according to this theory, obey covariance properties with respect to spatial affine transformations and Galilean transformations.

While overall qualitative comparisons between predictions from this principled theory have been successfully made to neurophysiological recordings of receptive fields of simple cells by DeAngelis et al. (1995; 2004), Conway and Livingstone (2006) and Johnson et al. (2008), publicly available data regarding full receptive field recordings are quite limited, why further experimental evidence would be needed to firmly either reject or support the stated hypotheses about affine covariance and Galilean covariance.

In the lack of such neurophysiological data regarding full receptive field recordings, one could, however, aim to instead obtain indirect cues regarding a possible variability in the degree of elongation of the receptive fields, by making use of the recordings of the orientation selectivity of visual neurons by Nauhaus et al. (2008), which show a substantial variability regarding broad vs. sharp tuning of the receptive fields in the primary visual cortex, as well as by Goris et al. (2015), who report comparably uniform distributions of the degree of orientation selectivity of simple and complex cells, in terms of histograms of the resultant of the orientation selectivity curves.

In a companion paper (Lindeberg 2024b), we have established a strong direct link between the orientation selectivity and the elongation of the receptive fields according to the idealized generalized Gaussian derivative model for visual receptive fields (as will be summarized in Section 2.2). If we would assume that that generalized Gaussian derivative model would constitute a sufficiently valid model for the population of simple and complex cells in the primary visual cortex, then we could logically infer possible indirect support for the working hypothesis, in that the observed variability in the orientation selectivity of the receptive fields would correspond to a variability in the degree of elongation of the receptive fields.

1.1 The hypothesis about affine covariant receptive fields

If we assume that the visual system should implement affine covariant receptive fields (Lindeberg 2023 Section 3.2), then the property of affine covariance would make it possible to compute better estimates of local surface orientation, compared to a visual system that does not implement affine covariance, or a sufficiently good approximation thereof.

A general motivation for the wider underlying working hypothesis about affine covariance is that, if the population of receptive fields would support affine covariance in the primary visual cortex, or sufficiently good approximations thereof, then such an ability would support the possibility of computing affine invariant image representations at higher levels in the visual hierarchy (Lindeberg 2013b), or more realistically sufficiently good approximations thereof, over restricted subspaces or subdomains of the most general forms of full variability under spatial affine transformations of the visual stimuli.

Fundamentally, we cannot expect the visual perception system to implement full affine invariance. For example, from the well-known experience, that it is much harder to read text upside-down, it is clear that the visual perception system cannot be regarded as invariant to spatial rotations in the image domain. However, from the expansion of the orientations of visual receptive fields according to the pinwheel structure of higher mammals, we can regard the population of receptive fields as supporting local rotational covariance.

When we look at a slanted surface in the world, we can get a robust and stable perception of its surface texture under substantial variations of the slant angle. This robustness of the visual perception system under stretchings of image patterns that correspond to non-uniform scaling transformations (the perspective effects on a slanted surface patch can, to first-order of approximation, be modelled as a stretching of the image pattern along the tilt direction in image space, complemented with a uniform scaling transformation). If the visual receptive fields would span a variability under such spatial stretching transformations, then such a variability would precisely correspond to a variability in the anisotropy, or the degree of elongation, of the receptive fields.

1.2 Variability over the elongation of receptive fields

The overall theme of this article is to, based on a theoretical analysis of a relationship between the orientation selectivity and the degree of anisotropy or degree of elongation of the receptive fields in the primary visual cortex, in combination with existing neurophysiological results concerning variabilities in the orientation selectivity of the receptive fields in relation to the pinwheel structure of higher mammals, address the question of whether the receptive fields in the primary visual cortex could be regarded as spanning a variability over the elongation of the receptive fields, to support covariant image measurements under such variabilities, or, at least, sufficiently good approximations thereof.

In particular, if we could assume that the spatial components of the receptive fields could be well modelled by affine Gaussian derivatives, then the property of affine covariance implies that there should be visual receptive fields for different degrees of anisotropy present in the visual system. If we further combine the theoretical results used in this paper, which state that the degree of orientation selectivity is strongly dependent on the anisotropy or the elongation of the receptive fields, with the biological results established by Nauhaus et al. (2008), which show that the degree of orientation selectivity for neurons in the primary visual cortex varies both strongly and in relationship with the position on the cortical surface in relation to the pinwheel structure. Then, these results together are fully consistent with the hypothesis that the visual receptive fields in the primary visual cortex should span a variability in their anisotropy, thus consistent with the hypothesis that the receptive fields in the pinwheel structure should span at least one more degree of freedom in the affine group, beyond mere rotations (as already established in previous neurophysiological measurements regarding the pinwheel structure of the oriented receptive fields in the primary visual cortex of higher mammals).

We will also, more generally, use predictions from the presented theoretical analysis to formulate a set of explicit, testable hypotheses, that could be either verified or rejected in future neurophysiological experiments. Additionally, we will formulate a set of quantitive measurements to be made, to characterize a possible variability in the anisotropy or elongation of receptive fields in the primary visual cortex, with special emphasis on the relationships between a possibly predicted variability in receptive field elongation and the pinwheel structure in the primary visual cortex of higher mammals.

1.3 Contributions and novelty

In summary, the purposes of this paper are twofold:

•

to in the current absence of biological measurements about a possible variability of the degree of elongation of receptive field shapes in the primary visual cortex, provide possible indirect support for such a hypothesis, based on a combination of previously established variability in the degree of the orientation selectivity of biological receptive fields with a model-based connection between the degree of elongation of the receptive fields and their orientation selectivity, express possible indirect support for that hypothesis, and
•

to formulate a set of theoretically motivated and experimentally testable predictions and quantitative measurements, that could be used by experimentalists for ultimately judging whether the formulated hypothesis about a variability over the degree of elongation of the receptive fields would hold in the primary visual cortex of higher mammals.

The main novel contributions of the paper are thus specifically:

•

the theoretical modelling based approach, that, in a theoretical neuroscience way, (in Section 3.2) establishes relations between biological measurements regarding another characteristic property of visual neurons in terms of orientation selectivity, for which experimental data are available, to the desirable property in terms of the degree of elongation of the receptive fields, and
•

the formulation of the set of biological predictions in Section 3.3, based on the combination of existing biological results with the results from the principled theoretical modelling-based analysis of relationships between the orientation selectivity and the degree of elongation for simple and complex cells in the primary visual cortex.

A further underlying motivation with this work is to lay out a conceptual foundation, by which theoreticians and experimentalists could join efforts, to establish to what extent the distributions of the shapes of the biological receptive fields would be compatible with an explanation from the fundamental constraint, that the family of receptive fields should be able to handle variabilities in the image data caused by geometric image transformations.

2 Methods

In this section, we: (i) give a theoretical background regarding the notion of affine covariant visual receptive fields, which constitutes the conceptual background for the hypotheses studied in this work; (ii) describe how the orientation selectivity of receptive fields is related to the degree of elongation of the receptive fields, based on an in-depth theoretical analysis of visual receptive fields according to the generalized Gaussian derivative model; and (iii) relate the computational modelling approaches taken and the contributions presented in this work to previous work in the field.

2.1 Affine covariant visual receptive fields

Let us represent spatial coordinates by $x=(x_{1},x_{2})^{T}$ and centered affine spatial transformations in the 2-D image domain as

x^{\prime}=A\,x,

(1)

where $A$ represents any non-singular $2\times 2$ matrix.

Then, an affine transformed image $f^{\prime}(x^{\prime})$ of an original image $f(x)$ is defined according to

f^{\prime}(x^{\prime})=f(x).

(2)

With the affine transformation operator ${\cal T}_{A}$ , we can write this relationship as

f^{\prime}={\cal T}_{A}\,f.

(3)

The property of affine covariance then means that the results of either:

•

applying an affine transformation $x^{\prime}=A\,x$ to an image $f(x)$ and then applying a receptive field ${\cal R}^{\prime}$ to the affine transformed image $f^{\prime}(x^{\prime})$ , or
•

applying a related receptive field ${\cal R}$ to the original image $f(x)$ and then applying an affine transformation ${\cal T}_{A}$ to that output,

will lead to the same result, such that

{\cal R}^{\prime}\,{\cal T}_{A}\,f={\cal T}_{A}\,{\cal R}\,f,

(4)

where the affine covariant property of the receptive field family means that for every receptive field ${\cal R}$ in the receptive field family, there exists a possibly transformed receptive field ${\cal R}^{\prime}$ within the same family, specifically determined according to the actual value of the affine transformation matrix $A$ , such that the above relationship is guaranteed to hold, for some transformed receptive field ${\cal R}^{\prime}$ as a function of the original receptive field ${\cal R}$ and the affine transformation matrix $A$ .

The property of affine covariance thus means that the family of receptive fields is well-behaved with regard to spatial affine transformations, in the sense that affine transformations commute with the operation of computing outputs from the family of receptive fields.

In Lindeberg (2021; 2023; 2024a), it is argued that such affine covariant properties constitute an essential property of spatial receptive fields, as well as for the spatial components in joint spatio-temporal receptive fields. Specifically, the receptive fields, according to the generalized Gaussian derivative model for visual receptive fields, to be used below, obey such affine covariant properties.

The property of the degree of elongation of the receptive fields, to be studied in detail in this work, spans a 1-D variability within the full 4-D variability of general affine transformations, thus constituting one of the degrees of freedom in the variability of transformed receptive field shapes of ${\cal R}^{\prime}$ , that will be generated by subjecting an original receptive field ${\cal R}$ to the 4-D variability of general affine transformation matrices $A$ .

2.2 Connections between the orientation selectivity and the degree of elongation of the receptive fields for the generalized Gaussian derivative model for visual receptive fields

For modelling the receptive fields in the primary visual cortex, we will use the generalized Gaussian derivative model for receptive fields (Lindeberg 2021).

2.2.1 Idealized models for simple cells

We will model the purely spatial component of the receptive fields for the simple cells as (Lindeberg 2021 Equation (23); see Figure 7 in that reference for illustrations)

T_{\scriptsize\mbox{simple}}(x_{1},x_{2};\;\sigma_{\varphi},\varphi,\Sigma_{% \varphi},m)\\ =T_{\varphi^{m},\scriptsize\mbox{norm}}(x_{1},x_{2};\;\sigma_{\varphi},\Sigma_% {\varphi})=\sigma_{\varphi}^{m}\,\partial_{\varphi}^{m}\left(g(x_{1},x_{2};\;% \Sigma_{\varphi})\right),

(5)

and with joint spatio-temporal receptive fields of the simple cells according to (Lindeberg 2021 Equation (25); see Figures 10-11 in that reference for illustrations)

	$\displaystyle\begin{split}T_{\scriptsize\mbox{simple}}(x_{1},x_{2},t;\;\sigma_% {\varphi},\sigma_{t},\varphi,v,\Sigma_{\varphi},m,n)\end{split}$
	$\displaystyle\begin{split}&=T_{{\varphi}^{m},{\bar{t}}^{n},\scriptsize\mbox{% norm}}(x_{1},x_{2},t;\;\sigma_{\varphi},\sigma_{t},v,\Sigma_{\varphi})\end{split}$
	$\displaystyle\begin{split}&=\sigma_{\varphi}^{m}\,\sigma_{t}^{n}\,\partial_{% \varphi}^{m}\,\partial_{\bar{t}}^{n}\left(g(x_{1}-v_{1}t,x_{2}-v_{2}t;\;\Sigma% _{\varphi})\,h(t;\;\sigma_{t})\right),\end{split}$			(6)

where

•

$\varphi$ is the preferred orientation of the receptive field,
•

$\sigma_{\varphi}$ is the amount of spatial smoothing,
•

$\partial_{\varphi}^{m}=(\cos\varphi\,\partial_{x_{1}}+\sin\varphi\,\partial_{x% _{2}})^{m}$ is an $m$ :th-order directional derivative operator, in the direction $\varphi$ ,
•

$\Sigma_{\varphi}$ is a symmetric positive definite covariance matrix, with one of its eigenvectors in the direction of $\varphi$ ,

•

$g(x;\;\Sigma_{\varphi})$ is a 2-D affine Gaussian kernel with its shape determined by the covariance matrix $\Sigma_{\varphi}$

g(x;\;\Sigma_{\varphi})=\frac{1}{2\pi\sqrt{\det\Sigma_{\varphi}}}e^{-x^{T}% \Sigma_{\varphi}^{-1}x/2}

(7)

for $x=(x_{1},x_{2})^{T}$ ,

•

$\sigma_{t}$ is the amount of temporal smoothing,
•

$v=(v_{1},v_{2})^{T}$ is a local motion vector, in the direction $\varphi$ of the spatial orientation of the receptive field,
•

$\partial_{\bar{t}}^{n}=(\partial_{t}+v_{1}\,\partial_{x_{1}}+v_{2}\,\partial_{% x_{2}})^{n}$ is an $n$ :th-order velocity-adapted temporal derivative operator, and
•

$h(t;\;\sigma_{t})$ is a temporal Gaussian kernel with standard deviation $\sigma_{t}$ .

This model builds upon the regular Gaussian derivative model for purely spatial receptive fields proposed by Koenderink and van Doorn (1984; 1987; 1992) and previously used for modelling biological fields by Young and his co-workers (1987; 2001; 2001). Here, that regular Gaussian derivative model is additionally generalized to affine covariance, according to Lindeberg (2013a; 2021).

2.2.2 Idealized models for complex cells

To model complex cells with a purely spatial dependency, we will use a quasi-quadrature measure of the form (Lindeberg 2020 Equation (39))

{\cal Q}_{\varphi,\scriptsize\mbox{spat},\scriptsize\mbox{norm}}L=\sqrt{L_{% \varphi,\scriptsize\mbox{norm}}^{2}+C_{\varphi}\,L_{\varphi\varphi,\scriptsize% \mbox{norm}}^{2}},

(8)

where

•

$L_{\varphi,\scriptsize\mbox{norm}}$ and $L_{\varphi\varphi,\scriptsize\mbox{norm}}$ constitute the results of convolving the input image with scale-normalized directional affine Gaussian derivative operators of orders 1 and 2:

L_{\varphi,\scriptsize\mbox{norm}}(x_{1},x_{2};\;\sigma_{\varphi},\Sigma_{% \varphi})=\\ =T_{\varphi,\scriptsize\mbox{norm}}(x_{1},x_{2};\;\sigma_{\varphi},\Sigma_{% \varphi})*f(x_{1},x_{2}),

(9)

L_{\varphi\varphi,\scriptsize\mbox{norm}}(x_{1},x_{2};\;\sigma_{\varphi},% \Sigma_{\varphi})=\\ =T_{\varphi\varphi,\scriptsize\mbox{norm}}(x_{1},x_{2};\;\sigma_{\varphi},% \Sigma_{\varphi})*f(x_{1},x_{2}),

(10)

•

$C_{\varphi}>0$ is a weighting factor between first and second-order information.

This model constitutes an affine Gaussian derivative analogue of the energy model of complex cells developed by Adelson and Bergen (1985) and Heeger (1992), and is consistent with the observation that receptive fields analogous to first- vs. second-order derivatives occur in pairs in biological vision (De Valois et al. 2000), with close analogies to quadrature pairs, as defined in terms of a Hilbert transform (Bracewell 1999, pp. 267–272).

Complex cells with a joint spatio-temporal dependency will, in turn, be modelled as

({\cal Q}_{\varphi,\scriptsize\mbox{vel},\scriptsize\mbox{norm}}L)=\sqrt{L_{% \varphi,\scriptsize\mbox{norm}}^{2}+\,C_{\varphi}\,L_{\varphi\varphi,% \scriptsize\mbox{norm}}^{2}},

(11)

where

L_{\varphi,\scriptsize\mbox{norm}}(x_{1},x_{2},t;\;\sigma_{\varphi},\sigma_{t}% ,v,\Sigma_{\varphi})=\\ =T_{\varphi,\scriptsize\mbox{norm}}(x_{1},x_{2},t;\;\sigma_{\varphi},\sigma_{t% },v,\Sigma_{\varphi})*f(x_{1},x_{2},t),

(12)

L_{\varphi\varphi,\scriptsize\mbox{norm}}(x_{1},x_{2},t;\;\sigma_{\varphi},% \sigma_{t},v,\Sigma_{\varphi})=\\ =T_{\varphi\varphi,\scriptsize\mbox{norm}}(x_{1},x_{2},t;\;\sigma_{\varphi},% \sigma_{t},v,\Sigma_{\varphi})*f(x_{1},x_{2},t),

(13)

with the underlying space-time separable spatio-temporal receptive fields $T_{\varphi^{m},t^{n},\scriptsize\mbox{norm}}(x_{1},x_{2},t;\;\sigma_{\varphi},% \sigma_{t},v,\Sigma_{\varphi})$ according to (2.2.1) for $n=0$ .

2.2.3 Orientation selectivity curves for the idealized receptive field models

In Lindeberg (2024b), the responses of the above purely spatial models of receptive fields are calculated with respect to a static sine wave of the form (see Figure 3)

f(x_{1},x_{2})=\sin\left(\omega\cos(\theta)\,x_{1}+\omega\sin(\theta)\,x_{2}+% \beta\right).

(14)

Additionally, the responses of the above joint spatio-temporal models of receptive fields are calculated with respect to a moving sine wave of the form

f(x_{1},x_{2},t)=\\ =\sin\left(\omega\cos(\theta)\,(x_{1}-u_{1}t)+\omega\sin(\theta)\,(x_{2}-u_{2}% t)+\beta\right),

(15)

with the velocity vector $(u_{1},u_{2})^{T}$ parallel to the inclination angle $\theta$ of the grating, such that $(u_{1},u_{2})^{T}=(u\cos\theta,u\sin\theta)^{T}$ .

In summary, the theoretical analysis in Lindeberg (2024b) shows that the resulting orientation selectivity curves for the first-order simple cells, second-order simple cells and complex cells, respectively, will be of the forms:

	$\displaystyle\begin{split}r_{\scriptsize\mbox{simple},1}(\theta)&=\frac{\left\|% \cos\theta\right\|}{\sqrt{\cos^{2}\theta+\kappa^{2}\sin^{2}\theta}},\end{split}$			(16)
	$\displaystyle\begin{split}r_{\scriptsize\mbox{simple},2}(\theta)&=\frac{\cos^{% 2}\theta}{\cos^{2}\theta+\kappa^{2}\sin^{2}\theta},\end{split}$			(17)
	$\displaystyle\begin{split}r_{\scriptsize\mbox{complex}}(\theta)&=\frac{\left\|% \cos\theta\right\|^{3/2}}{\left(\cos^{2}\theta+\kappa^{2}\sin^{2}\theta\right)^% {3/4}},\end{split}$			(18)

with similar angular dependencies within each class for both the purely spatial receptive fields and the joint spatio-temporal receptive fields.

Figure 4 shows graphs of these orientation selectivity curves, where we can clearly see how the orientation selectivity becomes more narrow for increasing values of the scale parameter ratio $\kappa$ , thus establishing a direct link between the elongation and the degree of orientation selectivity for the idealized models of the receptive fields.

2.3 Relations to previous work

Beyond the works by Nauhaus et al. (2008) and by Goris et al. (2015), that the treatment in Section 3 will largely build upon, there is a large body of work on characterizing the orientation selectivity of neurons, by Watkins and Berkley (1974), Rose and Blakemore (1974), Schiller et al. (1976), Albright (1984), Ringach et al. (2002), Nauhaus et al. (2008), Scholl et al. (2013), Sadeh and Rotter (2014) and Sasaki et al. (2015), as well as concerning biological mechanisms for achieving orientation selectivity by Somers et al. (1995), Sompolinsky and Shapley (1997), Carandini and Ringach (1997), Lampl et al. (2001), Ferster and Miller (2000), Shapley et al. (2003), Seriès et al. (2004), Hansel and van Vreeswijk (2012), Moldakarimov et al. (2014), Gonzalo Cogno and Mato (2015), Priebe (2016), Pattadkal et al. (2018), Nguyen and Freeman (2019), Merkt et al. (2019), Wei et al. (2022) and Wang et al. (2024). The focus of this paper, however, is not on the neural mechanisms that lead to orientation selectivity, but on purely functional properties at the macroscopic level.

Mathematical models of biological receptive fields have beyond in terms of Gaussian derivatives (Koenderink and van Doorn 1984; 1987; 1992; Young and his co-workers 1987; 2001; 2001; Lindeberg 2013a; 2021) also been formulated in terms of Gabor filters (Marcelja 1980; Jones and Palmer 1987a; 1987b; Porat and Zeevi 1988). Gaussian derivatives have, in turn, been used as primitives in theoretical models of early visual processing by Lowe (2000), May and Georgeson (2007), Hesse and Georgeson (2005), Georgeson et al. (2007), Hansen and Neumann (2008), Wallis and Georgeson (2009), Wang and Spratling (2016), Pei et al. (2016), Ghodrati et al. (2017), Kristensen and Sandberg (2021), Abballe and Asari (2022), Ruslim et al. (2023) and Wendt and Faul (2024).

Hubel and Wiesel (1959; 1962; 1968; 2005) pioneered the study of simple and complex cells. The properties of simple cells have been further characterized by DeAngelis et al. (1995; 2004), Ringach (2002; 2004), Conway and Livingstone (2006), Johnson et al. (2008), Walker et al. (2019) and De and Horwitz (2021), and the properties of complex cells investigated by Movshon et al. (1978), Emerson et al. (1987), Martinez and Alonso (2001), Touryan et al. (2002; 2005), Rust et al. (2005), van Kleef et al. (2010), Goris et al. (2015), Li et al. (2015) and Almasi et al. (2020), as well as modelled computationally by Adelson and Bergen (1985), Heeger (1992), Serre and Riesenhuber (2004), Einhäuser et al. (2002), Kording et al. (2004), Merolla and Boahen (2004), Berkes and Wiscott (2005), Carandini (2006), Hansard and Horaud (2011), Franciosini et al. (2019), Lindeberg (2020), Lian et al. (2021), Oleskiw et al. (2024) and Yedjour and Yedjour (2024). In this work, we have followed a specific way of modelling simple and complex cells in terms of affine Gaussian derivatives, according to the generalized affine Gaussian derivative model for visual receptive fields.

Properties of cortical maps in the primary visual cortex have, in turn, been studied in detail by Bonhoeffer and Grinvald (1991), Blasdel (1992), Maldonado et al. 1997, Koch et al. (2016), Kremkow et al. (2016), Najafian et al. (2022), Jung et al. (2022), Fang et al. (2022) and Vita et al. (2024).

3 Results

In this section, we will compare the results of the theoretical predictions in Section 2.2 with biological results concerning the orientation selectivity of visual neurons.

For later purposes, we will, however, first extend the above results concerning the orientation selectivity curves for idealized models of simple cells according to the generalized Gaussian derivative model for visual receptive fields, from first-order and second-order simple cells to also comprise third-order and fourth-order simple cells.

The results derived in Section 3.1 will then be used in Section 3.2.3, when extending the interpretation in the following Section 3.2.2, based on spatial derivatives up to order 2, to spatial derivatives up to order 4. Readers who are more interested in the functional results of the theory than the details of the mathematical derivations, should be able to, without major loss of continuity, skip the details in Section 3.1, while noting the result summary in Section 3.1.3, to then continue directly with Section 3.2.

3.1 Derivations of orientation selectivity properties for third-order and fourth-order simple cells

For simplicity, we will here restrict ourselves to purely static models of simple cells.

3.1.1 Third-order simple cell

Following the methodology in (Lindeberg 2024b) underlying the results summarized in Section 2.2.3, we will express an idealized model of a simple cell with four lobes along the preferred orientation of the simple cell as a third-order scale-normalized derivative of an affine Gaussian kernel (according to (5) for $m=3$ ), and for convenience of the calculations choose the preferred orientation as the horizontal $x_{1}$ -direction (for $\varphi=0$ ) with spatial scale parameter $\sigma_{1}$ in the horizontal $x_{1}$ -direction and spatial scale parameter $\sigma_{2}$ in the vertical $x_{2}$ -direction, and thus with a spatial covariance matrix of the form $\Sigma_{0}=\operatorname{diag}(\sigma_{1}^{2},\sigma_{2}^{2})$ :

	$\displaystyle\begin{split}&T_{000,\scriptsize\mbox{norm}}(x_{1},x_{2};\;\sigma% _{1},\sigma_{2})=\end{split}$
	$\displaystyle\begin{split}&=\frac{\sigma_{1}^{3}}{2\pi\sigma_{1}\sigma_{2}}\,% \partial_{x_{1}x_{1}x_{1}}\left(e^{-x_{1}^{2}/2\sigma_{1}^{2}-x_{2}^{2}/2% \sigma_{2}^{2}}\right)\end{split}$
	$\displaystyle\begin{split}&=\frac{(3\sigma_{1}^{2}x-x^{3})}{2\pi\sigma_{1}^{4}% \sigma_{2}}\,e^{-x_{1}^{2}/2\sigma_{1}^{2}-x_{2}^{2}/2\sigma_{2}^{2}}.\end{split}$			(19)

The corresponding receptive field response can then be expressed as, after solving the convolution integral in Mathematica,

	$\displaystyle\begin{split}L_{000,\scriptsize\mbox{norm}}(x_{1},x_{2};\;\sigma_% {1},\sigma_{2})=\end{split}$
	$\displaystyle\begin{split}&=\int_{\xi_{1}=-\infty}^{\infty}\int_{\xi_{2}=-% \infty}^{\infty}T_{000,\scriptsize\mbox{norm}}(\xi_{1},\xi_{2};\;\sigma_{1},% \sigma_{2})\end{split}$
	$\displaystyle\begin{split}&\phantom{==\int_{\xi_{1}=-\infty}^{\infty}\int_{\xi% _{2}=-\infty}^{\infty}}\times f(x_{1}-\xi_{1},x_{2}-\xi_{2})\,d\xi_{1}\xi_{2}% \end{split}$
	$\displaystyle\begin{split}&=-\omega^{3}\,\sigma_{1}^{3}\cos^{3}(\theta)\,e^{-% \frac{1}{2}\omega^{2}(\sigma_{1}^{2}\cos^{2}\theta+\sigma_{2}^{2}\sin^{2}% \theta)}\end{split}$
	$\displaystyle\begin{split}&\phantom{==}\times\cos(\omega\cos(\theta)\,x_{1}+% \omega\sin(\theta)\,x_{2}+\beta),\end{split}$			(20)

i.e., it corresponds to cosine wave with amplitude

A_{\varphi\varphi\varphi}(\theta,\omega;\;\sigma_{1},\sigma_{2})=\\ =\omega^{3}\,\sigma_{1}^{3}\cos^{3}(\theta)\,e^{-\frac{1}{2}\omega^{2}(\sigma_% {1}^{2}\cos^{2}\theta+\sigma_{2}^{2}\sin^{2}\theta)}.

(21)

If we, for this modelling situation, assume that the spatial receptive field is fixed, then it follows that the amplitude of the response will depend strongly on the angular frequency $\omega$ of the sine wave. Specifically, the magnitude of the response will first increase with the angular frequency of the input stimulus, because of the factor $\omega$ . Then, it will decrease with scale because of the strong exponential decrease with $\omega^{2}$ .

Let us consider that a biological experiment to measure the orientation selectivity properties of a visual neuron is performed in such a way that the angular frequency of the input stimulus is varied for each inclination angle $\theta$ , and that then the result for each value orientation $\theta$ of the stimulus is only reported for the angular frequency $\hat{\omega}$ that leads to the maximum response over all the image orientations. Then, we can determine this value of $\hat{\omega}$ by differentiating $A_{\varphi}(\theta,\omega;\;\sigma_{1},\sigma_{2})$ with respect to $\omega$ and setting the derivative to zero, which gives:

\hat{\omega}_{\varphi\varphi\varphi}=\frac{\sqrt{3}}{\sqrt{\sigma_{1}^{2}\cos^% {2}\theta+\sigma_{2}^{2}\sin^{2}\theta}}.

(22)

If we then insert this value into $A_{\varphi\varphi\varphi}(\theta,\omega;\;\sigma_{1},\sigma_{2})$ , and introduce a scale parameter ratio $\kappa$ such that

\sigma_{2}=\kappa\,\sigma_{1},

(23)

which gives

\hat{\omega}_{\varphi\varphi\varphi}=\frac{\sqrt{3}}{\sigma_{1}\sqrt{\cos^{2}% \theta+\kappa^{2}\sin^{2}\theta}}.

(24)

then this gives rise to an orientation selectivity curve of the form

A_{\varphi\varphi\varphi,\max}(\theta,\;\kappa)=\frac{3\sqrt{3}\left|\cos^{3}% \theta\right|}{e^{3/2}\left(\cos^{2}\theta+\kappa^{2}\sin^{2}\theta\right)^{3/% 2}}.

(25)

Notably, this amplitude measure is independent of the spatial scale parameter $\sigma_{1}$ of the receptive field. This property is a direct implication of the scale-invariant properties of differential expressions in terms of scale-normalized derivatives when using the specific value $\gamma=1$ for the scale normalization parameter.

3.1.2 Fourth-order simple cell

Let us next consider an idealized model of a simple cell with five lobes along the main orientation of the receptive field, which we model with as a fourth-order scale-normalized derivative of an affine Gaussian kernel (according to (5) for $m=4$ ), with its preferred orientation again for convenience chosen as the horizontal $x_{1}$ -direction (for $\varphi=0$ ), and with a spatial scale parameter $\sigma_{1}$ in the horizontal $x_{1}$ -direction and a spatial scale parameter $\sigma_{2}$ in the vertical $x_{2}$ -direction, and thus again with a spatial covariance matrix of the form $\Sigma_{0}=\operatorname{diag}(\sigma_{1}^{2},\sigma_{2}^{2})$ :

	$\displaystyle\begin{split}&T_{0000,\scriptsize\mbox{norm}}(x_{1},x_{2};\;% \sigma_{1},\sigma_{2})=\end{split}$
	$\displaystyle\begin{split}&=\frac{\sigma_{1}^{4}}{2\pi\sigma_{1}\sigma_{2}}\,% \partial_{x_{1}x_{1}x_{1}x_{1}}\left(e^{-x_{1}^{2}/2\sigma_{1}^{2}-x_{2}^{2}/2% \sigma_{2}^{2}}\right)\end{split}$
	$\displaystyle\begin{split}&=\frac{(3\sigma_{1}^{4}-6\sigma_{1}^{2}x^{2}+x^{4})% }{2\pi\sigma_{1}^{5}\sigma_{2}}\,e^{-x_{1}^{2}/2\sigma_{1}^{2}-x_{2}^{2}/2% \sigma_{2}^{2}}.\end{split}$			(26)

After solving the convolution integral in Mathematica, the corresponding receptive field response is then of the form

	$\displaystyle\begin{split}L_{0000,\scriptsize\mbox{norm}}(x_{1},x_{2};\;\sigma% _{1},\sigma_{2})=\end{split}$
	$\displaystyle\begin{split}&=\int_{\xi_{1}=-\infty}^{\infty}\int_{\xi_{2}=-% \infty}^{\infty}T_{0000,\scriptsize\mbox{norm}}(\xi_{1},\xi_{2};\;\sigma_{1},% \sigma_{2})\end{split}$
	$\displaystyle\begin{split}&\phantom{==\int_{\xi_{1}=-\infty}^{\infty}\int_{\xi% _{2}=-\infty}^{\infty}}\times f(x_{1}-\xi_{1},x_{2}-\xi_{2})\,d\xi_{1}\xi_{2}% \end{split}$
	$\displaystyle\begin{split}&=\omega^{4}\,\sigma_{1}^{4}\cos^{4}(\theta)\,e^{-% \frac{1}{2}\omega^{2}(\sigma_{1}^{2}\cos^{2}\theta+\sigma_{2}^{2}\sin^{2}% \theta)}\end{split}$
	$\displaystyle\begin{split}&\phantom{==}\times\sin(\omega\cos(\theta)\,x_{1}+% \omega\sin(\theta)\,x_{2}+\beta),\end{split}$			(27)

i.e., a sine wave with amplitude

A_{\varphi\varphi\varphi\varphi}(\theta,\omega;\;\sigma_{1},\sigma_{2})=\\ =\omega^{4}\,\sigma_{1}^{4}\cos^{4}(\theta)\,e^{-\frac{1}{2}\omega^{2}(\sigma_% {1}^{2}\cos^{2}\theta+\sigma_{2}^{2}\sin^{2}\theta)}.

(28)

As for the previous idealized receptive field model, this expression also first increases and then increases with the angular frequency $\omega$ . Again selecting the value of $\hat{\omega}$ at which the amplitude assumes its maximum over $\omega$ gives

\hat{\omega}_{\varphi\varphi\varphi\varphi}=\frac{2}{\sigma_{1}\sqrt{\cos^{2}% \theta+\kappa^{2}\sin^{2}\theta}},

(29)

which implies that the maximum amplitude over spatial scales as a function of the inclination angle $\theta$ and the scale parameter ratio $\kappa$ can be written

A_{\varphi\varphi\varphi\varphi,\max}(\theta;\;\kappa)=\frac{16\cos^{4}\theta}% {e^{2}\left(\cos^{2}\theta+\kappa^{2}\sin^{2}\theta\right)^{2}}.

(30)

3.1.3 Resulting orientation selectivity curves

If we additionally normalize the orientation selectivity curves (25) and (30) to have their maximum value equal to one for the preferred orientation $\theta=0$ , we then obtain normalized orientation selectivity curves of the forms

	$\displaystyle\begin{split}r_{\scriptsize\mbox{simple},3}(\theta)&=\frac{\left\|% \cos\theta\right\|^{3}}{(\cos^{2}\theta+\kappa^{2}\sin^{2}\theta)^{3/2}},\end{split}$			(31)
	$\displaystyle\begin{split}r_{\scriptsize\mbox{simple},4}(\theta)&=\frac{\cos^{% 4}\theta}{(\cos^{2}\theta+\kappa^{2}\sin^{2}\theta)^{2}},\end{split}$			(32)

and with examples of graphs of these curves, for a few values of the scale ratio parameter $\kappa$ , shown in Figure 5

As can be seen from a comparison with the orientation selectivity curves for the idealized models of first-order and second-order simple cells in Figure 4, the orientation selectivity curves of the idealized models of third-order and fourth-order simple cells follow the trends for the first-order and second-order simple cells, in that the orientation selectivity curves become more narrow both with increasing values of the scale parameter ratio $\kappa$ and with increasing order of spatial differentiation.

3.2 Interpretation of the connection between the orientation selectivity and the elongation of receptive fields in relation to biological measurements

3.2.1 Interpretation of the measurements about broad vs. sharp orientation selectivity of neurons by Nauhaus et al. (2008)

Nauhaus et al. (2008) measured the orientation tuning of neurons at different positions in the primary visual cortex for monkey and cat. They found that the orientation tuning is broader near the pinwheel centers and sharper in regions of homogeneous orientation preference. Figure 6 shows an overview of their results, where we can see how the degree of orientation selectivity changes rather continuously from broad to sharp with increasing distance from the pinwheel center (from top to bottom in the figure).

In view of our theoretical results, as summarized in Section 2.2, concerning the orientation selectivity of receptive fields, where the spatial smoothing part is performed based on affine Gaussian kernels, this qualitative behaviour is consistent with what would be the result if the ratio $\kappa$ between the two scale parameters of the underlying affine Gaussian kernels would increase from a lower to a higher value, when moving away from the centers of the pinwheels on the cortical surface. Thus, the presented theory leads to a prediction about a variability in the eccentricity or the elongation of the receptive fields in the primary visual cortex. In the case of pinwheel structures, the behaviour is specifically consistent with a variability in the eccentricity or the elongation of the receptive fields from the centers of the pinwheels towards the periphery.

Let us furthermore consider the theoretical prediction from Lindeberg (2023), that the shapes of the affine Gaussian derivative-based receptive fields ought to comprise a variability over a larger part of the affine group than mere rotations, to enable affine covariance and (partial) affine invariance at higher levels in the visual hierarchy. Then, if combined with the theoretical orientation selectivity analysis presented in the paper, those predictions are also consistent with the results by Nauhaus et al. (2008), with an additional explanatory power: If the theoretically motivated prediction would hold, then the underlying theoretical model may enable a deeper interpretation of those biological results, in terms of underlying computational mechanisms in the visual receptive fields, to enable specific functional covariance and invariance properties at higher levels in the visual hierarchy.

A highly interesting quantitative measurement to perform, in view of these theoretical results, would hence be to fit parameterized models of the orientation selectivity, according to Equations (16), (17), (18), (31) and (32)

r_{\lambda}(\theta)=\left(\frac{\left|\cos\theta\right|}{\sqrt{\cos^{2}\theta+% \kappa^{2}\sin^{2}\theta}}\right)^{\lambda}

(33)

to orientation tuning curves of the form recorded by Nauhaus et al. (2008), to get estimates of the distribution of the parameter $\kappa$ over a sufficiently large population of visual neurons, under the assumption that the spatial components of the biological receptive fields can be well modelled by affine Gaussian derivatives.¹¹1At the point of writing this article, the author does, however, not have access to the explicit data that would be needed to perform such an analysis.

In Lindeberg (2023; 2024a), theoretical treatments are given concerning covariance properties of visual receptive fields under natural image transformations, specifically geometric image transformations in terms of spatial scaling transformations, spatial affine transformations, Galilean transformations and temporal scaling transformations. According to that theory of spatial and spatio-temporal receptive fields, in terms of generalized Gaussian derivatives, the covariance properties of the receptive fields mean that the shapes of the receptive field families should span the degrees of freedom generated by the geometric image transformations. With regard to spatial affine transformations, which beyond spatial scaling transformations do also comprise spatial rotations and non-uniform scaling transformation with different amount of scaling in two orthogonal spatial directions, this theory implies that affine Gaussian kernels ought to be present in the receptive field families corresponding to different values of the ratio $\kappa$ between the spatial scale parameters, to support affine covariance. In Lindeberg (2023 Section 3.2) suggestions for new biological measurements were further proposed to support (or reject) those hypotheses.

If we would assume that it would be unlikely for the receptive fields to have as strong variability in their orientational selectivity properties as a function of the positions of the neurons in relation to the pinwheel structure, as reported in this study, without also having a strong variability in their eccentricity, then by combining the theoretical analysis in this article with the biological results by Nauhaus et al. (2008), that would serve as possible indirect support for the hypothesis concerning an expansion of receptive field shapes over variations in the ratio between the two scale parameters of spatially anisotropic receptive fields. Thus, if we would assume that the biological receptive fields can be well modelled by the generalized Gaussian derivative model based on affine Gaussian receptive fields, then the biological results by Nauhaus et al. (2008) are fully consistent with the prediction of such an explicit expansion over shapes of the visual receptive fields, based on the orientation selectivity of visual receptive fields, whose spatial smoothing component can be well modelled by affine Gaussian kernels.

3.2.2 Interpretation of the measurements of orientation selectivity histograms by Goris et al. (2015) based on spatial derivatives up to order 2

These predictions are furthermore consistent with existing biological results by Goris et al. (2015), concerning the distribution of the degree of orientation selectivity of the neurons in the primary visual cortex. By measuring the absolute value $|R|$ of the complex-valued resultant given by

R=\frac{\int_{\theta=-\pi}^{\pi}r(\theta)\,e^{2i\theta}d\theta}{\int_{\theta=-% \pi}^{\pi}r(\theta)\,d\theta},

(34)

for the orientation selectivity curve for each visual neuron, and then computing a normalized histogram of these measurements (see Figure 7), Goris et al. (2015) demonstrate a substantial variability in the orientation selectivity of the receptive fields of simple cells and complex cells in the primary visual cortex.

R_{\scriptsize\mbox{complex}}=\frac{\kappa^{2}\left(48\left(\kappa^{2}-1\right% )^{3/4}\Gamma\left(\frac{5}{4}\right)\,_{2}F_{1}\left(\frac{1}{2},1;\frac{3}{4% };\frac{1}{\kappa^{2}}\right)-16\left(\kappa^{2}-1\right)^{3/4}\Gamma\left(% \frac{5}{4}\right)+3\sqrt{2\pi}\,\kappa\,\Gamma\left(-\frac{1}{4}\right)\right% )}{2\left(\kappa^{2}-1\right)\left(16\left(\kappa^{2}-1\right)^{3/4}\Gamma% \left(\frac{5}{4}\right)\,_{2}F_{1}\left(\frac{1}{2},1;\frac{3}{4};\frac{1}{% \kappa^{2}}\right)+\sqrt{2\pi}\,\kappa\,\Gamma\left(-\frac{1}{4}\right)\right)}

(35)

Figure 8: Closed-form expression for the resultant

R

according to (34) calculated for the orientation selectivity curves (18) for our idealized models of complex cells, valid for the purely spatial model (8) and the joint spatio-temporal model (11). The function

{}_{2}F_{1}(a,b;c;z)

denotes the hypergeometric function

\operatorname{Hypergeometric2F1}[a,b,c,z]

in Mathematica, while

\Gamma(z)

represents Euler’s Gamma function.

This consistency can be demonstrated by computing the closed-form expression for the absolute value of the resultant of the orientation selectivity curves according to Equations (16)–(18), as done in (Lindeberg 2024b Section 5.1) for the first-order idealized models of simple cells

	$\displaystyle\begin{split}R_{\scriptsize\mbox{simple},1}&=\frac{\int_{\theta=-% \pi/2}^{\pi/2}\frac{\cos\theta}{\sqrt{\cos^{2}\theta+\kappa^{2}\sin^{2}\theta}% }\,\cos 2\theta\,d\theta}{\int_{\theta=-\pi/2}^{\pi/2}\frac{\cos\theta}{\sqrt{% \cos^{2}\theta+\kappa^{2}\sin^{2}\theta}}\,d\theta}\end{split}$
	$\displaystyle\begin{split}&=\frac{\kappa\left(\kappa\cosh^{-1}\kappa-\sqrt{% \kappa^{2}-1}\right)}{\left(\kappa^{2}-1\right)\cosh^{-1}\kappa},\end{split}$			(36)

for the second-order idealized models of simple cells

	$\displaystyle\begin{split}R_{\scriptsize\mbox{simple},2}&=\frac{\int_{\theta=-% \pi/2}^{\pi/2}\frac{\cos^{2}\theta}{\cos^{2}\theta+\kappa^{2}\sin^{2}\theta}\,% \cos 2\theta\,d\theta}{\int_{\theta=-\pi/2}^{\pi/2}\frac{\cos^{2}\theta}{\cos^% {2}\theta+\kappa^{2}\sin^{2}\theta}\,d\theta}\end{split}$
	$\displaystyle\begin{split}&=\frac{\kappa}{\kappa+1}.\end{split}$			(37)

as well as for the idealized models of complex cells

\displaystyle\begin{split}R_{\scriptsize\mbox{complex}}&=\frac{\int_{\theta=-% \pi/2}^{\pi/2}\frac{\cos^{3/2}\theta}{\left(\cos^{2}\theta+\kappa^{2}\sin^{2}% \theta\right)^{3/4}}\,\cos 2\theta\,d\theta}{\int_{\theta=-\pi/2}^{\pi/2}\frac% {\cos^{3/2}\theta}{\left(\cos^{2}\theta+\kappa^{2}\sin^{2}\theta\right)^{3/4}}% \,d\theta},\end{split}

(38)

with the explicit expression for the last result in Figure 8.

Let us additionally reparameterize these curves in terms of a logarithmic parameterization $K=\log\kappa$ of the scale parameter ratio $\kappa$ , which leads to orientation selectivity curves of the forms shown in Figure 9. Then, we can see that the experimentally obtained distributions in Figure 7 appear to be reasonably consistent with the assumption of a rather uniform distribution over the logarithmically parameterized scale parameter ratio $K=\log\kappa$ .

Such a parameterization would specifically constitute a canonical parameterization, if one would simplify²²2 More generally, one could instead conceive a uniform joint distribution on a hemisphere, as conceived in Figure 8 in (Lindeberg 2021), which regarding first-order spatial derivatives then leads to a distribution of spatial receptive field shapes of the form shown in Figure 15, and possibly complemented with additional priors to account for how important different local surface orientations in the environment would be for the perceptual process, as well as densely the space of combined image orientations $\varphi$ and scale parameter ratios $\kappa$ would need to be sampled, to support sufficiently good approximations of covariance over that submanifold for the local image measurements performed by the family of spatial receptive fields. In this treatment, we do, however, simplify this problem, by instead considering a uniform distribution over a logarithmic transformation of the scale parameter ratio $\kappa$ , which is also easier to handle in closed form calculations, and which may be regarded as a coarse approximation, to compensate for gross phenomena with regard to a non-uniform distribution of receptive field shapes over the scale parameter ratio $\kappa$ . the 2-D joint distribution of receptive field shapes over the scale parameter ratio $\kappa$ and the orientation $\varphi$ into two independent 1-D distributions over the scale parameter ratio $\kappa$ and the orientation $\varphi$ , respectively, in the idealized model of visual receptive fields according to the generalized Gaussian derivative framework.

Thus, also these biological results are consistent with the working hypothesis about an expansion over the degree of elongation of the receptive fields in the primary visual cortex, as would be implied from the assumption of a family of affine covariant visual receptive fields.

If we would aim at more detailed modelling of the experimentally recorded histograms of the resultant measure of the orientation selectivity curves, as reported by Goris et al. (2015) and as reproduced in Figure 7, we need to consider that there would be more free parameters in the modelling to determine, based on the following arguments:

•

One basic question concerns what range of values of the scale parameter ratio $\kappa$ would be spanned by the receptive fields in the primary visual cortex.

In the graphs that we have shown in Figure 9, we have used a range of the scale parameter ratio $\kappa$ over a factor 10 from the unit value 1, which results in a maximum value of $R$ for the idealized first-order model of a simple cell of about 0.67. In contrast, for the idealized second-order model of a simple cell, $R$ reaches a maximum value of $R$ about 0.91, while for the complex cell, $R$ assumes a maximum value of about 0.83.

If this range of scale parameter ratios would be reduced to a lower span, then the range of the possible values of $R$ would be reduced, while the range would be expanded if a wider range of scale ratios $\kappa$ would be implemented.
•

Another basic question concerns the distribution of receptive fields with respect to the order of spatial differentiation.

In our theoretical analysis so far, we have modelled the simple cells in terms of first- and second-order directional derivatives. Experimental results by Young (1985; 1987) do, however, indicate that receptive fields that can be modelled by Gaussian derivatives up to order 4 may be present in the primary visual cortex.

For such receptive fields of higher order, we will obtain additionally sharper orientation selectivity curves, which will then assume values of $R$ closer to 1 than for the first- and second-order models of simple cells.

To reproduce an idealized model of a histogram of the resultant $R$ for a population of simple cells, as shown in the top part of Figure 7, we would therefore have to a assume a distribution of receptive fields over different orders of spatial differentiation.

With ample reservations for to the possibility of substantial statistical fluctuations in the histograms, as could be influenced by the selection of the actual visual neurons from which the histograms were computed in the experiments, after noting that the histogram accumulated over the simple cells show much larger variability between the bins than the histogram accumulated over the complex cells, one may then ask if the peak in the bins number 2 and 3 from the left in the histogram for simple cells in Figure 7 could be influenced by contributions from simple cells whose spatial components of the receptive fields can be well modelled by first-order derivatives (who span a much lower range of the values of $R$ than derivatives of higher order), as well as if the heavy part in the right part of the histogram, especially around bin number 8 from the left, could be influenced by simple cells that may be well modelled by directional derivatives for higher orders than 1.

In the next section, we will consider idealized model of receptive fields corresponding to higher orders of spatial differentiation, to investigate these matters in more detail.

3.2.3 Interpretation of the measurements of orientation selectivity histograms by Goris et al. (2015) based on spatial derivatives up to order 4

Let us first compute the resultants for the orientation selectivity curves according to Equations (31) and (32) for the third-order idealized models of simple cells

	$\displaystyle\begin{split}R_{\scriptsize\mbox{simple},3}&=\frac{\int_{\theta=-% \pi/2}^{\pi/2}\frac{\cos^{3}\theta}{(\cos^{2}\theta+\kappa^{2}\sin^{2}\theta)^% {3/2}}\,\cos 2\theta\,d\theta}{\int_{\theta=-\pi/2}^{\pi/2}\frac{\cos^{3}% \theta}{(\cos^{2}\theta+\kappa^{2}\sin^{2}\theta)^{3/2}}\,d\theta}\end{split}$
	$\displaystyle\begin{split}&=\frac{\kappa\left(\sqrt{\kappa^{2}-1}\left(\kappa^% {2}+2\right)-3\kappa\cosh^{-1}(\kappa)\right)}{\left(\kappa^{2}-1\right)\left(% \kappa\sqrt{\kappa^{2}-1}-\cosh^{-1}(\kappa)\right)},\end{split}$			(39)

and for the fourth-order idealized models of simple cells

	$\displaystyle\begin{split}R_{\scriptsize\mbox{simple},4}&=\frac{\int_{\theta=-% \pi/2}^{\pi/2}\frac{\cos^{4}\theta}{(\cos^{2}\theta+\kappa^{2}\sin^{2}\theta)^% {2}}\,\cos 2\theta\,d\theta}{\int_{\theta=-\pi/2}^{\pi/2}\frac{\cos^{4}\theta}% {(\cos^{2}\theta+\kappa^{2}\sin^{2}\theta)^{2}}\,d\theta}\end{split}$
	$\displaystyle\begin{split}&=\frac{\kappa\,(\kappa+3)}{(\kappa+1)(\kappa+2)}.% \end{split}$			(40)

From the graphs of these resultant curves as function of the logarithm of the scale parameter ratio $\kappa$ shown in Figure 10, we can with comparison to Figure 9 note that the distributions of resultant $R$ are heavier towards larger values $R$ for the third-order and fourth-order models of simple cells than for the second-order simple cells or the complex cell model based on first-order and second-order directional derivatives. The maximum values of $R$ for the third-order and fourth-order models of simple cells are also higher than for the first-order and second-order models, with the maximum value being about 0.97 for the third-order model and about 0.98 for the fourth-order model.

From such a viewpoint, it seems plausible that a distribution of receptive fields over range of values of the scale parameter ratio $\kappa$ as well as over different orders of spatial differentiation could lead to a bump in the histogram of the resultant $R$ for somewhat larger values of $R$ , as can be seen in the experimental results for simple cells reported by Goris et al. (2015) and as reproduced in Figure 7.

Let us next compute the histograms over the resultant $R$ that will be the result if we would assume that the scale parameter ratio $\kappa$ would be uniformly distributed on a logarithmic scale $K=\log\kappa$ over some range delimited by $\kappa_{\min}$ and $\kappa_{\max}$ . As previously remarked in Footnote 2, such an assumption may constitute a simplification, since one may argue that a uniform distribution on a hemisphere, complemented with possible additional priors to take into account the accuracy of sampling in the parameter space, could be more appropriate. The assumption of a logarithmic distribution does, however, constitute a principled first assumption to use when to parameterize a strictly positive variable in terms of natural coordinates, in the absence of further information (Jaynes 1968).

To illustrate to what extent the distributions of the resultant will be influenced by receptive fields for different orders of spatial differentiation, Figure 11 shows histograms of the resultant accumulated for idealized models of simple cells of orders 1 and 2 as well as for the idealized model of complex cells based on a quasi-quadrature combination of the responses from first-order and second-order simple cells. For generating these graphs, we have created a uniform distribution of the scale parameter ratio $\kappa$ over a logarithmic scale, over the interval $\kappa\in[1/\kappa_{\max},\kappa_{\max}]$ for the arbitrary choice of $\kappa_{\max}=8$ . Figure 12 shows corresponding histograms of the resultant for third-order and fourth-order simple cells.

As can be seen from these graphs, the histogram over the first-order simple cells is delimited by a maximum value around $R_{\max,1}\approx 0.7$ , while the distributions for third-order and fourth-order simple cells are heavier for larger values of the resultant approaching $R\rightarrow 1$ , and in line with the above arguments. The model used for computing histograms of the resultant for the idealized models of complex cells does, however, not very well reproduce the shape of the biologically obtained histogram, thus indicating that the model for the complex cells may be overly simplified³³3In this context, it should be remarked that there is strong conceptual difference between the idealized models for simple cells vs. the idealized models of complex cells. The idealized model for simple cells has been determined in a theoretically principled manner from axiomatic derivations, and also been matched to biological measurements of simple cells, whereas the idealized model for complex cells has been chosen as an as straightforward way as possible for combining the responses of odd-shaped and even-shaped simple cells. for the purpose of reproducing the shape of the resultant histogram, see the discussion in Section 6 in (Lindeberg 2024b) for a number of suggestions concerning ways to extend that model.

Figure 13 shows additional results of combining the resultant for populations of simple cells over different orders of spatial differentiation, either up to order 2 or up to order 4, here assuming the same number of neurons for all the different orders of spatial differentiation.

With ample reservation from the facts that: (i) in the current state, we have not principled biological arguments for choosing particular values of the parameters $\kappa_{\min}$ and $\kappa_{\max}$ that determine the range of the scale parameter ratio $\kappa$ , where different choices of these parameters may affect the shapes of the combined histograms of the resultant $R$ , (ii) the choice of a logarithmic distribution over the scale parameter $\kappa$ does, as previously mentioned, neglect any co-dependency with respect to the distribution over the orientation angle $\varphi$ , (iv) the assumption about equal numbers of receptive fields for the different orders of spatial differentiation may not necessarily hold in reality, and additionally expressing reservation from the fact that (v) our computations of the resultant $R$ for the receptive fields are based on idealized noise free model, while there additionally could be sources to noise in the biological experiments as well as modelling errors between the receptive fields of the actual biological neurons and our idealized receptive fields, as can be seen from these results, these combined histograms give rise to a bump in the histograms for lower values of the resultant $R$ , in qualitative agreement with the biological results by Goris et al. (2015) and as reproduced in Figure 7. Furthermore, to obtain something that would look like a small bump for larger values of the resultant $R$ , the modelling situation with receptive fields up to order 4 would give a closer similarity to the biological results by Goris et al. (2015) compared to the model based on receptive fields up to order 2.

From this analysis, it thus appears as if (i) the histogram of the resultant of the simple cells in the biological experiments et al. (2015) could be rather well explained by the receptive fields in the primary visual cortex of macaque monkeys having a variability over the degree of elongation, and that additionally (ii) the non-uniform nature of the experimentally obtained histogram of the resultant $R$ for the simple cells could be explained better by assuming that receptive fields should be present up to a spatial differentiation order up to 4 than up to a spatial differentiation order of 2.

3.2.4 Summary of the interpretations of the biological experiments by Nauhaus et al. (2008) and by Goris et al. (2015)

With regard to the working hypothesis that we set out to investigate the possible validity of, we can conclude that the experimental results by both Nauhaus et al. (2008) and by Goris et al. (2015) are, in combination with the theoretical results in Sections 2.2 and 3.1, concerning a direct connection between the degree of elongation of a receptive field with the orientation selectivity of the receptive field, clearly consistent with an expansion over the degree of elongation of the receptive field shapes in the primary visual cortex.

Based on these results we propose that, beyond an expansion over rotations, as is performed in current models of the pinwheel structure of visual receptive fields (Bonhoeffer and Grinvald 1991, Blasdel 1992, Swindale 1996, Petitot 2003, Koch et al. 2016, Kremkow et al. 2016, Baspinar et al. 2018, Najafian et al. 2022, Liu and Robinson 2022), also an explicit expansion over the eccentricity $\epsilon$ of the receptive fields (the inverse of the parameter $\kappa$ ) should be included, when modelling the pinwheel structure in the visual cortex.

Possible ways, by which an explicit dependency on the eccentricity of the receptive fields could be incorporated into the modelling of pinwheel structures, will be outlined in more detail in the following treatment regarding more specific biological hypotheses.

3.3 Explicit testable hypotheses for biological experiments

Based on the above theoretical analysis, with its associated theoretical predictions, we propose that it would be highly interesting to perform experimental characterization and analysis based on joint estimation of

•

orientational selectivity,
•

receptive field eccentricity,
•

orientational homogeneity and
•

location of the neuron in the visual cortex in relation to the pinwheel structure,

in the primary visual cortex of animals with clear pinwheel structures, to determine if there is a variability in the eccentricity or elongation of the receptive fields, and specifically if the degree of elongation increases with the distance from the centres of the pinwheels towards periphery, as arising as one possible interpretation of combining the theoretical results about orientation selectivity of affine Gaussian receptive fields in this article with the biological results by Nauhaus et al. (2008).

If additionally, reconstructions of the receptive field shapes could be performed for the receptive fields probed during such a systematic investigation of the difference in response characteristics with the distance from the pinwheel centers, and if the receptive fields could additionally be reasonably well modelled according to the generalized Gaussian model for receptive fields studied and used in this paper, it would be interesting to investigate if the shapes of the affine Gaussian components of these receptive fields would span a larger part of the affine group, than the span over mere image orientations, as already established in the orientation maps of the visual cortex, as characterized by Bonhoeffer and Grinvald (1991), Blasdel (1992) and others, see Figure 15 for an illustration.

If we would lay out the shapes of affine Gaussian receptive fields according to the shapes of their underlying spatial covariance matrices $\Sigma$ , then we would for a fixed value of their size (the spatial scale parameter) obtain a distribution of the form shown in Figure 15. That directional distribution is, however, in a certain aspect redundant, since opposite orientations on the unit circle are represented by two explicit copies, where the corresponding receptive fields are either equal, for receptive fields corresponding to spatial directional derivatives of even order, or of opposite sign for derivatives of even order. Could it be established that the receptive fields shapes, if expanded over a variability over eccentricity or elongation, for animals that have a clear pinwheel structure, have a spatial distribution that can somehow be related to such an idealized distribution, if we collapse opposite image orientations to the same image orientation, by e.g. a double-angle mapping $\varphi\mapsto 2\varphi$ ?

Notably the variability of the spatial covariance matrices in the affine Gaussian derivative model comprises a variability over two spatial scale parameters $\sigma_{1}$ and $\sigma_{2}$ , while the theoretical analysis of the orientation selectivity properties studied in this article has mainly concerned their ratio $\kappa=\sigma_{2}/\sigma_{1}$ . Hence, the illustration in Figure 15 should not be taken as a literal prediction, even if reduced by a double-angle representation. In Figure 15, the larger scale parameter $\sigma_{1}$ is held constant, for convenience of graphical illustration, as obtained by mapping the uniformly sized receptive fields from a uniform distribution on the hemisphere. More generally, one could also conceive other distributions as possible, such as, for example, instead keeping the smaller eigenvalue $\sigma_{2}$ constant from the center towards the periphery.

To conclude, we propose to state the following testable hypotheses for biological experiments:

Hypothesis 1 (Variability in eccentricity) Let $\sigma_{\varphi}$ and $\sigma_{\bot\varphi}$ be the characteristic lengths in the preferred directions of an orientation selective simple cell in the primary visual cortex. Then, over a population of such simple cells, there is a substantial variability in their eccentricity ratio $\epsilon=\sigma_{\varphi}/\sigma_{\bot\varphi}$ .

Hypothesis 2 (Variability in eccentricity coupled to orientational homogeneity) Assuming that Hypothesis 1 holds, let $\epsilon$ denote the eccentricity of a simple cell in the primary visual cortex, and let $H$ be a measure of the homogeneity in the orientation preference of its surrounding neurons. Then, over a population of simple cells, there is a systematic connection between $\epsilon$ and $H$ .

Hypothesis 3 (Variability in eccentricity coupled to the pinwheel structure) Assuming that Hypothesis 1 holds, let $\epsilon$ denote the eccentricity of a simple cell in the primary visual cortex. Then, over a population of simple cells, there is a systematic connection between $\epsilon$ and the distance from the nearest pinwheel center.

If Hypothesis 3 would hold, then we could also sharpen this hypothesis further as:

Hypothesis 4 (Increase in elongation with increasing distance from the centers of the pinwheels) Assuming that Hypothesis 3 holds, let $\epsilon$ denote the eccentricity measure of a simple cell in the primary visual cortex defined such that $\epsilon=1$ if the characteristic lengths of the spatial receptive fields are equal, and tending towards zero as the characteristic lengths differ more and more. Then, over a population of simple cells, the eccentricity measure decreases from the center of the pinwheel towards the periphery.

Note that the latter explicit hypotheses have been expressed on a general form, of not explicitly assuming that the biological receptive fields can be well modelled according to the generalized Gaussian derivative model for receptive fields. The essential factor in the definitions is only that it should be possible to define estimates of the characteristic lengths $\sigma_{\varphi}$ and $\sigma_{\bot\varphi}$ , so as to be able to define a measure of the eccentricity $\epsilon$ .

If either Hypothesis 2 or Hypotesis 3 would hold, then we could also explicitly state the following hypothesis:

Hypothesis 5 (Pinwheel structure more structured than a mere expansion over spatial orientations) The pinwheel structure comprises an, at least, two-dimensional variability of receptive field shapes, beyond an expansion over spatial orientations, also an expansion over the eccentricity of the receptive fields in the primary visual cortex.

For simplicity, we have above expressed these hypotheses for the case of simple cells, for which it is easiest to define the measures $\sigma_{\varphi}$ and $\sigma_{\bot\varphi}$ of the characteristic lengths, because of the linearity of the receptive fields. Provided that corresponding measures of characteristic lengths could also be in a sufficiently well-established way be defined also for non-linear complex cells, corresponding explicit hypotheses could also be formulated for complex cells.

It should finally be stressed that, we have in this treatment not considered the binocular aspects of the pinwheel structure. In Hypothesis 5, the variability of the pinwheel structure over contributions from the left and the right eyes should therefore not be counted as a property to contribute to the terminology “more structured”.

3.4 Quantitative measurements for detailed characterization

To further characterize possible relationships between the orientational selectivity, receptive field eccentricity, orientational homogeneity, and the location of the neuron in relation to the pinwheel structure in the primary visual cortex, we would also propose to characterize the possible relationships between these entities in terms of:

Quantitative measurement 1: (Relationship between orientational selectivity and receptive field eccentricity) Graph or scatter diagram showing how a quantitative measure of orientational selectivity is related to a quantitative measure of receptive field eccentricity, accumulated over a sufficiently large population of neurons.

Quantitative measurement 2: (Relationship between orientational homogeneity and receptive field eccentricity) Graph or scatter diagram showing how a quantitative measure of orientational homogeneity is related to a quantitative measure receptive field eccentricity, accumulated over a sufficiently large population of neurons.

Quantitative measurement 3: (Relationship between receptive field eccentricity and the pinwheel structure) Graph or scatter diagram showing how a quantitative measure of receptive field eccentricity depends on the distance to the nearest pinwheel center, accumulated over a sufficiently large population of neurons.

Quantitative measurement 4: (Relationship between receptive field eccentricity and the pinwheel structure) Two-dimensional map showing how a quantitative measure of receptive field eccentricity relates to a two-dimensional map of the orientation preference over the same region in the primary visual cortex, with the center of the pinwheel structure explicitly marked, again accumulated over a sufficiently large population of neurons.

If the above theoretically motivated biological hypotheses could be investigated experimentally, and if the above quantitative measurements of receptive field characteristics could be performed, it could be judged if the prediction from the presented theoretical analysis about a systematic variability in receptive field eccentricity, with a possible relationship to the pinwheel structure, could be either experimentally supported or rejected. In a corresponding manner, such a judgement could also answer if the receptive fields in the primary visual cortex could be regarded as spanning a larger part of the affine group, than an expansion over mere rotations in the image domain.

4 Discussion

By comparing the theoretical analysis of the orientation selectivity properties of the affine Gaussian derivative model (Section 2.2) with the experimental results from Nauhaus et al. (2008) on broadly vs. sharply tuned visual neurons (Figure 6) and from Goris et al. (2015) on rather uniform distributions of the resultant values from orientation selectivity curves (Figure 7), we find potential support for one of the dimensions of variability in a biological hypothesis formulated in Lindeberg (2023), stating that the family of receptive field shapes should span the degrees of freedom in the natural geometric image transformations. This potential support rests on the assumption, that it should be unlikely for the population of receptive fields to show strong variability in orientation selectivity, without also showing similar variability in eccentricity or elongation.

Such an assumption-based reasoning would then specifically imply indirect support for the hypothesis that the receptive field shapes should span a sufficiently wide range of ratios between the scale parameters in the directions perpendicular to versus parallel with the preferred orientation of the receptive field, to support affine covariance of the family of visual receptive fields.

Without explicitly relying on expressing such an explicit assumption, regarding whether the visual receptive fields in the primary visual cortex could be well modelled by affine Gaussian derivative based receptive fields, we can, however, firmly state that the biological measurements performed by Nauhaus et al. (2008) and by Goris et al. (2015) are, in combination with the theoretical results summarized in Section 2.2, consistent with the hypothesis that the receptive fields should span a variability in the eccentricity of the receptive fields. In this respect, the measurements that demonstrate a strong variability in orientation selectivity would specifically be consistent with the theoretically based hypothesis formulated in (Lindeberg 2023), that the receptive fields in the primary visual cortex should span the variability of receptive field shapes under spatial affine transformations.

If we would apply a similar type of assumption-based logical reasoning to the pinwheel structure in the primary visual cortex, then such a reasoning, based on the results by Nauhaus et al. (2008), that the orientation selectivity appears to vary strongly from the centers of the pinwheels towards the periphery, would imply that the pinwheel structure in the visual cortex would, beyond an explicit expansion over image orientations, also comprise an explicit expansion over the eccentricity or the degree of elongation of the receptive fields. Based on these predictions, we propose to consider explicit dependencies on a variability in the eccentricity of the receptive fields, when modelling the pinwheel structures in the primary visual cortex.

Strictly, and formally, the results from such logical inference could, however, only be regarded as theoretical predictions, to generate explicit hypothesis concerning the distribution of receptive field characteristics in these respects. To raise the question of determining if these theoretical predictions would firmly hold in reality, we propose that the testable explicit biological hypotheses formulated in Section 3.3 could be used to, in neurophysiological experiments, either verify or reject the overall hypothesis, concerning possible variabilities in the eccentricity of the receptive fields in the primary visual cortex of higher mammals, as well as hypotheses about possible connections between such variabilities in the eccentricity or the elongation and other receptive field characteristics, in particular in relation to the pinwheel structure in the primary visual cortex of higher mammals. Furthermore, if those hypothesis would hold, then the proposed quantitative measurements formulated in Section 3.4 could be used, to characterise how a possible variability in the eccentricity of the receptive field could be related to other receptive field characteristics, including the pinwheel structure in the visual cortex.

Concerning possible limitations in the hypothetical reasoning stages used for possible logical inference and for formulating the explicit biological hypotheses above, based on explicitly stated assumptions regarding whether the biological receptive fields could be reasonably well modelled by affine Gaussian derivative based receptive fields, to be able to draw possible further conclusions, the possible validity of those hypothetical logical reasoning stages could, however, break down, if there would be other external factors, not covered by the theoretical model, that could also strongly influence the orientation selectivity of the receptive fields. The possible applicability of the hypothetical logical reasoning stages above thus, ultimately, depends on the possible agreement between the model and biological data, and can only be taken further by performing more detailed actual model fitting (not performed here, because of lack of access to the data by Nauhaus et al. (2008) as well as lack of access to data with receptive field recordings over a sufficiently large population of visual neurons in the primary visual cortex) and/or performing complementary neurophysiological experiments, to ultimately judge if the theoretically based predictions, stated more explicitly in Section 3.3, would be applicable to actual biological neurons.

Author contributions

TL defined the scope of the work, developed the theory, performed the analysis, formulated the biological hypotheses, generated the illustrations in Figures 1, 2, 3, 4, 5, 8, 9, 10, 11, 12, 13, and 15, and wrote the paper.

Data availability statement

This paper does not contain any new experimental data, specifically neither on humans nor on cell cultures.

The data used for the analysis presented in this paper are based on existing sources in the scientific literature, explicitly cited in the text.

Additional information

The author declares no competing interests.

References

Abballe and Asari [2022] L. Abballe and H. Asari. Natural image statistics for mouse vision. PLoS ONE, 17(1):e0262763, 2022.
Adelson and Bergen [1985] E. Adelson and J. Bergen. Spatiotemporal energy models for the perception of motion. Journal of Optical Society of America, A 2:284–299, 1985.
Albright [1984] T. D. Albright. Direction and orientation selectivity of neurons in visual area MT of the macaque. Journal of Neurophysiology, 52(6):1106–1130, 1984.
Almasi et al. [2020] A. Almasi, H. Meffin, S. L. Cloherty, Y. Wong, M. Yunzab, and M. R. Ibbotson. Mechanisms of feature selectivity and invariance in primary visual cortex. Cerebral Cortex, 30(9):5067–5087, 2020.
Baspinar et al. [2018] E. Baspinar, G. Citti, and A. Sarti. A geometric model of multi-scale orientation preference maps via Gabor functions. Journal of Mathematical Imaging and Vision, 60:900–912, 2018.
Berkes and Wiskott [2005] P. Berkes and L. Wiskott. Slow feature analysis yields a rich repertoire of complex cell properties. Journal of Vision, 5(6):579–602, 2005.
Blasdel [1992] G. G. Blasdel. Orientation selectivity, preference and continuity in monkey striate cortex. Journal of Neuroscience, 12(8):3139–3161, 1992.
Bonhoeffer and Grinvald [1991] T. Bonhoeffer and A. Grinvald. Iso-orientation domains in cat visual cortex are arranged in pinwheel-like patterns. Nature, 353:429–431, 1991.
Bracewell [1999] R. N. Bracewell. The Fourier Transform and its Applications. McGraw-Hill, New York, 1999. 3rd edition.
Carandini [2006] M. Carandini. What simple and complex cells compute. The Journal of Physiology, 577(2):463–466, 2006.
Carandini and Ringach [1997] M. Carandini and D. L. Ringach. Predictions of a recurrent model of orientation selectivity. Vision Research, 37(21):3061–3071, 1997.
Cogno and Mato [2015] S. G. Cogno and G. Mato. The effect of synaptic plasticity on orientation selectivity in a balanced model of primary visual cortex. Frontiers in Neural Circuits, 9:42, 2015.
Conway and Livingstone [2006] B. R. Conway and M. S. Livingstone. Spatial and temporal properties of cone signals in alert macaque primary visual cortex. Journal of Neuroscience, 26(42):10826–10846, 2006.
De and Horwitz [2021] A. De and G. D. Horwitz. Spatial receptive field structure of double-opponent cells in macaque V1. Journal of Neurophysiology, 125(3):843–857, 2021.
DeAngelis and Anzai [2004] G. C. DeAngelis and A. Anzai. A modern view of the classical receptive field: Linear and non-linear spatio-temporal processing by V1 neurons. In L. M. Chalupa and J. S. Werner, editors, The Visual Neurosciences, volume 1, pages 704–719. MIT Press, 2004.
DeAngelis et al. [1995] G. C. DeAngelis, I. Ohzawa, and R. D. Freeman. Receptive field dynamics in the central visual pathways. Trends in Neuroscience, 18(10):451–457, 1995.
Einhäuser et al. [2002] W. Einhäuser, C. Kayser, P. König, and K. P. Körding. Learning the invariance properties of complex cells from their responses to natural stimuli. European Journal of Neuroscience, 15(3):475–486, 2002.
Emerson et al. [1987] R. C. Emerson, M. C. Citron, W. J. Vaughn, and S. A. Klein. Nonlinear directionally selective subunits in complex cells of cat striate cortex. Journal of Neurophysiology, 58(1):33–65, 1987.
Fang et al. [2022] C. Fang, X. Cai, and H. D. Lu. Orientation anisotropies in macaque visual areas. Proceedings of the National Academy of Sciences, 119(15):e2113407119, 2022.
Ferster and Miller [2000] D. Ferster and K. D. Miller. Neural mechanisms of orientation selectivity in the visual cortex. Annual Review of Neuroscience, 23(1):441–471, 2000.
Franciosini et al. [2019] A. Franciosini, V. Boutin, and L. Perrinet. Modelling complex cells of early visual cortex using predictive coding. In Proc. 28th Annual Computational Neuroscience Meeting, 2019. Available from https://laurentperrinet.github.io/publication/franciosini-perrinet-19-cns/franciosini-perrinet-19-cns.pdf.
Georgeson et al. [2007] M. A. Georgeson, K. A. May, T. C. A. Freeman, and G. S. Hesse. From filters to features: Scale-space analysis of edge and blur coding in human vision. Journal of Vision, 7(13):7.1–21, 2007.
Ghodrati et al. [2017] M. Ghodrati, S.-M. Khaligh-Razavi, and S. R. Lehky. Towards building a more complex view of the lateral geniculate nucleus: Recent advances in understanding its role. Progress in Neurobiology, 156:214–255, 2017.
Goris et al. [2015] R. L. T. Goris, E. P. Simoncelli, and J. A. Movshon. Origin and function of tuning diversity in Macaque visual cortex. Neuron, 88(4):819–831, 2015.
Hansard and Horaud [2011] M. Hansard and R. Horaud. A differential model of the complex cell. Neural Computation, 23(9):2324–2357, 2011.
Hansel and van Vreeswijk [2012] D. Hansel and C. van Vreeswijk. The mechanism of orientation selectivity in primary visual cortex without a functional map. Journal of Neuroscience, 32(12):4049–4064, 2012.
Hansen and Neumann [2008] T. Hansen and H. Neumann. A recurrent model of contour integration in primary visual cortex. Journal of Vision, 8(8):8.1–25, 2008.
Heeger [1992] D. J. Heeger. Normalization of cell responses in cat striate cortex. Visual Neuroscience, 9:181–197, 1992.
Hesse and Georgeson [2005] G. S. Hesse and M. A. Georgeson. Edges and bars: where do people see features in 1-D images? Vision Research, 45(4):507–525, 2005.
Hubel and Wiesel [1959] D. H. Hubel and T. N. Wiesel. Receptive fields of single neurones in the cat’s striate cortex. J Physiol, 147:226–238, 1959.
Hubel and Wiesel [1962] D. H. Hubel and T. N. Wiesel. Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J Physiol, 160:106–154, 1962.
Hubel and Wiesel [1968] D. H. Hubel and T. N. Wiesel. Receptive fields and functional architecture of monkey striate cortex. The Journal of Physiology, 195(1):215–243, 1968.
Hubel and Wiesel [2005] D. H. Hubel and T. N. Wiesel. Brain and Visual Perception: The Story of a 25-Year Collaboration. Oxford University Press, 2005.
Jaynes [1968] E. T. Jaynes. Prior probabilities. Trans. on Systems Science and Cybernetics, 4(3):227–241, 1968.
Johnson et al. [2008] E. N. Johnson, M. J. Hawken, and R. Shapley. The orientation selectivity of color-responsive neurons in Macaque V1. The Journal of Neuroscience, 28(32):8096–8106, 2008.
Jones and Palmer [1987a] J. Jones and L. Palmer. The two-dimensional spatial structure of simple receptive fields in cat striate cortex. J. of Neurophysiology, 58:1187–1211, 1987a.
Jones and Palmer [1987b] J. Jones and L. Palmer. An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. J. of Neurophysiology, 58:1233–1258, 1987b.
Jung et al. [2022] Y. J. Jung, A. Almasi, S. H. Sun, M. Yunzab, S. L. Cloherty, S. H. Bauquier, M. Renfree, H. Meffin, and M. R. Ibbotson. Orientation pinwheels in primary visual cortex of a highly visual marsupial. Science Advances, 8(39):eabn0954, 2022.
Koch et al. [2016] E. Koch, J. Jin, J. M. Alonso, and Q. Zaidi. Functional implications of orientation maps in primary visual cortex. Nature Communications, 7(1):13529, 2016.
Koenderink [1984] J. J. Koenderink. The structure of images. Biological Cybernetics, 50(5):363–370, 1984.
Koenderink and van Doorn [1987] J. J. Koenderink and A. J. van Doorn. Representation of local geometry in the visual system. Biological Cybernetics, 55(6):367–375, 1987.
Koenderink and van Doorn [1992] J. J. Koenderink and A. J. van Doorn. Generic neighborhood operators. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(6):597–605, Jun. 1992.
Kording et al. [2004] K. P. Kording, C. Kayser, W. Einhäuser, and P. Konig. How are complex cell properties adapted to the statistics of natural stimuli? Journal of Neurophysiology, 91(1):206–212, 2004.
Kremkow et al. [2016] J. Kremkow, J. Jin, Y. Wang, and J. M. Alonso. Principles underlying sensory map topography in primary visual cortex. Nature, 533(7601):52–57, 2016.
Kristensen and Sandberg [2021] D. G. Kristensen and K. Sandberg. Population receptive fields of human primary visual cortex organised as dc-balanced bandpass filters. Scientific Reports, 11(1):22423, 2021.
Lampl et al. [2001] I. Lampl, J. S. Anderson, D. C. Gillespie, and D. Ferster. Prediction of orientation selectivity from receptive field architecture in simple cells of cat visual cortex. Neuron, 30(1):263–274, 2001.
Li et al. [2015] Y.-T. Li, B.-H. Liu, X.-L. Chou, L. I. Zhang, and H. W. Tao. Synaptic basis for differential orientation selectivity between complex and simple cells in mouse visual cortex. Journal of Neuroscience, 35(31):11081–11093, 2015.
Lian et al. [2021] Y. Lian, A. Almasi, D. B. Grayden, T. Kameneva, A. N. Burkitt, and H. Meffin. Learning receptive field properties of complex cells in V1. PLoS Computational Biology, 17(3):e1007957, 2021.
Lindeberg [2013a] T. Lindeberg. A computational theory of visual receptive fields. Biological Cybernetics, 107(6):589–635, 2013a.
Lindeberg [2013b] T. Lindeberg. Invariance of visual operations at the level of receptive fields. PLOS ONE, 8(7):e66990, 2013b.
Lindeberg [2020] T. Lindeberg. Provably scale-covariant continuous hierarchical networks based on scale-normalized differential expressions coupled in cascade. Journal of Mathematical Imaging and Vision, 62(1):120–148, 2020.
Lindeberg [2021] T. Lindeberg. Normative theory of visual receptive fields. Heliyon, 7(1):e05897:1–20, 2021. doi: 10.1016/j.heliyon.2021.e05897.
Lindeberg [2023] T. Lindeberg. Covariance properties under natural image transformations for the generalized Gaussian derivative model for visual receptive fields. Frontiers in Computational Neuroscience, 17:1189949:1–23, 2023.
Lindeberg [2024a] T. Lindeberg. Joint covariance properties under geometric image transformations for spatio-temporal receptive fields according to the generalized Gaussian derivative model for visual receptive fields. arXiv preprint arXiv:2311.10543, 2024a.
Lindeberg [2024b] T. Lindeberg. Orientation selectivity properties for the affine Gaussian derivative and the affine Gabor models for visual receptive fields. arXiv preprint arXiv:2304.11920, 2024b.
Liu and Robinson [2022] X. Liu and P. A. Robinson. Analytic model for feature maps in the primary visual cortex. Frontiers in Computational Neuroscience, 16:2, 2022.
Lowe [2000] D. G. Lowe. Towards a computational model for object recognition in IT cortex. In Biologically Motivated Computer Vision, volume 1811 of Springer LNCS, pages 20–31. Springer, 2000.
Maldonado et al. [1997] P. E. Maldonado, I. Godecke, C. M. Gray, and T. Bonhoeffer. Orientation selectivity in pinwheel centers in cat striate cortex. Science, 276(5318):1551–1555, 1997.
Marcelja [1980] S. Marcelja. Mathematical description of the responses of simple cortical cells. Journal of Optical Society of America, 70(11):1297–1300, 1980.
Martinez and Alonso [2001] L. M. Martinez and J.-M. Alonso. Construction of complex receptive fields in cat primary visual cortex. Neuron, 32(3):515–525, 2001.
May and Georgeson [2007] K. A. May and M. A. Georgeson. Blurred edges look faint, and faint edges look sharp: The effect of a gradient threshold in a multi-scale edge coding model. Vision Research, 47(13):1705–1720, 2007.
Merkt et al. [2019] B. Merkt, F. Schüßler, and S. Rotter. Propagation of orientation selectivity in a spiking network model of layered primary visual cortex. PLoS Computational Biology, 15(7):e1007080, 2019.
Merolla and Boahn [2004] P. Merolla and K. Boahn. A recurrent model of orientation maps with simple and complex cells. In Advances in Neural Information Processing Systems (NIPS 2004), pages 995–1002, 2004.
Moldakarimov et al. [2014] S. Moldakarimov, M. Bazhenov, and T. J. Sejnowski. Top-down inputs enhance orientation selectivity in neurons of the primary visual cortex during perceptual learning. PLoS Computational Biology, 10(8):e1003770, 2014.
Movshon et al. [1978] J. A. Movshon, E. D. Thompson, and D. J. Tolhurst. Receptive field organization of complex cells in the cat’s striate cortex. The Journal of Physiology, 283(1):79–99, 1978.
Najafian et al. [2022] S. Najafian, E. Koch, K. L. Teh, J. Jin, H. Rahimi-Nasrabadi, Q. Zaidi, J. Kremkow, and J.-M. Alonso. A theory of cortical map formation in the visual brain. Nature Communications, 13(1):2303, 2022.
Nauhaus et al. [2008] I. Nauhaus, A. Benucci, M. Carandini, and D. L. Ringach. Neuronal selectivity and local map structure in visual cortex. Neuron, 57(5):673–679, 2008.
Nguyen and Freeman [2019] G. Nguyen and A. W. Freeman. A model for the origin and development of visual orientation selectivity. PLoS Computational Biology, 15(7):e1007254, 2019.
Oleskiw et al. [2024] T. D. Oleskiw, J. D. Lieber, E. P. Simoncelli, and J. A. Movshon. Foundations of visual form selectivity for neurons in macaque V1 and V2. bioRxiv, 2024.03.04.583307, 2024.
Pattadkal et al. [2018] J. J. Pattadkal, G. Mato, C. van Vreeswijk, N. J. Priebe, and D. Hansel. Emergent orientation selectivity from random networks in mouse visual cortex. Cell Reports, 24(8):2042–2050, 2018.
Pei et al. [2016] Z.-J. Pei, G.-X. Gao, B. Hao, Q.-L. Qiao, and H.-J. Ai. A cascade model of information processing and encoding for retinal prosthesis. Neural Regeneration Research, 11(4):646, 2016.
Petitot [2003] J. Petitot. The neurogeometry of pinwheels as a sub-Riemannian contact structure. Journal of Physiology-Paris, 97(2–3):265–309, 2003.
Porat and Zeevi [1988] M. Porat and Y. Y. Zeevi. The generalized Gabor scheme of image representation in biological and machine vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 10(4):452–468, 1988.
Priebe [2016] N. J. Priebe. Mechanisms of orientation selectivity in the primary visual cortex. Annual Review of Vision Science, 2:85–107, 2016.
Ringach [2002] D. L. Ringach. Spatial structure and symmetry of simple-cell receptive fields in macaque primary visual cortex. Journal of Neurophysiology, 88:455–463, 2002.
Ringach [2004] D. L. Ringach. Mapping receptive fields in primary visual cortex. Journal of Physiology, 558(3):717–728, 2004.
Ringach et al. [2002] D. L. Ringach, R. M. Shapley, and M. J. Hawken. Orientation selectivity in macaque V1: Diversity and laminar dependence. Journal of Neuroscience, 22(13):5639–5651, 2002.
Rose and Blakemore [1974] D. Rose and C. Blakemore. An analysis of orientation selectivity in the cat’s visual cortex. Experimental Brain Research, 20:1–17, 1974.
Ruslim et al. [2023] M. A. Ruslim, A. N. Burkitt, and Y. Lian. Learning spatio-temporal V1 cells from diverse LGN inputs. bioRxiv, pages 2023–11, 2023.
Rust et al. [2005] N. C. Rust, O. Schwartz, J. A. Movshon, and E. P. Simoncelli. Spatiotemporal elements of macaque V1 receptive fields. Neuron, 46(6):945–956, 2005.
Sadeh and Rotter [2014] S. Sadeh and S. Rotter. Statistics and geometry of orientation selectivity in primary visual cortex. Biological Cybernetics, 108:631–653, 2014.
Sasaki et al. [2015] K. S. Sasaki, R. Kimura, T. Ninomiya, Y. Tabuchi, H. Tanaka, M. Fukui, Y. C. Asada, T. Arai, M. Inagaki, T. Nakazono, M. Baba, K. Daisuke, S. Nishimoto, T. M. Sanada, T. Tani, K. Imamura, S. Tanaka, and I. Ohzawa. Supranormal orientation selectivity of visual neurons in orientation-restricted animals. Scientific Reports, 5(1):16712, 2015.
Schiller et al. [1976] P. H. Schiller, B. L. Finlay, and S. F. Volman. Quantitative studies of single-cell properties in monkey striate cortex. II. Orientation specificity and ocular dominance. Journal of Neurophysiology, 39(6):1320–1333, 1976.
Scholl et al. [2013] B. Scholl, A. Y. Y. Tan, J. Corey, and N. J. Priebe. Emergence of orientation selectivity in the mammalian visual pathway. Journal of Neuroscience, 33(26):10616–10624, 2013.
Seriès et al. [2004] P. Seriès, P. E. Latham, and A. Pouget. Tuning curve sharpening for orientation selectivity: coding efficiency and the impact of correlations. Nature Neuroscience, 7(10):1129–1135, 2004.
Serre and Riesenhuber [2004] T. Serre and M. Riesenhuber. Realistic modeling of simple and complex cell tuning in the HMAX model, and implications for invariant object recognition in cortex. Technical Report AI Memo 2004-017, MIT Computer Science and Artifical Intelligence Laboratory, 2004.
Shapley et al. [2003] R. Shapley, M. Hawken, and D. L. Ringach. Dynamics of orientation selectivity in the primary visual cortex and the importance of cortical inhibition. Neuron, 38(5):689–699, 2003.
Somers et al. [1995] D. C. Somers, S. B. Nelson, and M. Sur. An emergent model of orientation selectivity in cat visual cortical simple cells. Journal of Neuroscience, 15(8):5448–5465, 1995.
Sompolinsky and Shapley [1997] H. Sompolinsky and R. Shapley. New perspectives on the mechanisms for orientation selectivity. Current Opinion in Neurobiology, 7(4):514–522, 1997.
Swindale [1996] N. V. Swindale. The development of topography in the visual cortex: A review of models. Network: Computation in Neural Systems, 7(2):161–247, 1996.
Touryan et al. [2002] J. Touryan, B. Lau, and Y. Dan. Isolation of relevant visual features from random stimuli for cortical complex cells. Journal of Neuroscience, 22(24):10811–10818, 2002.
Touryan et al. [2005] J. Touryan, G. Felsen, and Y. Dan. Spatial structure of complex cell receptive fields measured with natural images. Neuron, 45(5):781–791, 2005.
Valois et al. [2000] R. L. D. Valois, N. P. Cottaris, L. E. Mahon, S. D. Elfer, and J. A. Wilson. Spatial and temporal receptive fields of geniculate and cortical cells and directional selectivity. Vision Research, 40(2):3685–3702, 2000.
van Kleef et al. [2010] J. P. van Kleef, , S. L. Cloherty, and M. R. Ibbotson. Complex cell receptive fields: evidence for a hierarchical mechanism. The Journal of Physiology, 588(18):3457–3470, 2010.
Vita et al. [2024] D. J. Vita, F. S. Orsi, N. G. Stanko, N. A. Clark, and A. Tiriac. Development and organization of the retinal orientation selectivity map. bioRxiv:2024.03.27.585774, 2024.
Walker et al. [2019] E. Y. Walker, F. H. Sinz, E. Cobos, T. Muhammad, E. Froudarakis, P. G. Fahey, A. S. Ecker, J. Reimer, X. Pitkow, and A. S. Tolias. Inception loops discover what excites neurons most using deep predictive models. Nature Neuroscience, 22(12):2060–2065, 2019.
Wallis and Georgeson [2009] S. A. Wallis and M. A. Georgeson. Mach edges: Local features predicted by 3rd derivative spatial filtering. Vision Research, 49(14):1886–1893, 2009.
Wang et al. [2024] H. Wang, O. Dey, W. N. Lagos, N. Behnam, E. M. Callaway, and B. K. Stafford. Parallel pathways carrying direction-and orientation-selective retinal signals to layer 4 of the mouse visual cortex. Cell Reports, 43(3), 2024.
Wang and Spratling [2016] Q. Wang and M. W. Spratling. Contour detection in colour images using a neurophysiologically inspired model. Cognitive Computation, 8(6):1027–1035, 2016.
Watkins and Berkley [1974] D. W. Watkins and M. A. Berkley. The orientation selectivity of single neurons in cat striate cortex. Experimental Brain Research, 19:433–446, 1974.
Wei et al. [2022] W. Wei, B. Merkt, and S. Rotter. A theory of orientation selectivity emerging from randomly sampling the visual field. bioRxiv, pages 2022–07, 2022.
Wendt and Faul [2024] G. Wendt and F. Faul. Binocular luster elicited by isoluminant chromatic stimuli relies on mechanisms similar to those in the achromatic case. Journal of Vision, 24(3):7–7, 2024.
Yedjour and Yedjour [2024] H. Yedjour and D. Yedjour. A spatiotemporal energy model based on spiking neurons for human motion perception. Cognitive Neurodynamics, pages 1–15, 2024.
Young [1985] R. A. Young. The Gaussian derivative theory of spatial vision: Analysis of cortical cell receptive field line-weighting profiles. Technical Report GMR-4920, Computer Science Department, General Motors Research Lab., Warren, Michigan, 1985.
Young [1987] R. A. Young. The Gaussian derivative model for spatial vision: I. Retinal mechanisms. Spatial Vision, 2(4):273–293, 1987.
Young and Lesperance [2001] R. A. Young and R. M. Lesperance. The Gaussian derivative model for spatio-temporal vision: II. Cortical data. Spatial Vision, 14(3, 4):321–389, 2001.
Young et al. [2001] R. A. Young, R. M. Lesperance, and W. W. Meyer. The Gaussian derivative model for spatio-temporal vision: I. Cortical model. Spatial Vision, 14(3, 4):261–319, 2001.

	First-order simple cell	Second-order simple cell	Complex cell
$\kappa=1$
$\kappa=2$
$\kappa=4$
$\kappa=8$

	Third-order simple cell	Fourth-order simple cell
$\kappa=1$
$\kappa=2$
$\kappa=4$
$\kappa=8$

Do the receptive fields in the primary visual cortex span a variability over the degree of elongation of the receptive fields?††thanks: The support from the Swedish Research Council (contract 2022-02969) is gratefully acknowledged.

Abstract

Keywords:

1 Introduction

1.1 The hypothesis about affine covariant receptive fields

1.2 Variability over the elongation of receptive fields

1.3 Contributions and novelty

2 Methods

2.1 Affine covariant visual receptive fields

2.2 Connections between the orientation selectivity and the degree of elongation of the receptive fields for the generalized Gaussian derivative model for visual receptive fields

2.2.1 Idealized models for simple cells

2.2.2 Idealized models for complex cells

2.2.3 Orientation selectivity curves for the idealized receptive field models

2.3 Relations to previous work

3 Results

3.1 Derivations of orientation selectivity properties for third-order and fourth-order simple cells

3.1.1 Third-order simple cell

3.1.2 Fourth-order simple cell

3.1.3 Resulting orientation selectivity curves

3.2 Interpretation of the connection between the orientation selectivity and the elongation of receptive fields in relation to biological measurements

3.2.1 Interpretation of the measurements about broad vs. sharp orientation selectivity of neurons by Nauhaus et al. (2008)

3.2.2 Interpretation of the measurements of orientation selectivity histograms by Goris et al. (2015) based on spatial derivatives up to order 2

3.2.3 Interpretation of the measurements of orientation selectivity histograms by Goris et al. (2015) based on spatial derivatives up to order 4

3.2.4 Summary of the interpretations of the biological experiments by Nauhaus et al. (2008) and by Goris et al. (2015)

3.3 Explicit testable hypotheses for biological experiments

3.4 Quantitative measurements for detailed characterization

4 Discussion

Author contributions

Data availability statement

Additional information

References

Do the receptive fields in the primary visual cortex span a variability over the degree of elongation of the receptive fields?^†^†thanks: The support from the Swedish Research Council (contract 2022-02969) is gratefully acknowledged.