Cortical areas

Single areas in the networks were composed of mean-field attractors that have been used to model working memory, perceptual decision-making, as well as to model individual areas in multi-area cortcal simulations [42, 45–48]. Each area consisted of two populations, such that c = A, B, where each population was associated with a choice alternative. The dynamics of a given population is described by a single synaptic gating variable representing the fraction of activated N-methyl-D-aspartate receptor (NMDA) conductance and governed by the equation [45], (1) where the NMDA time constant τ_NMDA = 60 ms, and the rate of saturation is controlled by γ = 0.641. The firing rate r was a function of the input current I, as defined by the curve relation first described in [49]: (2) where a = 270 Hz/nA, b = 108 Hz, and d = 0.154 seconds. The total synaptic current consisted of recurrent (I_rec), noisy (I_noise), background (I_o = 0.3297 nA) and external components such that, (3) where the coupling constant g = 0.0011 nA/Hz converts the firing rate I_input to current. As described in section 0.2, specific values of I_input are notated as I_A,1, I_A,2, I_B,1, or I_B,2. For a given population i in area n of the the network, the recurrent current was given by the equation, (4) where is the strength of the connection from population j in area m to population i in area n. The use of this equation in the Linear Network and Hierarchical Network is described in the following sections.

The noise currents for each population were defined by an Ornstein-Uhlenbeck process: (5) where the time constant τ_AMPA = 2 ms, and η_I is a Gaussian white noise term with mean zero, and variance .

Inputs

External inputs were provided as firing rates, spanning the range of 0 to 40 Hz. Notationally, they are given by I, such that the input of attribute 1 for choice alternative A was written I_A,1, etc. An unspecified choice alternative is indicated with the population subscript “c” and unspecified attribute by the subscript “x.”

When varying environmental uncertainty, a value η_I was randomly drawn from independently for each attribute on each trial, such that I_c,x = I_c,x + η_I. The variance was used to control the amount of environmental uncertainty in a given block of trials.

In several places, we refer to a normalized value of the input, . In those cases, we specify the quantity used in normalization.

Linear network

The Linear Network consisted of two layers. The first layer provided inputs from the individual attributes, in the form of firing rates I_input. The attribute inputs “a” to a given population were linearly summed, such that (6) where w = 0.5. These values were then fed into the final area described in section 0.1. As there was only one area, Eq 4 simplified to, (7) where for i = j, the connection strength was 0.3725 nA, and where i ≠ j the connection strength was −0.1137 nA [45]. The network choice was determined when one of the final area populations passed the firing rate threshold of 35 Hz.

Hierarchical networks

The input layer to the Hierarchical Networks was also specified by I_c,x. In the Hierarchical Network, inputs were segregated by attribute then transmitted to an intermediate transform layer, consisting of two-population areas specific for each attribute. These areas were structured along the lines described in section 0.1.

In the intermediate layer areas of the Hierarchical Network, the connection strengths in Eq 4 were defined as follows. J₊ controlled the strength of the excitatory connection of a population to itself (A → A, etc.), and J₋ the strength of the inhibitory connection from one population to another (A → B, etc.). The strength of J between areas in the intermediate layer areas was set to 0. Connectivity from the intermediate transform layer (TL) areas to the final layer (FL) was restricted to excitation between populations with the same selectivity, such that , , etc.

The excitatory and inhibitory weights in this intermediate layer (J₊ and J₋) were parametrically varied to achieve the specific Hierarchical Network E/I tones described in the Results section. For simplicity, weights in all areas in the processing layer were symmetrically varied, such that when the recurrent excitation or cross-inhibition were changed, all areas in the processing layer assumed the specified values. The outputs from this processing layer were continuously fed into a final area of the same type as the final area of the Linear Network in section 0.1. To clarify notation, when an input r is passed through an intermediate transform layer area, its associated transformed output is referred to as T. The value T is that of the synaptic gating variable from Eq 1.

Simplified non-dynamical models

We also used non-dynamical linear and max operations to investigate the separation of offer distributions. These models used the input values and uncertainty level to create distributions of decision values.

Model behavior

Fixed point analysis

To compute the fixed points and phase portraits of a processing layer area, we used the python package pydstool [52], parameterized by the equations described in 0.1, with the internal noise set to 0. We first defined an attribute input as a coordinate in euclidean space space whose positions are defined by the level of each input. We then defined a phase plane by applying these inputs to each E/I tone configuration. A starting position in each phase plane was selected near the origin at (0.06, 0.06), and a noiseless trajectory was computed. The end point of that trajectory was taken as the output of the transform layer.

Neurotypical and restricted inhibitory tone

We introduced a module that assessed the level of uncertainty in the environment, then used a continuous function to set the E/I tone in the intermediate layer areas to a tone that optimized performance. To do this, we ran the task on all E/I tones configurations as described in Section 0.3.2. Given the extensive run-time associated with simulations, we selected the E/I tone closest to that given by the continuous function. The neurotypical model was able to utilize the full range of excitation and inhibition. The limited inhibitory tone model was able to utilize the full range of excitation, but the inhibitory tone was limited to a maximum of −0.01 nA. We then presented the models with blocks of trials at different uncertainty levels, and calculated the P(Larger Chosen) (section 0.3.2) during each block.

Hierarchical networks adopt distinct decision regimes dictated by intermediate layer E/I tone

To investigate the behavior of multi-layer hierarchical neural networks, we used a multi-attribute decision task. On any given trial, the networks were presented with two choice alternatives (A or B). These choice alternatives were each composed of two attributes (1 or 2). For simplicity, attributes were assumed to have equal contributions to subjective value and action costs were assumed to zero. When indexing, the first value refers to the choice alternative, and the second to the attribute. Thus, choice alternative A was presented to the network using input I_A,1 and input I_A,2, while choice alternative B was presented to the network using input I_B,1 and input I_B,2. Inputs were translated from offer values to firing rates. As competition within the final area was between alternatives, the signal at that stage is denoted by F_A or F_B, where the subscript indicates the choice alternative. This integration of value attributes into a decision signal is similar to that recently observed in the dlPFC [22]. On a given trial, the choice between A or B was determined by the first population in the final area (F_A or F_B) to cross a firing rate threshold (35 Hz).

We created two basic network frameworks: the Linear Network and the Hierarchical Network. Areas in these networks were composed of mean-field approximations of population firing rates, parameterized by recurrent excitation and cross-inhibition (see Materials and methods). As the name suggests, the Linear Network computed an exact linear weighted sum of the attributes (weight = 0.5). Thus, with the Linear Network, the final area received a signal that was linearly translated from the offer values (Fig 1A). The Linear Network has a similar architecture to that presented in [53], and represents the case where all attribute signals are fully available to a final area.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Network schematics and sample trial.

I: input; F final layer synaptic gating variable; T: intermediate transform layer synaptic gating variable; red: alternative A; blue: alternative B; black: attribute 1; white: attribute 2; arrows: excitation; circles: inhibition. Both networks include an input layer and a final layer. (A) The Linear Network consists of two layers, with attribute signals from the input layer directly transmitted to the final layer. (B) The Hierarchical Network includes an additional intermediate layer that performs a functional transformation on the attribute signals prior to their passing to the final area. (C) A sample trial of the Hierarchical Network, with intermediate layer weights J₊ = 0.34 nA and J₋ = −0.02. For each area, the X-axis indicates time, while the Y-axis indicates the population firing rate. The first vertical line indicates the onset of the offer value signal, and the second vertical line indicates termination of the offer value signal.

https://doi.org/10.1371/journal.pcbi.1008791.g001

Several prior computational models have proposed parallel processing of attributes [30, 31, 54–58]. Furthermore, recent studies involving humans directly examined attribute-specific processing prior to integration in downstream areas [59–61]. Therefore, in the Hierarchical Network, inputs were first transformed by intermediate, attribute-specific areas (Fig 1B). The intermediate layer transforming the input signal was a nonlinear dynamical system, composed of mean-field populations associated with each choice alternative (see Materials and methods). To denote the transformation performed by an intermediate layer transform area on the input signal, its output signal will be referred to as T, such that I_A,1 is related to T_A,1, etc. These continuous output signals were integrated by the final winner-take-all area that determined the decision on a given trial. A single trial of the Hierarchical Network is shown in Fig 1C. This framework, similar in structure to that presented in [56], represents the simplest form of a network with parallel processing streams, where transformations are performed on attribute signals passing through specialized areas prior to the final decision.

To verify that the Hierarchical Network was able to perform the task, we computed simple psychometric functions of choice behavior. This was done by linearly increasing the input values associated with choice alternative A (I_A,1 and I_A,2), while keeping inputs associated with B (I_B,1, and I_B,2) fixed as a reference. We then varied the intermediate transform layer excitatory and inhibitory weights (colored gold in Fig 2A) and measured the proportion of trials that networks chose A (referred to as P(A)). A sigmoid function was fit to the resultant P(A) (see Materials and methods). Fig 2B shows the results for three intermediate transform layer E/I tone configurations. The networks were able to generate expected psychometric functions, increasingly choosing A as its offer value increased.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Hierarchical networks produce distinct decision regimes.

I: input; : input normalized by the reference choice alternative values’ linear sum; T: transform layer synaptic gating variable; P(A): proportion of 1,000 trials where alternative A was chosen; J₊: excitatory weight; J₋: inhibitory weight; yellow: varied weights; forest green: regime II; tan: regime I; sky blue: regime III. The first subscript indicates choice A or B, and the second subscript x indicates a generic attribute. Weights were systematically varied identically in all intermediate layer areas. (A) A diagram of a generic intermediate layer area and its inputs. (B) The Hierarchical Network produced expected psychometric functions. (C) Modification of intermediate layer E/I tone produced distinct indifference curve regimes. Examples of regime I (linear), regime II (convex) and regime III (concave) decision regimes. (D) The space of E/I tones can be partitioned by decision regime according to their indifference curves.

https://doi.org/10.1371/journal.pcbi.1008791.g002

Having verified the Hierarchical Network is able to perform the task, we next examined how it utilizes attribute information. While the Linear Network performs a simple linear sum of the attributes, the Hierarchical Network first passes the input signals through attribute-specific intermediate areas parameterized by E/I tone (Figs 1B and 2A).

To investigate how the transform of attribute inputs changed with E/I tone, we used indifference curves. The Linear Network by definition computes a weighted sum of the attributes, so its indifference curve is trivially linear, following the line where I_A,1 + I_A,2 = I_B,1 + I_B,2. For the Hierarchical Network, however, the indifference curve provides a tool for analyzing how it weighs and combines attribute information (Fig 2). A linear indifference curve indicates that the network is performing a linear combination (similar to that of the Linear Network). A convex indifference curve indicates that the network places greater weight on attribute values composing a choice alternative that is balanced between the attributes, and less weight on attribute values composing an unbalanced choice alternative. A concave indifference curve indicates the opposite: specifically, the network underweighs the smaller attribute value, and instead places preferentially greater weight on the salience of larger attributes.

When generating an indifference curve for each E/I tone, we again held I_B,1 and I_B,2 constant as a reference. We then independently varied the magnitudes of I_A,1 and I_A,2 (see Materials and methods). For each offer, we calculated the proportion of 1,000 trials where the network chose alternative A, producing P(A). For some values of I_A,1 and I_A,2, (for example, when both are much less than their counterparts) P(A) will be low. For other values of I_A,1 and I_A,2, (for example, when both are much greater than their counterparts) P(A) will be high. However, there will be a set of values of I_A,1 and I_A,2 where the subject is indifferent between the choices and P(A) ≈ 0.5. The values where P(A) ≈ 0.5 were used to fit the indifference curve.

We parametrically varied the levels of excitatory (0.30 nA to 0.40 nA, spaced 0.05 nA) and inhibitory weights (0.0 nA to −0.10 nA, spaced 0.05 nA) within intermediate layer transform areas (Fig 2A). For each E/I tone configuration, we presented the offers and then fit a single-parameter exponential function to the points where P(A) ≈ 0.5 (see Materials and methods). That single parameter determined the curvature, such that a value of 1 indicates a linear indifference curve, a value less than one indicates convex, and a value greater than 1 indicates concave. We found that at different levels of excitation and inhibition, the network adopted distinct decision regimes, corresponding to linear (regime I), convex (regime II) and concave (regime III) indifference curves (Fig 2C). The appearance of these linear-input decision regimes formed distinct regions, such that we were able to use the regimes to partition the space of E/I tones (Fig 2D). This revealed that by configuring the levels of excitation and inhibition in the intermediate layer, the Hierarchical Network can adopt distinct decision-making regimes.

Regime III arises from a winner-take-all operation

Having established that the E/I tone can define decision-making regimes, we next examined the functional transformation taking place within an intermediate layer attribute-specific transform area.

The phase planes of dynamical systems are a highly useful tool for analyzing the functional properties of decision networks [45, 46]. For an intermediate layer transform area in the Hierarchical Network, each combination of E/I tone and input produces a unique phase plane. We calculated phase planes for all combinations of an intermediate layer transform area’s excitatory tone (0.30 nA to 0.40 nA, spaced 0.05 nA) and inhibitory tone (0.0 nA to −0.10 nA, spaced 0.05 nA), and all input combinations of 0 to 40 Hz, spaced 0.5 Hz. This produced 793,881 total phase planes. For each phase plane, we computed a noiseless trajectory starting near the origin. The endpoint of that trajectory was taken as the output of the specific combination of network structure and inputs. We used these fixed point trajectory endpoints to approximate T_A,1, T_A,2, T_B,1, and T_B,2. The decision was determined by linearly summing these T values to obtain F_A and F_B, then choosing the maximum of these two choice values.

Though the intermediate layer outputs to the final area in the full dynamical model often did not reach the fixed points (due to the decision threshold), the fixed points did a reasonable job of approximating behavior. To measure this, for each E/I tone we fit a sigmoid to the choice behavior of the full model as a function of the difference between choice alternative fixed points (see Materials and methods). As over 90% of E/I tone configurations had an R² > 0.70, we were confident in the use of fixed points as an approximation for analysis of intermediate layer function.

We then used fixed points to compute indifference curves, similar to those shown in Fig 2C. In Fig 3A, we show the decision-making regimes as computed from the intermediate layer fixed points (as in Fig 2D). Its qualitatively similar form to that of Fig 2D further supports that the fixed points provide a reasonable approximation of the full network.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Hierarchical network decision regimes produce linear and nonlinear operations.

I: input; : input normalized by the maximum input value; T: the transformed output of an intermediate layer; J₊: excitatory weight; J₋: inhibitory weight; red: choice alternative A; blue: choice alternative B; green: trajectory from a starting point of (0.06, 0.06); solid circles: stable fixed points; large solid circle: trajectory endpoint. (A) Using fixed point approximations, the E/I tone space can be partitioned according to decision regimes similar to the full model. The color scheme is the same as Fig 2D. (B) Phase portraits and bifurcation diagrams show that regimes III performs a highly-nonlinear winner-take-all operation. The E/I tone is provided both in the heading, and indicated in (A) by the lower-case marker. The top row contains phase portraits produced by a specific set of inputs (I_A,x = 20, I_B,x = 18). The relative value of the inputs is indicated by the vertical dashed line in the bifurcation diagrams of the second row. To produce the bifurcation diagrams, the input-output of a single attribute value is varied (I_A,x) while the other (I_B,x) is fixed. The difference between the normalized fixed () and the varied value () is on the X-axis. The T value of the endpoints is on the Y-axis. Individual trajectory endpoint values T_A,x (red) and T_B,x (blue) values are plotted, along with lines joining them when needed to aid visualization.

https://doi.org/10.1371/journal.pcbi.1008791.g003

Confident in the representativeness of the intermediate layer fixed points, we used them to investigate the input-output relationship of various intermediate layer E/I tone configurations. First, we examined individual phase planes from E/I tone configurations in the three regimes (top row of Fig 3B), receiving inputs I_A,x = 18 Hz and I_B,x = 20 Hz. While the endpoints of of the intermediate transform layer areas in regimes I and II qualitatively maintained the relationship of the input (left and middle of the top row in Fig 3B), regime III performed a highly-nonlinear winner-take-all operation.

To better characterize this operation, we first looked at how the output changes as one input is increased while all else is held constant. We then assessed the how multiple attributes interact as they pass through the intermediate layer.

To investigate how output changes as only one input is varied, we kept one of the attribute inputs (I_B,x) to an intermediate layer transform area fixed, while increasing the level of the other attribute input (I_A,x). The resultant T_A,x and T_B,x for the three regions is shown in the bottom row of Fig 3B. Note that in regimes I and II the constant-input does not change as the magnitude of the variable input increases (Fig 3B, bottom left and middle). In regime III, when the variable input is less than the fixed input, the output-value of the variable input is suppressed; however, when the variable input is greater than the fixed input, the output-value of the constant input is suppressed, while the output-value of the varied input grows approximately linearly (Fig 3B, bottom row on the right). Thus, in regime I and II, each input value passes through the intermediate layer, with a degree of linearity defined by the E/I tone. In regime III, however, a qualitatively different function is implemented such that the smaller input value is suppressed by a max operation.

Hierarchical network performance relative to the linear network improves under uncertainty

Having characterized the decision-properties in three regimes of the Hierarchical Network and the basic operations performed by under these regimes, we investigated how performance of the Hierarchical Network compares to that of a conventional Linear Network. Uncertainty as to information has been a been widely studied in the context of multi-attribute decision-making, as well as value-based decision-making [23]. Indeed, many real-world decisions are made in environments where the true value of an attribute has a known degree of uncertainty. One could divide the source of an attribute’s known uncertainty into three categories: 1) external representation in the environment, 2) transmission of externally represented values into the brain, 3) internal generation of values within the brain. Case 1 could occur when bargaining for a good in a market, when there is an estimated uncertainty as to whether the value being quoted by the other party is accurate. Case 2 could arise when reading prices on a menu that are listed in a foreign currency; the numbers on the menu are accurate, but there is an estimated uncertainty as to the ability to convert the currency (from Rupees to Yuan) in one’s head. Case 3 could take place when one is assigning value to the amount of happiness a toy will bring for a young niece on her birthday, when there is an estimated uncertainty as to her preferences. While each of these cases involves a unique processing of individual attribute values prior to entering our network models, once they reach the network models they are identical.

Such uncertainty could be incorporated into a model’s attribute weights, or as a separate attribute unto itself. Yet, both approaches would fail to significantly alter performance in environments where the level of uncertainty rises uniformly. For example, if one incorporates uniform environmental uncertainty in attribute weights as the inverse of the variance, all attributes will decrease by the same proportion, thus having negligible effect on the decision process. Similarly, if one incorporates uncertainty as a separate attribute, the same attribute value will contribute to both alternatives; while this may affect the reaction time in a dynamical model, it again will fail to shift the decision strategy.

Instead, a global increase in environmental uncertainty should be addressed with a shift in the underlying function applied to attributes. We therefore investigated performance of our networks in environments where varying degrees of value stochasticity created uniform uncertainty as to the true attribute values.

To control the level of uncertainty in the environment, we added a stochasticity term η_I to each input at the beginning of the trial, independently drawn from a normal distribution centered on 0, with a variance of . We then sequentially presented networks with an array of 630 offers, which were translated to firing rates of 10 to 20 Hz, spaced by 2 Hz. The inputs were produced combinatorially, such that on one trial the alternative A attributes might be [14 Hz, 12 Hz] and alternative B attributes [12 Hz, 18 Hz]. On the next trial, A could be composed of [10 Hz, 12 Hz] and B [20 Hz, 18 Hz], etc. These choices were presented in blocks, where each block had a fixed that ranged from 0 to 2 Hz. To simplify the analysis, we assumed linear subjective value functions.

For the Linear Network and each E/I tone configuration of the Hierarchical Network, we first calculated the proportion of trials for each offer where the larger combined alternative value was chosen. We then compared the proportion of offers where the Hierarchical Network was more likely than the Linear Network to select the choice alternative with the larger combined value. We found when there is no environmental uncertainty, the Linear Network out-performs the Hierarchical Network on the vast majority of offers (Fig 4A, top). However, at a high level of environmental uncertainty, the proportion of offers increases where the Hierarchical Network outperforms the Linear Network (Fig 4A, bottom). This improvement in relative performance as uncertainty increases is observed both in the mean relative performance of all E/I tones, and in the relative performance of the best E/I tone (Fig 4B). When examining the best performing E/I tones, we found that the excitatory tone remains relatively constant, while the inhibitory tone increases (Fig 4C). Crucially, this shift in E/I tone behaviorally manifested in a shift of observable indifference curvature, where concavity increases along with uncertainty (Fig 4D).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Hierarchical network in regime III outperforms linear network as uncertainty increases.

HN: Hierarchical Network; LN: Linear Network; J₊/purple: HN intermediate layer excitatory weight; J₋/turquoise: HN intermediate layer inhibitory weight; : environmental uncertainty; white outline: regime I. (A) Performance of HN relative to LN improves across offers as uncertainty increases. Networks were presented with 640 unique offers. The proportion of offers for which networks chose the larger combined alternative was measured. Heatmaps show the proportion of offers where the HN is more likely than the LN to chose the larger alternative. HN E/I tone are given on the X-axis and Y-axis respectively. (Top) Relative performance with no environmental uncertainty. (Bottom) Relative performance at high environmental uncertainty. (B) The mean of HN E/I tones and the best HN E/I tone both improve in performance relative to the LN as uncertainty increases. The X-axis indicates the amount of environmental uncertainty, while the Y-axis indicates the proportion of offers the HN displays better performance than the LN. The ‘x’ markers indicate the mean of all E/I tones in (A), while the dots indicate relative performance of the best E/I tone. (C) Best-performing inhibitory tone increases with uncertainty. Excitatory and inhibitory weights are given for the best performing HN intermediate layer E/I tone at each uncertainty level. A sigmoid function was fit to the points. (D) The indifference curvature of HN high-performing E/I tones becomes increasingly concave. The curvature is given as a function of environmental uncertainty for the best-performing E/I tone. (E) Performance of regime III degrades less with uncertainty relative to regimes I and II or the linear network. Values and linear fits are given for the LN, and as well as weight configurations representing the three HN network regimes (regime I: J₊ = 0.34, J₋ = −0.01; regime II: J₊ = 0.36, J₋ = −0.0; regime III: J₊ = 0.32, J₋ = −0.1).

https://doi.org/10.1371/journal.pcbi.1008791.g004

To better understand this transition in the indifference curvature, we examined the overall proportion of trials where the larger offer was chosen as a function of uncertainty for the Linear Network and several E/I tones from Hierarchical Network regimes (Fig 4E). Though the regime III E/I tones under-perform at low uncertainty, they display less degradation as uncertainty increases, eventually crossing that of the Linear Network and regime I. Thus, the Hierarchical Network displays distinct decision regimes that maximize choice performance under a variety of environmental uncertainty levels. Furthermore, the shift in regimes with uncertainty is primarily achieved through increasing inhibitory tone, and is behaviorally differentiable.

Hierarchical network regime III selectively increases offer separability and attends to the salience of attribute differences

Having established that different decision-making regimes maximize reward at different uncertainty levels, as well as how the network produces these regimes, the natural question arises as to why this occurs.

Given the results of the fixed point analysis (as shown in the right column of Fig 3B), regime III appears to be performing a winner-take-all function, with the larger of the attribute input values transmitted and the smaller quashed. This resembles max operations, where the final area operates on the maximum within-attribute values of the intermediate transform areas. To gain intuition as to how the Hierarchical Network could out-perform the Linear Network, we compared a simplified linear combination to one involving attribute-level max operations.

For the linear operation, the final decision value was simply a linear sum of attribute distributions. The max operation, however, first took a maximum of the attribute distributions while at the attribute level. An example offer is given in Fig 5A, and the set of equations in Fig 5B.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Hierarchical network regime III winner-take-all operation increases offer separability.

(A) Example offer where Regime III improves in relative performance as uncertainty increases. (B) Linear and max operations used to calculate decision value distributions for the offer shown in (A). In a linear operation, the mean and variance of both attribute are used to calculate distribution from which the final offer value is drawn. The max operation first selects the larger within-attribute alternative, which then becomes the decision value distribution. (C) Decision value distributions produced by the max operation maintain greater separability as uncertainty increases. Distributions are shown for the decision values calculated in (B) using (top row), (middle row) and (bottom row). In the left column are results for the linear operation, and in the right from the max operation. (D) For those offers where regime III performance (J₊ = 0.35, J₋ = −0.1) exceeds that of the LN, a max operation provides greater separability between decision value distributions. For offers that transitioned to better HN performance, at each uncertainty level we calculated the d′ of decision value distributions produced by the linear operation and the max operation. Shown are the mean and standard error of the d′s.

https://doi.org/10.1371/journal.pcbi.1008791.g005

We then examined the set of offers where, as uncertainty increased, the Hierarchical Network began to out-perform the Linear Network (indicated in Fig 4B). When the transition in performance occurred, the Linear Operation decision value distributions began to overlap, while the Max Operation decision value distributions remained separable (Fig 5C). We quantified this by calculating the mean and standard error of the d′ values for the linear and max operation distributions at each uncertainty level (Fig 5D). The d′ value quantifies the separability of two distributions. As anticipated, for those offers where Hierarchical Network performance overtook that of the Linear Network, the max operation produced significantly higher separability between the offer distributions.

We then asked how the distribution of offer attribute values affected the relative performance of the decision regimes. Indeed, if one simply used the max operation as a heuristic on a pure combinatorial sampling of offers, one would correctly select the largest choice alternative on 88.9% of trials. We therefore focused on offers that could maximally differentiate the three Hierarchical Network regimes. Offers used to assess indifference curves can be divided into four quadrants (Fig 6A, inset). In quadrants I and III, all three regimes are predicted to choose the same alternative. Quadrants II and IV, however the three regimes predict differentiable choice behavior. For example, a pure max operation would choose the smaller alternative for 50% of offers. We thus sampled 91 offers from quadrants II and IV (see Materials and methods). Surprisingly, in the Hierarchical Network intermediate layer, the best-performing E/I tone configurations again became increasingly concave as uncertainty increased (Fig 6A). Furthermore, the E/I tone displayed a similar trend of relatively stable excitation, with a monotonic increase in inhibition (Fig 6B). If regime III were performing a pure max operation, this would not be observed.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Hierarchical network regime III preferentially attends to attributes with larger cross-alternative differences.

Lime green: choice behavior matching that of a max operation; khaki: choice behavior choosing higher combined value; gray: region prior to threshold crossing. (Inset A) Offers were selected from quadrants of the indifference curve offers plane where the three regimes display differentiable choice behavior. Quadrants II and IV, from which offers were selected, is given in white. Example indifference curves from the three regimes are shown. (A) As in Fig 4E, the best performing indifference curvature becomes more concave as environmental uncertainty increases. (B) As in Fig 4C, the best performing E/I tone as a function of environmental uncertainty shows increasing inhibitory tone. (C) A regime III E/I tone performs better than expected of a pure sequential-max network. (D) An example offer where a pure max operation would select the smaller combined value, but the regime III area selects the larger combined value. (E) Phase portraits from the values in (D), showing that the regime III areas still perform a winner-take-all function. (F) On trials where the regime III area out-performed a pure max operation, the area with greater attribute value differences has population firing rates that more quickly diverged (leaving the gray area of (E)). Shown is a histogram of firing rate divergence time differences between attribute areas for trials when the network performed as predicted by a max operation, and a histogram of trials where it performed superior to it.

https://doi.org/10.1371/journal.pcbi.1008791.g006

To interrogate why the network was performing better than expected, we focused on the subset of the 640 offers where a pure max operation would choose the smaller choice alternative. We selected a single regime III E/I tone configuration (J₊ = 0.32nA, J₋ = −0.02 nA), then calculated the proportion of trials where the network performs as predicted of a max operation, as well as the proportion of trials where the network is superior to predictions. On 66.7% of trials, the network was superior to the pure max operation (Fig 6C).

We then examined a typical trial where the Hierarchical Network performed better than expected (Fig 6D). On this trial, the combined value of alternative A attributes was greater than the combined value of alternative B attributes (I_A,1 + I_A,2 > I_B,1 + I_B,2), but I_A,1 > I_B,1 and I_A,2 < I_B,2. Thus, in a pure sequential-max operation, the final choice would be between I_A,1 and I_B,2. As I_A,1 < I_B,2, a pure sequential-max operation would chose alternative B, which is the smaller combined alternative. However, the regime III networks chose alternative A. This could be achieved if, rather than just attending to the largest attribute value, the network considered the magnitude of the difference within-attributes. For example, if the network attended to the fact a ≫ b, it could arrive at choice alternative A, which is the higher valued offer.

To achieve this more sophisticated operation, the network may simply on select offers perform an input-transformation different from winner-take-all. To test this possibility, we examined the phase portraits produced with these inputs by intermediate layer transform areas (Fig 6E). Rather than selectively shifting to a different operation, both areas consistently performed a winner-take-all function.

We then asked if the dynamics of the network were accomplishing a more sophisticated nonlinear computation. To do this, we set a firing-rate threshold difference of 12 Hz (gray in Fig 6E) within each attribute-specific intermediate layer transform area (see Materials and methods). We presented the network with offers where a sequential-max operation would choose the smaller choice alternative (Fig 6C). For these offers, we measured in each area the time at which the threshold was crossed. Finally, we compared the time of threshold crossing between attribute-specific areas. This produced a “threshold crossing difference,” for each set of choice alternatives. The difference in threshold crossing was consistently larger in those cases where the network performed better than expected (Fig 6F). Mechanistically, the earlier the threshold is crossed in an intermediate area, the more it biases the competition in the final area [62]. This indicated that the network codes the magnitude of attribute difference, and does so using time dynamics.

Thus, advantageous performance in regime III was achieved through a combination of increasing offer separability, and selective attention to the salience of attributes with a larger cross-alternative difference.

Readout from intermediate layer transform areas is accurate, and under uncertainty improves relative to the final area

The use of time dynamics by the intermediate layer transform areas to perform sophisticated nonlinear computations suggests more can be learned by examining the readout from lower areas in the hierarchy. Experimentally, decision signals can be read out from numerous brain areas. This has prompted a larger conversation as to where decisions actually take place, how brain areas contribute to decision-making, and the relevance of experimental findings [9, 10, 10, 19–21, 63].

To read out from intermediate layer transform areas, we extended our previous threshold analysis, such that the choice was determined by the first population to exceed the threshold difference within an area. An example trial is given in Fig 7A. Importantly, we assumed that the network decision was still being determined by the final area, meaning that our readout from intermediate layer transform areas was similar to that done during an electrophysiology or imaging experiment. Because of this, if no intermediate layer transform area crossed the threshold prior to the final area crossing its threshold, we classified that trial as a failed readout.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Intermediate layer areas provide accurate readout.

TA: intermediate layer transform areas; FA: final area; light red: firing rate for population T_A,1; light blue: T_B,1; dark red: T_A,2; dark blue: T_B,2. (A) A decision is read out when the firing rate difference within an area exceeds a threshold of 12 Hz. Inputs and E/I tone as in Fig 6D and 6E. Vertical dotted lines indicate The onset of the offer and the time at which the firing-rate difference crosses the threshold. (B) Decisions are consistently read out from regime III. Heatmap shows the proportion of trials for different HN E/I tones where the intermediate layer TAs crosses the firing rate threshold before the FA reaches a decision. (C) Regime III TA readout agrees with FA. Proportion of trials (at zero environmental uncertainty) where the readout from the intermediate layer TA agrees with that of the FA. (D) TA readout is more likely than FA to select higher valued option as uncertainty increases. The relative performance at choosing the larger offer by the TA and FA at low (left) and high (right) environmental uncertainty. Green indicates superior performance by the TA, while purple indicates superior performance by the FA. (E) Summary plot of performance differences between the TA and FA as a function of environmental uncertainty for the mean of all TA E/I tones, as well as the best performing TA E/I tone.

https://doi.org/10.1371/journal.pcbi.1008791.g007

We first examined the proportion of trials for which a successful transform-area readout was obtained, finding that regime III areas with high E/I tone were successful on nearly every trial (Fig 7B). This was due to high recurrent excitation driving an increase in firing rate, while high inhibition drove a resolution of the competition. We further found that at low uncertainty, readout from many E/I tones in regime III agreed on nearly every trial with that of the final area (Fig 7C). This has a crucial experimental interpretation: even with the decision determined by a later area, it is possible to readout that decision with near 100% accuracy from an earlier area; furthermore, the early area readout can be obtained prior to that recorded from the final area. Thus, the mere presence of a decision signal and its relative timing cannot definitively be used to indicate the role of a brain area in decision-making.

Experimentally, neurons are occasionally identified that are more selective than the actual subject for stimulus value or motion coherence. Explanations have been proposed that such neurons provide specialized signals, or are an analytical artifact [64, 65]. The relatively simple feed-forward, mean-field architecture of the Hierarchical Network provides another avenue of investigation for this phenomenon. To do so, we compared the accuracy of readout from an intermediate layer transform area with readout from the final area. This was done by first computing the proportion of trials in each layer for which the larger choice alternative was selected; then, we subtracted P(Larger Chosen) by the final area from P(Larger Chosen) by the transform area. We found that in the absence of uncertainty, readout from the final area is always better than that of the intermediate layer, though the high-performing E/I tone configurations from Fig 7B and 7C comes close to matching the final area (Fig 7D, left). However, at high uncertainty, readout from the intermediate layer transform areas exceeds that of the final area (Fig 7D, right). The difference is summarized in Fig 7E for the mean of all E/I tones, and the best E/I tone. The improvement in relative performance was largely driven by regime I and II E/I tone configurations (as seen in the heatmap of Fig 7D, right). These regions have less inhibition to slow ramping activity. Thus, attribute pairs with larger differences between offers more quickly crossed the FR difference threshold; In effect, these regions provided an alternative mechanism for the filtering of less-informative attribute information in situations of high uncertainty. From this result arises two important experimental implications: first, it is possible to obtain superior readout from lower-hierarchy areas, even in a feed-forward network without specialized functions; second, this readout can occur in regimes where attribute value information is linearly transmitted.

Choice patterns of model with restricted inhibitory tone differs from control as environment uncertainty increases

Having shown that the Hierarchical Network can adopt multiple decision-making regimes, that the regimes maximize performance in different environments, and how E/I tone defines regime-specific attribute transforms, we next hypothesized a framework where E/I tone can be modulated according to the degree of environmental uncertainty. We then applied this framework to investigate altered decision-making in conditions with limits on inhibitory tone.

The theory of E/I imbalance is one of the leading biophysical frameworks for understanding psychiatric disorders such as ASD and SCZ. Yet given the vast number of manners in which E/I imbalance can arise, it is important to identify meaningful mechanisms on which specific etiologies converge [24]. Though the biological details in each condition likely differ (reduced PV interneuron expression in ASD, and reduced excitation of inhibitory cells in SCZ), and they appear to involve unique sets of brain areas (targeted areas in ASD, and widespread alterations in SCZ), both conditions show evidence of a shared mechanism involved reduced inhibitory tone [25, 28, 29]. While reduced inhibitory tone has been shown to impact visual processing in both ASD and SCZ, far less work has been done on its cognitive implications [25–27]. This is all the more important as the symptomology of both disorders is less characterized by basic sensory deficits than altered cognition.

In addition to restricted inhibitory tone in SCZ and ASZ, there is evidence for a general intolerance of uncertainty [66–69], and altered performance with environmental uncertainty [70–73]. Furthermore, differences have been found in multi-attribute decision tasks [74–76].

Therefore, to investigate novel connections between altered cognition and abnormal E/I tone, we tested variants of the Hierarchical Network on a multi-attribute decision task with uniform environmental uncertainty as to the true value of attributes. Sessions were composed of blocks of trials, during which the uncertainty level of offers was stable. At the beginning of each block, the network was given the level of environmental uncertainty. It then chose between offers composed of two attributes (assumed to be of equal subjective value), with the goal of accumulating the greatest total reward.

The E/I tone of a brain area is not necessarily fixed. Indeed, there are several mechanisms by which the tone can be temporarily modulated [77–79]. The cholinergic diffuse modulatory system, in particular, provides a plausible mechanism for the source of E/I tone shift due to expected environmental uncertainty [80–87].

To allow the network to adapt to uncertainty levels, we added a module that used a continuous function to identify the optimal intermediate layer E/I tone for a given environmental uncertainty level, then could shift weights in the attribute-specific intermediate layers to that tone (Fig 8A). The control model was able to explore the full range of possible weights. The model with restricted local inhibition was implemented by limiting the possible inhibitory tone in intermediate attribute-specific areas to a maximum magnitude of −0.01 nA.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Task performance under uncertainty differs between neurotypical and restricted local inhibition models.

(A) A module was added (forest green) that modulates the intermediate layer E/I tone according to the uncertainty level. The E/I tone were determined using continuous functions, similar to those shown in Figs 4C and 6B. In the restricted inhibitory tone model, inhibition was limited to −0.01. (B) A single session of the simulation, where the X-axis is trial number, and Y-axis is the proportion of trials where the models choose the larger option. (C) Performance between the models diverges as uncertainty increases. The proportion of the trials where the larger alternative was chose is given as a function of uncertainty for both models. (D) The optimal indifference curvature differs between models as uncertainty increases. To the right are example indifference curves for neurotypical (top) and restricted local inhibition (bottom) models at their maximum level of concavity. Input values () are normalized by the reference choice alternative values’ linear sum.

https://doi.org/10.1371/journal.pcbi.1008791.g008

Sessions were composed of trial blocks with variable levels of uncertainty. Observing performance in a single session, we could already see a separation of performance levels during high-uncertainty blocks (Fig 8B). Analyzing P(Larger Chosen) as a function of uncertainty, we found the restricted local inhibition model did not differ from the neurotypical model when the uncertainty level was low, but began to diverge as uncertainty increased (Fig 8C). We also examined the change in the indifference curve as a function of uncertainty, finding a ceiling to the concavity of the restricted local inhibition model’s decision behavior (Fig 8D). Thus, the models make two strong experimental predictions: 1) performance will be identical at low uncertainty, while subjects with restricted local inhibition will show a greater fall-off in total reward as uncertainty increases; and 2) at high uncertainty, subjects with restricted local inhibition will continue to show more linear-weighting of attribute values than neurotypicals, who shift to a nonlinear strategy of weighting the attribute by their difference.