To show how control variables and controlled coherence variables can be leveraged to structure complex probabilistic models, we rely on an illustrative example. BRAID is a series of models of visual word recognition, reading and reading acquisitionFootnote 3. The BRAID models were developed incrementally: the initial version of BRAID is purely visuo-orthographic, to simulate letter recognition, word recognition, and orthographic lexical decision [35, 36, 41, 43, 44]; BRAID-Learn includes visuo-attentional exploration of the stimulus and word learning, to simulate orthographic learning [37, 45]; BRAID-Phon includes phonological representations, to simulate reading aloud [46]; finally, BRAID-Acq combines BRAID-Learn and BRAID-Phon and includes contextual semantics and phonological familiarity evaluation, to simulate incidental word learning [47, 48]. The BRAID family of models has been shown to account for a wide variety of behavioral effects, with simulation and analyses which are beyond the scope of the current paper. Nevertheless, it is worth mentioning them and where they can be found. The initial version of BRAID was shown to account for benchmark effects in letter or word recognition, or lexical decision, such as frequency [49], neighborhood frequency [43], context and priming effects [36]. It was also confronted to effects more specifically related to the role of visual attention on word recognition and lexical decision, and shown to be able to account for length effects [35, 36], optimal viewing position effects in typical [36] and pathological readers [44], and crowding effects [36]. The BRAID-Phon model was shown to account for length and frequency effects in lexical decision, word recognition, and naming [46]. The BRAID-Learn model was shown to account for eye movement patterns during orthographic learning [37, 45]. Finally, the BRAID-Acq model was shown to account for the effect of various languages’ orthographic depth on word and pseudo-word reading [48].

In the current paper, we present a simplified version of the BRAID model, to focus on its control and controlled coherence variables, with three use cases: first, using them as information potentiometers, we define a visuo-attentional mechanism modulating the amount of sensory information that feeds perceptual representations; second, using them as information voltmeters, we define a familiarity evaluation mechanism for detecting novel words; third and finally, using them as information diodes, we define a mechanism modulating the flow of top-down lexical information in the model. For each, we provide illustrative experimental results highlighting properties of the defined mechanisms.

Overall Architecture of the BRAID Model

Figure 5 illustrates the architecture of the simplified BRAID model. Variables are either noted with uppercase letters or lowercase Greek letters, and are doubly indexed, with the lower index denoting spatial position (letter position, from 1 to N), and the upper index denoting time steps (from 0 to T). For instance, variable \(_1^t\) represents the perceived letter at the first position and time step t. We use a shorthand to denote series of variables: \(_^t\) are perceived letter at positions 1 to N at time t.

Fig. 5The alternative text for this image may have been generated using AI.

Graphical representations of a simplified version of the BRAID model, using the classical convention for representing graphical probabilistic models (augmented by the use of self-looping arrows on nodes to represent temporal self-dependencies; e.g., the self-looping arrow on node \(_1^t\) indicates a term \(P(_1^t \mid _1^)\)); Colored rectangles delineate the 4 sub-models: the sensory submodel (green), the visuo-attentional submodel (orange), the letter perceptual submodel (blue), and the lexical submodel (red). The dashed line indicates a deterministic dependency (i.e., a value is propagated, not a probability distribution)

Our graphical representation (Fig. 5) features self-looping arrows on nodes, which is a custom, non-standard way to represent temporal dependencies. For instance, the self-looping arrow on variable \(_1^t\) indicates that the model includes a probabilistic term \(P(_1^t \mid _1^)\). The model contains dynamic terms of these forms on 3 types of variables: on all variables \(_^t\), on variable \(D^t_\), and on variable \(W^t\). Therefore, the model is a dynamic Bayesian model [12] with \(N+2\) temporal models in parallel. In other words, the model contains \(N+2\) dynamic models, that accumulate, maintain, leak and exchange information over time: these are the “perceptual, memory-like components” of the model.

A second feature of the model is its structural organization around submodels (colored rectangles in Fig. 5). The letter sensory submodel (Fig. 5, green) includes variables \(_^t\) (“Letters in the Stimulus”) representing the visual stimulus, that is, letters from position 1 to N at time t. Visual processing considers acuity effects (as a function of the difference between letter position and eye position \(G^t\) (“Gaze position”), interference between adjacent letters, and visual similarity between letters. In all simulations used later on, the sensory submodel “outputs” its result into probability distributions on variables \(_^t\) (“Letters Internally represented”) , with terms of the form \(P(_n^t \mid _^t~G^t)\) , for all position n. Variables \(_^t\) and \(_^t\) have discrete domains, representing the 26 possible letters of the English alphabet. Overall, the model was calibrated so that a simulated time step (i.e., between \(t-1\) and t) corresponds to a millisecond of physical time; for this to be true, the output of the sensory submodel is a weakly informed probability distribution over letter identity. In other words, the submodel “outputs” a probability distribution \(P(_n^t \mid _^t~G^t)\) of high entropy, that is, not very different from the uniform distribution, but that nevertheless assigns its highest probability to the correct letter.

The second submodel is the letter perceptual submodel (Fig. 5, blue), which features variables \(_1^t\) (\(\) for “Letter Perceived”) to \(_N^t\) , and independent temporal models on each position, which implement perceptual, “memory-like” models. Variables \(_^t\) have the same domain as variables \(_^t\) , that is, the set of all possible letters: therefore, the letter perceptual model is used to compute probability distributions over the perceived identity of letters at each position of the stimulus. As such, during inference, their distributions can be interpreted as “recognizing letters” from the stimulus. Mathematically, they start with uniform distributions over letter identity at time 0, and, when a stimulus letter sequence is presented, they evolve, gradually increasing probability of the correct letter. When stimulation is removed, the probability distributions gradually leak information, thus decaying back to their original, uniform state.

The third submodel is the lexical submodel (Fig. 5, red), which features four noticeable ingredients. The first is a probabilistic implementation of the lexicon, using a naïve Bayes fusion model (as in the Bayesian reader model [50]): for each word w in the space of all known words \(W^t\), a probability distribution \(P(_i^t \mid [W^t=w])\) (\(L_L\) for “Letter at the Lexical level”) is a discrete probability distribution over the letter space, that assigns a very high probability (\(1-26\epsilon \)) to the correct letter (with an \(\epsilon \), small probability value assigned to each other letter). The second ingredient is a dynamic temporal model over variable \(W^t\), to implement a perceptual model over words, to be used for word recognition. As with variables \(_^t\) in the letter perceptual model, it accumulates sensory evidence about word identity when a stimulus is present, and gradually leaks information otherwise. Its initial state \(P(W^0)\) is the frequency of known words.

The third and fourth ingredients of the lexical submodel mirror are similar but concern word familiarity instead of word identity. The third component is a probabilistic model to define what is a known word: if the stimulus is a known word, a word of the lexicon should match with perceived letters in all positions (Boolean variables \(_n^t=\textit\) for all positions n); otherwise, all words of the lexicon should differ from the stimulus in at least one position (one of the \(_n^t\) is \(\textit\)). The fourth ingredient is the last “perceptual dynamic” model, on Boolean variable \(D^t_\) (D as this variable is involved in “lexical Decision”). During inference, the probability distribution over this variable can be interpreted as performing online familiarity evaluation of the stimulus, to be used in a lexical decision task [35], or for novelty detection, in the context of word learning [37, 45].

With these three submodels (sensory, letter perceptual and lexical), we recognize the three-layer architecture of classical models of visual word recognition, such as the IA model [51, 52]. Between these three layers, the BRAID model features a fourth named submodel, the visuo-attentional submodel (Fig. 5, orange), and a final layer between the letter perceptual and lexical submodels: these heavily feature coherence and controlled coherence variables, and are used to modulate the information flows in the model’s architecture. As such, they are the main focus of interest in the current paper, and we describe them below in detail.

Modeling Visual Attention

Visual attention is a cognitive mechanism by which sensory processing of some feature (e.g., shape or color) or portion of the visual scene is enhanced, to the detriment of other features or portions. Several previous models have been concerned with feature processing modulation [53,54,55,56,57], whereas BRAID only considers the spatial component of visual attention. This is inspired by classical conceptions of visuo-spatial attention, such as the spotlight model [58], its gradient variant [59], or the zoom-lens model [60]. In such models, a portion of the 2D visual scene is considered “inside the spotlight”, or “under the magnification” and thus processed efficiently, whereas the rest of the visual scene is not.

Model Definition

Since the model only deals with letter sequences as input, we restrict ourselves to a 1D mathematical model of the spatial distribution of attention. A smoothly varying 1D probability distribution to describe how attention is distributed naturally provides the mathematical form of a gradual spotlight, with continuous modulation of processing resource as a function of distance to the attentional focus.

The BRAID model’s visual attention submodel (Fig. 5, orange) features a layer of coherence variables, \(_^t\), controlled by a layer of control variables \(_^t\). Each coherence variable \(_n^t\) links variable \(_n^t\), the output of the sensory submodel, to variable \(_n^t\), the perceptual variable on the letter at position n. The coherence and control variables at position n implement an information potentiometer (see Section “Use Cases: Inference”), controlling the flow of information from the sensory to the perceptual submodel.

An additional variable \(A^t\) (“Attention”), defined over ℝ, constrains the information potentiometers over positions. A Normal probability distribution \(P(A^t \mid \mu _A^t~\sigma _A^t)\) is defined: its mean \(\mu _A^t\) is the position of attention focus (where maximum probability and thus, maximum attention is assigned), and its standard-deviation \(\sigma _A^t\) is the dispersion of attention (a small standard-deviation for a narrowly focused attention, with locally efficient processing; a large standard-deviation for a diffuse attention allocation). Therefore, in the BRAID model, the probability distribution \(P(A^t \mid \mu _A^t~\sigma _A^t)\) is a direct model of the spatial distribution of attention. This leverages the fact that both probabilities and attention are limited resources, and that allocating probability or attention somewhere can only be done to the detriment of allocating it elsewhere.

The potentiometer states are conditioned on variable \(A^t\): at each position n, \(P([_n^t=1] \mid A^t)\) is the probability value of the Normal distribution \(P(A^t \mid \mu _A^t~\sigma _A^t)\), discretized and renormalized over positions 1 to N. Therefore, the potentiometers are constrained across positions so that, altogether, they implement an information filter, spatially modulating the flow of information between the sensory and perceptual submodels.

More precisely, consider the inference to simulate letter recognition in the BRAID model. We consider, to start with, a simplified version that only features the bottom-up information flow between the sensory and perceptual submodels, modulated by visual attentionFootnote 4. We compute:

$$\begin Q__n}^t = P(_n^t} \mid _^}~G^~\mu _A^~\sigma _A^~[_^=1])~. \end$$

In questionFootnote 5\(Q__n}^t\), we consider a given position n and time step t, and we compute the probability distribution over perceptual variable \(_n^t\), given the known stimulus letter sequence \(_^\), the known gaze position \(G^\), the known parameters \(\mu _A^, \sigma _A^\) of visual attention distribution (and setting appropriately the coherence and control variables to obtain the desired information potentiometer properties, that is, coherence variables \(_^\) to 1 and control variables \(_^\) undefined). In the following, we use a shorthand \(K^t\) to refer to the set of all variables that are “known” during inference, except coherence variables: \(K^t= _^}~G^~\mu _A^~\sigma _A^\); this yields a more compact notation:

$$\begin Q_^t = P(_n^t} \mid K^t~[_^=1])~. \end$$

(11)

Computing \(Q__n}^t\) with Bayesian inference yields [36, 41]:

$$\begin \begin Q__n}^t & = & \displaystyle \sum __n^} \left[ P(_n^t \mid _n^) Q__n}^ \right] & \quad \textit \\ & & \times \displaystyle \left[ \alpha _n P(_n^t \mid _^t~G^t) + (1-\alpha _n) U \right] ~, & \quad \textit \end \end$$

(12)

with \(\alpha _n=P([_n^t=1] \mid A^t)\). This equation features the usual components of inference in temporal models, such as in Hidden Markov models [15, 16] or Dynamic Bayesian models [12]: the summation is the prediction step or dynamic update, featuring the recursive estimation \(Q__n}^\) at prior time step \(t-1\) and temporal model \(P(_n^t \mid _n^)\); the result of this summation is multiplied, in the update step, with a term representing sensory evidence. It is a compound term, with the output of the sensory letter submodel, \(P(_n^t \mid _^t~G^t)\), partially transmitted through the information potentiometer controlled by attention, \(P([_n^t=1] \mid A^t)\). This shows how the visuo-attentional distribution mathematically affects letter recognition.

We also simulate word recognition; to start with, we only consider the bottom-up transfer of information, from sensory processing, to letter perception, to word perception. We compute:

$$\begin Q_W^t = P(W^t \mid K^t~[_^=1]~[}_^=1])~, \end$$

(13)

that is, the probability distribution over words, given the known stimulus letter sequence, the known gaze position and visual attention distribution, and given that coherence variables and controlled coherence variables are set appropriately. Computing \(Q_W^t\) yields [36, 41]:

$$\begin \begin Q_W^t & = & \displaystyle \sum _} \left[ P(W^t \mid W^) Q_W^ \right] & \quad \textit \\ & & \times \displaystyle \prod _^N \langle P(_n^t \mid W^t) , Q__n}^t \rangle ~. & \quad \textit \end \end$$

(14)

This equation has the same structure as above: the summation is the update step, featuring the recursive estimation and temporal model, which is multiplied with sensory evidence. In this case however, sensory evidence is the comparison of probability distributions over letters predicted from the lexicon, \(P(_n^t \mid W^t)\) , and perceived letters, \(Q__n}^t\) . This comparison takes the form of a dot product, with the coherence variables \(}_^\) used as information voltmeters (see (7)).

Simulation Results

Figure 6 illustrates the effect of the visual attention distribution on letter and word recognition, with the 5-letter sequence “BRAID” as stimulus. Default parameters [36] position visual attention and gaze to the center of the stimulus (\(G^t=\mu _A^t=3\)), and visual attention dispersion slightly favors center letters (\(\sigma _A^t=1.75\)). In this situation (Fig. 6, top row, left plot), simulation yields rapid and correct letter recognition, with slight modulation of recognition speed as a function of letter position: after roughly 500 iterationsFootnote 6, all probability distributions over letters assign more than 0.8 probability to the correct hypothesis in letter space (Fig. 6, top row, middle plot). Word recognition also proceeds quickly and correctly: after roughly 400 iterations, the correct word (\(W^t=\text \)) reaches a probability above 0.8 (Fig. 6, top row, right plot). In this particular example, we observe an initial lexical competition against the word “BRAIN”, an alternative that is initially more probable because of its higher base frequency.

Fig. 6The alternative text for this image may have been generated using AI.

Simulation results of letter and word recognition when processing the stimulus letter sequence “BRAID”, with a centered attention and default dispersion. Left: distribution of attention (y-axis) on the stimulus letter sequence (x-axis). Middle: evolution of the probability value of letters at each position (y-axis) as a function of simulated time (x-axis). Letter position is indicated by the dash type for the curves, and letter identity by the line color of each curve. Only the most likely hypothesis is shown for each position (i.e., not all probability distributions are shown, only the letter hypothesis with highest probability). Right: evolution of the probability distribution over words (y-axis) as a function of simulated time (x-axis). Only the 10 most likely hypotheses are shown

The parameters of the visuo-attentional distribution affect letter and word recognition dynamics [36]. To study how, and tease this effect apart from lexical effects, we consider another simulation, where the stimulus is the letter sequence “AAAAA”. Figure 7 shows simulation results. The first three rows illustrate the influence of position \(\mu _A^t\) of visual distribution and eye position \(G^t\), which is either centered (first row), to the left (second row) or to the right (third row). Simulation results show that attended letters are processed faster, to the detriment of others letters; this also affects lexical hypotheses, in a straightforward manner. For instance, with focus on the left of “AAAAA”, the most likely hypotheses in the lexical space are words with initial As (e.g., “AGAIN”, “APART”, “AWARE”); in contrast, with focus on the right, likely words are “DRAMA”, “GHANA” and “PLAZA”. Note that, since the stimulus is not a word, probability values in the lexical space remain small (compare with Fig. 6).

Fig. 7The alternative text for this image may have been generated using AI.

Simulation results of letter and word recognition when processing the stimulus letter sequence “AAAAA”, with 5 different distributions of attention (Rows, from top to bottom: centered attention and default dispersion; attention on the first letter; attention on the last letter; centered and narrowly focused attention; maximally dispersed attention). Plot organization is identical to Fig. 6. In plots where attention is centered, it is actually set slightly off-center (2.97 instead of 3.0), in order to slightly separate curves that would otherwise be superposed.

Modulating visuo-attentional dispersion also affects letter recognition. With a centered and concentrated (\(\sigma _A^t=.5\)) distribution of attention (Fig. 7, fourth row), the central letter “A” receives almost all available attentional resources, and it is rapidly recognized. However, this massively impairs the recognition of other letters, especially external ones. This also impairs word recognition, with several hypotheses among the most likely having central As (“YEARS”, “PLACE”, “SMALL”), but with the overall lexical probability distribution remaining close to its initial state.

In the last simulation (Fig. 7, fifth row), attention is maximally dispersed, so that all letters receive the same amount of attentional resource. In this situation, modulation of letter recognition dynamics is minimal. Since bottom-up processing and its propagation through attention is made equal for all positions, the slight difference in processing speed can only be attributed to the small top-down, lexical propagation of information, that remains despite the stimulus not being a word (see below).

Modeling Familiarity Evaluation and Novelty Detection

So far, we have detailed how the BRAID model could recognize letters and words. To link letter and word perception mathematically, coherence variables \(}_^t\) were all assumed to be 1 during inference (see the right-hand side of (13)). This can be interpreted as the assumption that there is a word whose letters match stimulus letters. This results in a mathematical form that finds the best possible match (see the dot product of (14)). This also applies when the stimulus is not a word in the known lexicon, resulting in recognizing the known word that best matches the stimulus.

Contrasting this assumption with another computation that allows “errors in the input” provides a mechanism to assess whether the stimulus matches a known word or not. This yields a familiarity evaluation mechanism, briefly mentioned above, and defined mathematically here using coherence variables.

Model Definition

We first define a Boolean variable \(D_^t\), to represent whether the stimulus is familiar (i.e., a known word, \(D_^t=\textit\)) or not (\(D_^t=\textit\)) and associate it with “expected patterns of values for coherence variables”: if the stimulus is familiar, all \(}_^t\) variables should be 1, otherwise, at least one of them should be 0. These “expected patterns” are stored in probability distributions over binary variables \(_^t\). Mathematically:

$$\begin P([_^t=\]~|~[D_^t=d])= & \left\ 1 & \text \,\, d=True\,\, \text \,\, \forall n, c_n=1\\ 1/N & \text \,\, d=False\,\, \text \,\, \exists ! n, c_n=0 \\ 0 & \text ~. \\ \end\right. \end$$

We recall that \(\exists ! n, c_n=0\) reads “there exists a single n such that \(c_n=0\)”. In other words, \(P(_^t~|~[D_^t=\textit])\) is uniformly distributed over the N cases that contain a single error \(c_n=0\) at position n: we only consider non-word stimuli which have exactly 1 letter that differs from a real word. This reduces computation time (for a word of length N, only N cases are considered instead of \(2^N-1\)) and it has a negligible impact on model behavior, as having two errors is also recognized as at least 1 error.

To connect these expected patterns over variables \(_^t\) to coherence variables \(}_^t\), an additional layer of binary variables (\(}_^t\)) is used. These have interesting interpretations: as “connecting glue” between variables \(_^t\) and variables \(}_^t\), they are coherence variables; however, from the point of view of coherence variables \(}_^t\), they are control variables.

These are assumed to be 1 for familiarity evaluation, which is written:

$$\begin Q_D^t = P(D_^t \mid K^t~[}_^=1]~[_^=1])~. \end$$

Connecting the familiarity evaluation component to the BRAID architecture in this manner has several consequences. First, it connects the lexical familiarity component to the letter and word perception components, so that the dynamic model evaluating familiarity is activated. Second, it creates an “assumption check” as the “sensory term” of familiarity evaluation. The dot product between perceived letters and letters of known words is performed \(N+1\) times: once assuming that all \(}_^t\) are 1, and N times with one of them set to 0. When the best dot product has all \(}_^t\) to 1, this is an indication that a known word matches the stimulus in all positions, so that the probability that the stimulus is familiar increases; otherwise, to go to the stimulus from a known word, at least one error must by hypothesized, so that the probability that the stimulus is familiar decreases.

To present the mathematical expression for familiarity evaluation, we consider separately the two cases, where the input is familiar, \(Q_}^t\), and where it is not, \(Q_}^t\) . Bayesian inference yields [36, 41]:

$$\beginQ_^t & \propto \sum _^} \left[ P([D_^t=True] \mid D_^)~Q_D^ \right] \ \ \textit \\& \times \displaystyle \sum _ \left[ \begin \displaystyle P\left( W^t \bigg | \begin K^~[_^=1] \\ }_^=1]} \end \right) \\ \displaystyle \prod _^ \left\langle P(_n^t \mid W^t), Q__n}^t \right\rangle \end \right] ~. \ \ \textit \end$$

(15)

The innermost bracket, under the summation over variable \(W^t\), can be recognized as \(Q_W^t\) of (14); the reason for writing this in this manner will be apparent when considering \(Q_}^t\) (see below). During familiarity evaluation, we are not interested in any particular word, but in whether or not some word is being recognized: this is an interpretation of the inner marginalization, which serves as the sensory term. The top line, with the recurrence term \(Q_D^\) multiplied by the temporal model, is the dynamic update step.

The case \(Q_}^t\), an instance of using coherence variables set to 0, features a more complex sensory term. Bayesian inference yields [36, 41]:

$$\begin&Q_^t \\& \propto \sum _^} \left[ P([D_^t=False] \mid D_^)~Q_D^ \right] \ \ \textit \\& \times \displaystyle \frac~\sum _^N \left[ \begin \displaystyle \sum _ \left[ \begin \displaystyle P\left( W^t \bigg | \begin K^~[_^=1] \\ }_i^=0]} \\ }_^=1]} \end\right) \\ \displaystyle \prod _ n=1\\ n \ne i \end}^ \left\langle P(_n^t~|~W^t), Q__n}^t \right\rangle \\ \displaystyle \left\langle P(_i^t~|~W^t), (1-Q__i}^t) / |\mathcal _L| \right\rangle \end \right] \end \right] ~. \ \ \textit \end$$

(16)

Indeed, variables \(_^t\) now enumerate all possible error positions in the stimulus. This yields N cases to consider, which are equally probable (see ()).

The sensory term enumerates the possible position for an “error” between predicted and perceived letters (the summation over positions 1 to N), and, for each of these possible error positions (the i-th position, \([}_i^=0]\)), assume a match everywhere else (\([}_^=1]\))Footnote 7. Finally, consider the update term: it is similar to the update term for word recognition \(Q_W^t\) (see (14)), but assumes a mismatch in the i-th position. In other words, whereas familiarity evaluation involves word recognition \(Q_W^t\) in the True case \(Q_}^t\), in the False case \(Q_}^t\), N processes of “almost-word-recognition” are computed in parallel.

Overall, these equations provide a familiarity evaluation mechanism interpreted as an information voltmeter: whereas during letter and word recognition coherence variables “let probabilistic information flow”, here, the manner in which it flows is measured. If it “flows better” (in the sense of (7)) under the assumption that the input is a word, this increases lexical familiarity; otherwise, it decreases it.

Simulation Results

Figure 8 illustrates the resulting familiarity evaluation mechanism over a variety of stimuli letter sequences. First, the model is successful in these illustrative examples, correctly assigning high probability that the stimulus is a word when it is (Fig. 8, left column), and conversely (Fig. 8, right column). Second, we observe that the dynamics of convergence is sensitive to the stimulus characteristics: convergence is fast for a high-frequency word (such as “HOUSE”), slower for a lower-frequency word (“MOUSE”), slower still for a very low-frequency word (“QUIRE”). The example of a “non-word-like word” (“ULCER”, the only 5-letter English word that starts with “ULC”) shows a non-monotonous behavior, with initial hesitancy before a sharp increase of the probability that the stimulus is a word.

Fig. 8