From Computational Cognitive Neuroscience Wiki
Jump to: navigation, search
Project Name dyslex
Filename File:dyslex.proj Open Project in emergent
Author Randall C. O'Reilly
Publication OReillyMunakataFrankEtAl12
First Published Oct 26 2016
Tags Reading, Orthography, Semantics, Phonology, Acquired Dyslexia, Deep Dyslexia, Surface Dyslexia
Description This model simulates normal and disordered (dyslexic) reading performance in terms of a distributed representation of word-level knowledge across Orthography, Semantics, and Phonology.
Updated 26 October 2016, 14 January 2017, 16 January 2018
Versions 8.0.0, 8.0.2, 8.0.3, 8.0.5
Emergent Versions 8.0.1, 8.0.4, 8.5.1
Other Files File:dyslex trained.wts.gz

Back to CCNBook/Sims/All or Language Chapter.


This model simulates normal and disordered (dyslexic) reading performance in terms of a distributed representation of word-level knowledge across Orthography, Semantics, and Phonology. It is based on a model by Plaut and Shallice (1993). Note that this form of dyslexia is aquired (via brain lesions such as stroke) and not the more prevalent developmental variety.

It is recommended that you click here to undock this document from the main project window. Use the Window menu to find this window if you lose it, and you can always return to this document by browsing to this document from the docs section in the left browser panel of the project's main window.

Because the network takes some time to train (for 250 epochs), we will just load in a pre-trained network to begin with. Be sure to download the weights from the panel at the right.

Do Load Weights in the overall ControlPanel.

Normal Reading Performance

For our initial exploration, we will just observe the behavior of the network as it "reads" the words presented to the orthographic input layer. Note that the letters in the input are ordered left-to-right, bottom to top.

Press Step TrlTst in the control panel.

You will see the activation flow through the network, and it should settle into the correct pronunciation and semantics for the first word, "tart" (the bakery food). In the network data at the bottom of the network view, you can see the output_name showing what the network produced over the phonology output layer.

Click on the TrialTestOutputData tab to see a record of the network's performance on this word.

The group column shows whether this item is concrete (Con) or abstract (Abs), the trial_name column indicates the input pattern, and closest_name column shows the word that is closest to what the network produced, and the columns after that indicate what type of error the network makes, if any. All of these should be 0, as the network is intact and well trained. Concrete words have more distinctive features, whereas abstract words have fewer.

Switch back to the DyslexiaNet view and continue to Step TrlTst through the words, paying particular attention to the timing of when the Phonological layer gets active relative to the Semantic one (you can do Step CycTst to see this more clearly).

Question 9.1: Do you think the initial phonological activation is caused by the "direct" input via orthography or the "indirect" input via semantics? Did you see any cases where the initial phonological pattern is subsequently altered when the later input arrives? List the words where this happened.

Reading with Complete Pathway Lesions

Figure ds.1: Cluster plot of semantic similarity for words in the simple triangle model of reading and dyslexia. Words that are semantically close (e.g., within the same terminal cluster) are sometimes confused for each other in simulated deep dyslexia.

We next explore the network's ability to read with one of the two pathways to phonology removed from action. This relatively simple manipulation provides some insight into the network's behavior, and can be mapped onto two of the three dyslexias. Specifically, when we remove the semantic pathway, leaving an intact direct pathway, we reproduce the characteristics of surface dyslexia, where words can be read but access to semantic representations is impaired and visual errors are made. When we remove the direct pathway, reading must go through the semantic pathway, and we reproduce the effects of deep dyslexia by finding both semantic and visual errors. Note that phonological dyslexia is a milder form of deep dyslexia, which we explore when we perform incremental amounts of partial damage instead of lesioning entire pathways.

We begin by lesioning the semantic pathway.

Set lesion_type on the control panel to SEMANTICS_FULL (Apply), and do Test: Init and Step TrlTst.

This does not actually remove any units or other network structure; it just flips a "lesion" flag that (reversibly) deactivates an entire layer (its color frame changes from emerald to grey to indicate this inactive status). Note that by removing an entire pathway, we make the network rely on the remaining intact pathway. This means that the errors one would expect are those associated with the properties of the intact pathway, not the lesioned one. For example, lesioning the direct pathway makes the network rely on semantics, allowing for the possibility of semantic errors to the extent that the semantic pathway doesn't quite get things right without the assistance of the missing direct pathway. Completely lesioning the semantic pathway itself does not lead to semantically related errors -- there is no semantic information left for such errors to be based on!

Do Run to test the network on all items, and look at the results in the TrialTestOutputData view.

Question 9.2: How many times did the network with only the direct pathway make a reading mistake overall (you can count the number of 1's in the various error columns, or look at the AllEpochTestOutputData view which has counted them for you)? Notice that the network does not produce any blend outputs, indicating that the phonological output closely matched a known word.

For each of the errors, compare the word the network produced (closest_name) with the input word (trial_name). If the produced word is very similar orthographically (and phonologically) to the input word, this is called a visual error, because the error is based on the visual properties instead of the semantic properties of the word. The simulation automatically scores errors as visual if the input orthography and the response orthography (determined from the response phonology) overlap by two or more letters. You should see this reflected in the vis column in the grid view.

Question 9.3: How many of the semantically lesioned network's errors were visual, broken down by concrete and abstract, and overall?

Now, let's try the direct pathway lesion and retest the network

Set lesion_type to DIRECT_FULL, Apply, and do a Run.

Question 9.4: What was the total number of errors this time, and how many of these errors were visual, how many were semantic, and how many were "other" for the concrete and abstract categories (as reported in AllEpochTestOutputData)?.

The simulation does automatic coding of semantic errors, but they are somewhat more difficult to code because of the variable amount of activity in each pattern. We use the criterion that if the input and response semantic representations overlap by .4 or more as measured by the cosine or normalized inner product between the patterns, then errors are scored as semantic. The cosine goes from 0 for totally non-overlapping patterns to 1 for completely overlapping ones. The value of .4 does a good job of including just the nearest neighbors in the cluster plot of semantic relationships ( Figure ds.1Figure ds.1). Nevertheless, because of the limited semantics, the automatically coded semantic errors do not always agree with our intuitions, which you might see if you check out the individual semantic errors in TrialTestOutputData.

To summarize the results so far, we have seen that a lesion to the semantic pathway results in purely visual errors, while a lesion to the direct pathway results in a combination of visual and semantic errors. To a first order of approximation, this pattern is observed in surface and deep dyslexia, respectively. As simulated in the PMSP model, people with surface dyslexia are actually more likely to make errors on low-frequency irregular words, but we cannot examine this aspect of performance because frequency and regularity are not manipulated in our simple corpus of words. Thus, the critical difference for our model is that surface dyslexia does not involve semantic errors, while the deep dyslexia does. Visual errors are made in both cases.

Reading with Partial Pathway Lesions

We next explore the effects of more realistic types of lesions that involve partial, random damage to the units in the various pathways, where we systematically vary the percentage of units damaged. There are six different lesion types, corresponding to damaging different layers in the semantic and direct pathways. For each type of lesion, one can specify the percent of units removed from the layer in question with the lesion_pct value.

The first two lesion types damage the semantic pathway hidden layers (OS_Hid and SP_Hid), to simulate the effects of surface dyslexia. The next type damages the direct pathway (OP_Hid), to simulate the effects of phonological dyslexia, and at high levels, deep dyslexia. The next two lesion types damage the semantic pathway hidden layers again (OS_Hid and SP_Hid) but with a simultaneous complete lesion of the direct pathway, which corresponds to the model of deep dyslexia explored by Plaut & Shallice (1993). Finally, the last lesion type damages the direct pathway hidden layer again (OP_Hid) but with a simultaneous complete lesion of the semantic pathway, which should produce something like an extreme form of surface dyslexia. This last condition is included more for completeness than for any particular neuropsychological motivation.

Semantic Pathway Lesions

Set lesion_type to OS_HID and lesion_pct to .1 and do Run -- then play with different lesion pct values.

You should observe that the network makes almost exclusively visual errors (like the network with a full semantic pathway lesion). Results with 25 samples per lesion level are shown in Figure dd.2 in Dyslexia Details -- corresponding results from this model can be found in the BatchTestOutputData tab.

Click on BatchTestOutputData -- it should be showing results from OS_HID lesions as indicated by the labels in the graph. This graph is configured to only show 10 data points at a time (view rows = 10 in the corresponding control panel), so to see the other data you have to use the scroll bar at the bottom of the graph -- drag that to the right to explore the full set of data, then return to the OS_HID case.

Your results should also show this general pattern of purely visual errors (or perhaps some "other" errors at high lesion levels), which is generally consistent with surface dyslexia, as expected. It is somewhat counterintuitive that semantic errors are not made when lesioning the semantic pathway, but remember that the intact direct pathway provides orthographic input directly to the phonological pathway. This input generally constrains the phonological output to be something related to the orthographic input, and it prevents any visually unrelated semantic errors from creeping in. In other words, any tendency toward semantic errors due to damage to the semantic pathway is preempted by the direct orthographic input. We will see that when this direct input is removed, semantic errors are indeed made.

Do some SP_HID lesions at various lesion_pct levels, and then look at the BatchTestOutputData results for SP_HID lesions.

You should observe lots of visual errors, but interestingly, the network also makes some semantic errors in this case. This is due to being much closer to the phonological output, such that the damage can have a more direct effect where incorrect semantic information influences the output.

Next, skip ahead to the OP_HID_DIRECT_FULL and SP_HID_DIRECT_FULL cases, both for your lesion tests and in the batch output data.

Results across 25 repetitions can be found in Dyslexia Details for these same semantic pathway lesions in conjunction with a complete lesion of the direct pathway. This corresponds to the type of lesion studied by Plaut & Shallice (1993) in their model of deep dyslexia. For all levels of semantic pathway lesion, we now see semantic errors, together with visual errors and a relatively large number of "other" (uncategorizable) errors. This pattern of errors is generally consistent with that of deep dyslexia, where all of these kinds of errors are observed. Comparing the effects of these lesions relative to the previous case, we see that the direct pathway was playing an important role in generating correct responses, particularly in overcoming the semantic confusions that the semantic pathway would have otherwise made.

Question 9.5: Compare the first bar in each graph of Figure dd.3 in Dyslexia Details (corresponding to the case with only a direct pathway lesion, and no damage to the semantic pathway) with the subsequent bars, which include increasing amounts of semantic pathway damage: Does additional semantic pathway damage appear to be necessary to produce the semantic error symptoms of deep dyslexia?

Figure dd.3 in Dyslexia Details also shows the relative number of semantic errors for the concrete versus abstract words. One characteristic of deep dyslexia is that patients make more semantic errors on abstract words relative to concrete words.

Question 9.6: Is there evidence in Figure dd.3 in Dyslexia Details that the model also makes a differential number of errors for concrete vs. abstract words?

Direct Pathway Lesions

Figure dd.1 in Dyslexia Details shows the effects of direct pathway lesions, both with and without an intact semantic pathway. Let's focus first on the case with the intact semantic pathway (the Full Sem graphs in the figure).

Set lesion_type to OP_HID and try some different lesion_pct lesions, and scroll to this data in the batch test graph view.

Notice that for smaller levels of damage more of the errors are visual than semantic. This pattern corresponds well with phonological dyslexia, especially assuming that this damage to the direct pathway interferes with the pronunciation of nonwords, which can presumably only be read via this direct orthography to phonology pathway. Unfortunately, we can't test this aspect of the model because the small number of training words provides an insufficient sampling of the regularities that underlie successful nonword generalization, but the large-scale model of the direct pathway described in the next section produces nonword pronunciation deficits with even relatively small amounts of damage.

Interestingly, as the level of damage increases, the model makes increasingly more semantic errors, such that the profile of performance at high levels of damage provides a good fit to deep dyslexia, which is characterized by the presence of semantic and visual errors, plus the inability to pronounce nonwords. The semantic errors result from the learning-based division of labor effect as described previously (Section 10.3.2 in the text). Furthermore, we see another aspect of deep dyslexia in this data, namely a greater proportion of semantic errors in the abstract words than in the concrete ones (especially when you add together semantic and visual + semantic errors).

Set lesion_type to OP_HID_SEMANTIC_FULL, and try some lesions for this case (and view the batch graph data).

This case of partial direct pathway damage with a completely lesioned semantic pathway produces mostly visual and "other" errors.

You may now close the project (use the window manager close button on the project window or File/Close Project menu item) and then open a new one, or just quit emergent entirely by doing Quit emergent menu option or clicking the close button on the root window.