Testing neurocognitive predictions of the hub-and-spoke model of semantic memory with network representational similarity analysis

Neuropsychology, transcranial magnetic stimulation, and functional neuroimaging with PET, fMRI, MEG, and electrocorticography (ECoG) provide convergent evidence that the anterior temporal lobes (ATL) serve a critical role in semantic cognition. The hub and spoke theory of semantic cognition casts the ATL as a hub, responsible for supporting generalization, categorization, and inference that is unrestricted by perceptual modality (e.g., visual or aural experience) or conceptual domain (e.g., living or non-living things). While quite a lot is know about the range of semantic content and processes that the ATL helps to subserve, less is known about how it contributes to representing semantic knowledge. In a series of experiments supported by fMRI and ECoG, I tested explicit hypotheses about how patterns of functional activity over the ATL and the rest of cortex relate to semantic similarity among concepts. Semantic similarity structure is expressed by patterns of activity in the ATL for concepts cued by simple images (e.g., a drawing of a frog) or characteristic sounds (e.g., a recording of a frog croaking), consistent with it being a cross-modal semantic hub. Semantic structure is also expressed by parts of cortex associated with modality-specific processing, but in visual areas when drawings are shown and audio areas when sounds are played. The ECoG dataset revealed that sub-category structure among both living and non-living things are expressed in the ventral ATL, consistent with its contribution across domains. Finally, while semantic representations are expressed both within and beyond the ATL, structure present in the ATL appeared to support a full semantic space, rather than binding together concepts expressed elsewhere in the brain or coding for the interactions among modalities. Many of the analyses were conducted by applying network representational similarity analysis (network RSA), a novel whole-brain technique that discovers patterns of functional units which jointly express a target similarity structure. This permitted the first test of whether a single similarity structure is encoded over multiple regions of cortex working together, and enabled us to predict where concepts exist in semantic space based on estimates of neural activity.