A new approach, based on the typology of signs
I would like to suggest a new approach to the study of word-image relationships based on the Peircean trichotomy of the iconic, the indexical, and the symbolic sign. For readers acquainted with the fundamentals of semiotics, it is unnecessary to repeat that pictures are predominantly iconic and words are predominantly symbolic signs. However, it is necessary to underline that our topic is not the extratextual (referential) relationship between verbal or pictorial signs on the one and their referential objects on the other hand, but with the intratextual relation between words and pictures in one multimedial message, i.e., with the way words relate to the pictures in juxtaposition and pictures to words. My argument is that the trichotomy of icon, index, and symbol also applies to the intratextual relationship between words and images.

Intratextual iconicity
Iconic signs are signs based on a similarity between the sign and its object. There is intratextual iconicity between words and pictures when the verbal text conveys the same message as the picture. The picture is hence an icon of the text, and the text is an icon of the picture. In fact, a redundant illustration of a text or a redundant verbal paraphrase of a picture are the clearest examples of intratextual iconicity.
With Peirce, we can furthermore distinguish between images, diagrams, and metaphors in the intratextual word-image relationship: The homology between text and picture is of the type of an image when both evoke the same mental image without any other semiotic mediation. It is of the type of a diagram when one of the two messages represents the other by means of merely structural relationships, and the relationship is metaphorical when there is a semiotic mediation via a third sign.

Intratextual indexicality
Indexical sign relationships are important in the pragmatic dimension of word-image relationships (see 2.3). There are five major kinds of intratextual indexicality, by which words and images are typically related:
(1) ostension, a mere showing, as in the message "The new Mercedes!" in the context of the picture showing what the words announce.
(2) deixis, a relationship of indicating or pointing at. There is ·verbal deixis in messages of the type "This is the new Cadillac!" referring to a picture of the new car model, ·symbolic deixis, when picture and text are connected by means of other conventional indicators, such as lines or arrows, and less frequently ·nonverbal-pictorial deixis, when the image depicts gestures or other nonverbal indices pointing at a verbal message.
(3) indexicality by contiguity: the mere spatial contiguity (juxtaposition) between word and picture serves as an index that connects the verbal with the pictorial sign. The message is simply: this verbal text refers to that picture (and not to any other picture on the same page). Traditionally, the text appears below or above the picture to which it pertains (as in a caption or legend), but in advertising any space in the vicinity of the picture is being used.
(4) indexicality by pars-pro-toto relationship: the pictorial message represents only a part of the message conveyed by the verbal message, or vice versa. For instance, the verbal message advertises "New York", but the picture only represents the Statue of Liberty. (5) exemplification: the picture gives only an example of what the verbal message refers to (and frequently vice versa). For instance, a supermarket advertises only one of its products without mentioning any of the other products for sale. Exemplification is closely related to ostension and to pars-pro-toto indexicality, and there are many other overlaps between the various subclasses of indexicality.
more