Intermedial overlaps between word and images
My paper deals with some aspects of the relationship between words and images. The concepts of 'word' and 'image' are not synonymous with 'verbal' and 'visual communication', although they are often restricted to these modalities of sign use. Words and images are cross-medially related, and there are many overlaps between them.
Words are communicated both via the acoustic and the visual channels. The acoustic channel is the one of spoken words, but words are also communicated visually in writing. Even in acoustic (oral) communication, words mostly occur in a visual setting, for example, in the context of gestures, facial and eye communication, and in a situational setting of objects to which the words often refer indexically.
The concept of 'image', on the other hand, does not necessarily exclude words as they occur in oral or written communication. In spoken language, we use 'verbal images', which are, of course, not visual, but mental images. Even in writing, we come across visual images, for example in pictography, where pictures are used to represent words visually, or in picture poems, where a written text takes the shape of a picture, or in iconic interpretaion of letters, as in words such as U-turn or T-shirt. In contrast to such broader implications of the concepts of 'word' and 'image', we will have to restrict ourselves in this paper to words in the form of writing and to images in the form of pictures.
There are not many media who are restricted to the use of written words or pictures only. The radio, telephone, telegram, letter, book, or e-mail are media in which communication may or must take place by means of words only. Paintings, drawings or photography are media typically restricted to pictures. However, in most of the typically verbal media we find the presence of pictorial elements, and in most of the pictorial media we find the presence of verbal elements. Words are combined with pictures to a multimedial message. On the radio, words go together with music or other acoustic signs. In newspapers or newsmagazines they combine with pictures, and in the movies, television, or the hypermedia, words combine with pictures, music, and visusally represented nonverbal communication. Moreover, words are also able to represent pictures in a verbal description. In short, communication by words and by pictures overlap in a plurality of intermedial contexts.
more