Art Science city:ASC2015 Valencia

The image takes the word: construction of a visual vocabulary

Pilar Rosado, Eva figueras y Ferran Reverter


En ASC’15 (ed.), 2º Congreso Internacional Arte Ciencia y Ciudad (pp. 63-71).

ISBN: 978-84-9048-456-2 (versión impresa)




Conflict between writing and image is ancient. At present, there are many evidences that a return to the image is needed. To find analogies between language and visual information is important. If it has been possible to decompose language in elements and structures, why not with images? The opportunity offered by digital image to describe lines and shapes in mathematical terms provides us the ability to decipher the problem of meaning contained in image. The objective of our research is to develop a series of computer vision programs to search for analogies in large datasets—in this case, collections of images of abstract paintings—based solely on their visual content without textual annotation. We have programmed an algorithm based on a specific model of image description used in computer vision. This approach involves placing a regular grid over the image and selecting a pixel region around each node. Dense features computed over this regular grid with overlapping patches are used to represent the images. Analysing the distances between the whole set of image descriptors we are able to group them according to their similarity and each resulting group will determines what we call “visual words”. Considering the whole collection of images, the total collection of “visual words” will define his “visual vocabulary”. This model is called Bag-of-Words representation of an image because does not contain information concerning the spatial relationships among the visual words which make it up. Given the frequency with which each visual word occurs in each image, we apply the method pLSA (Probabilistic Latent Semantic Analysis), a statistical model that classifies fully automatically, without any textual annotation, images according to their formal patterns. In this way, the researchers hope to develop a tool both for producing and analysing works of art.