Introduction:

I am currently involved in the Cinecast project supported by the French government. The goal of this project is to develop softwares and apps for film analysis and audiovisual content annotation.

From a technological point of view, the project is managed by NETIA which is a subsidiary of GlobeCast, one of the largest French operator for telecommunication. Major French movie libraries are associated to the project (INA, BPI, La Cinemathèque française, Le Forum des Images or the BNF). Important social media dedicated to cinema, like Allociné, Vodkaster, leSite.tv or Universciné are also associated to Cinecast. At the academic level, the project is finally supervised by the LIRIS, a leading computer lab based in Lyon, the IRI (a research and innovation center based in Paris) and my institution (Telecom ParisTech), which is specialized in the analysis of usages of new technologies.

Let me now explain my focus: I will discuss the audiovisual annotation based on 2 analysis:

• the analysis of online annotation services dedicated to cinema & videos
• the analysis of professional annotation softwares.

First, I’ll draw up a typology and, in a second part, I will focus my attention on the social value of the annotation. Here, I would like to discuss the concept of “curatorial economy”.

Text:

What is annotation? Annotation means 2 things:

• From a sociological point of view, this is an activity that consists in adding text or graphic signs on a pre-existing content, whether or not digital
• From a semiological point of view, it is also the result of this activity

Annotation is an old practice. We can think about:

• scribes, who did not hesitate to annotate the books they were working on by including glosses or commentaries
• today, there are annotations in second-hand printed books (underlining, brackets or paraphrases)
• there are also annotations in digital documents as in Adobe documents or Word documents

But let’s focus on audiovisual annotation. Where do we find it?

• First, on websites like Youtube or Vodkaster which offer spaces dedicated to a fun-oriented annotation
• There are also softwares such as Lignes de Temps developed by the Centre Pompidou’s research lab “the IRI” which offer you to annotate content in a more analytic or educational way

To begin, there are 2 forms of textual-audiovisual annotation:

1 – The first and the most widespread type of annotation is the textual one, put aside the audiovisual content. Here, the screen economy is based on the distinction between the audiovisual content, the annotations and the annotative tools. This screen economy is closely linked to the hand-written annotation described previously. The informational and recreational website Youtube or Vodkaster as such the pedagogical software Lignes de Temps, also previously introduced, are based on this model.

2 – There is a second way of making textual-audiovisual annotations: the writing on the audiovisual content. YouTube or the Japanese video sharing website Nico Nico Douga allow to produce textual annotation directly on the flow. Here, the annotation refers to a comic strip speech bubble. It can also take the form of a text embedded on the movie stream. Sometimes, we can find graphic annotations pointing an item or a movie detail.

 3 – We have a third hybrid and complex way to engaged an annotative activity with the Polemic Tweet app. With Polemic Tweet, experienced during congresses or conferences, you can text your approval or your disapproval to the speaker, introducing a reference or asking a question to him, using your Twitter account. The visual result of this annotative activity is an annotation that is recorded on the video stream broadcasted live on the Internet. In the end, you can see all the annotations put on a timeline, showing levels of activity (bar height) and types of activity (color codes).

 Comments – that I made aside about the graphical annotation – lead me to talk about this specific type of annotation which is the graphical one. Indeed, if the audiovisual annotation is largely based on a whole literate culture, there is more widely a whole graphic culture that is reflected on the screen when we turn our attention to the annotation.

4 – There are basic forms of evaluative activity such as pushing a button. Here, we think about the Facebook Like button… that allows us to show that we love a comment, an image or a video.

5 – There are also some graphical annotations such as banding or score (or pointing).The program Faux Raccord proposed by Allociné website is based on this model. We can hear a voiceover commentary associated with the film clips which reveals [editing] errors in the movie by inserting circles, arrows or crosses.

 6 – Closely link to the textual annotation, because it is like a quotation, we can find a sixth type of audiovisual content. This specific type of annotation is based on the cutting of movie content and the mediation of sequences. In the case of Vodkaster, this gesture is central, the retrieval of a movie sequence playing as a comment on the film itself (because the scene is emblematic, because it is funny or because it is an important moment of the story).

Finally, I would like to mention 2 types of creative annotation.

 7 – First, but in fact, seventh, we can encountered a graphical annotative activity having a creative value. The members of the electronic band The Subs make a use of annotation tools provided by Youtube this way. The result, The Famous Videocast, is a short program in which the musicians present video clips. At the end of each introduction, a graphical form modeling a hyperlink appears on the screen. Clicking on it, we will switch on the URL of the video that was presented.

8 – And heighth, I wish to deal with the mashup, this type of audiovisual creation which “proliferates” today on the Internet, and which we have – also – to consider as a creative form of annotation, each part of the mashup can be caught sometimes as an annotation, sometimes as an annotated content (a movie sequence commenting another movie sequence).

Here, a relevant example of mashup that creates a dialogue between, on one hand, a movie scene from Star Wars and, on the other hand, a clip from Ace Ventura with Jim Carrey: Princess Leia vs Ace Ventura

I mentioned similarities between the textual annotation and the audiovisual annotation, but there is also a difference between them. With the textual one, we are in front of a text (the annotation) that said something of another text (the annotated content). With the audiovisual annotation, we say something about a film with a text or a graphical sign.

Moreover, the audiovisual annotation is often less a comment than a film description or a film reformulation. So, this type of annotation can be defined as an intermedial one because the same thing is repeated twice: once with the film and once with the text.

Here an example referring to a Youtube video in which we can see a young Chinese girl trying to stop a fight in a bus. And the annotation says: “This Younger Chinese lady who stopped the fight. Total hero!”:

Various scenarios that I’ve been described show us that softwares and apps but, also, movies themselves (as in the case of mashup) drive the annotators to acting in a certain way. Sometimes annotators are forced to write short texts, such as on the website Vodkaster which follows the 140 characters’ Twitter rule. Youtube implicitly suggests taking the same position. On the opposite, other sofwares and apps suggest to the annotator to perform a complex annotation work as in Lignes de Temps (cutting movies, pasting, writing, coloring sequences, tagging…). Moreover, these websites, softwares and apps lead the annotators to annotate next to the movies or into the movies. Annotators don’t have the choice. Thus, one can say that digital tools give to the audiovisual content a strong connotative value because they require users to bring some kind of emotions, of values, according to certain rules.

But whatever form the annotation takes, it has always a documentary value. It informs us about the film which it is associated to. Why? Because an annotator always establishes a close relationship between technical rules set by sofwares, websites and apps and the freedom that it is offered to him to express himself (by writing, cutting, pasting, coloring…), a bit like a curator in a museum who selects both works of art, exhibits them and documents them with texts (labels and catalogues). This is why I chose to talk about “curatorial economy” to conceptualize the annotative activity.

In addition, the analysis of the audiovisual annotation shows us, that the film content is no longer alone on the screen. We are in a process of getting away from a past computer life post-web 2.0 in which the staging of the screen-space depended entirely on a webmaster or a software designer. Now people can watch, on the screen, a movie and a sum of annotations, including their own. This is why I’m talking about “curatorial economy”. By cutting a content, introducing a comment or graphic elements, an annotator reconfigures the communicative scope of videos and movies. He or she assigns them a value and participates to their exhibitions.

I suggest to illustrate this idea with an example extracted from Vodkaster website. I want to tell you about a webpage dedicated to a scene from Evan Glodell’s movie Bellflower (2011). Extracting and entitling the clip ” Experience with the radiator of the car,” the annotator has immediately revealed something about the movie and, moreover, he or she has exposed his or her point of view. Comment and tags associated with the clip confirmed this hypothesis.

But other annotations could have been written. The annotator could have entitled the scene, for instance: “drinking and driving” or “a couple in the sunset” as the scene deals with a story of tinkering a mini-bar within a car, but it also deals with two young people flirting in the sunset. These two last titles could have participated in other forms of mediation of the movie scene!

Conclusion:

We are now entering the era of a screen economy in which content produced by users cohabit with content available on the platforms. This leads to the emergence of what I call a “curatorial economy”. This curatorial economy manages the development of digital softwares and Internet today. One can already seen it on Internet plateforms suchs as Youtube, Vevo, Dailymotion, Nico Nico Douga and whatever… But, this principle is about to appear on our televisions with the development of connecting-TV (or IPTV).

This paper was presented in Avanca, Portugal, on 26th of July 2012: “AVANCA 2012 International Conference Cinema: Art, Technology, Communication”