VISTA is a Computer Graphics and Computer Vision group developing novel high level representations of visual and virtual contents for either fully automatic analysis and/or interactive real-time modification of 2D videos and 3D animated content. On the one hand, we aim at proposing fully automatic understanding of existing visual material - including multiple modalities vastly available in videos-, extract spatio-temporal information, descriptive narrations from them and infer semantic concepts. On the second hand, we wish to place efficient new visual representations for 2D and 3D virtual models at the center of the interactive process with humans. These representations should be seen as a responsive, animation ready medium on which information is automatically extracted and analysed, while humans can freely interact, add elements and refine them in space and time according to their wish.