Visual Worlds: Temporal Analysis, Animation and Authoring
VISTA - Research team in Computer Graphics & Vision on Animated Visual and Virtual Worlds.
We are part of the laboratory LIX, at Ecole Polytechnique/CNRS, Institut Polytechnique de Paris.
Keywords - Computer Graphics, Computer Vision with Deep Learning, AI for Visual Computing, Animated Content, Shape and Motion, Interactive Creation, Visual Simulation.
Application - Movies, Video Games, Animation Cinema, Natural Science, Medical Imaging, Archeology, Art & Sciences, Design, Fashion, CAD.
Specialized Keywords
- Graphics: Sketch-based Modeling, Virtual Sculpting, Character Animation, Natural Phenomenon, Real-Time, Implicit Surface, Hybrid and Procedural Models.
- Vision: Human-centric Video understanding, Cinematography analysis, Visual Transformers, NeRF, Interior Scenes.
- Learning: Multi-Modal Learning, Generative Models, Gans, Diffusion Models, Reinforcement Learning, Lightweight Learning.

Research Axes

Our team develop new methods for the analysis and creation of Visual and Virtual Worlds with a specific focus on Animated Content. Our methods spans fully-automatic understanding of videos, up to the interactive creation of populated 3D virtual worlds. To this ends we are proposing methods improving the (i) Analysis of visual content, (ii) Shape and Motion representation, and (iii) the Creation of Visual Worlds.

We first propose fully Automatic AI-based Analysis of 2D Videos and 3D Animated Content that leverage Deep-Learning technics with a specific focus on time and multimodal input data. We are specifically developing methods for automatic human recognition, pose estimation, and behavior understanding. We also propose lightweight learning based on statistical approches to extract spatial relation between shapes from a single input.

Second, we develop Interactive Models to efficiently represent Shape and Motion. We are specialized in integrating spatio-temporal constraint into real-time reactive virtual models for game-like application using,either, explicit procedural models, or discovering them via Reinforcement-Learning. We also propose alternative, volume-based, representation for shapes modeling relying on implicit surface. These models are suited for complex shape synthesis or advanced interactive behaviors (precise collision, deformation). We finally develop layered and coupled models of different spatial/temporal nature adapted to simulate efficiently large and multi-scale natural scenes.

Third our models and analysis are aimed at the Creation and Authoring of Visual and Virtual Worlds. To this ends, we propose Expressive Creation Methodology, relying on Sketching or Sculpting Gestures, as well as Sound and Multimodal Systems. These steps are supported by the scene analysis allowing to provide suggestion system, up to helping the narrative design of the scene. We further propose transfert medodologies between geometry, animation, and style in complement to generative models in order to create lively and populated worlds with sufficient variety, or to explore the impact of parameters into a simulated world.

  • 1. Analysis and Understanding of Visual Content
    Deep CNN, Human-centric video learning
    Automatic & multimodal understanding
    Light learning, spatial representation
  • 2. Interactive Models for Shape and Motion
    Alternative representation (Field based, Implicit surfaces, ...)
    Spatio-temporal constraints
    Visual simulation, Layered models
    Behavioral simulation, Reinforcement learning
  • 3. Creating and Authoring Visual Worlds.
    Expressive creation: Sketching or Sculpting gestures, Sound, Multimodal system
    A-priori/learned knowledge constraints
    Narrative design, suggestion system
    Generation and style transfert, Visual transformers

Team Expertise

We are researchers with mixed expertises and backgrounds in Computer Graphics and Computer Vision. We jointly develop AI-based approaches and efficient representation to improve 2D video analysis and 3D animated virtual world generation.

The specific aspect of our team-based methodology is propose a global Visual Computing approach coupling Automatic Vision and Interactive Graphics methodologies. This allows to tackle complex open scientific problems mixing the analysis of 2D and the synthesis of 3D content. For instance, we develop generative-based approaches ranging from automatic-learning fom data (GAN, diffusion, etc), reinforcement-learning, as well as alternative lightweight and efficient model relying on a-priori knowledge and user-centric design.

At the LIX level, our speciality relies on
- Video Analysis and Understanding
- Human Representation and Virtual Character Animation
- Interactive Creation
- Interactive Simulation of Multi-Scale Natural Scenes.
More events

VISTA Recent Events

Conference: Eurographics 2023
Jiayi Wei will present her work on the use of Line Process for Robust Pointset Denoising at Eurographics.
Newcomer: James Gain
We are glad to receive James Gain (Cape Town University) as invited professor. James will be in the teamr 3 month to collaborate on natural phenomenon and VR application, in association with his master student James Burnes.
Conference: CLIPE ITN Workshop
Eduardo Alvarado and Ariel Kwiatkowski are participating to the CLIPE-ITN Workshop in Tubingen. Their work is presented at the FMX2023.
Newcomer: Pascal Guehl
We welcome Pascal Guehl who is joining our team as a research engineer for one year. Pascal got a PhD in interactive material synthesis from University of Strasbourg, and will work in our team on improving the Matisse Software.
Teaching with AI
We participated to the discussion of the teaching & AI organized at Polytechnique. Vicky Kalogeiton, in collaboration with Michalis Vazirgiannis animated the discussion and presented a demonstration of the use of generative AI such as Chat-GPT in teaching context.

VISTA Seminars

Application domain

We have being developing our recent contributions in the following typical domains:
- Human recognition on Videos and 3D virtual Character Animation
- Cloth and Garment analysis and synthesis
- Natural environment simulation (terrain, volcano, flaura and fauna)
- Medical Imaging Analysis and biological shape design
Our research is highly application-driven where we aim at providing scientific support to enhance creativity with application in entertainement (movies and games), design as well as art in general. Our development of interactive visual representation can also find application for general public experience to help understanding time-related phenomenon (ex. terrain evolution, impact of climate change), or for expert public via serious games. Finally, we further provide dedicated analysis, interactive models and visualization for other scientific disciplines such as medical imaging, biology, or archeology where our models can help analysis, or serve as virtual test bench.
- Improve Creative and Entertainment Industries
video games/animation, Movies, VFX, creative arts, design
- Interactive Representation and Experience for the general Public or Experts
Museography, Archeology, Serious games
- Support for Natural Sciences
Medical, Biology, Environment

We have ongoing (or recent) research collaboration with the following companies.

Research environment

We are located on the campus of Institut Polytechnique de Paris in the Alan Turing building [Contact].
We are collocated and working in close collaboration with the GeomeriX team at LIX regarding Geometry Analysis and Processing.
At the LIX level, we are part of the Modeling, Simulation and Learning Pole.
At the IP Paris, we are part of GeoVISTA - regrouping the Graphics and Vision teams on the Plateau de Saclay.


Software & Code
Job Offers
Funded Projects