Visual Worlds: Temporal Analysis, Animation and Authoring
VISTA - Research team in Computer Graphics & Vision at LIX, at Ecole Polytechnique/CNRS, Institut Polytechnique de Paris
Objectives: Analyse & Generate Animated Visual Scenes and Interactive 3D Virtual Worlds
Applications: Entertainment, Design, Natural Sciences.

Research Axes

Our team develop new methods for the analysis and creation of Visual and Virtual Worlds with a specific focus on Animated Content. Our methods spans fully-automatic understanding of videos, up to the interactive creation of populated 3D virtual worlds. To this ends we are proposing methods improving the (i) Analysis of visual content, (ii) Shape and Motion representation, and (iii) the Creation of Visual Worlds.

We first propose fully Automatic AI-based Analysis of 2D Videos and 3D Animated Content that leverage Deep-Learning technics with a specific focus on time and multimodal input data. We are specifically developing methods for automatic human recognition, pose estimation, and behavior understanding. We also propose lightweight learning based on statistical approches to extract spatial relation between shapes from a single input.

Second, we develop Interactive Models to efficiently represent Shape and Motion. We are specialized in integrating spatio-temporal constraint into real-time reactive virtual models for game-like application using,either, explicit procedural models, or discovering them via Reinforcement-Learning. We also propose alternative, volume-based, representation for shapes modeling relying on implicit surface. These models are suited for complex shape synthesis or advanced interactive behaviors (precise collision, deformation). We finally develop layered and coupled models of different spatial/temporal nature adapted to simulate efficiently large and multi-scale natural scenes.

Third our models and analysis are aimed at the Creation and Authoring of Visual and Virtual Worlds. To this ends, we propose Expressive Creation Methodology, relying on Sketching or Sculpting Gestures, as well as Sound and Multimodal Systems. These steps are supported by the scene analysis allowing to provide suggestion system, up to helping the narrative design of the scene. We further propose transfert medodologies between geometry, animation, and style in complement to generative models in order to create lively and populated worlds with sufficient variety, or to explore the impact of parameters into a simulated world.

  • 1. Analysis and Understanding of Visual Content
    Deep CNN, Human-centric video learning
    Automatic & multimodal understanding
    Light learning, spatial representation
  • 2. Interactive Models for Shape and Motion
    Alternative representation (Field based, Implicit surfaces, ...)
    Spatio-temporal constraints
    Visual simulation, Layered models
    Behavioral simulation, Reinforcement learning
  • 3. Creating and Authoring Visual Worlds.
    Expressive creation: Sketching or Sculpting gestures, Sound, Multimodal system
    A-priori/learned knowledge constraints
    Narrative design, suggestion system
    Generation and style transfert, Visual transformers
Keywords - Computer Graphics, Computer Vision with Deep Learning, Generative AI, Animated Content, Shape and Motion, Interactive Creation, Visual Simulation, AI for Visual Computing.
Application - Movies, Video Games, Animation Cinema, Natural Science, Medical Imaging, Archeology, Art & Sciences, Design, Fashion, CAD.
Specialized Keywords
- Graphics: Sketch-based Modeling, Virtual Sculpting, Character Animation, Natural Phenomenon, Real-Time, Implicit Surface, Hybrid and Procedural Models.
- Vision: Human-centric Video understanding, Cinematography analysis, Visual Transformers, NeRF, Interior Scenes.
- Learning: Multi-Modal Learning, Generative Models, GANs, Diffusion Models, Reinforcement Learning, Lightweight Learning.

Team Expertise

The specific aspect of our team-based methodology is propose a global Visual Computing approach coupling Automatic Vision and Interactive Graphics methodologies. This allows to tackle complex open scientific problems mixing the analysis of 2D and the synthesis of 3D content. For instance, we develop generative-based approaches ranging from automatic-learning fom data (GAN, diffusion, etc), reinforcement-learning, as well as alternative lightweight and efficient model relying on a-priori knowledge and user-centric design.

We are researchers with mixed expertises and backgrounds in Computer Graphics and Computer Vision. We jointly develop AI-based approaches and efficient representation to improve 2D video analysis and 3D animated virtual world generation.

At the LIX level, our speciality relies on
- Video Analysis and Understanding
- Human Representation and Virtual Character Animation
- Interactive Creation
- Interactive Simulation of Multi-Scale Natural Scenes.
More events

VISTA Recent Events

Event: Generative AI at LIX open day
Xi Wang presents some of the team's recent result on Generative AI in Visual Computing to the 1st year Polytechniciens students visiting the LIX laboratory.
Event: David-Henri Garnier PhD
Congratulation to David-Henri Garnier who succesfully defended his PhD on Interactive 3D Modeling of Evolutionary and Emergent Bio-Inspired Shapes that was developed in collaboration with Dassault Systèmes.
Event: AI presentation at Archimedes workshop
Vicky Kalogeiton presents her work on Multimodality in Computer Vision at the Archimedes project workshop dedicated to Artificial Intelligence, Computer Vision, and Precision Medicine.
Event: Training Days in Animation and Simulation for Graphics
We are organizing a research school on 3D Animation in Computer Graphics on the 10-12th of April with the association of the GDR-IGRV.
Event: AI presentation to the French Ministry of Defense
Vicky Kalogeiton presents our research in AI during the visit of the French Ministry of Defense at polytechnique.

VISTA Seminars

Application domain

We have being developing our recent contributions in the following typical domains:
- Human recognition on Videos and 3D virtual Character Animation
- Cloth and Garment analysis and synthesis
- Natural environment simulation (terrain, volcano, flaura and fauna)
- Medical Imaging Analysis and biological shape design
Our research is highly application-driven where we aim at providing scientific support to enhance creativity with application in entertainement (movies and games), design as well as art in general. Our development of interactive visual representation can also find application for general public experience to help understanding time-related phenomenon (ex. terrain evolution, impact of climate change), or for expert public via serious games. Finally, we further provide dedicated analysis, interactive models and visualization for other scientific disciplines such as medical imaging, biology, or archeology where our models can help analysis, or serve as virtual test bench.
- Improve Creative and Entertainment Industries
video games/animation, Movies, VFX, creative arts, design
- Interactive Representation and Experience for the general Public or Experts
Museography, Archeology, Serious games
- Efficient Virtual Test Bench for Natural Sciences
Medical, Biology, Climatology, Natural Environment

We have ongoing (or recent) research collaboration with the following companies.

Research environment

We are located on the campus of Institut Polytechnique de Paris in the Alan Turing building [Contact].
We are collocated and working in close collaboration with the GeomeriX team at LIX regarding Geometry Analysis and Processing.
At the LIX level, we are part of the Modeling, Simulation and Learning Pole.
At the IP Paris, we are part of GeoVISTA - regrouping the Graphics and Vision teams on the Plateau de Saclay.


Software & Code
Job Offers
Funded Projects