logo-vista

VISTA

Visual Worlds: Temporal Analysis, Animation and Authoring
VISTA - Research team in Computer Graphics & Vision at LIX, at Ecole Polytechnique/CNRS, Institut Polytechnique de Paris
Objectives: Analyse & Generate Animated Visual Scenes and Interactive 3D Virtual Worlds
Scientific Approach: Generative AI, Reinforcement Learning, Expressive Modeling & Authoring, Real-Time Simulation, Geometric Constraints, Field-Based Representation.
Applications: Entertainment, Design, Natural Sciences.

Research Axes

Our team develop new methods for the creation of Visual and Virtual Worlds with a specific focus on Storytelling for Animated Content. Our methods spans fully-automatic understanding of videos, up to the interactive creation of populated 3D virtual worlds. To this ends we are proposing methods improving the (i) Analysis of visual content, (ii) Shape and Motion representation, and (iii) the Creation of Visual Worlds.

We first propose fully Automatic AI-based Analysis of 2D Videos and 3D Animated Content that leverage Deep-Learning technics with a specific focus on time and multimodal input data. We are specifically developing methods for automatic human recognition, pose estimation, and behavior understanding. We also propose lightweight learning based on statistical approches to extract spatial relation between shapes from a single input.

Second, we develop Interactive Models to efficiently represent Shape and Motion. We are specialized in integrating spatio-temporal constraint into real-time reactive virtual models for game-like application using,either, explicit procedural models, or discovering them via Reinforcement-Learning. We also propose alternative, volume-based, representation for shapes modeling relying on implicit surface. These models are suited for complex shape synthesis or advanced interactive behaviors (precise collision, deformation). We finally develop layered and coupled models of different spatial/temporal nature adapted to simulate efficiently large and multi-scale natural scenes.

Third our models and analysis are aimed at the Creation and Authoring of Visual and Virtual Worlds. To this ends, we propose Expressive Creation Methodology, relying on Sketching or Sculpting Gestures, as well as Sound and Multimodal Systems. These steps are supported by the scene analysis allowing to provide suggestion system, up to helping the narrative design of the scene. We further propose transfert medodologies between geometry, animation, and style in complement to generative models in order to create lively and populated worlds with sufficient variety, or to explore the impact of parameters into a simulated world.

  • 1. Analysis and Understanding of Visual Content
    Deep CNN, Human-centric video learning
    Automatic & multimodal understanding
    Light learning, spatial representation
  • 2. Interactive Models for Shape and Motion
    Alternative representation (Field based, Implicit surfaces, ...)
    Spatio-temporal constraints
    Visual simulation, Layered models
    Behavioral simulation, Reinforcement learning
  • 3. Creating and Authoring Visual Worlds.
    Expressive creation: Sketching or Sculpting gestures, Sound, Multimodal system
    A-priori/learned knowledge constraints
    Narrative design, suggestion system
    Generation and style transfert, Visual transformers
Keywords - Computer Graphics, Computer Vision with Deep Learning, Generative AI, Animated Content, Shape and Motion, Interactive Creation, Visual Simulation, AI for Visual Computing.
Application - Movies, Video Games, Animation Cinema, Natural Science, Medical Imaging, Archeology, Art & Sciences, Design, Fashion, CAD.
Specialized Keywords
- Graphics: Sketch-based Modeling, Virtual Sculpting, Character Animation, Natural Phenomenon, Real-Time, Implicit Surface, Hybrid and Procedural Models.
- Vision: Human-centric Video understanding, Cinematography analysis, Visual Transformers, NeRF, Interior Scenes.
- Learning: Multi-Modal Learning, Generative Models, GANs, Diffusion Models, Reinforcement Learning, Lightweight Learning.


Team Expertise

The specific aspect of our team-based methodology is propose a global Visual Computing approach coupling Automatic Vision and Interactive Graphics methodologies. This allows to tackle complex open scientific problems mixing the analysis of 2D and the synthesis of 3D content. For instance, we develop generative-based approaches ranging from automatic-learning fom data (GAN, diffusion, etc), reinforcement-learning, as well as alternative lightweight and efficient model relying on a-priori knowledge and user-centric design.

We are researchers with mixed expertises and backgrounds in Computer Graphics and Computer Vision. We jointly develop AI-based approaches and efficient representation to improve 2D video analysis and 3D animated virtual world generation.

At the LIX level, our speciality relies on
- Video Analysis and Understanding
- Human Representation and Virtual Character Animation
- Interactive Creation
- Interactive Simulation of Multi-Scale Natural Scenes.
More events

VISTA Recent Events

event-thumbnail
2025/05/22
Event: vicky Kalogeiton VISTA team leader
Vicky Kalogeiton is now the new team leader of VISTA
event-thumbnail
2025/05/18
Award: Honorable Best Paper Award Eurographics
Congratulations to Théo Cheynel for receiving an Honorable Best Paper Award at Eurographics conference for the work "ReConForM: Real-time Contact-aware Motion Retargeting for more Diverse Character Morphologies" developed in collaboration with Kinetix (Thomas Rossi and Baptiste Bello-Gurlet), Damien Rohmer and Marie-Paule Cani.
event-thumbnail
2025/05/08
Event: Invited Researcher: Dimitrios Samaras
Dimitrios Samaras (Stony Brook University) will join our team on an Hi! Paris Invited Professor position from May to June 2025 to collaborate on "Human attention-guided video representations".
event-thumbnail
2025/02/06
Event: AI Summit: AI, Science and Society
Vicky Kalogeiton will be part of the Symposium on Frontiers in Generative AI in the AI Summit (AI, Science and Society) taking place at Ecole polytechnique.
event-thumbnail
2024/12/02
Event: Vicky Kalogetion HDR
Congratulation to Vicky Kalogeiton who succesfully defended her HDR on Story-level multimodal generativeAI: from understanding to generating visual data using multiple modalities that presented to the jury composed of Raoul de Charette, Dimitris Samaras, Josef Sivic, Matthieu Cord, Juergen Gall, Vincent Lepetit, Elisa Ricci, Jakob Verbeek.

VISTA Seminars

Application domain

We have being developing our recent contributions in the following typical domains:
- Human recognition on Videos and 3D virtual Character Animation
- Cloth and Garment analysis and synthesis
- Natural environment simulation (terrain, volcano, flaura and fauna)
- Medical Imaging Analysis and biological shape design
Our research is highly application-driven where we aim at providing scientific support to enhance creativity with application in entertainement (movies and games), design as well as art in general. Our development of interactive visual representation can also find application for general public experience to help understanding time-related phenomenon (ex. terrain evolution, impact of climate change), or for expert public via serious games. Finally, we further provide dedicated analysis, interactive models and visualization for other scientific disciplines such as medical imaging, biology, or archeology where our models can help analysis, or serve as virtual test bench.
- Improve Creative and Entertainment Industries
video games/animation, Movies, VFX, creative arts, design
- Interactive Representation and Experience for the general Public or Experts
Museography, Archeology, Serious games
- Efficient Virtual Test Bench for Natural Sciences
Medical, Biology, Climatology, Natural Environment


We have ongoing (or recent) research collaboration with the following companies.
team-application

Research environment

We are located on the campus of Institut Polytechnique de Paris in the Alan Turing building [Contact].
We are collocated and working in close collaboration with the GeomeriX team at LIX regarding Geometry Analysis and Processing.
At the LIX level, we are part of the Modeling, Simulation and Learning Pole.
At the IP Paris, we are part of GeoVISTA - regrouping the Graphics and Vision teams on the Plateau de Saclay.

Links

publications
Publications
software
Software & Code
job
Job Offers
projects
Funded Projects