logo-vista

VISTA

Visual Worlds: Temporal Analysis, Animation and Authoring
VISTA - Research team in Computer Graphics & Vision on Animated Visual and Virtual Worlds.
We are part of the laboratory LIX, at Ecole Polytechnique/CNRS, Institut Polytechnique de Paris.
Keywords - Computer Graphics, Computer Vision with Deep Learning, AI for Visual Computing, Animated Content, Shape and Motion, Interactive Creation, Visual Simulation.
Application - Movies, Video Games, Animation Cinema, Natural Science, Medical Imaging, Archeology, Art & Sciences, Design, Fashion, CAD.
Specialized Keywords
- Graphics: Sketch-based Modeling, Virtual Sculpting, Character Animation, Natural Phenomenon, Real-Time, Implicit Surface, Hybrid and Procedural Models.
- Vision: Human-centric Video understanding, Cinematography analysis, Visual Transformers, NeRF, Interior Scenes.
- Learning: Multi-Modal Learning, Generative Models, Gans, Diffusion Models, Reinforcement Learning, Lightweight Learning.

Research Axes

Our team develop new methods for the analysis and creation of Visual and Virtual Worlds with a specific focus on Animated Content. Our methods spans fully-automatic understanding of videos, up to the interactive creation of populated 3D virtual worlds. To this ends we are proposing methods improving the (i) Analysis of visual content, (ii) Shape and Motion representation, and (iii) the Creation of Visual Worlds.

We first propose fully Automatic AI-based Analysis of 2D Videos and 3D Animated Content that leverage Deep-Learning technics with a specific focus on time and multimodal input data. We are specifically developing methods for automatic human recognition, pose estimation, and behavior understanding. We also propose lightweight learning based on statistical approches to extract spatial relation between shapes from a single input.

Second, we develop Interactive Models to efficiently represent Shape and Motion. We are specialized in integrating spatio-temporal constraint into real-time reactive virtual models for game-like application using,either, explicit procedural models, or discovering them via Reinforcement-Learning. We also propose alternative, volume-based, representation for shapes modeling relying on implicit surface. These models are suited for complex shape synthesis or advanced interactive behaviors (precise collision, deformation). We finally develop layered and coupled models of different spatial/temporal nature adapted to simulate efficiently large and multi-scale natural scenes.

Third our models and analysis are aimed at the Creation and Authoring of Visual and Virtual Worlds. To this ends, we propose Expressive Creation Methodology, relying on Sketching or Sculpting Gestures, as well as Sound and Multimodal Systems. These steps are supported by the scene analysis allowing to provide suggestion system, up to helping the narrative design of the scene. We further propose transfert medodologies between geometry, animation, and style in complement to generative models in order to create lively and populated worlds with sufficient variety, or to explore the impact of parameters into a simulated world.

  • 1. Analysis and Understanding of Visual Content
    Deep CNN, Human-centric video learning
    Automatic & multimodal understanding
    Light learning, spatial representation
  • 2. Interactive Models for Shape and Motion
    Alternative representation (Field based, Implicit surfaces, ...)
    Spatio-temporal constraints
    Visual simulation, Layered models
    Behavioral simulation, Reinforcement learning
  • 3. Creating and Authoring Visual Worlds.
    Expressive creation: Sketching or Sculpting gestures, Sound, Multimodal system
    A-priori/learned knowledge constraints
    Narrative design, suggestion system
    Generation and style transfert, Visual transformers


Team Expertise

We are researchers with mixed expertises and backgrounds in Computer Graphics and Computer Vision. We jointly develop AI-based approaches and efficient representation to improve 2D video analysis and 3D animated virtual world generation.

The specific aspect of our team-based methodology is propose a global Visual Computing approach coupling Automatic Vision and Interactive Graphics methodologies. This allows to tackle complex open scientific problems mixing the analysis of 2D and the synthesis of 3D content. For instance, we develop generative-based approaches ranging from automatic-learning fom data (GAN, diffusion, etc), reinforcement-learning, as well as alternative lightweight and efficient model relying on a-priori knowledge and user-centric design.

At the LIX level, our speciality relies on
- Video Analysis and Understanding
- Human Representation and Virtual Character Animation
- Interactive Creation
- Interactive Simulation of Multi-Scale Natural Scenes.
More events

VISTA Recent Events

event-thumbnail
2023/07/03
Defense: Efficient visual simulation of volcanic phenomena
Maud Lastic will defend her PhD work on Efficient visual simulation of volcanic phenomena on Monday, 3rd July 2pm in Amphi Sophie Germain.
event-thumbnail
2023/06/11
Award: ACM SIGGRAPH Coons Award to Marie-Paule Cani
Congratulations to Marie-Paule Cani who is the recipient of the prestigious ACM SIGGRAPH Steven Anson Coons Award 2023 for her outstanding contributions to computer graphics in shape modeling, computer animation, and content creation tools.
event-thumbnail
2023/06/12
Project: Hi! Paris project Plant Growth
Tim Scheller obtained a Hi! Paris scholarship to pursue his PhD on Capturing 4D plant growth. This PhD will strengthen our projects on Natural Environment modeling and simulation.
event-thumbnail
2023/06/05
Newcomer: Ridouane Ghermi
We are welcoming Ridouane Ghermi, who starts his PhD with Vicky Kalogeiton on the topic of AI-based requests on movies.
event-thumbnail
2023/05/10
Conference: Eurographics 2023
Jiayi Wei will present her work on the use of Line Process for Robust Pointset Denoising at Eurographics.

VISTA Seminars

Application domain

We have being developing our recent contributions in the following typical domains:
- Human recognition on Videos and 3D virtual Character Animation
- Cloth and Garment analysis and synthesis
- Natural environment simulation (terrain, volcano, flaura and fauna)
- Medical Imaging Analysis and biological shape design
Our research is highly application-driven where we aim at providing scientific support to enhance creativity with application in entertainement (movies and games), design as well as art in general. Our development of interactive visual representation can also find application for general public experience to help understanding time-related phenomenon (ex. terrain evolution, impact of climate change), or for expert public via serious games. Finally, we further provide dedicated analysis, interactive models and visualization for other scientific disciplines such as medical imaging, biology, or archeology where our models can help analysis, or serve as virtual test bench.
- Improve Creative and Entertainment Industries
video games/animation, Movies, VFX, creative arts, design
- Interactive Representation and Experience for the general Public or Experts
Museography, Archeology, Serious games
- Efficient Virtual Test Bench for Natural Sciences
Medical, Biology, Climatology, Natural Environment


We have ongoing (or recent) research collaboration with the following companies.
team-application

Research environment

We are located on the campus of Institut Polytechnique de Paris in the Alan Turing building [Contact].
We are collocated and working in close collaboration with the GeomeriX team at LIX regarding Geometry Analysis and Processing.
At the LIX level, we are part of the Modeling, Simulation and Learning Pole.
At the IP Paris, we are part of GeoVISTA - regrouping the Graphics and Vision teams on the Plateau de Saclay.

Links

publications
Publications
software
Software & Code
job
Job Offers
projects
Funded Projects