logo-vista

VISTA

Visual Worlds: Temporal Analysis, Animation and Authoring
VISTA - Research team in Computer Graphics & Vision at LIX, at Ecole Polytechnique/CNRS, Institut Polytechnique de Paris
Objectives: Analyse & Generate Animated Visual Scenes and Interactive 3D Virtual Worlds
Applications: Entertainment, Design, Natural Sciences.

Research Axes

Our team develop new methods for the analysis and creation of Visual and Virtual Worlds with a specific focus on Animated Content. Our methods spans fully-automatic understanding of videos, up to the interactive creation of populated 3D virtual worlds. To this ends we are proposing methods improving the (i) Analysis of visual content, (ii) Shape and Motion representation, and (iii) the Creation of Visual Worlds.

We first propose fully Automatic AI-based Analysis of 2D Videos and 3D Animated Content that leverage Deep-Learning technics with a specific focus on time and multimodal input data. We are specifically developing methods for automatic human recognition, pose estimation, and behavior understanding. We also propose lightweight learning based on statistical approches to extract spatial relation between shapes from a single input.

Second, we develop Interactive Models to efficiently represent Shape and Motion. We are specialized in integrating spatio-temporal constraint into real-time reactive virtual models for game-like application using,either, explicit procedural models, or discovering them via Reinforcement-Learning. We also propose alternative, volume-based, representation for shapes modeling relying on implicit surface. These models are suited for complex shape synthesis or advanced interactive behaviors (precise collision, deformation). We finally develop layered and coupled models of different spatial/temporal nature adapted to simulate efficiently large and multi-scale natural scenes.

Third our models and analysis are aimed at the Creation and Authoring of Visual and Virtual Worlds. To this ends, we propose Expressive Creation Methodology, relying on Sketching or Sculpting Gestures, as well as Sound and Multimodal Systems. These steps are supported by the scene analysis allowing to provide suggestion system, up to helping the narrative design of the scene. We further propose transfert medodologies between geometry, animation, and style in complement to generative models in order to create lively and populated worlds with sufficient variety, or to explore the impact of parameters into a simulated world.

  • 1. Analysis and Understanding of Visual Content
    Deep CNN, Human-centric video learning
    Automatic & multimodal understanding
    Light learning, spatial representation
  • 2. Interactive Models for Shape and Motion
    Alternative representation (Field based, Implicit surfaces, ...)
    Spatio-temporal constraints
    Visual simulation, Layered models
    Behavioral simulation, Reinforcement learning
  • 3. Creating and Authoring Visual Worlds.
    Expressive creation: Sketching or Sculpting gestures, Sound, Multimodal system
    A-priori/learned knowledge constraints
    Narrative design, suggestion system
    Generation and style transfert, Visual transformers
Keywords - Computer Graphics, Computer Vision with Deep Learning, AI for Visual Computing, Animated Content, Shape and Motion, Interactive Creation, Visual Simulation.
Application - Movies, Video Games, Animation Cinema, Natural Science, Medical Imaging, Archeology, Art & Sciences, Design, Fashion, CAD.
Specialized Keywords
- Graphics: Sketch-based Modeling, Virtual Sculpting, Character Animation, Natural Phenomenon, Real-Time, Implicit Surface, Hybrid and Procedural Models.
- Vision: Human-centric Video understanding, Cinematography analysis, Visual Transformers, NeRF, Interior Scenes.
- Learning: Multi-Modal Learning, Generative Models, GANs, Diffusion Models, Reinforcement Learning, Lightweight Learning.


Team Expertise

The specific aspect of our team-based methodology is propose a global Visual Computing approach coupling Automatic Vision and Interactive Graphics methodologies. This allows to tackle complex open scientific problems mixing the analysis of 2D and the synthesis of 3D content. For instance, we develop generative-based approaches ranging from automatic-learning fom data (GAN, diffusion, etc), reinforcement-learning, as well as alternative lightweight and efficient model relying on a-priori knowledge and user-centric design.

We are researchers with mixed expertises and backgrounds in Computer Graphics and Computer Vision. We jointly develop AI-based approaches and efficient representation to improve 2D video analysis and 3D animated virtual world generation.

At the LIX level, our speciality relies on
- Video Analysis and Understanding
- Human Representation and Virtual Character Animation
- Interactive Creation
- Interactive Simulation of Multi-Scale Natural Scenes.
More events

VISTA Recent Events

event-thumbnail
2023/12/07
Defense: Successfull PhDs
Congratulation to Ariel Kwiatkovski, Eduardo Alvarado and Jiayi Wei for their successfull PhD defenses this week.
event-thumbnail
2023/11/17
Award: Best Poster MIG
Paul Boursin was awarded best poster of the MIG conference 2023 (Motion, Interaction and Games) that took place in Rennes from 15-17 November 2023 for his work on Enabling Physical VR Interaction with Deep RL Agents.
event-thumbnail
2023/11/10
Award: 2nd Best Paper JFIG
Théo Cheynel was awarded second best paper of the JFIG conference 2023 that took place in Montpellier from 8-10 November 2023 for his work on Sparse Motion Sematics for Contact Aware Retargeting.
event-thumbnail
2023/10/05
Conference: IMET Conference
Damien Rohmer was co-chair of the IMET conference 2023 that took place in Barcelona from 5-6 October 2023.
event-thumbnail
2023/10/04
Award: Best Poster ICCV CVAAD
We received the Best Poster award at the ICCV Workshop CVAAD for our work on Floor Plan Reconstruction from Sparse Views, authored by Arnaud Gueze, Mathieu Ospici (Homiwoo), Damien Rohmer and Marie-Paule Cani.

VISTA Seminars

Application domain

We have being developing our recent contributions in the following typical domains:
- Human recognition on Videos and 3D virtual Character Animation
- Cloth and Garment analysis and synthesis
- Natural environment simulation (terrain, volcano, flaura and fauna)
- Medical Imaging Analysis and biological shape design
Our research is highly application-driven where we aim at providing scientific support to enhance creativity with application in entertainement (movies and games), design as well as art in general. Our development of interactive visual representation can also find application for general public experience to help understanding time-related phenomenon (ex. terrain evolution, impact of climate change), or for expert public via serious games. Finally, we further provide dedicated analysis, interactive models and visualization for other scientific disciplines such as medical imaging, biology, or archeology where our models can help analysis, or serve as virtual test bench.
- Improve Creative and Entertainment Industries
video games/animation, Movies, VFX, creative arts, design
- Interactive Representation and Experience for the general Public or Experts
Museography, Archeology, Serious games
- Efficient Virtual Test Bench for Natural Sciences
Medical, Biology, Climatology, Natural Environment


We have ongoing (or recent) research collaboration with the following companies.
team-application

Research environment

We are located on the campus of Institut Polytechnique de Paris in the Alan Turing building [Contact].
We are collocated and working in close collaboration with the GeomeriX team at LIX regarding Geometry Analysis and Processing.
At the LIX level, we are part of the Modeling, Simulation and Learning Pole.
At the IP Paris, we are part of GeoVISTA - regrouping the Graphics and Vision teams on the Plateau de Saclay.

Links

publications
Publications
software
Software & Code
job
Job Offers
projects
Funded Projects