With the help of artificial intelligence, Samsung has designed a system that can be used to create animations based on images. For example, a person can be animated based on characteristics that are taken from photos.
The software was made by a Samsung development center in Moscow, in collaboration with the Skolkovo Institute of Science and Technology. The makers have put a scientific paper online on Arxiv, but in a video they briefly explain how the system works. Three neural networks are created that can build a model based on single images, or frames from videos, with which images can be animated. The system has been tested with images of several people, as demonstrated in the video.
In an example, the makers show how an animation can be made of a person based on just eight frames. For this purpose, one of the neural networks converts the frames used into vectors with which animations can be made. Another neural network looks at facial characteristics in the images to be analyzed, while the third network looks at the ‘realism’ and the pose of the animated person.
Incidentally, it is also possible to build an animated model with only one frame, although the makers point out that more images produce an animation that is more realistic. In addition, the software also works with paintings to a limited extent; for example, the developers managed to generate moving images from the Mona Lisa.
Traditionally, a 3D model of the image to be animated is built for creating animations. That is not necessary with this software: the Samsung AI itself extracts the relevant information from images and uses it to build animations. There is more software that builds animations based on artificial intelligence; such software is used, among other things, for making so-called deepfakes.