Framerate – Herz
Any moving image format, whether digital or analog, has a certain frame rate. This defines how many frames we see per second and is almost always referred to as FPS.It doesn't work the other way around. A classic video game that runs at 24fps feels sluggish and slow. On the other hand, many people find movies that are played at a higher frame rate disturbing. A very good example is "The Hobbit", which was shown at 48fps in some cinemas. The so-called "soap opera" effect[3] was disturbing for many viewers.
Hobbiton and Middle Earth didn’t feel like a different universe, it felt like a special effect, a film set with actors in costumes. It looked like behind the scenes footage. The movement of the actors looked… strange. Almost as if the performances had been partly sped up…. It didn’t look cinematicSource: https://www.studiodaily.com/2012/04/the-hobbit-the-soap-opera-effect-and-the-48fps-and-faster-future-of-movies/
Frame Interpolation
If you now want to include slow motion in your 24fps cinema film, the respective sequence must be filmed with a higher frame rate. So, for example, for a 4x slowdown, it makes sense to film a scene at 96fps. If you slow down footage that does not have a higher frame rate, this results in unpleasant judder. Even in expensive cinema films this is, unfortunately, often to observe. For example, because the director decides to show a scene in slow motion during the editing process, which was not intended for this beforehand.
Clear stuttering at the end of the table
And this is where frame interpolation comes into play. In this process, artificial intermediate images are calculated with the help of software algorithms.
Source: https://en.wikipedia.org/wiki/Motion_interpolation
This determines which areas in the image have moved where. To do this, the algorithm may not only compare two images, but look at a large number of individual images to better determine the movement. Depending on the video source, this process can lead to very good, to terrible results. These are then visible as distracting artifacts or unnatural-looking movements.
Source: Artifacts from conventional motion interpolation
Video interpolation is supported by pretty much every video editing or compositing program nowadays, and is usually done with a few mouse clicks. Unfortunately, you often come up against the limits of classic algorithms. That's what @ninki.vienna and I had to realize during a video project. That's why we decided to take a new approach.
Flowframes – Video-Interpolation with AI (artificial intelligence)
During my thesis, when I was looking for examples of AI applications in video production, I came across FlowFrames. In fact, I first read their research paper before noticing that they also make their algorithms available in a free open source project. Here is a link to the research paper: RIFE: Real-Time Intermediate Flow Estimation for Video Frame Interpolation for Video Frame Interpolation. There you can also see that the results with their algorithm gives far better results than classical methods.
Left: Original, right with Frame Interpolation using Flowframes
Fortunately, although the underlying algorithms are highly complex, FlowFrames is very easy to use. This is already made clear by the minimalist interface:
Use the button "Browse (Video)" to select the desired video file, in the field below you can define where the newly created file should be saved. At "Set FPS / Interpolation Factor" you define how much you want to increase the frame rate. This is a great strength of Flowframes, many classic software products only allow a maximum of doubling. As "AI Model" you use the default selected model. You should only adjust this tab if you get incorrect results. In the "Output Mode" you can set the preferred video format. After that you only have to press the button "Interpolate!
And the computer starts working, and working hard. Like pretty much all classic AI applications, Flowframes needs a decent graphics card. As a minimum requirement I would recommend 4GB VRAM, but 6GB or more is optimal. If you are not sure what kind of graphics card you have, you can easily check this in Windows 10 via the Task Manager.
If you do not have an Nvidia graphics card, you will have to resort to the "RIFF - Vulkan" in the "Interpolation AI" tab. The larger the image resolution of the input video, the more enormous the required computing power. Up to FullHD (1920x1080), the interpolation is still feasible with classic gaming graphics cards without any problems, but from 4K (3840x2160) onwards, a really enormous amount of performance is required.
If you don't have the hardware but would like to try Flowframes for a clip, just write me and I can calculate it for you with my computer.
Use cases
The classic cases are, of course, people who edit video or simply publish videos on a regular basis. I don't know how many times it happened to me that I would have liked to use slow motion in a scene, but the corresponding source material prevented this. However, in my opinion, there is another obvious use for it: restoring old footage.Maybe you, your parents or grandparents still have really old pictures. For example, from family celebrations or first trips. Super8 film, which used to be very common, was usually filmed at 18fps[4]. These stuttering shots can be given new life by frame interpolation. Here is an example:
Summary
As a video editor, I find it exciting how more and more AI software applications are diving into my field. However, often these are for such specific use cases that they are of no use to people who are not in that area of expertise. I think this is a bit different with Flowframes, and that's why I didn't want to keep it from you. Especially the restoration approach is worth a look. Because you can not only increase the frame rate, but also dramatically increase the image resolution (for example with the trial version of Video Enhance AI). In combination, this can already lead to amazing shots and provide an exciting glimpse into the past:If you have any questions about the topic, just ask!
Source
[1] https://flauntdigital.com/blog/video-frame-rates/[2]https://www.nvidia.com/de-at/geforce/technologies/360-hz/
[3]https://www.studiodaily.com/2012/04/the-hobbit-the-soap-opera-effect-and-the-48fps-and-faster-future-of-movies/
[4]https://www.straight8.net/cheat-sheet
Congratulations @lichtcatchtoby! You have completed the following achievement on the Hive blockchain and have been rewarded with new badge(s) :
Your next target is to reach 80 posts.
You can view your badges on your board and compare yourself to others in the Ranking
If you no longer want to receive notifications, reply to this comment with the word
STOPCheck out the last post from @hivebuzz:
Support the HiveBuzz project. Vote for our proposal!