Day 16: Recording a movie from libGDX game

I thought about posting a YouTube video of the gameplay. I used recordmydesktop program to record videos in the past, but the problem is that libGDX and rmd do not sync so I get a lot of artifacts on the screen, sprites are cut in half, etc. I searched and found some useful articles about it. Basically, one would record each frame to a PNG file and then create a video out of it. Of course, this requires a lot of disk space, but that's not an issue. I found this page really helpful:

http://www.wendytech.de/2012/07/opengl-screen-capture-in-real-time/

However, their code has some weak points. For some reason, when I overlay background with sprites with semi-transparent pixels, the resulting PNG file has semi-transparent pixels in that area. Producing a video from it yields some really ugly artifacts. I tried different settings, even changed my render code, but the problem remained. Now, a simple post-processing step using ImageMagick (adding black background) fixes the problem, so I figured if I'm going to post-process it anyway, I might do the vertical flip in ImageMagick as well. So I turned off the y-flip in the code. This makes it more efficient: there is no need to allocate w*h*4 bytes of memory on each frame. On 800x480 screen, that's about 1.5MB allocate on each frame!

Also, the code that deals with framerate (skipping frames) is really sub-optimal. It skips the file numbers, which is not a big deal, but it also creates a new ScreenShot object each frame, which is completely not needed. For example, if you are recording at 30fps and game runs at 60fps, you're creating new object 50% of the time.

Finally, the code seems to forget to dispose of pixmap, so if you run it for a really long time, if would eat all your RAM.

So, I yanked out the whole FPS code out of ScreenShot class, and leave that to the class that sets up countinous screenshots instead. I also noted that some of the variables are initialized but never used. My ScreenShot class is much simpler and straightforward:

public class ScreenShot implements Runnable
{
    private static int fileCounter = 0;
    private Pixmap pixmap;

    @Override
    public void run()
    {
        saveScreenshot();
    }

    public void prepare()
    {
        getScreenshot(0, 0, Gdx.graphics.getWidth(), Gdx.graphics.getHeight(), false);
    }

    public void saveScreenshot()
    {
        FileHandle file = new FileHandle("/tmp/shot_"+ String.format("%06d",  fileCounter++) + ".png");
        PixmapIO.writePNG(file, pixmap);
        pixmap.dispose();
    }

    public void getScreenshot(int x, int y, int w, int h, boolean flipY)
    {
        Gdx.gl.glPixelStorei(GL10.GL_PACK_ALIGNMENT, 1);
        pixmap = new Pixmap(w, h, Pixmap.Format.RGBA8888);
        Gdx.gl.glReadPixels(x, y, w, h, GL10.GL_RGBA, GL10.GL_UNSIGNED_BYTE, pixmap.getPixels());
    }
}

Yep, that's all. In render loop I use this at and of each render:

ScreenShot worker = new ScreenShot();
worker.prepare();			// grab screenshot
executor.execute(worker);	// delayed save in other thread

For completeness, executor added to my Screen subclass:

private ExecutorService executor;
...
executor = Executors.newFixedThreadPool(25);

Now, on my Core2Duo it has a hard time keeping up frame rate. On one hand, it's good, because gameplay slows down and I'm able to play a nice, partly scripted video. On the other hand, it would be nicer to just record and later cut out what you need. So I added a hotkey for screenshots. It only records shots while S is pressed on the keyboard. When you record a few interesting seconds, just release S and let the PNG writer catch up. When it does, and CPU load goes back to normal, you can record again.

Videos created like this are super easy to edit. Just remove the unneeded PNG files and compress video without them. It's also easy to sync with music, because you can add/remove frames at will.

Turning screenshots into YouTube video

Since Android screen I'm using by default is 480x800 pixels, the closest fit is YouTube's 1280x720 format. We need to scale the image to 432x720 to maintain aspect ratio. This leaves us with a lot of unused area. You can put your logo there, or even show 2 videos running side-by-side ;) I decided to put the video inside another rectangle, that resembles a handheld device, so it's even smaller, it's 372x620.

Anyway, I created a 1280x720 static image with my logo and now I'm blending the gameplay into it, and also flipping it vertically. On Linux, I'm using a command like this:

for i in shot*png; do echo $i; convert $i -flip -filter Lanczos -resize 372x620 temp1.png; composite temp1.png back.png -geometry +126+56 $i; done

Once all the images are ready, we can run mencoder to produce the video. YouTube recommends using H.264 format and bitrate up to 5000 for 720p videos. They also recommend two B-frames. Here's my command:

mencoder mf://shot*.png -mf w=1080:h=720:fps=25:type=png -ovc x264 -audiofile music.mp3 -oac copy -o movie.avi -x264encopts bitrate=5000:bframes=2:subq=6:frameref=3:pass=1:nr=2000

This yields a solid quality YT video of your gameplay. You can see the result at the start of this post. As for the audio, I just slapped over the game's soundtrack. I'm not capturing the audio of the actual game.

UPDATE Jun 2016: Guys from Lifemakers Studio tried my method and made some improvements to solve the transparency problem and make it run faster. Please read their analysis

read more... Tweet to @bigosaur Bigosaur, 2013-10-15