fast video encoding

[gregorth]

Hi all,

for a scientific application I need to save a video stream to disc for
further post processing. My cam can deliver 8bit grayscale images with
resolution 640x480 with a framerate up to 100Hz, this is a data rate
of 30MB/s. Writing the data uncompressed to disc hits the data
transfer limits of my current system and creates huge files. Therefore
I would like to use video compression, preferably fast and high
quality to lossless encoding. Final file size is not that important.
Because of the hardware I am bound to WinXP.

I already tried pymedia for encoding to mpeg2, however I only managed
to get a framerate of about 30-40fps (on a 1.8GHz dual core). There is
still room for improvements in my code, but before trying hard I want
to ask for advices or other possibilities. I also found gstreamer with
pygst python bindings, which seems to be more modern (and performant?)
package than pymedia. I did not yet try it, especially since I didn't
find a simple code example of how to use it for my use case. Can
somebody give me a hint?

I also found huffyuv or lagarith which is provided as a directshow
codec for Windows. Can somebody give me a hint how to use a directshow
codec with python?

I am a novice with video encoding. I found that few codecs support
gray scale images. Any hints to take advantage of the fact that I only
have gray scale images?

Thanks for any help

Gregor

 

[David Bolen]

I am a novice with video encoding. I found that few codecs support
gray scale images. Any hints to take advantage of the fact that I only
have gray scale images?

I don't know that there's any good way around the fact that video
encoding is simply one of the heavier CPU-bound activities you're
likely to encounter. So I suspect that codec choice (barring letting
quality drop a bit) is going to move the bar less than picking the
beefiest CPU you can.

If I were looking to do this, I'd probably include investigating
pumping the raw camera images into an ffmpeg encoding subprocess and
let it handle all the encoding. There's about a gazillion options and
a variety of codec options to play with for performance. You could
grab and store a short video sample from the camera and use it as a
benchmark to compare encoding options.

ffmpeg does have a -pix_fmt option that can be used to indicate the
input pixel type - "gray" would be 8-bit, and result in a 4:0:0 image
with a YUV-based encoder, for example. Not sure how much, if any,
impact it would have on encoding speed though.

To be honest, with your data rate, I might even consider getting
Python out of the pure encoding path once it starts - depending on the
raw network format delivered by the camera you might be able to have
ffmpeg read directly from it. Might not be worth it, since the data
transfer is probably I/O bound, but a 640x480x1 stream at 100Hz is
still nothing to sniff at, and even managing the raw data flow in
Python might eat up some CPU that you could better allocate to the
encoding process, or require extra data copies along the way that
would be best to avoid.

-- David

 

[sturlamolden]

for a scientific application I need to save a video stream to disc for
further post processing.

I have worked a bit on this as well. There are two things that make
scientific applications different form common video encoding:

First, a scientific video stream is often very different from a
'movie': There are usually very little 'movement'. For example, when I
have filmed a mouse swimming in a water maze (a pool of milky white
water), the only thing that moves is the rat. So I could achieve
excellent compression just by saving the pixels that changed.

Second, scientific data should be stored with lossless compression if
possible.

Common video codecs are aimed at films and home video, not scientific
video streams.

I don't know how your data are, but I would be very picky about the
codec. And chances are you will be better off home-brewing your own
domain specific compression.

If you need to store large amounts of scientific data, consider using
something like a HDF5 database for storage. There are two Python
wrappers for HDF5 that I know of (h5py and PyTables).

 

[Rhodri James]

I have worked a bit on this as well. There are two things that make
scientific applications different form common video encoding:

First, a scientific video stream is often very different from a
'movie': There are usually very little 'movement'. For example, when I
have filmed a mouse swimming in a water maze (a pool of milky white
water), the only thing that moves is the rat. So I could achieve
excellent compression just by saving the pixels that changed.

These are referred to as P-frames; any encoder that gets further than
the most basic version of any video protocol will do this for you.
Unfortunately, water pixels change a lot from frame to frame, even
when the camera is static, so it doesn't gain you as much as you might
hope in cases like you mention.

Second, scientific data should be stored with lossless compression if
possible.

If data storage is no object, be my guest. Of course, you're going to
have to write your own codecs for this, since pretty much every video
protocol in existence is lossy.

In reality, lossless compression isn't necessary. As long as your
quantisation isn't completely up the spout, the added precision isn't
actually going to gain you anything unless you're working in very
specialised situations; even then, if it does make a significant
difference then you needed greater resolution anyway.

 

[greg]

Unfortunately, water pixels change a lot from frame to frame, even
when the camera is static, so it doesn't gain you as much as you might
hope in cases like you mention.

In the case mentioned, the water is of no interest, so it
could be removed altogether by a suitable custom compressor.