如何将图像从流中解码为现有图像

I have an application which plays audio with NAudio. A known limitation with NAudio is that every time the Garbage Collector runs, every thread is paused until it is done.

我有一个与NAudio播放音频的应用程序。 NAudio的一个已知限制是,每次垃圾收集器运行时,每个线程都会暂停,直到完成为止。

The application runs fine, all GC is done within acceptable time and there is no stuttering.

应用程序运行正常,所有GC都在可接受的时间内完成,没有口吃。

But we also have an separate application that sends a thumbnail to the main application (with the audio player) over TCP every second. The Thumbnail is about 1300 bytes when encoded as JPEG.

但是我们还有一个单独的应用程序,它每秒通过TCP向主应用程序(带有音频播放器)发送缩略图。当编码为JPEG时,缩略图大约为1300字节。

This is the code we are currently using to decode the image:

这是我们目前用于解码图像的代码:

MemoryStream ms = new MemoryStream(data);
BitmapDecoder bdec = BitmapDecoder.Create(ms, BitmapCreateOptions.None, BitmapCacheOption.Default);
BitmapSource source = bdec.Frames[0];
imgPreview.Source = source;

And to encode:

并编码:

JpegBitmapEncoder jpgEncoder = new JpegBitmapEncoder();
jpgEncoder.QualityLevel = quality;
jpgEncoder.Frames.Add(BitmapFrame.Create(renderTarget));

byte[] imageArray;
using (MemoryStream outputStream = new MemoryStream())
{
    jpgEncoder.Save(outputStream);
    imageArray = outputStream.ToArray();
}

Where RenderTarget is a RenderTargetBitmap that have the image content.

其中RenderTarget是具有图像内容的RenderTargetBitmap。

Right now we are creating and throwing away a MemoryStream, a BitmapDecoder, and a BitmapSource every second. I have commented out lines from the code and it looks like the MemoryStream and the BitmapDecoder constructor does not create any stutter, but once it's accessed through Frames[0], it starts to stutter.

现在我们每秒都在创建并丢弃一个MemoryStream,一个BitmapDecoder和一个BitmapSource。我已经注释掉了代码中的行,看起来MemoryStream和BitmapDecoder构造函数没有创建任何断言,但是一旦通过Frames [0]访问它,它就会开始断断续续。

We have also tried this approach instead of the BitmapDecoder, but with same results:

我们也尝试过这种方法而不是BitmapDecoder,但结果相同:

img.BeginInit();
img.StreamSource = ms;
img.EndInit();

Surely there is a better way to continuously update an image?

当然有更好的方法来不断更新图像?

The best way would be to just send over the raw image data, and create a WriteableBitmap, which are just rewritten every second. But the raw image is 170 kb, over 100 times more than the encoded image, and we really don't want to do that. Is it possible to decode a JPEG stream into an existing byte array, or existing image?

最好的方法是只发送原始图像数据,然后创建一个WriteableBitmap,它们每秒都会重写一次。但是原始图像是170 kb,比编码图像多100多倍,我们真的不想这样做。是否可以将JPEG流解码为现有字节数组或现有图像?

2 个解决方案

#1

Ok, so I have found the solution to my problem.

好的,所以我找到了解决问题的方法。

Instead of using the WPF BitmapDecoder to decode the images, I'm using the Windows Forms Bitmap. It is disposable, and much nicer to the Garbage Collector.

我没有使用WPF BitmapDecoder来解码图像,而是使用Windows窗体位图。它是一次性的,对垃圾收集器来说更好。

So the solution is as follows:

所以解决方案如下:

var stream = new MemoryStream(data);
var formsBitmap = new Bitmap(stream);

var width = formsBitmap.Width;
var height = formsBitmap.Height;
if (bitmap == null || height != bitmap.PixelHeight || width != bitmap.PixelWidth)
{
    bitmap = new WriteableBitmap(width, height, 96, 96, PixelFormats.Pbgra32, null);
    imgPreview.Source = bitmap;
}

BitmapData data = formsBitmap.LockBits(new Rectangle(0, 0, formsBitmap.Width, formsBitmap.Height),
                                        ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb);

try
{
    bitmap.WritePixels(new Int32Rect(0, 0, width, height), data.Scan0, data.Stride * data.Height, data.Stride);
}
finally
{
    formsBitmap.UnlockBits(data);
}

formsBitmap.Dispose();

Instead of decoding every JPEG frame I receive, I create a new Bitmap (from WinForms) from the data. Then I simply copy the pixels to only WriteableBitmap I use.

我没有解码我收到的每个JPEG帧,而是从数据中创建一个新的Bitmap(来自WinForms)。然后我只是将像素复制到我使用的WriteableBitmap。

Hopefully this can be of help to others as well.

希望这对其他人也有帮助。

#2

Profiling the code confirms that bdec.Frames[0]; takes a relatively large chunk of CPU time. Looking at the code in ILSpy the 'Frames' getter has an empty implementation (virtual method not overridden by the JpgBitmapDecodersub-class), so I assume there is some kind of call made to the underlying Windows API going on there(?)

分析代码确认bdec.Frames [0];需要相当大的CPU时间。看看ILSpy中的代码,'Frames'getter有一个空实现(虚方法没有被JpgBitmapDecodersub类覆盖),所以我假设有一些对底层Windows API的调用(?)

The bottom line is that decoding JPG is going to be slower than say PNG or GIF; I would try PNG encoding as it should still give a good compression ratio but with better performance.

底线是解码JPG比PNG或GIF慢;我会尝试PNG编码,因为它仍然应该提供良好的压缩比,但具有更好的性能。

#1