当试图使用scipy.io.loadmat读取.mat文件时，“类型错误:缓冲区太小，无法请求数组”。

I have such a code:

我有这样的代码:

import tempfile
import subprocess
import shlex
import os
import numpy as np
import scipy.io

script_dirname = os.path.abspath(os.path.dirname(__file__))


def get_windows(image_fnames, cmd='selective_search'):
    f, output_filename = tempfile.mkstemp(suffix='.mat')
    os.close(f)
    fnames_cell = '{' + ','.join("'{}'".format(x) for x in image_fnames) + '}'
    command = "{}({}, '{}')".format(cmd, fnames_cell, output_filename)
    print(command)

    mc = "matlab -nojvm -r \"try; {}; catch; exit; end; exit\"".format(command)
    pid = subprocess.Popen(
        shlex.split(mc), stdout=open('/dev/null', 'w'), cwd=script_dirname)
    retcode = pid.wait()
    if retcode != 0:
        raise Exception("Matlab script did not exit successfully!")

    all_boxes = list(scipy.io.loadmat(output_filename)['all_boxes'][0])
    subtractor = np.array((1, 1, 0, 0))[np.newaxis, :]
    all_boxes = [boxes - subtractor for boxes in all_boxes]

    os.remove(output_filename)
    if len(all_boxes) != len(image_fnames):
        raise Exception("Something went wrong computing the windows!")
    return all_boxes

if __name__ == '__main__':

    import time

    image_filenames = [
        script_dirname + '/000015.jpg',
        script_dirname + '/cat.jpg'
    ] * 4
    t = time.time()
    boxes = get_windows(image_filenames)
    print(boxes[:2])
    print("Processed {} images in {:.3f} s".format(
        len(image_filenames), time.time() - t))

The code is tested, and it has to work.

代码是经过测试的，它必须工作。

When I run the code, I get the following error:

当我运行代码时，我得到以下错误:

Traceback (most recent call last):
  File "selective_search.py", line 62, in <module>
    boxes = get_windows(image_filenames)

  File "selective_search.py", line 42, in get_windows
    all_boxes = list(scipy.io.loadmat(output_filename)['all_boxes'][0])

  File "/usr/lib/python2.7/dist-packages/scipy/io/matlab/mio.py", line 131, in loadmat
    MR = mat_reader_factory(file_name, appendmat, **kwargs)

  File "/usr/lib/python2.7/dist-packages/scipy/io/matlab/mio.py", line 55, in mat_reader_factory
    mjv, mnv = get_matfile_version(byte_stream)

  File "/usr/lib/python2.7/dist-packages/scipy/io/matlab/miobase.py", line 218, in get_matfile_version
    buffer = fileobj.read(4))

TypeError: buffer is too small for requested array

I'm using MATLAB 2015.

我用MATLAB 2015。

How can I fix the problem?

我该如何解决这个问题?

2 个解决方案

#1

This script constructs a command line, calls MATLAB with that, and then reads the .mat produced.

这个脚本构造一个命令行，用它调用MATLAB，然后读取。mat生成的。

I think you need to test the pieces:

我觉得你需要测试一下

does the MATLAB calling command look right?

MATLAB调用命令看起来对吗?
does the MATLAB run fine?

MATLAB运行正常吗?
is the .mat valid (read with MATLAB)?

.mat是否有效(用MATLAB阅读)?
can you read it from Python - just a plain loadmat?

你能从Python里读到它吗——只是一个普通的loadmat?

How long ago was it when it ran? For what MATLAB version? You may need to change this script so it does not destroy the temporary file, giving you a chance to test it interactively.

它跑了多久了?对MATLAB版本吗?您可能需要更改此脚本，以便它不会破坏临时文件，从而给您一个交互式测试它的机会。

One possibility is that the .mat is in a format that loadmat can't handle. MATLAB keeps changing the .mat format, the latest being some form of hd5?. You might need to change the MATLAB script so it uses an earlier format. I don't recall what kind error loadmat produces when loading a newer incompatible version.

一种可能性是，.mat的格式是loadmat无法处理的。MATLAB一直在改变。mat格式，最新的是某种形式的hd5?。您可能需要修改MATLAB脚本，以便使用更早的格式。我不记得加载一个更新不兼容的版本时，loadmat产生了什么类型的错误。

The TypeError: buffer is too small for requested array error is something I'd expect from a np.ndarray() call, not the usual np.array. But digging into the loadmat code, scipy/io/matlab/mio5.py I see that it does use ndarray. That does point to some sort of file format incompatibility, either the MATLAB file version, or maybe a 32/64 bit machine difference.

TypeError:缓冲区对于请求的数组错误来说太小了，这是我从npndarray()调用中期望得到的，而不是通常的np.array。但是要深入研究loadmat代码，scipy/io/matlab/mio5。py，我看到它用的是ndarray。这确实指出了某种文件格式的不兼容性，或者是MATLAB文件版本，或者可能是32/64位机器的差异。

The error is in the

误差在。

def get_matfile_version(fileobj):

function, right at the start where it tries to read the 1st 4 bytes of the file:

函数，在开始时它试图读取文件的第4个字节:

# Mat4 files have a zero somewhere in first 4 bytes
fileobj.seek(0)
mopt_bytes = np.ndarray(shape=(4,),
                       dtype=np.uint8,
                       buffer = fileobj.read(4))

It's reading the bytes, and trying to create an array directly from them. That looks like a straight forward operation. Except, what would happen if the file was empty? The buffer would be 0 bytes, too small. If so, then the problem is the MATLAB failed to run or to save its file. I'll have to experiment.

它读取字节，并试图直接从它们创建数组。这看起来是一个直接的操作。但是，如果文件是空的，会发生什么?缓冲区是0字节，太小了。如果是这样，那么问题是MATLAB无法运行或保存它的文件。我要尝试。

BINGO - the .mat file is empty

BINGO - .mat文件为空。

In [270]: with open('test.mat','w') as f:pass  # empty file
In [271]: matlab.loadmat('test.mat')
---------------------------------------------------------------------------
...
/usr/lib/python3/dist-packages/scipy/io/matlab/miobase.py in get_matfile_version(fileobj)
    216     mopt_bytes = np.ndarray(shape=(4,),
    217                            dtype=np.uint8,
--> 218                            buffer = fileobj.read(4))
    219     if 0 in mopt_bytes:
    220         fileobj.seek(0)

TypeError: buffer is too small for requested array

So for some reason, the MATLAB script failed. mkstemp creates an empty temporary file. Normally the MATLAB script would over write it (or append?). But if the script fails (to run), then this file remains empty, producing this error when you try to read it.

由于某种原因，MATLAB脚本失败了。mkstemp创建一个空的临时文件。通常，MATLAB脚本会重写它(或附加?)但是如果脚本失败(运行)，那么这个文件仍然是空的，当您试图读取它时产生这个错误。

If you used tempfile to get a file name, rather than create the file, you'd get an OSError, 'no such file'.

如果您使用tempfile来获取文件名，而不是创建文件，则会得到一个OSError，“没有这样的文件”。

I don't think the scipy developers anticipated someone would try to load an empty .mat file, otherwise they would have caught and translated this error.

我不认为scipy开发人员预期有人会尝试加载一个空的.mat文件，否则他们会发现并翻译这个错误。

#2

I have solved the problem.

我已经解决了这个问题。

I told you, I have run the code in different computer which has MATLAB 2014, and it worked.

我告诉过你，我已经在不同的计算机上运行了这个代码，它有MATLAB 2014，而且它工作了。

In that computer, gcc version was 4.7, but in the current computer which has MATLAB 2015, gcc version is 4.9.

在这台计算机中，gcc的版本是4.7，但是在现在的计算机中有MATLAB 2015, gcc版本是4.9。

I have removed gcc 4.9, and installed 4.7, and the problem has been solved. :)

我已经删除了gcc 4.9，安装了4.7，问题已经解决了。:)

#1