Python：如何保持浮动数组的大块数组相对较小？

I have a numpy array created from reading many images with the cv2 package. I read the image in grayscale so the pixel values are from 0 to 255 in which case the type of the data is uint8. This means each data element is of size 1 byte. I create a list using each image and then want to transform the list of arrays into an array of arrays. Afterwards, I need to feed this data into a model but the model needs the image pixel values to be floats between 1 and 0. Now each float in Python is 8 bytes. So I tried to transform each array using this cv2 function

我有一个通过使用cv2包读取许多图像而创建的numpy数组。我以灰度读取图像,因此像素值从0到255,在这种情况下,数据的类型是uint8。这意味着每个数据元素的大小为1个字节。我使用每个图像创建一个列表,然后想要将数组列表转换为数组数组。之后,我需要将这些数据提供给模型,但模型需要图像像素值在1到0之间浮动。现在Python中的每个浮点数都是8个字节。所以我尝试使用这个cv2函数转换每个数组

unlabeled_img_array = cv2.normalize(unlabeled_img_array.astype('float'), None, 0.0, 1.0, cv2.NORM_MINMAX)

It works and I can create the list of arrays. The problem arises when I try to turn the list of arrays into an array of arrays with this:

它工作,我可以创建数组列表。当我尝试将数组列表转换为数组数组时出现问题:

unlabeled_img_array_arrays = np.array(unlabeled_img_list_arrays)

I then get a Memory Error, evidently because the matrix is too big. If I do it with datatype uint8 there's no error.

然后我得到一个内存错误,显然是因为矩阵太大了。如果我用数据类型uint8这样做,那就没有错误。

My question is. Is there a way to get over this problem or would I have to stick with using uint8 instead of float values?

我的问题是。有没有办法解决这个问题,还是我必须坚持使用uint8而不是浮点值?

Edit:

I also tried using this

我也试过用这个

cv2.normalize(unlabeled_img_array.astype(np.float16), None, 0.0, 1.0, cv2.NORM_MINMAX)

but it gives me this error

但它给了我这个错误

TypeError: src data type = 23 is not supported

Is there a way to make the array to float16? Maybe that will cut the size enough. Although I'm not sure if the model will accept it.

有没有办法让数组浮动16?也许这会缩小尺寸。虽然我不确定该模型是否会接受它。

1 个解决方案

#1

You could try 'float16' instead of 'float', this would nominally save 3/4 of the memory.

您可以尝试'float16'而不是'float',这将名义上节省3/4的内存。