My issue is that a custom class has been saved with pickle.dump, since these files were saved the custom class has been changed and now when I use pickle.load I am getting this error. Is it a problem with the saved file?
我的问题是自定义类被保存为pickle。转储,因为这些文件被保存,所以自定义类已经被更改,现在当我使用pickle时。负载我得到这个错误。保存的文件有问题吗?
The error:
错误:
File "/cprprod/extern/lib/python2.7/pickle.py", line 1378, in load
return Unpickler(file).load()
File "/cprprod/extern/lib/python2.7/pickle.py", line 858, in load
dispatch[key](self)
file "/cprprod/extern/lib/python2.7/pickle.py", line 1070, in load_inst
self._instantiate(klass, self.marker())
File "/cprprod/extern/lib/python2.7/pickle.py", line 1060, in _instantiate
value = klass(*args)
Is there anything I can do to load the file?
我能做些什么来加载文件吗?
The code
的代码
file = open(filename,'rb')
obj = pickle.load(file)
will give me the error.
会给我错误。
Here is some minimal code which can reproduce the error:
下面是一些可以重现错误的最小代码:
import pickle
class foo:
def __init__(self,a):
self.a = a
def __str__(self):
return str(self.a)
obj = foo(1)
with open('junk','wb') as f:
pickle.dump(obj,f)
class foo:
def __init__(self,a,b):
self.a = a
self.b = b
def __str__(self):
return '%s %s'%(self.a,self.b)
def __getinitargs__(self):
return (self.a,self.b)
with open('junk','rb') as f:
obj = pickle.load(f)
print str(obj)
3 个解决方案
#1
2
Given the contrived code that I posted on your behalf in the question, we can "fix" this error as:
鉴于我在问题中以你的名义发布的设计代码,我们可以“修复”这个错误,如下所示:
with open('junk','rb') as f:
try:
obj = pickle.load(f)
except Exception as e:
print e
position = f.tell()
a = foo.__getinitargs__
del foo.__getinitargs__
f.seek(position)
obj = pickle.load(f)
foo.__getinitargs__ = a
print str(obj)
Now we see that the instance has been unpickled and no longer has attribute b
.
现在我们看到实例已经被取消pickle,并且不再具有属性b。
#2
2
If you added __getinitargs__()
then it is up to you to make sure your new class can handle the arguments passed to __init__()
. Old data that doesn't have the __getinitargs__
data will still lead to __init__
to be called but with no arguments.
如果您添加了__getinitargs__(),那么您应该确保您的新类能够处理传递给__init__()的参数。没有__getinitargs__数据的旧数据仍然会导致__init__被调用,但没有参数。
Make the arguments to __init__
optional via keyword arguments:
通过关键字参数使__init__可选:
def __init__(self, otherarg=None):
if otherarg is None:
# created from an old-revision pickle. Handle separately.
# The pickle will be loaded *normally* and data will still be set normally
return
self.otherarg = otherarg
When loading the old-style pickle, the data for these classes will still be restored. You can use __setstate__()
to transform the internal state as needed.
当加载旧式pickle时,这些类的数据仍然会被恢复。您可以使用__setstate__()来转换需要的内部状态。
Alternatively, temporarily remove the __getinitargs__
method from the class:
或者,临时从类中删除__getinitargs__方法:
initargs = foo.__getinitargs__.__func__
del foo.__getinitargs__
obj = pickle.load(f)
foo.__getinitargs__ = initargs
and re-dump your pickles from the now-loaded objects with __getinitargs__
reinstated.
然后用恢复的__getinitargs__重新从当前加载的对象中转储泡菜。
I've tested both methods and in both cases the old data is loaded correctly and you can then dump your objects again to a new pickle file with __getinitargs__
just fine.
我已经测试了这两种方法,在这两种情况下,旧数据都被正确地加载了,然后您可以将对象再次转储到一个新的pickle文件中,并使用__getinitargs__。
#3
1
You might want to modify the custom class to optionally require a second parameter. This would keep back award compatibility with your pickled objects.
您可能希望修改自定义类,以便可以选择地需要第二个参数。这将保留与您的pickle对象的兼容性。
#1
2
Given the contrived code that I posted on your behalf in the question, we can "fix" this error as:
鉴于我在问题中以你的名义发布的设计代码,我们可以“修复”这个错误,如下所示:
with open('junk','rb') as f:
try:
obj = pickle.load(f)
except Exception as e:
print e
position = f.tell()
a = foo.__getinitargs__
del foo.__getinitargs__
f.seek(position)
obj = pickle.load(f)
foo.__getinitargs__ = a
print str(obj)
Now we see that the instance has been unpickled and no longer has attribute b
.
现在我们看到实例已经被取消pickle,并且不再具有属性b。
#2
2
If you added __getinitargs__()
then it is up to you to make sure your new class can handle the arguments passed to __init__()
. Old data that doesn't have the __getinitargs__
data will still lead to __init__
to be called but with no arguments.
如果您添加了__getinitargs__(),那么您应该确保您的新类能够处理传递给__init__()的参数。没有__getinitargs__数据的旧数据仍然会导致__init__被调用,但没有参数。
Make the arguments to __init__
optional via keyword arguments:
通过关键字参数使__init__可选:
def __init__(self, otherarg=None):
if otherarg is None:
# created from an old-revision pickle. Handle separately.
# The pickle will be loaded *normally* and data will still be set normally
return
self.otherarg = otherarg
When loading the old-style pickle, the data for these classes will still be restored. You can use __setstate__()
to transform the internal state as needed.
当加载旧式pickle时,这些类的数据仍然会被恢复。您可以使用__setstate__()来转换需要的内部状态。
Alternatively, temporarily remove the __getinitargs__
method from the class:
或者,临时从类中删除__getinitargs__方法:
initargs = foo.__getinitargs__.__func__
del foo.__getinitargs__
obj = pickle.load(f)
foo.__getinitargs__ = initargs
and re-dump your pickles from the now-loaded objects with __getinitargs__
reinstated.
然后用恢复的__getinitargs__重新从当前加载的对象中转储泡菜。
I've tested both methods and in both cases the old data is loaded correctly and you can then dump your objects again to a new pickle file with __getinitargs__
just fine.
我已经测试了这两种方法,在这两种情况下,旧数据都被正确地加载了,然后您可以将对象再次转储到一个新的pickle文件中,并使用__getinitargs__。
#3
1
You might want to modify the custom class to optionally require a second parameter. This would keep back award compatibility with your pickled objects.
您可能希望修改自定义类,以便可以选择地需要第二个参数。这将保留与您的pickle对象的兼容性。