Python实现文件的全备份和差异备份

时间:2021-05-13 14:10:46

转自

http://www.75271.com/1258.html


Python实现文件的全备份和差异备份

之前有写利用md5方式来做差异备份,但是这种md5方式来写存在以下问题:

  • md5sum获取有些软连接的MD5值存在问题
  • 不支持对空目录进行备份,因为md5sum无法获取空目录的md5值
  • 权限的修改md5sum无法判断

解决方案

利用文件的mtime ctime

mtime(Modified time)是在写入文件时随文件内容的更改而更改的

ctime(Create time)是在写入文件、更改所有者、权限或链接设置时随Inode的内容更改而更改的

废话不多说直接上代码:

 
  1. #!/usr/bin/env <a href="http://www.75271.com/tag/python" title="查看更多关于python的文章" target="_blank">python</a>
  2. import time,os,sys,cPickle
  3. fileInfo = {}
  4. def logger(time,fileName,status,fileNum):
  5. f = open('backup.log','a')
  6. f.write("%s\t%s\t%s\t\t%s\n" % (time,fileName,status,fileNum))
  7. def tar(sDir,dDir,fileNum):
  8. command = "tar zcf %s %s >/dev/null 2>&1" % (dDir + ".tar.gz",sDir)
  9. if os.system(command) == 0:
  10. logger(time.strftime('%F %X'),dDir + ".tar.gz",'success',fileNum)
  11. else:
  12. logger(time.strftime('%F %X'),dDir + ".tar.gz",'failed',fileNum)
  13. def fullBak(path):
  14. fileNum = 0
  15. for root,dirs,files in os.walk(path):
  16. for name in files:
  17. file = os.path.join(root, name)
  18. mtime = os.path.getmtime(file)
  19. ctime = os.path.getctime(file)
  20. fileInfo[file] = (mtime,ctime)
  21. fileNum += 1
  22. f = open(P,'w')
  23. cPickle.dump(fileInfo,f)
  24. f.close()
  25. tar(S,D,fileNum)
  26. def diffBak(path):
  27. for root,dirs,files in os.walk(path):
  28. for name in files:
  29. file = os.path.join(root,name)
  30. mtime = os.path.getmtime(file)
  31. ctime = os.path.getctime(file)
  32. fileInfo[file] = (mtime,ctime)
  33. if os.path.isfile(P) == 0:
  34. f = open(P,'w')
  35. f.close()
  36. if os.stat(P).st_size == 0:
  37. f = open(P,'w')
  38. cPickle.dump(fileInfo,f)
  39. fileNum = len(fileInfo.keys())
  40. f.close()
  41. print fileNum
  42. tar(S,D,fileNum)
  43. else:
  44. f = open(P)
  45. old_fileInfo = cPickle.load(f)
  46. f.close()
  47. difference = dict(set(fileInfo.items())^set(old_fileInfo.items()))
  48. fileNum = len(difference)
  49. print fileNum
  50. difference_file = ' '.join(difference.keys())
  51. print difference_file
  52. tar(difference_file,D,fileNum)
  53. f = open(P,'w')
  54. cPickle.dump(fileInfo,f)
  55. f.close()
  56. def Usage():
  57. print '''
  58. Syntax: <a href="http://www.75271.com/tag/python" title="查看更多关于python的文章" target="_blank">python</a> file_bakcup.py pickle_file model source_dir filename_bk
  59. model: 1:Full backup 2:Differential backup
  60. example: <a href="http://www.75271.com/tag/python" title="查看更多关于python的文章" target="_blank">python</a> file_backup.py fileinfo.pk 2 /etc etc_$(date +%F)
  61. explain: Automatically add '.tar.gz' suffix
  62. '''
  63. sys.exit()
  64. if len(sys.argv) != 5:
  65. Usage()
  66. P = sys.argv[1]
  67. M = int(sys.argv[2])
  68. S = sys.argv[3]
  69. D = sys.argv[4]
  70. if M == 1:
  71. fullBak(S)
  72. elif M == 2:
  73. diffBak(S)
  74. else:
  75. print "\033[;31mDoes not support this mode\033[0m"
  76. Usage()

测试使用:

python file_backup.py data.pk 1 data data_$(date +%F) #全备份
$ > data/www.75271.com #测试创建文件,修改文件权限
$ chmod 777 data/py/75271com_bk/data.db
python file_backup.py data.pk 2 data data_$(date +%F)_1 #备份改变的文件
2
data/py/75271com_bk/data.db data/www.75271.com