python运维常用相关模块
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
一.模块的分类
模块，用一砣代码实现了某个功能的代码集合。
类似于函数式编程和面向过程编程，函数式编程则完成一个功能，其他代码用来调用即可，提供了代码的重用性和代码间的耦合。而对于一个复杂的功能来，可能需要多个函数才能完成（函数又可以在不同的.py文件中），n个 .py 文件组成的代码集合就称为模块。
如：os 是系统相关的模块；file是文件操作相关的模块
模块分为三种：
1>.自定义模块:自定义的模块，顾名思义，就是你自己写的python程序，我们知道python的代码都存在一个以".py"结尾的文件中的，我们这样命名一个python脚本，吧后缀去掉就是模块名称，这个就是自定义模块，我举个例子：我写了一个yinzhengjie.py的文件。里面的内容我们可以忽略，如果我们要导入这个模块的话直接导入yinzhengjie这个模块名称就好了；
2>.内置模块：那么问题来了，我们学的cha(),id()等等所有的内置函数是模块吗？答案是否定的！不是！对内置函数不是内置模块，他们只是python解释器自带的一些功能，那么什么是内置模块呢？一会我会再我的博客中提到一些常用的内置模块；
3>.开源模块：这个就很好解释了，python语言的官网提供了一个供应开发人员上传你的代码到服务器上（https://pypi.python.org/pypi），然后客户端只要在命令行中输入安装命令就可以随意的在shell或者cmd的python解释器中调用这个第三方模块，比如：pip install paramiko.
二.内置模块解析：
1.OS模块详解
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import os
#1.获取当前工作目录，即当前python脚本工作的目录路径
print(os.getcwd())
#2.改变当前脚本工作目录；相当于shell下cd,记住，这个是没有返回值的哟！
print(os.chdir(r"D:\python\daima\DAY1"))
#3.返回当前目录: ('.')
print(os.curdir)
#4.获取当前目录的父目录字符串名：('..')
print(os.pardir)
#5.可生成多层递归目录(创建目录),生产完毕后返回一个None值
print(os.makedirs("D:\python\daima\DAY10"))
#6.若目录为空，则删除，并递归到上一级目录，如若也为空，则删除，依此类推（和上面的相反，就是删除目录。）
print(os.removedirs("D:\python\daima\DAY10"))
#7.生成单级目录；相当于shell中mkdir dirname，如果当前目录已经存在改目录就会报错！
print(os.mkdir("DAY10"))
#8.删除单级空目录，若目录不为空则无法删除，报错；相当于shell中rmdir dirname,如果当前目录没有改目录就会报错！
print(os.rmdir("DAY10"))
#9.列出指定目录下的所有文件和子目录，包括隐藏文件，并以列表方式打印
print(os.listdir("D:\python\daima"))
#10.删除一个文件
# os.remove("locked.txt")
#11.重命名文件/目录
# os.rename("oldname","newname")
#12.os.stat('path/filename') 获取文件/目录信息
print(os.stat("D:\python\daima\DAY4"))
#13.输出操作系统特定的路径分隔符，win下为"\\",Linux下为"/"
print(os.sep)
#14.输出当前平台使用的行终止符，win下为"\r\n",Linux下为"\n"
print(os.linesep)
#15.输出用于分割文件路径的字符串
print(os.pathsep)
#16.输出字符串指示当前使用平台。win->'nt'; Linux->'posix'
print(os.name)
#17.运行shell或者windows命令，直接显示命令的输出结果，可以将这个数据存放在一个变量中哟
# print(os.system("dir"))
#18.返回path规范化的绝对路径
print(os.path.abspath("user_info.txt"))
#19.将path分割成目录和文件名二元组返回
print(os.path.split(r"D:\python\daima\DAY1\user_info.txt"))
#20.返回path的目录。其实就是os.path.split(path)的第一个元素
print(os.path.dirname(r"D:\python\daima\DAY1\user_info.txt"))
#21.os.path.basename(path) 返回path最后的文件名。如何path以／或\结尾，那么就会返回空值。即os.path.split(path)的第二个元素
print(os.path.basename(r"D:\python\daima\DAY1\user_info.txt"))
#22.os.path.exists(path) 如果path存在，返回True；如果path不存在，返回False
print(os.path.exists(r"D:\python\daima\DAY1\user_info.txt"))
#23.os.path.isabs(path) 如果path是绝对路径，返回True
print(os.path.isabs(r"D:\python\daima\DAY1\user_info.txt"))
#24.os.path.isfile(path) 如果path是一个存在的文件，返回True。否则返回False
print(os.path.isfile(r"D:\python\daima\DAY1\user_info.txt"))
#25.os.path.isdir(path) 如果path是一个存在的目录，则返回True。否则返回False
print(os.path.isdir(r"D:\python\daima\DAY1\user_info.txt"))
#26.os.path.join(path1[, path2[, ...]]) 将多个路径组合后返回，第一个绝对路径之前的参数将被忽略
print(os.path.join(r"user_info.txt",r"D:\python\daima\DAY1\user_info.txt"))
#27.os.path.getatime(path) 返回path所指向的文件或者目录的最后存取时间
print(os.path.getatime(r"D:\python\daima\DAY1\user_info.txt"))
#28.os.path.getmtime(path) 返回path所指向的文件或者目录的最后修改时间
print(os.path.getmtime(r"D:\python\daima\DAY1\user_info.txt"))
'''
更多关于os模块的使用方法请参考：https://docs.python.org/2/library/os.html?highlight=os#module-os
''' #以上代码执行结果如下：
D:\python\daima\DAY4
None
.
..
None
None
None
None
['.idea', 'DAY1', 'DAY2', 'DAY3', 'DAY4', 'DAY5', '__pycache__']
os.stat_result(st_mode=16895, st_ino=22799473113577966, st_dev=839182139, st_nlink=1, st_uid=0, st_gid=0, st_size=4096, st_atime=1487743397, st_mtime=1487743397, st_ctime=1486692902)
\ ;
nt
D:\python\daima\DAY1\user_info.txt
('D:\\python\\daima\\DAY1', 'user_info.txt')
D:\python\daima\DAY1
user_info.txt
True
True
True
False
D:\python\daima\DAY1\user_info.txt
1483869109.7747889
1483869109.7758367
os模块常用方法详解
2.sys模块常用方法
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import sys
#1.获取Python解释程序的版本信息
print(sys.version)
#2.返回操作系统平台名称
print(sys.platform)
#3.返回模块的搜索路径，初始化时使用PYTHONPATH环境变量的值
print(sys.path)
#4.退出程序，正常退出时exit(0),如果不写数字的话，默认就是0
# print(sys.exit(100))
#5.命令行参数List，第一个元素是程序本身路径
# path_info = sys.argv[1]
#6.显示当前系统最大的Int值
print(sys.maxsize) '''
更多使用方法请参考：https://docs.python.org/2/library/sys.html?highlight=sys#module-sys
''' #以上代码执行结果如下： 3.5.2 (v3.5.2:4def2a2901a5, Jun 25 2016, 22:01:18) [MSC v.1900 32 bit (Intel)]
win32
['D:\\python\\daima\\DAY4', 'D:\\python\\daima', 'C:\\Users\\yzj\\AppData\\Local\\Programs\\Python\\Python35-32\\python35.zip', 'C:\\Users\\yzj\\AppData\\Local\\Programs\\Python\\Python35-32\\DLLs', 'C:\\Users\\yzj\\AppData\\Local\\Programs\\Python\\Python35-32\\lib', 'C:\\Users\\yzj\\AppData\\Local\\Programs\\Python\\Python35-32', 'C:\\Users\\yzj\\AppData\\Local\\Programs\\Python\\Python35-32\\lib\\site-packages']
2147483647
sys模块常用方法详解
3.json和pickle模块详解
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import pickle
'''
pickle:
1.>用于python特有的类型 和 python的数据类型间进行转换
2.>pickle模块提供了四个功能：dumps、dump、loads、load.
补充说明：将数据通过特殊的形式转换成只有python解释器识别的字符串，这个过程我们叫做序列化，而把哪些python能够识别的字符串转换成我们能看懂的叫做反序列化。
'''
data_info = {"name":"尹正杰","password":""}
#1.将数据通过特殊的形式转换为只有python语言知识的字符串并写入文件
# pickle_str = pickle.dumps(data_info)
# print(pickle_str)
# f = open("test.txt","wb")
# f.write(pickle_str)
#2.上面的写入文件的方法也可以这么玩,看起来更简单
# with open("test_1.txt","wb") as fb:
# pickle.dump(data_info,fb)
#我们知道将数据存入文件，那么我们怎么把存入文件的东西读出来呢？
#方法一：
# f = open("test_1.txt","rb")
# print(pickle.loads(f.read()))
#方法二:
f = open("test_1.txt","rb")
print(pickle.load(f))
python自带pickle模块用法
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import json
'''
用于序列化的两个模块
1>.json:用于字符串 和 python数据类型间进行转换
2>.pickle:用于python特有的类型和python的数据类型间进行转换
json模块提供了四个功能：dumps、dump、loads、load
pickle模块提供了四个功能：dumps、dump、loads、load
'''
accounts = {
"id":521,
"name":"yinzhengjie",
"banlance":""
}
#存数据方式一：
# f = open(r"D:\python\daima\DAY4\test_2.txt","w")
# json_str = json.dumps(accounts)
# f.write(json_str)
#存数据方式二：
# with open(r"D:\python\daima\DAY4\test_2.txt","w") as fp:
# json.dump(accounts,fp)
#读取数据的方法一：
# f = open("test_2.txt","r")
# print(json.loads(f.read()))
#方法二:
f = open("test_2.txt","r")
print(json.load(f))
Python自带的json模块用法
对比json和pickle的异同：
1>.相同点：都是用于系列化和反序列化的模块。
2>.不同点：json是在所有语言都通用的数据存储格式，而pickle是仅仅只有python语言独有的存储格式。
4.time模块与datetime模块详解
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import time
#1.测量处理器运算时间,不包括sleep时间,不稳定,mac上测不出来
print(time.process_time())
#2.返回与utc时间的时间差,以秒计算
print(time.altzone)
#3.返回默认时间格式
print(time.asctime())
#4.返回本地时间的struct_time对象格式
print(time.localtime())
#5.返回utc时间的struc时间对象格式
print(time.gmtime(time.time()-800000))
#6.返回本地时间格式，
print(time.asctime(time.localtime()))
#7.返回时间格式，同上
print(time.ctime())
#8.将日期字符串转成struct时间对象格式
string_2_struct = time.strptime("2016/05/22","%Y/%m/%d")
print(string_2_struct)
#9.将struct时间对象转成时间戳
struct_2_stamp = time.mktime(string_2_struct)
print(struct_2_stamp)
#10.将utc时间戳转换成struct_time格式
print(time.gmtime(time.time()-86640))
#11.将utc struct_time格式转成指定的字符串格式
print(time.strftime("%Y-%m-%d %H:%M:%S",time.gmtime()))
time模块演示
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import time,datetime
#1.打印当前系统时间
print(datetime.datetime.now())
#2.时间戳直接转成日期格式如：2017-02-22
print(datetime.date.fromtimestamp(time.time()))
#3.当前时间+3天
print(datetime.datetime.now() + datetime.timedelta(3))
#4.当前时间-3天
print(datetime.datetime.now() + datetime.timedelta(-3))
#5.当前时间+3小时
print(datetime.datetime.now() + datetime.timedelta(hours=3))
#6.当前时间+30分
print(datetime.datetime.now() + datetime.timedelta(minutes=30))
#7.时间替换
c_time = datetime.datetime.now()
print(c_time.replace(minute=3,hour=2))
datetime模块用法
关于时间的一个转换流程图：
测试代码如下：
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import time
#1.将日期转换成struct_time格式，会以一个tuple的形式打印出来
# struct_time = time.strptime("2017/2/22","%Y/%m/%d") #注意分割的符号要保持一致哟，我这里分割的符号是“/”
struct_time = time.strptime("2017-2-22 17:29:30","%Y-%m-%d %H:%M:%S") #ye ky 注意分割的符号要保持一致哟，我这里分割的符号是“-”
print(struct_time)
#2.将struct_time格式转换成时间戳的形式
stamp_time = time.mktime(struct_time)
print(stamp_time)
#3.将时间戳的形式，转换成日期格式
date_time = time.gmtime(stamp_time)
print(date_time)
print(time.strftime("%Y-%m-%d %H:%M:%S",date_time))
关于参数的详细说明如下：
| Directive | Meaning | Notes |
|---|---|---|
%a |
Locale’s abbreviated weekday name. | |
%A |
Locale’s full weekday name. | |
%b |
Locale’s abbreviated month name. | |
%B |
Locale’s full month name. | |
%c |
Locale’s appropriate date and time representation. | |
%d |
Day of the month as a decimal number [01,31]. | |
%H |
Hour (24-hour clock) as a decimal number [00,23]. | |
%I |
Hour (12-hour clock) as a decimal number [01,12]. | |
%j |
Day of the year as a decimal number [001,366]. | |
%m |
Month as a decimal number [01,12]. | |
%M |
Minute as a decimal number [00,59]. | |
%p |
Locale’s equivalent of either AM or PM. | (1) |
%S |
Second as a decimal number [00,61]. | (2) |
%U |
Week number of the year (Sunday as the first day of the week) as a decimal number [00,53]. All days in a new year preceding the first Sunday are considered to be in week 0. | (3) |
%w |
Weekday as a decimal number [0(Sunday),6]. | |
%W |
Week number of the year (Monday as the first day of the week) as a decimal number [00,53]. All days in a new year preceding the first Monday are considered to be in week 0. | (3) |
%x |
Locale’s appropriate date representation. | |
%X |
Locale’s appropriate time representation. | |
%y |
Year without century as a decimal number [00,99]. | |
%Y |
Year with century as a decimal number. | |
%z |
Time zone offset indicating a positive or negative time difference from UTC/GMT of the form +HHMM or -HHMM, where H represents decimal hour digits and M represents decimal minute digits [-23:59, +23:59]. | |
%Z |
Time zone name (no characters if no time zone exists). | |
%% |
A literal '%' character. |
5.random模块详解
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import random
'''
random：
是用来生成随机数字的。
'''
#举例子：
print(random.random())
print(random.randint(1,20))
print(random.randrange(1,10))
'''
random 都有哪些应用呢？
'''
#生成随机验证码，版本一：
checkcode = ''
for i in range(6): #修改后面的数字表示随机生成的数字个数，因为要循环6次
current = random.randrange(0,4)
if current != i:
temp = chr(random.randint(65,90))
else:
temp = random.randint(0,9)
checkcode += str(temp)
print(checkcode)
#生成随机验证码，版本二
import string
source = string.digits + string.ascii_lowercase
print("".join(random.sample(source,6))) #修改后面的数字表示随机生成的数字个数
random常见方法及应用
6.logging模块
很多程序都有记录日志的需求，并且日志中包含的信息即有正常的程序访问日志，还可能有错误、警告等信息输出，python的logging模块提供了标准的日志接口，你可以通过它存储各种格式的日志，logging的日志可以分为 debug(), info(), warning(), error() and critical() 5个级别，从左往右依次增加告警级别，下面我们看一下怎么用。
看一下这几个日志级别分别代表什么意思
| Level | When it’s used |
|---|---|
DEBUG |
Detailed information, typically of interest only when diagnosing problems. |
INFO |
Confirmation that things are working as expected. |
WARNING |
An indication that something unexpected happened, or indicative of some problem in the near future (e.g. ‘disk space low’). The software is still working as expected. |
ERROR |
Due to a more serious problem, the software has not been able to perform some function. |
CRITICAL |
A serious error, indicating that the program itself may be unable to continue running. |
日志格式
|
%(name)s |
Logger的名字 |
|
%(levelno)s |
数字形式的日志级别 |
|
%(levelname)s |
文本形式的日志级别 |
|
%(pathname)s |
调用日志输出函数的模块的完整路径名，可能没有 |
|
%(filename)s |
调用日志输出函数的模块的文件名 |
|
%(module)s |
调用日志输出函数的模块名 |
|
%(funcName)s |
调用日志输出函数的函数名 |
|
%(lineno)d |
调用日志输出函数的语句所在的代码行 |
|
%(created)f |
当前时间，用UNIX标准的表示时间的浮 点数表示 |
|
%(relativeCreated)d |
输出日志信息时的，自Logger创建以 来的毫秒数 |
|
%(asctime)s |
字符串形式的当前时间。默认格式是 “2003-07-08 16:49:45,896”。逗号后面的是毫秒 |
|
%(thread)d |
线程ID。可能没有 |
|
%(threadName)s |
线程名。可能没有 |
|
%(process)d |
进程ID。可能没有 |
|
%(message)s |
用户输出的消息 |
A.简单的logging模块案例演示：
1>.初探logging模块:
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import logging
logging.warning("user [尹正杰] attempted wrong password more than 3 times")
logging.critical("server is down") 以上代码执行结果如下：
WARNING:root:user [尹正杰] attempted wrong password more than 3 times
CRITICAL:root:server is down
2>.如果想把日志写到文件里，也很简单:
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import logging
logging.basicConfig(filename='yinzhengjie.log', level=logging.INFO)
logging.debug('This message should go to the log file')
logging.info('So should this')
logging.warning('And this, too') '''
补充说明：
logging.basicConfig中的参数level=loggin.INFO意思是，把日志纪录级别设置为INFO，也就是说，只有比日志是INFO或比INFO级别更高的日志才会被纪录到文件里，在这个例子，第一条日志是不会被纪录的，如果希望纪录debug的日志，那把日志级别改成DEBUG就行了。
''' #查看'yinzhengjie.log'文件内容如下：
INFO:root:So should this
WARNING:root:And this, too
B.如果想同时把log打印在屏幕和文件日志里，就需要了解一点复杂的知识了：
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
'''
Python使用logging模块记录日志涉及四个主要类，使用官方文档中的概括最为合适：
1>.logger提供了应用程序可以直接使用的接口；
2>.handler将(logger创建的)日志记录发送到合适的目的输出；
3>.filter提供了细度设备来决定输出哪条日志记录；
4>.formatter决定日志记录的最终输出格式。
'''
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
'''
Python使用logging模块记录日志涉及四个主要类，使用官方文档中的概括最为合适：
1>.logger提供了应用程序可以直接使用的接口；
2>.handler将(logger创建的)日志记录发送到合适的目的输出；
3>.filter提供了细度设备来决定输出哪条日志记录；
4>.formatter决定日志记录的最终输出格式。
''' #logger
'''
每个程序在输出信息之前都要获得一个Logger。Logger通常对应了程序的模块名.
#1>.比如聊天工具的图形界面模块可以这样获得它的Logger：
LOG=logging.getLogger(”chat.gui”)
#2>.而核心模块可以这样：
LOG=logging.getLogger(”chat.kernel”)
#3>.指定最低的日志级别，低于lel的级别将被忽略。debug是最低的内置级别，critical为最高
Logger.setLevel(lel)
#4>.添加或删除指定的filter
Logger.addFilter(filt)、Logger.removeFilter(filt)
#5>.增加或删除指定的handler
Logger.addHandler(hdlr)、Logger.removeHandler(hdlr)
#6>.可以设置的日志级别
Logger.debug()、Logger.info()、Logger.warning()、Logger.error()、Logger.critical()
''' #handler
'''
handler对象负责发送相关的信息到指定目的地。Python的日志系统有多种Handler可以使用。有些Handler可以把信息输出到控制台，有些Logger可以把信息输出到文件，还有些 Handler可以把信息发送到网络上。如果觉得不够用，还可以编写自己的Handler。可以通过addHandler()方法添加多个多handler
#1>.指定被处理的信息级别，低于lel级别的信息将被忽略
Handler.setLevel(lel)
#2>.给这个handler选择一个格式
Handler.setFormatter()
#3>.新增或删除一个filter对象
Handler.addFilter(filt)、Handler.removeFilter(filt)
每个Logger可以附加多个Handler。接下来我们就来介绍一些常用的Handler：
#1>.logging.StreamHandler
使用这个Handler可以向类似与sys.stdout或者sys.stderr的任何文件对象(file object)输出信息。它的构造函数是：StreamHandler([strm]),其中strm参数是一个文件对象。默认是sys.stderr
2) logging.FileHandler
和StreamHandler类似，用于向一个文件输出日志信息。不过FileHandler会帮你打开这个文件。它的构造函数是：FileHandler(filename[,mode]),filename是文件名，必须指定一个文件名。mode是文件的打开方式。参见Python内置函数open()的用法。默认是’a'，即添加到文件末尾。
3) logging.handlers.RotatingFileHandler
这个Handler类似于上面的FileHandler，但是它可以管理文件大小。当文件达到一定大小之后，它会自动将当前日志文件改名，然后创建 一个新的同名日志文件继续输出。比如日志文件是chat.log。当chat.log达到指定的大小之后，RotatingFileHandler自动把 文件改名为chat.log.1。不过，如果chat.log.1已经存在，会先把chat.log.1重命名为chat.log.2。。。最后重新创建 chat.log，继续输出日志信息。它的构造函数是：RotatingFileHandler( filename[, mode[, maxBytes[, backupCount]]]),其中filename和mode两个参数和FileHandler一样。maxBytes用于指定日志文件的最大文件大小。如果maxBytes为0，意味着日志文件可以无限大，这时上面描述的重命名过程就不会发生。backupCount用于指定保留的备份文件的个数。比如，如果指定为2，当上面描述的重命名过程发生时，原有的chat.log.2并不会被更名，而是被删除。
4) logging.handlers.TimedRotatingFileHandler
这个Handler和RotatingFileHandler类似，不过，它没有通过判断文件大小来决定何时重新创建日志文件，而是间隔一定时间就 自动创建新的日志文件。重命名的过程与RotatingFileHandler类似，不过新的文件不是附加数字，而是当前时间。它的构造函数是：TimedRotatingFileHandler( filename [,when [,interval [,backupCount]]]),其中filename参数和backupCount参数和RotatingFileHandler具有相同的意义,interval是时间间隔。when参数是一个字符串。表示时间间隔的单位，不区分大小写。它有以下取值：(S[秒],M[分],H[小时],D[天],W[每星期（interval==0时代表星期一）],midnight[每天凌晨]
'''
常用的功能介绍
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import logging
# create logger
logger = logging.getLogger('TEST-LOG') #定义报警的标题
logger.setLevel(logging.DEBUG) #设置在屏幕中打印的报警级别和写入文件的文件的最低级别这个相当于总开关，下面2个调试都得在这基础之上对告警级别做处理。
# create console handler and set level to debug
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG) #设置在屏幕中打印的报警级别，（优先权没有第一个权限高）
# create file handler and set level to warning #创建文件处理程序并设置告警级别。
fh = logging.FileHandler("access.log") #定义保存日志的文件
fh.setLevel(logging.WARNING) #定义将报警信息写入文件中的级别（优先权没有第一个权限高））
# create formatter #定义输出格式
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
# add formatter to ch and fh #添加格式花输出
ch.setFormatter(formatter)
fh.setFormatter(formatter)
# add ch and fh to logger #添加记录器
logger.addHandler(ch)
logger.addHandler(fh)
# 'application' code #定义各种报警级别输出的内容。
logger.debug('debug message')
logger.info('info message')
logger.warn('warn message')
logger.error('error message')
logger.critical('critical message') #屏幕输入内容如下：
2017-02-23 10:08:28,184 - TEST-LOG - DEBUG - debug message
2017-02-23 10:08:28,185 - TEST-LOG - INFO - info message
2017-02-23 10:08:28,185 - TEST-LOG - WARNING - warn message
2017-02-23 10:08:28,185 - TEST-LOG - ERROR - error message
2017-02-23 10:08:28,185 - TEST-LOG - CRITICAL - critical message #文件存入内容如下：
2017-02-23 10:08:28,185 - TEST-LOG - WARNING - warn message
2017-02-23 10:08:28,185 - TEST-LOG - ERROR - error message
2017-02-23 10:08:28,185 - TEST-LOG - CRITICAL - critical message
控制报警级别的输出案例
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import logging
from logging import handlers
logger = logging.getLogger(__name__)
log_file = "timelog.log"
#fh = handlers.RotatingFileHandler(filename=log_file,maxBytes=10,backupCount=3)
fh = handlers.TimedRotatingFileHandler(filename=log_file,when="S",interval=5,backupCount=3) #filename定义将信息输入到指定的文件，when指定单位是s(秒),interval是时间间隔的频率,单位是when所指定的哟（所以，你可以理解频率是5s）；backupCount表示备份的文件个数，我这里是指定的3个文件。
formatter = logging.Formatter('%(asctime)s %(module)s:%(lineno)d %(message)s') #定义输出格式
fh.setFormatter(formatter) #添加格式化输出
logger.addHandler(fh)
logger.warning("test1")
logger.warning("test2")
logger.warning("test3")
logger.warning("test4")
按照时间自动截断并保存指定文件个数的案例
7.shutil模块3
改模块可以处理高级的 文件、文件夹、压缩包
1>.将文件内容拷贝到另一个文件中，可以部分内容
def copyfileobj(fsrc, fdst, length=16*1024): #需要制定一个源文件，目标文件，以及每次读取的长度.
"""copy data from file-like object fsrc to file-like object fdst"""
while 1:
buf = fsrc.read(length)
if not buf:
break
fdst.write(buf)
shutil.copyfileobj函数的源代码
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import shutil
f_1 = open("file_1","r+",encoding="utf-8")
f_2 = open("file_2","a+",encoding="utf-8") #如果这里是“r+”的方式打开的话那么在下面调用copyfileobj函数的时候，这个文件会被重定向的哟！（也就是之前的内容会被覆盖掉）
shutil.copyfileobj(f_1,f_2) #将f_1文件的内容追加到f_2文件中.
调用方法展示
2>.拷贝文件
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
def copyfile(src, dst):
"""Copy data from src to dst"""
if _samefile(src, dst):
raise Error("`%s` and `%s` are the same file" % (src, dst)) for fn in [src, dst]:
try:
st = os.stat(fn)
except OSError:
# File most likely does not exist
pass
else:
# XXX What about other special files? (sockets, devices...)
if stat.S_ISFIFO(st.st_mode):
raise SpecialFileError("`%s` is a named pipe" % fn) with open(src, 'rb') as fsrc:
with open(dst, 'wb') as fdst:
copyfileobj(fsrc, fdst)
shutil.copyfile函数的源代码
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import shutil
shutil.copyfile("file_1","file_2") #直接写源文件和目标文件，不需要像上面那样费劲的打开一个文件了，如果没有文件就创建一个，如果有的话就会直接覆盖源文件的内容有哟
调用方法展示
3>.仅拷贝权限。内容、组、用户均不变
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
def copymode(src, dst):
"""Copy mode bits from src to dst"""
if hasattr(os, 'chmod'):
st = os.stat(src)
mode = stat.S_IMODE(st.st_mode)
os.chmod(dst, mode)
shutil.copymode函数的源代码
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import shutil
shutil.copymode("file_1","file_3") #首先，这2个文件必须存在，仅仅拷贝的是权限！要注意哟！
调用方法展示
4>.拷贝状态的信息，包括：mode bits, atime, mtime, flags
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
def copystat(src, dst):
"""Copy all stat info (mode bits, atime, mtime, flags) from src to dst"""
st = os.stat(src)
mode = stat.S_IMODE(st.st_mode)
if hasattr(os, 'utime'):
os.utime(dst, (st.st_atime, st.st_mtime))
if hasattr(os, 'chmod'):
os.chmod(dst, mode)
if hasattr(os, 'chflags') and hasattr(st, 'st_flags'):
try:
os.chflags(dst, st.st_flags)
except OSError, why:
for err in 'EOPNOTSUPP', 'ENOTSUP':
if hasattr(errno, err) and why.errno == getattr(errno, err):
break
else:
raise
shutil.copystat函数的源代码
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import shutil
shutil.copystat("file_1","file_3") #将前面的状态信息拷贝给后面的文件。
调用方法展示
5>.拷贝文件和权限
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
def copy(src, dst):
"""Copy data and mode bits ("cp src dst"). The destination may be a directory. """
if os.path.isdir(dst):
dst = os.path.join(dst, os.path.basename(src))
copyfile(src, dst)
copymode(src, dst)
shutil.copy函数的源代码
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import shutil
shutil.copy("file_1","file_11") #这个就是拷贝一个的内容还有权限一起拷贝
调用方法展示
6>.拷贝文件和状态信息
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
def copy2(src, dst):
"""Copy data and all stat info ("cp -p src dst"). The destination may be a directory. """
if os.path.isdir(dst):
dst = os.path.join(dst, os.path.basename(src))
copyfile(src, dst)
copystat(src, dst)
shutil.copy2函数的源代码
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import shutil
shutil.copy2("file_1","file_22") #这个就是拷贝一个的内容还有状态信息也一并拷贝
调用方法展示
7>.递归的去拷贝文件
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
def ignore_patterns(*patterns):
"""Function that can be used as copytree() ignore parameter. Patterns is a sequence of glob-style patterns
that are used to exclude files"""
def _ignore_patterns(path, names):
ignored_names = []
for pattern in patterns:
ignored_names.extend(fnmatch.filter(names, pattern))
return set(ignored_names)
return _ignore_patterns def copytree(src, dst, symlinks=False, ignore=None):
"""Recursively copy a directory tree using copy2(). The destination directory must not already exist.
If exception(s) occur, an Error is raised with a list of reasons. If the optional symlinks flag is true, symbolic links in the
source tree result in symbolic links in the destination tree; if
it is false, the contents of the files pointed to by symbolic
links are copied. The optional ignore argument is a callable. If given, it
is called with the `src` parameter, which is the directory
being visited by copytree(), and `names` which is the list of
`src` contents, as returned by os.listdir(): callable(src, names) -> ignored_names Since copytree() is called recursively, the callable will be
called once for each directory that is copied. It returns a
list of names relative to the `src` directory that should
not be copied. XXX Consider this example code rather than the ultimate tool. """
names = os.listdir(src)
if ignore is not None:
ignored_names = ignore(src, names)
else:
ignored_names = set() os.makedirs(dst)
errors = []
for name in names:
if name in ignored_names:
continue
srcname = os.path.join(src, name)
dstname = os.path.join(dst, name)
try:
if symlinks and os.path.islink(srcname):
linkto = os.readlink(srcname)
os.symlink(linkto, dstname)
elif os.path.isdir(srcname):
copytree(srcname, dstname, symlinks, ignore)
else:
# Will raise a SpecialFileError for unsupported file types
copy2(srcname, dstname)
# catch the Error from the recursive copytree so that we can
# continue with other files
except Error, err:
errors.extend(err.args[0])
except EnvironmentError, why:
errors.append((srcname, dstname, str(why)))
try:
copystat(src, dst)
except OSError, why:
if WindowsError is not None and isinstance(why, WindowsError):
# Copying file access times may fail on Windows
pass
else:
errors.append((src, dst, str(why)))
if errors:
raise Error, errors
shutil.ignore_patterns与shutil.copytree函数的源代码
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import shutil
shutil.copytree(r"D:\python\daima\DAY5",r"D:\python\daima\DAY6\test",ignore=shutil.ignore_patterns("atm","*.log")) #需要输入原路径，目标路径，利用ignore函数可以对需要拷贝的东西进行过滤，我这里过滤掉源目录中所有的包含“atm”，“*log”关键字的目录或者文件。（换句话说，就是新拷贝的路径中，不包含被过滤掉的文件信息。）
调用方法展示
8>.递归的去删除文件
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
def rmtree(path, ignore_errors=False, onerror=None):
"""Recursively delete a directory tree. If ignore_errors is set, errors are ignored; otherwise, if onerror
is set, it is called to handle the error with arguments (func,
path, exc_info) where func is os.listdir, os.remove, or os.rmdir;
path is the argument to that function that caused it to fail; and
exc_info is a tuple returned by sys.exc_info(). If ignore_errors
is false and onerror is None, an exception is raised. """
if ignore_errors:
def onerror(*args):
pass
elif onerror is None:
def onerror(*args):
raise
try:
if os.path.islink(path):
# symlinks to directories are forbidden, see bug #1669
raise OSError("Cannot call rmtree on a symbolic link")
except OSError:
onerror(os.path.islink, path, sys.exc_info())
# can't continue even if onerror hook returns
return
names = []
try:
names = os.listdir(path)
except os.error, err:
onerror(os.listdir, path, sys.exc_info())
for name in names:
fullname = os.path.join(path, name)
try:
mode = os.lstat(fullname).st_mode
except os.error:
mode = 0
if stat.S_ISDIR(mode):
rmtree(fullname, ignore_errors, onerror)
else:
try:
os.remove(fullname)
except os.error, err:
onerror(os.remove, fullname, sys.exc_info())
try:
os.rmdir(path)
except os.error:
onerror(os.rmdir, path, sys.exc_info())
shutil.rmtree函数的源代码
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import shutil
shutil.rmtree(r"D:\python\daima\DAY6\test")
调用方法展示
9>.递归的去移动文件
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
def move(src, dst):
"""Recursively move a file or directory to another location. This is
similar to the Unix "mv" command. If the destination is a directory or a symlink to a directory, the source
is moved inside the directory. The destination path must not already
exist. If the destination already exists but is not a directory, it may be
overwritten depending on os.rename() semantics. If the destination is on our current filesystem, then rename() is used.
Otherwise, src is copied to the destination and then removed.
A lot more could be done here... A look at a mv.c shows a lot of
the issues this implementation glosses over. """
real_dst = dst
if os.path.isdir(dst):
if _samefile(src, dst):
# We might be on a case insensitive filesystem,
# perform the rename anyway.
os.rename(src, dst)
return real_dst = os.path.join(dst, _basename(src))
if os.path.exists(real_dst):
raise Error, "Destination path '%s' already exists" % real_dst
try:
os.rename(src, real_dst)
except OSError:
if os.path.isdir(src):
if _destinsrc(src, dst):
raise Error, "Cannot move a directory '%s' into itself '%s'." % (src, dst)
copytree(src, real_dst, symlinks=True)
rmtree(src)
else:
copy2(src, real_dst)
os.unlink(src)
shutil.move函数源代码如下
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import shutil
shutil.move(r"D:\python\daima\DAY5\atm","D:\python\daima\DAY6") #前面是原路径，后面是默认路径
调用方法展示
10>.创建压缩包并返回文件路径，例如：zip、tar(改方法其实质上就是调用的zipfile和tarfile函数的，可以了解一下这2个函数的用法)
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import shutil
shutil.make_archive("day5","zip",r"D:\python\daima\DAY5") #第一个参数需要传入归档后的文件名称，可以指定绝对路径，第二个参数表示打包的类型，tar表示归档不压缩，但是zip表示压缩，我这里用了压缩类型，第三个参数传递的是被被压缩的对象。还可以传递所属者，所属组等等。
'''
补充：
base_name： 压缩包的文件名，也可以是压缩包的路径。只是文件名时，则保存至当前目录，否则保存至指定路径，
如：www =>保存至当前路径
如：/Users/yinzhengjie/www =>保存至/Users/yinzhengjie/
format： 压缩包种类，“zip”, “tar”, “bztar”，“gztar”
root_dir： 要压缩的文件夹路径（默认当前目录）
owner： 用户，默认当前用户
group： 组，默认当前组
logger： 用于记录日志，通常是logging.Logger对象
'''
将目录打包的案例
11>.zipfile用法扩充
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
class ZipFile(object):
""" Class with methods to open, read, write, close, list zip files. z = ZipFile(file, mode="r", compression=ZIP_STORED, allowZip64=False) file: Either the path to the file, or a file-like object.
If it is a path, the file will be opened and closed by ZipFile.
mode: The mode can be either read "r", write "w" or append "a".
compression: ZIP_STORED (no compression) or ZIP_DEFLATED (requires zlib).
allowZip64: if True ZipFile will create files with ZIP64 extensions when
needed, otherwise it will raise an exception when this would
be necessary. """ fp = None # Set here since __del__ checks it def __init__(self, file, mode="r", compression=ZIP_STORED, allowZip64=False):
"""Open the ZIP file with mode read "r", write "w" or append "a"."""
if mode not in ("r", "w", "a"):
raise RuntimeError('ZipFile() requires mode "r", "w", or "a"') if compression == ZIP_STORED:
pass
elif compression == ZIP_DEFLATED:
if not zlib:
raise RuntimeError,\
"Compression requires the (missing) zlib module"
else:
raise RuntimeError, "That compression method is not supported" self._allowZip64 = allowZip64
self._didModify = False
self.debug = 0 # Level of printing: 0 through 3
self.NameToInfo = {} # Find file info given name
self.filelist = [] # List of ZipInfo instances for archive
self.compression = compression # Method of compression
self.mode = key = mode.replace('b', '')[0]
self.pwd = None
self._comment = '' # Check if we were passed a file-like object
if isinstance(file, basestring):
self._filePassed = 0
self.filename = file
modeDict = {'r' : 'rb', 'w': 'wb', 'a' : 'r+b'}
try:
self.fp = open(file, modeDict[mode])
except IOError:
if mode == 'a':
mode = key = 'w'
self.fp = open(file, modeDict[mode])
else:
raise
else:
self._filePassed = 1
self.fp = file
self.filename = getattr(file, 'name', None) try:
if key == 'r':
self._RealGetContents()
elif key == 'w':
# set the modified flag so central directory gets written
# even if no files are added to the archive
self._didModify = True
elif key == 'a':
try:
# See if file is a zip file
self._RealGetContents()
# seek to start of directory and overwrite
self.fp.seek(self.start_dir, 0)
except BadZipfile:
# file is not a zip file, just append
self.fp.seek(0, 2) # set the modified flag so central directory gets written
# even if no files are added to the archive
self._didModify = True
else:
raise RuntimeError('Mode must be "r", "w" or "a"')
except:
fp = self.fp
self.fp = None
if not self._filePassed:
fp.close()
raise def __enter__(self):
return self def __exit__(self, type, value, traceback):
self.close() def _RealGetContents(self):
"""Read in the table of contents for the ZIP file."""
fp = self.fp
try:
endrec = _EndRecData(fp)
except IOError:
raise BadZipfile("File is not a zip file")
if not endrec:
raise BadZipfile, "File is not a zip file"
if self.debug > 1:
print endrec
size_cd = endrec[_ECD_SIZE] # bytes in central directory
offset_cd = endrec[_ECD_OFFSET] # offset of central directory
self._comment = endrec[_ECD_COMMENT] # archive comment # "concat" is zero, unless zip was concatenated to another file
concat = endrec[_ECD_LOCATION] - size_cd - offset_cd
if endrec[_ECD_SIGNATURE] == stringEndArchive64:
# If Zip64 extension structures are present, account for them
concat -= (sizeEndCentDir64 + sizeEndCentDir64Locator) if self.debug > 2:
inferred = concat + offset_cd
print "given, inferred, offset", offset_cd, inferred, concat
# self.start_dir: Position of start of central directory
self.start_dir = offset_cd + concat
fp.seek(self.start_dir, 0)
data = fp.read(size_cd)
fp = cStringIO.StringIO(data)
total = 0
while total < size_cd:
centdir = fp.read(sizeCentralDir)
if len(centdir) != sizeCentralDir:
raise BadZipfile("Truncated central directory")
centdir = struct.unpack(structCentralDir, centdir)
if centdir[_CD_SIGNATURE] != stringCentralDir:
raise BadZipfile("Bad magic number for central directory")
if self.debug > 2:
print centdir
filename = fp.read(centdir[_CD_FILENAME_LENGTH])
# Create ZipInfo instance to store file information
x = ZipInfo(filename)
x.extra = fp.read(centdir[_CD_EXTRA_FIELD_LENGTH])
x.comment = fp.read(centdir[_CD_COMMENT_LENGTH])
x.header_offset = centdir[_CD_LOCAL_HEADER_OFFSET]
(x.create_version, x.create_system, x.extract_version, x.reserved,
x.flag_bits, x.compress_type, t, d,
x.CRC, x.compress_size, x.file_size) = centdir[1:12]
x.volume, x.internal_attr, x.external_attr = centdir[15:18]
# Convert date/time code to (year, month, day, hour, min, sec)
x._raw_time = t
x.date_time = ( (d>>9)+1980, (d>>5)&0xF, d&0x1F,
t>>11, (t>>5)&0x3F, (t&0x1F) * 2 ) x._decodeExtra()
x.header_offset = x.header_offset + concat
x.filename = x._decodeFilename()
self.filelist.append(x)
self.NameToInfo[x.filename] = x # update total bytes read from central directory
total = (total + sizeCentralDir + centdir[_CD_FILENAME_LENGTH]
+ centdir[_CD_EXTRA_FIELD_LENGTH]
+ centdir[_CD_COMMENT_LENGTH]) if self.debug > 2:
print "total", total def namelist(self):
"""Return a list of file names in the archive."""
l = []
for data in self.filelist:
l.append(data.filename)
return l def infolist(self):
"""Return a list of class ZipInfo instances for files in the
archive."""
return self.filelist def printdir(self):
"""Print a table of contents for the zip file."""
print "%-46s %19s %12s" % ("File Name", "Modified ", "Size")
for zinfo in self.filelist:
date = "%d-%02d-%02d %02d:%02d:%02d" % zinfo.date_time[:6]
print "%-46s %s %12d" % (zinfo.filename, date, zinfo.file_size) def testzip(self):
"""Read all the files and check the CRC."""
chunk_size = 2 ** 20
for zinfo in self.filelist:
try:
# Read by chunks, to avoid an OverflowError or a
# MemoryError with very large embedded files.
with self.open(zinfo.filename, "r") as f:
while f.read(chunk_size): # Check CRC-32
pass
except BadZipfile:
return zinfo.filename def getinfo(self, name):
"""Return the instance of ZipInfo given 'name'."""
info = self.NameToInfo.get(name)
if info is None:
raise KeyError(
'There is no item named %r in the archive' % name) return info def setpassword(self, pwd):
"""Set default password for encrypted files."""
self.pwd = pwd @property
def comment(self):
"""The comment text associated with the ZIP file."""
return self._comment @comment.setter
def comment(self, comment):
# check for valid comment length
if len(comment) > ZIP_MAX_COMMENT:
import warnings
warnings.warn('Archive comment is too long; truncating to %d bytes'
% ZIP_MAX_COMMENT, stacklevel=2)
comment = comment[:ZIP_MAX_COMMENT]
self._comment = comment
self._didModify = True def read(self, name, pwd=None):
"""Return file bytes (as a string) for name."""
return self.open(name, "r", pwd).read() def open(self, name, mode="r", pwd=None):
"""Return file-like object for 'name'."""
if mode not in ("r", "U", "rU"):
raise RuntimeError, 'open() requires mode "r", "U", or "rU"'
if not self.fp:
raise RuntimeError, \
"Attempt to read ZIP archive that was already closed" # Only open a new file for instances where we were not
# given a file object in the constructor
if self._filePassed:
zef_file = self.fp
should_close = False
else:
zef_file = open(self.filename, 'rb')
should_close = True try:
# Make sure we have an info object
if isinstance(name, ZipInfo):
# 'name' is already an info object
zinfo = name
else:
# Get info object for name
zinfo = self.getinfo(name) zef_file.seek(zinfo.header_offset, 0) # Skip the file header:
fheader = zef_file.read(sizeFileHeader)
if len(fheader) != sizeFileHeader:
raise BadZipfile("Truncated file header")
fheader = struct.unpack(structFileHeader, fheader)
if fheader[_FH_SIGNATURE] != stringFileHeader:
raise BadZipfile("Bad magic number for file header") fname = zef_file.read(fheader[_FH_FILENAME_LENGTH])
if fheader[_FH_EXTRA_FIELD_LENGTH]:
zef_file.read(fheader[_FH_EXTRA_FIELD_LENGTH]) if fname != zinfo.orig_filename:
raise BadZipfile, \
'File name in directory "%s" and header "%s" differ.' % (
zinfo.orig_filename, fname) # check for encrypted flag & handle password
is_encrypted = zinfo.flag_bits & 0x1
zd = None
if is_encrypted:
if not pwd:
pwd = self.pwd
if not pwd:
raise RuntimeError, "File %s is encrypted, " \
"password required for extraction" % name zd = _ZipDecrypter(pwd)
# The first 12 bytes in the cypher stream is an encryption header
# used to strengthen the algorithm. The first 11 bytes are
# completely random, while the 12th contains the MSB of the CRC,
# or the MSB of the file time depending on the header type
# and is used to check the correctness of the password.
bytes = zef_file.read(12)
h = map(zd, bytes[0:12])
if zinfo.flag_bits & 0x8:
# compare against the file type from extended local headers
check_byte = (zinfo._raw_time >> 8) & 0xff
else:
# compare against the CRC otherwise
check_byte = (zinfo.CRC >> 24) & 0xff
if ord(h[11]) != check_byte:
raise RuntimeError("Bad password for file", name) return ZipExtFile(zef_file, mode, zinfo, zd,
close_fileobj=should_close)
except:
if should_close:
zef_file.close()
raise def extract(self, member, path=None, pwd=None):
"""Extract a member from the archive to the current working directory,
using its full name. Its file information is extracted as accurately
as possible. `member' may be a filename or a ZipInfo object. You can
specify a different directory using `path'.
"""
if not isinstance(member, ZipInfo):
member = self.getinfo(member) if path is None:
path = os.getcwd() return self._extract_member(member, path, pwd) def extractall(self, path=None, members=None, pwd=None):
"""Extract all members from the archive to the current working
directory. `path' specifies a different directory to extract to.
`members' is optional and must be a subset of the list returned
by namelist().
"""
if members is None:
members = self.namelist() for zipinfo in members:
self.extract(zipinfo, path, pwd) def _extract_member(self, member, targetpath, pwd):
"""Extract the ZipInfo object 'member' to a physical
file on the path targetpath.
"""
# build the destination pathname, replacing
# forward slashes to platform specific separators.
arcname = member.filename.replace('/', os.path.sep) if os.path.altsep:
arcname = arcname.replace(os.path.altsep, os.path.sep)
# interpret absolute pathname as relative, remove drive letter or
# UNC path, redundant separators, "." and ".." components.
arcname = os.path.splitdrive(arcname)[1]
arcname = os.path.sep.join(x for x in arcname.split(os.path.sep)
if x not in ('', os.path.curdir, os.path.pardir))
if os.path.sep == '\\':
# filter illegal characters on Windows
illegal = ':<>|"?*'
if isinstance(arcname, unicode):
table = {ord(c): ord('_') for c in illegal}
else:
table = string.maketrans(illegal, '_' * len(illegal))
arcname = arcname.translate(table)
# remove trailing dots
arcname = (x.rstrip('.') for x in arcname.split(os.path.sep))
arcname = os.path.sep.join(x for x in arcname if x) targetpath = os.path.join(targetpath, arcname)
targetpath = os.path.normpath(targetpath) # Create all upper directories if necessary.
upperdirs = os.path.dirname(targetpath)
if upperdirs and not os.path.exists(upperdirs):
os.makedirs(upperdirs) if member.filename[-1] == '/':
if not os.path.isdir(targetpath):
os.mkdir(targetpath)
return targetpath with self.open(member, pwd=pwd) as source, \
file(targetpath, "wb") as target:
shutil.copyfileobj(source, target) return targetpath def _writecheck(self, zinfo):
"""Check for errors before writing a file to the archive."""
if zinfo.filename in self.NameToInfo:
import warnings
warnings.warn('Duplicate name: %r' % zinfo.filename, stacklevel=3)
if self.mode not in ("w", "a"):
raise RuntimeError, 'write() requires mode "w" or "a"'
if not self.fp:
raise RuntimeError, \
"Attempt to write ZIP archive that was already closed"
if zinfo.compress_type == ZIP_DEFLATED and not zlib:
raise RuntimeError, \
"Compression requires the (missing) zlib module"
if zinfo.compress_type not in (ZIP_STORED, ZIP_DEFLATED):
raise RuntimeError, \
"That compression method is not supported"
if not self._allowZip64:
requires_zip64 = None
if len(self.filelist) >= ZIP_FILECOUNT_LIMIT:
requires_zip64 = "Files count"
elif zinfo.file_size > ZIP64_LIMIT:
requires_zip64 = "Filesize"
elif zinfo.header_offset > ZIP64_LIMIT:
requires_zip64 = "Zipfile size"
if requires_zip64:
raise LargeZipFile(requires_zip64 +
" would require ZIP64 extensions") def write(self, filename, arcname=None, compress_type=None):
"""Put the bytes from filename into the archive under the name
arcname."""
if not self.fp:
raise RuntimeError(
"Attempt to write to ZIP archive that was already closed") st = os.stat(filename)
isdir = stat.S_ISDIR(st.st_mode)
mtime = time.localtime(st.st_mtime)
date_time = mtime[0:6]
# Create ZipInfo instance to store file information
if arcname is None:
arcname = filename
arcname = os.path.normpath(os.path.splitdrive(arcname)[1])
while arcname[0] in (os.sep, os.altsep):
arcname = arcname[1:]
if isdir:
arcname += '/'
zinfo = ZipInfo(arcname, date_time)
zinfo.external_attr = (st[0] & 0xFFFF) << 16L # Unix attributes
if compress_type is None:
zinfo.compress_type = self.compression
else:
zinfo.compress_type = compress_type zinfo.file_size = st.st_size
zinfo.flag_bits = 0x00
zinfo.header_offset = self.fp.tell() # Start of header bytes self._writecheck(zinfo)
self._didModify = True if isdir:
zinfo.file_size = 0
zinfo.compress_size = 0
zinfo.CRC = 0
zinfo.external_attr |= 0x10 # MS-DOS directory flag
self.filelist.append(zinfo)
self.NameToInfo[zinfo.filename] = zinfo
self.fp.write(zinfo.FileHeader(False))
return with open(filename, "rb") as fp:
# Must overwrite CRC and sizes with correct data later
zinfo.CRC = CRC = 0
zinfo.compress_size = compress_size = 0
# Compressed size can be larger than uncompressed size
zip64 = self._allowZip64 and \
zinfo.file_size * 1.05 > ZIP64_LIMIT
self.fp.write(zinfo.FileHeader(zip64))
if zinfo.compress_type == ZIP_DEFLATED:
cmpr = zlib.compressobj(zlib.Z_DEFAULT_COMPRESSION,
zlib.DEFLATED, -15)
else:
cmpr = None
file_size = 0
while 1:
buf = fp.read(1024 * 8)
if not buf:
break
file_size = file_size + len(buf)
CRC = crc32(buf, CRC) & 0xffffffff
if cmpr:
buf = cmpr.compress(buf)
compress_size = compress_size + len(buf)
self.fp.write(buf)
if cmpr:
buf = cmpr.flush()
compress_size = compress_size + len(buf)
self.fp.write(buf)
zinfo.compress_size = compress_size
else:
zinfo.compress_size = file_size
zinfo.CRC = CRC
zinfo.file_size = file_size
if not zip64 and self._allowZip64:
if file_size > ZIP64_LIMIT:
raise RuntimeError('File size has increased during compressing')
if compress_size > ZIP64_LIMIT:
raise RuntimeError('Compressed size larger than uncompressed size')
# Seek backwards and write file header (which will now include
# correct CRC and file sizes)
position = self.fp.tell() # Preserve current position in file
self.fp.seek(zinfo.header_offset, 0)
self.fp.write(zinfo.FileHeader(zip64))
self.fp.seek(position, 0)
self.filelist.append(zinfo)
self.NameToInfo[zinfo.filename] = zinfo def writestr(self, zinfo_or_arcname, bytes, compress_type=None):
"""Write a file into the archive. The contents is the string
'bytes'. 'zinfo_or_arcname' is either a ZipInfo instance or
the name of the file in the archive."""
if not isinstance(zinfo_or_arcname, ZipInfo):
zinfo = ZipInfo(filename=zinfo_or_arcname,
date_time=time.localtime(time.time())[:6]) zinfo.compress_type = self.compression
if zinfo.filename[-1] == '/':
zinfo.external_attr = 0o40775 << 16 # drwxrwxr-x
zinfo.external_attr |= 0x10 # MS-DOS directory flag
else:
zinfo.external_attr = 0o600 << 16 # ?rw-------
else:
zinfo = zinfo_or_arcname if not self.fp:
raise RuntimeError(
"Attempt to write to ZIP archive that was already closed") if compress_type is not None:
zinfo.compress_type = compress_type zinfo.file_size = len(bytes) # Uncompressed size
zinfo.header_offset = self.fp.tell() # Start of header bytes
self._writecheck(zinfo)
self._didModify = True
zinfo.CRC = crc32(bytes) & 0xffffffff # CRC-32 checksum
if zinfo.compress_type == ZIP_DEFLATED:
co = zlib.compressobj(zlib.Z_DEFAULT_COMPRESSION,
zlib.DEFLATED, -15)
bytes = co.compress(bytes) + co.flush()
zinfo.compress_size = len(bytes) # Compressed size
else:
zinfo.compress_size = zinfo.file_size
zip64 = zinfo.file_size > ZIP64_LIMIT or \
zinfo.compress_size > ZIP64_LIMIT
if zip64 and not self._allowZip64:
raise LargeZipFile("Filesize would require ZIP64 extensions")
self.fp.write(zinfo.FileHeader(zip64))
self.fp.write(bytes)
if zinfo.flag_bits & 0x08:
# Write CRC and file sizes after the file data
fmt = '<LQQ' if zip64 else '<LLL'
self.fp.write(struct.pack(fmt, zinfo.CRC, zinfo.compress_size,
zinfo.file_size))
self.fp.flush()
self.filelist.append(zinfo)
self.NameToInfo[zinfo.filename] = zinfo def __del__(self):
"""Call the "close()" method in case the user forgot."""
self.close() def close(self):
"""Close the file, and for mode "w" and "a" write the ending
records."""
if self.fp is None:
return try:
if self.mode in ("w", "a") and self._didModify: # write ending records
pos1 = self.fp.tell()
for zinfo in self.filelist: # write central directory
dt = zinfo.date_time
dosdate = (dt[0] - 1980) << 9 | dt[1] << 5 | dt[2]
dostime = dt[3] << 11 | dt[4] << 5 | (dt[5] // 2)
extra = []
if zinfo.file_size > ZIP64_LIMIT \
or zinfo.compress_size > ZIP64_LIMIT:
extra.append(zinfo.file_size)
extra.append(zinfo.compress_size)
file_size = 0xffffffff
compress_size = 0xffffffff
else:
file_size = zinfo.file_size
compress_size = zinfo.compress_size if zinfo.header_offset > ZIP64_LIMIT:
extra.append(zinfo.header_offset)
header_offset = 0xffffffffL
else:
header_offset = zinfo.header_offset extra_data = zinfo.extra
if extra:
# Append a ZIP64 field to the extra's
extra_data = struct.pack(
'<HH' + 'Q'*len(extra),
1, 8*len(extra), *extra) + extra_data extract_version = max(45, zinfo.extract_version)
create_version = max(45, zinfo.create_version)
else:
extract_version = zinfo.extract_version
create_version = zinfo.create_version try:
filename, flag_bits = zinfo._encodeFilenameFlags()
centdir = struct.pack(structCentralDir,
stringCentralDir, create_version,
zinfo.create_system, extract_version, zinfo.reserved,
flag_bits, zinfo.compress_type, dostime, dosdate,
zinfo.CRC, compress_size, file_size,
len(filename), len(extra_data), len(zinfo.comment),
0, zinfo.internal_attr, zinfo.external_attr,
header_offset)
except DeprecationWarning:
print >>sys.stderr, (structCentralDir,
stringCentralDir, create_version,
zinfo.create_system, extract_version, zinfo.reserved,
zinfo.flag_bits, zinfo.compress_type, dostime, dosdate,
zinfo.CRC, compress_size, file_size,
len(zinfo.filename), len(extra_data), len(zinfo.comment),
0, zinfo.internal_attr, zinfo.external_attr,
header_offset)
raise
self.fp.write(centdir)
self.fp.write(filename)
self.fp.write(extra_data)
self.fp.write(zinfo.comment) pos2 = self.fp.tell()
# Write end-of-zip-archive record
centDirCount = len(self.filelist)
centDirSize = pos2 - pos1
centDirOffset = pos1
requires_zip64 = None
if centDirCount > ZIP_FILECOUNT_LIMIT:
requires_zip64 = "Files count"
elif centDirOffset > ZIP64_LIMIT:
requires_zip64 = "Central directory offset"
elif centDirSize > ZIP64_LIMIT:
requires_zip64 = "Central directory size"
if requires_zip64:
# Need to write the ZIP64 end-of-archive records
if not self._allowZip64:
raise LargeZipFile(requires_zip64 +
" would require ZIP64 extensions")
zip64endrec = struct.pack(
structEndArchive64, stringEndArchive64,
44, 45, 45, 0, 0, centDirCount, centDirCount,
centDirSize, centDirOffset)
self.fp.write(zip64endrec) zip64locrec = struct.pack(
structEndArchive64Locator,
stringEndArchive64Locator, 0, pos2, 1)
self.fp.write(zip64locrec)
centDirCount = min(centDirCount, 0xFFFF)
centDirSize = min(centDirSize, 0xFFFFFFFF)
centDirOffset = min(centDirOffset, 0xFFFFFFFF) endrec = struct.pack(structEndArchive, stringEndArchive,
0, 0, centDirCount, centDirCount,
centDirSize, centDirOffset, len(self._comment))
self.fp.write(endrec)
self.fp.write(self._comment)
self.fp.flush()
finally:
fp = self.fp
self.fp = None
if not self._filePassed:
fp.close()
zipfile的源代码
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import zipfile
z = zipfile.ZipFile("ziptest.zip","w") #创建一个叫”ziptest.zip“的压缩文件
z.write(r"D:\python\daima\DAY5\README",arcname="README") #将该文件放入到”ziptest.zip“的压缩文件中,后面的arcname参数的意思是指压缩这个文件的即可，不用压缩这个文件的所在的绝对路径。如果不加这个参数的话，会把该文件的当前位置的绝对路径都一起压缩了
z.write(r"D:\python\daima\DAY3\modify.txt",arcname="modify.txt") #同上
z.write("day5.zip") #将当前路径的文件压缩到”ziptest.zip“的压缩文件中
z.close() #关闭压缩文件，这个时候就可以将文件存入进去了。
zipfile压缩的用法
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import zipfile
z = zipfile.ZipFile("ziptest.zip","r")
z.extract("README") #进行解压一个叫“README”的文件
z.extractall(path=r"D:\python\daima\DAY5\test_1") #将z这个压缩包的内容全部解压出来，path表示执行解压后的存放路径
z.extractall(members=["modify.txt"]) #表示将z这个压缩包全部解压出来，出了列表中的文件，注意不支持模糊匹配哟！
z.close()
zipfile解压的用法
12>.tarfile用法扩充
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
class TarFile(object):
"""The TarFile Class provides an interface to tar archives.
""" debug = 0 # May be set from 0 (no msgs) to 3 (all msgs) dereference = False # If true, add content of linked file to the
# tar file, else the link. ignore_zeros = False # If true, skips empty or invalid blocks and
# continues processing. errorlevel = 1 # If 0, fatal errors only appear in debug
# messages (if debug >= 0). If > 0, errors
# are passed to the caller as exceptions. format = DEFAULT_FORMAT # The format to use when creating an archive. encoding = ENCODING # Encoding for 8-bit character strings. errors = None # Error handler for unicode conversion. tarinfo = TarInfo # The default TarInfo class to use. fileobject = ExFileObject # The default ExFileObject class to use. def __init__(self, name=None, mode="r", fileobj=None, format=None,
tarinfo=None, dereference=None, ignore_zeros=None, encoding=None,
errors=None, pax_headers=None, debug=None, errorlevel=None):
"""Open an (uncompressed) tar archive `name'. `mode' is either 'r' to
read from an existing archive, 'a' to append data to an existing
file or 'w' to create a new file overwriting an existing one. `mode'
defaults to 'r'.
If `fileobj' is given, it is used for reading or writing data. If it
can be determined, `mode' is overridden by `fileobj's mode.
`fileobj' is not closed, when TarFile is closed.
"""
modes = {"r": "rb", "a": "r+b", "w": "wb"}
if mode not in modes:
raise ValueError("mode must be 'r', 'a' or 'w'")
self.mode = mode
self._mode = modes[mode] if not fileobj:
if self.mode == "a" and not os.path.exists(name):
# Create nonexistent files in append mode.
self.mode = "w"
self._mode = "wb"
fileobj = bltn_open(name, self._mode)
self._extfileobj = False
else:
if name is None and hasattr(fileobj, "name"):
name = fileobj.name
if hasattr(fileobj, "mode"):
self._mode = fileobj.mode
self._extfileobj = True
self.name = os.path.abspath(name) if name else None
self.fileobj = fileobj # Init attributes.
if format is not None:
self.format = format
if tarinfo is not None:
self.tarinfo = tarinfo
if dereference is not None:
self.dereference = dereference
if ignore_zeros is not None:
self.ignore_zeros = ignore_zeros
if encoding is not None:
self.encoding = encoding if errors is not None:
self.errors = errors
elif mode == "r":
self.errors = "utf-8"
else:
self.errors = "strict" if pax_headers is not None and self.format == PAX_FORMAT:
self.pax_headers = pax_headers
else:
self.pax_headers = {} if debug is not None:
self.debug = debug
if errorlevel is not None:
self.errorlevel = errorlevel # Init datastructures.
self.closed = False
self.members = [] # list of members as TarInfo objects
self._loaded = False # flag if all members have been read
self.offset = self.fileobj.tell()
# current position in the archive file
self.inodes = {} # dictionary caching the inodes of
# archive members already added try:
if self.mode == "r":
self.firstmember = None
self.firstmember = self.next() if self.mode == "a":
# Move to the end of the archive,
# before the first empty block.
while True:
self.fileobj.seek(self.offset)
try:
tarinfo = self.tarinfo.fromtarfile(self)
self.members.append(tarinfo)
except EOFHeaderError:
self.fileobj.seek(self.offset)
break
except HeaderError, e:
raise ReadError(str(e)) if self.mode in "aw":
self._loaded = True if self.pax_headers:
buf = self.tarinfo.create_pax_global_header(self.pax_headers.copy())
self.fileobj.write(buf)
self.offset += len(buf)
except:
if not self._extfileobj:
self.fileobj.close()
self.closed = True
raise def _getposix(self):
return self.format == USTAR_FORMAT
def _setposix(self, value):
import warnings
warnings.warn("use the format attribute instead", DeprecationWarning,
2)
if value:
self.format = USTAR_FORMAT
else:
self.format = GNU_FORMAT
posix = property(_getposix, _setposix) #--------------------------------------------------------------------------
# Below are the classmethods which act as alternate constructors to the
# TarFile class. The open() method is the only one that is needed for
# public use; it is the "super"-constructor and is able to select an
# adequate "sub"-constructor for a particular compression using the mapping
# from OPEN_METH.
#
# This concept allows one to subclass TarFile without losing the comfort of
# the super-constructor. A sub-constructor is registered and made available
# by adding it to the mapping in OPEN_METH. @classmethod
def open(cls, name=None, mode="r", fileobj=None, bufsize=RECORDSIZE, **kwargs):
"""Open a tar archive for reading, writing or appending. Return
an appropriate TarFile class. mode:
'r' or 'r:*' open for reading with transparent compression
'r:' open for reading exclusively uncompressed
'r:gz' open for reading with gzip compression
'r:bz2' open for reading with bzip2 compression
'a' or 'a:' open for appending, creating the file if necessary
'w' or 'w:' open for writing without compression
'w:gz' open for writing with gzip compression
'w:bz2' open for writing with bzip2 compression 'r|*' open a stream of tar blocks with transparent compression
'r|' open an uncompressed stream of tar blocks for reading
'r|gz' open a gzip compressed stream of tar blocks
'r|bz2' open a bzip2 compressed stream of tar blocks
'w|' open an uncompressed stream for writing
'w|gz' open a gzip compressed stream for writing
'w|bz2' open a bzip2 compressed stream for writing
""" if not name and not fileobj:
raise ValueError("nothing to open") if mode in ("r", "r:*"):
# Find out which *open() is appropriate for opening the file.
for comptype in cls.OPEN_METH:
func = getattr(cls, cls.OPEN_METH[comptype])
if fileobj is not None:
saved_pos = fileobj.tell()
try:
return func(name, "r", fileobj, **kwargs)
except (ReadError, CompressionError), e:
if fileobj is not None:
fileobj.seek(saved_pos)
continue
raise ReadError("file could not be opened successfully") elif ":" in mode:
filemode, comptype = mode.split(":", 1)
filemode = filemode or "r"
comptype = comptype or "tar" # Select the *open() function according to
# given compression.
if comptype in cls.OPEN_METH:
func = getattr(cls, cls.OPEN_METH[comptype])
else:
raise CompressionError("unknown compression type %r" % comptype)
return func(name, filemode, fileobj, **kwargs) elif "|" in mode:
filemode, comptype = mode.split("|", 1)
filemode = filemode or "r"
comptype = comptype or "tar" if filemode not in ("r", "w"):
raise ValueError("mode must be 'r' or 'w'") stream = _Stream(name, filemode, comptype, fileobj, bufsize)
try:
t = cls(name, filemode, stream, **kwargs)
except:
stream.close()
raise
t._extfileobj = False
return t elif mode in ("a", "w"):
return cls.taropen(name, mode, fileobj, **kwargs) raise ValueError("undiscernible mode") @classmethod
def taropen(cls, name, mode="r", fileobj=None, **kwargs):
"""Open uncompressed tar archive name for reading or writing.
"""
if mode not in ("r", "a", "w"):
raise ValueError("mode must be 'r', 'a' or 'w'")
return cls(name, mode, fileobj, **kwargs) @classmethod
def gzopen(cls, name, mode="r", fileobj=None, compresslevel=9, **kwargs):
"""Open gzip compressed tar archive name for reading or writing.
Appending is not allowed.
"""
if mode not in ("r", "w"):
raise ValueError("mode must be 'r' or 'w'") try:
import gzip
gzip.GzipFile
except (ImportError, AttributeError):
raise CompressionError("gzip module is not available") try:
fileobj = gzip.GzipFile(name, mode, compresslevel, fileobj)
except OSError:
if fileobj is not None and mode == 'r':
raise ReadError("not a gzip file")
raise try:
t = cls.taropen(name, mode, fileobj, **kwargs)
except IOError:
fileobj.close()
if mode == 'r':
raise ReadError("not a gzip file")
raise
except:
fileobj.close()
raise
t._extfileobj = False
return t @classmethod
def bz2open(cls, name, mode="r", fileobj=None, compresslevel=9, **kwargs):
"""Open bzip2 compressed tar archive name for reading or writing.
Appending is not allowed.
"""
if mode not in ("r", "w"):
raise ValueError("mode must be 'r' or 'w'.") try:
import bz2
except ImportError:
raise CompressionError("bz2 module is not available") if fileobj is not None:
fileobj = _BZ2Proxy(fileobj, mode)
else:
fileobj = bz2.BZ2File(name, mode, compresslevel=compresslevel) try:
t = cls.taropen(name, mode, fileobj, **kwargs)
except (IOError, EOFError):
fileobj.close()
if mode == 'r':
raise ReadError("not a bzip2 file")
raise
except:
fileobj.close()
raise
t._extfileobj = False
return t # All *open() methods are registered here.
OPEN_METH = {
"tar": "taropen", # uncompressed tar
"gz": "gzopen", # gzip compressed tar
"bz2": "bz2open" # bzip2 compressed tar
} #--------------------------------------------------------------------------
# The public methods which TarFile provides: def close(self):
"""Close the TarFile. In write-mode, two finishing zero blocks are
appended to the archive.
"""
if self.closed:
return if self.mode in "aw":
self.fileobj.write(NUL * (BLOCKSIZE * 2))
self.offset += (BLOCKSIZE * 2)
# fill up the end with zero-blocks
# (like option -b20 for tar does)
blocks, remainder = divmod(self.offset, RECORDSIZE)
if remainder > 0:
self.fileobj.write(NUL * (RECORDSIZE - remainder)) if not self._extfileobj:
self.fileobj.close()
self.closed = True def getmember(self, name):
"""Return a TarInfo object for member `name'. If `name' can not be
found in the archive, KeyError is raised. If a member occurs more
than once in the archive, its last occurrence is assumed to be the
most up-to-date version.
"""
tarinfo = self._getmember(name)
if tarinfo is None:
raise KeyError("filename %r not found" % name)
return tarinfo def getmembers(self):
"""Return the members of the archive as a list of TarInfo objects. The
list has the same order as the members in the archive.
"""
self._check()
if not self._loaded: # if we want to obtain a list of
self._load() # all members, we first have to
# scan the whole archive.
return self.members def getnames(self):
"""Return the members of the archive as a list of their names. It has
the same order as the list returned by getmembers().
"""
return [tarinfo.name for tarinfo in self.getmembers()] def gettarinfo(self, name=None, arcname=None, fileobj=None):
"""Create a TarInfo object for either the file `name' or the file
object `fileobj' (using os.fstat on its file descriptor). You can
modify some of the TarInfo's attributes before you add it using
addfile(). If given, `arcname' specifies an alternative name for the
file in the archive.
"""
self._check("aw") # When fileobj is given, replace name by
# fileobj's real name.
if fileobj is not None:
name = fileobj.name # Building the name of the member in the archive.
# Backward slashes are converted to forward slashes,
# Absolute paths are turned to relative paths.
if arcname is None:
arcname = name
drv, arcname = os.path.splitdrive(arcname)
arcname = arcname.replace(os.sep, "/")
arcname = arcname.lstrip("/") # Now, fill the TarInfo object with
# information specific for the file.
tarinfo = self.tarinfo()
tarinfo.tarfile = self # Use os.stat or os.lstat, depending on platform
# and if symlinks shall be resolved.
if fileobj is None:
if hasattr(os, "lstat") and not self.dereference:
statres = os.lstat(name)
else:
statres = os.stat(name)
else:
statres = os.fstat(fileobj.fileno())
linkname = "" stmd = statres.st_mode
if stat.S_ISREG(stmd):
inode = (statres.st_ino, statres.st_dev)
if not self.dereference and statres.st_nlink > 1 and \
inode in self.inodes and arcname != self.inodes[inode]:
# Is it a hardlink to an already
# archived file?
type = LNKTYPE
linkname = self.inodes[inode]
else:
# The inode is added only if its valid.
# For win32 it is always 0.
type = REGTYPE
if inode[0]:
self.inodes[inode] = arcname
elif stat.S_ISDIR(stmd):
type = DIRTYPE
elif stat.S_ISFIFO(stmd):
type = FIFOTYPE
elif stat.S_ISLNK(stmd):
type = SYMTYPE
linkname = os.readlink(name)
elif stat.S_ISCHR(stmd):
type = CHRTYPE
elif stat.S_ISBLK(stmd):
type = BLKTYPE
else:
return None # Fill the TarInfo object with all
# information we can get.
tarinfo.name = arcname
tarinfo.mode = stmd
tarinfo.uid = statres.st_uid
tarinfo.gid = statres.st_gid
if type == REGTYPE:
tarinfo.size = statres.st_size
else:
tarinfo.size = 0L
tarinfo.mtime = statres.st_mtime
tarinfo.type = type
tarinfo.linkname = linkname
if pwd:
try:
tarinfo.uname = pwd.getpwuid(tarinfo.uid)[0]
except KeyError:
pass
if grp:
try:
tarinfo.gname = grp.getgrgid(tarinfo.gid)[0]
except KeyError:
pass if type in (CHRTYPE, BLKTYPE):
if hasattr(os, "major") and hasattr(os, "minor"):
tarinfo.devmajor = os.major(statres.st_rdev)
tarinfo.devminor = os.minor(statres.st_rdev)
return tarinfo def list(self, verbose=True):
"""Print a table of contents to sys.stdout. If `verbose' is False, only
the names of the members are printed. If it is True, an `ls -l'-like
output is produced.
"""
self._check() for tarinfo in self:
if verbose:
print filemode(tarinfo.mode),
print "%s/%s" % (tarinfo.uname or tarinfo.uid,
tarinfo.gname or tarinfo.gid),
if tarinfo.ischr() or tarinfo.isblk():
print "%10s" % ("%d,%d" \
% (tarinfo.devmajor, tarinfo.devminor)),
else:
print "%10d" % tarinfo.size,
print "%d-%02d-%02d %02d:%02d:%02d" \
% time.localtime(tarinfo.mtime)[:6], print tarinfo.name + ("/" if tarinfo.isdir() else ""), if verbose:
if tarinfo.issym():
print "->", tarinfo.linkname,
if tarinfo.islnk():
print "link to", tarinfo.linkname,
print def add(self, name, arcname=None, recursive=True, exclude=None, filter=None):
"""Add the file `name' to the archive. `name' may be any type of file
(directory, fifo, symbolic link, etc.). If given, `arcname'
specifies an alternative name for the file in the archive.
Directories are added recursively by default. This can be avoided by
setting `recursive' to False. `exclude' is a function that should
return True for each filename to be excluded. `filter' is a function
that expects a TarInfo object argument and returns the changed
TarInfo object, if it returns None the TarInfo object will be
excluded from the archive.
"""
self._check("aw") if arcname is None:
arcname = name # Exclude pathnames.
if exclude is not None:
import warnings
warnings.warn("use the filter argument instead",
DeprecationWarning, 2)
if exclude(name):
self._dbg(2, "tarfile: Excluded %r" % name)
return # Skip if somebody tries to archive the archive...
if self.name is not None and os.path.abspath(name) == self.name:
self._dbg(2, "tarfile: Skipped %r" % name)
return self._dbg(1, name) # Create a TarInfo object from the file.
tarinfo = self.gettarinfo(name, arcname) if tarinfo is None:
self._dbg(1, "tarfile: Unsupported type %r" % name)
return # Change or exclude the TarInfo object.
if filter is not None:
tarinfo = filter(tarinfo)
if tarinfo is None:
self._dbg(2, "tarfile: Excluded %r" % name)
return # Append the tar header and data to the archive.
if tarinfo.isreg():
with bltn_open(name, "rb") as f:
self.addfile(tarinfo, f) elif tarinfo.isdir():
self.addfile(tarinfo)
if recursive:
for f in os.listdir(name):
self.add(os.path.join(name, f), os.path.join(arcname, f),
recursive, exclude, filter) else:
self.addfile(tarinfo) def addfile(self, tarinfo, fileobj=None):
"""Add the TarInfo object `tarinfo' to the archive. If `fileobj' is
given, tarinfo.size bytes are read from it and added to the archive.
You can create TarInfo objects using gettarinfo().
On Windows platforms, `fileobj' should always be opened with mode
'rb' to avoid irritation about the file size.
"""
self._check("aw") tarinfo = copy.copy(tarinfo) buf = tarinfo.tobuf(self.format, self.encoding, self.errors)
self.fileobj.write(buf)
self.offset += len(buf) # If there's data to follow, append it.
if fileobj is not None:
copyfileobj(fileobj, self.fileobj, tarinfo.size)
blocks, remainder = divmod(tarinfo.size, BLOCKSIZE)
if remainder > 0:
self.fileobj.write(NUL * (BLOCKSIZE - remainder))
blocks += 1
self.offset += blocks * BLOCKSIZE self.members.append(tarinfo) def extractall(self, path=".", members=None):
"""Extract all members from the archive to the current working
directory and set owner, modification time and permissions on
directories afterwards. `path' specifies a different directory
to extract to. `members' is optional and must be a subset of the
list returned by getmembers().
"""
directories = [] if members is None:
members = self for tarinfo in members:
if tarinfo.isdir():
# Extract directories with a safe mode.
directories.append(tarinfo)
tarinfo = copy.copy(tarinfo)
tarinfo.mode = 0700
self.extract(tarinfo, path) # Reverse sort directories.
directories.sort(key=operator.attrgetter('name'))
directories.reverse() # Set correct owner, mtime and filemode on directories.
for tarinfo in directories:
dirpath = os.path.join(path, tarinfo.name)
try:
self.chown(tarinfo, dirpath)
self.utime(tarinfo, dirpath)
self.chmod(tarinfo, dirpath)
except ExtractError, e:
if self.errorlevel > 1:
raise
else:
self._dbg(1, "tarfile: %s" % e) def extract(self, member, path=""):
"""Extract a member from the archive to the current working directory,
using its full name. Its file information is extracted as accurately
as possible. `member' may be a filename or a TarInfo object. You can
specify a different directory using `path'.
"""
self._check("r") if isinstance(member, basestring):
tarinfo = self.getmember(member)
else:
tarinfo = member # Prepare the link target for makelink().
if tarinfo.islnk():
tarinfo._link_target = os.path.join(path, tarinfo.linkname) try:
self._extract_member(tarinfo, os.path.join(path, tarinfo.name))
except EnvironmentError, e:
if self.errorlevel > 0:
raise
else:
if e.filename is None:
self._dbg(1, "tarfile: %s" % e.strerror)
else:
self._dbg(1, "tarfile: %s %r" % (e.strerror, e.filename))
except ExtractError, e:
if self.errorlevel > 1:
raise
else:
self._dbg(1, "tarfile: %s" % e) def extractfile(self, member):
"""Extract a member from the archive as a file object. `member' may be
a filename or a TarInfo object. If `member' is a regular file, a
file-like object is returned. If `member' is a link, a file-like
object is constructed from the link's target. If `member' is none of
the above, None is returned.
The file-like object is read-only and provides the following
methods: read(), readline(), readlines(), seek() and tell()
"""
self._check("r") if isinstance(member, basestring):
tarinfo = self.getmember(member)
else:
tarinfo = member if tarinfo.isreg():
return self.fileobject(self, tarinfo) elif tarinfo.type not in SUPPORTED_TYPES:
# If a member's type is unknown, it is treated as a
# regular file.
return self.fileobject(self, tarinfo) elif tarinfo.islnk() or tarinfo.issym():
if isinstance(self.fileobj, _Stream):
# A small but ugly workaround for the case that someone tries
# to extract a (sym)link as a file-object from a non-seekable
# stream of tar blocks.
raise StreamError("cannot extract (sym)link as file object")
else:
# A (sym)link's file object is its target's file object.
return self.extractfile(self._find_link_target(tarinfo))
else:
# If there's no data associated with the member (directory, chrdev,
# blkdev, etc.), return None instead of a file object.
return None def _extract_member(self, tarinfo, targetpath):
"""Extract the TarInfo object tarinfo to a physical
file called targetpath.
"""
# Fetch the TarInfo object for the given name
# and build the destination pathname, replacing
# forward slashes to platform specific separators.
targetpath = targetpath.rstrip("/")
targetpath = targetpath.replace("/", os.sep) # Create all upper directories.
upperdirs = os.path.dirname(targetpath)
if upperdirs and not os.path.exists(upperdirs):
# Create directories that are not part of the archive with
# default permissions.
os.makedirs(upperdirs) if tarinfo.islnk() or tarinfo.issym():
self._dbg(1, "%s -> %s" % (tarinfo.name, tarinfo.linkname))
else:
self._dbg(1, tarinfo.name) if tarinfo.isreg():
self.makefile(tarinfo, targetpath)
elif tarinfo.isdir():
self.makedir(tarinfo, targetpath)
elif tarinfo.isfifo():
self.makefifo(tarinfo, targetpath)
elif tarinfo.ischr() or tarinfo.isblk():
self.makedev(tarinfo, targetpath)
elif tarinfo.islnk() or tarinfo.issym():
self.makelink(tarinfo, targetpath)
elif tarinfo.type not in SUPPORTED_TYPES:
self.makeunknown(tarinfo, targetpath)
else:
self.makefile(tarinfo, targetpath) self.chown(tarinfo, targetpath)
if not tarinfo.issym():
self.chmod(tarinfo, targetpath)
self.utime(tarinfo, targetpath) #--------------------------------------------------------------------------
# Below are the different file methods. They are called via
# _extract_member() when extract() is called. They can be replaced in a
# subclass to implement other functionality. def makedir(self, tarinfo, targetpath):
"""Make a directory called targetpath.
"""
try:
# Use a safe mode for the directory, the real mode is set
# later in _extract_member().
os.mkdir(targetpath, 0700)
except EnvironmentError, e:
if e.errno != errno.EEXIST:
raise def makefile(self, tarinfo, targetpath):
"""Make a file called targetpath.
"""
source = self.extractfile(tarinfo)
try:
with bltn_open(targetpath, "wb") as target:
copyfileobj(source, target)
finally:
source.close() def makeunknown(self, tarinfo, targetpath):
"""Make a file from a TarInfo object with an unknown type
at targetpath.
"""
self.makefile(tarinfo, targetpath)
self._dbg(1, "tarfile: Unknown file type %r, " \
"extracted as regular file." % tarinfo.type) def makefifo(self, tarinfo, targetpath):
"""Make a fifo called targetpath.
"""
if hasattr(os, "mkfifo"):
os.mkfifo(targetpath)
else:
raise ExtractError("fifo not supported by system") def makedev(self, tarinfo, targetpath):
"""Make a character or block device called targetpath.
"""
if not hasattr(os, "mknod") or not hasattr(os, "makedev"):
raise ExtractError("special devices not supported by system") mode = tarinfo.mode
if tarinfo.isblk():
mode |= stat.S_IFBLK
else:
mode |= stat.S_IFCHR os.mknod(targetpath, mode,
os.makedev(tarinfo.devmajor, tarinfo.devminor)) def makelink(self, tarinfo, targetpath):
"""Make a (symbolic) link called targetpath. If it cannot be created
(platform limitation), we try to make a copy of the referenced file
instead of a link.
"""
if hasattr(os, "symlink") and hasattr(os, "link"):
# For systems that support symbolic and hard links.
if tarinfo.issym():
if os.path.lexists(targetpath):
os.unlink(targetpath)
os.symlink(tarinfo.linkname, targetpath)
else:
# See extract().
if os.path.exists(tarinfo._link_target):
if os.path.lexists(targetpath):
os.unlink(targetpath)
os.link(tarinfo._link_target, targetpath)
else:
self._extract_member(self._find_link_target(tarinfo), targetpath)
else:
try:
self._extract_member(self._find_link_target(tarinfo), targetpath)
except KeyError:
raise ExtractError("unable to resolve link inside archive") def chown(self, tarinfo, targetpath):
"""Set owner of targetpath according to tarinfo.
"""
if pwd and hasattr(os, "geteuid") and os.geteuid() == 0:
# We have to be root to do so.
try:
g = grp.getgrnam(tarinfo.gname)[2]
except KeyError:
g = tarinfo.gid
try:
u = pwd.getpwnam(tarinfo.uname)[2]
except KeyError:
u = tarinfo.uid
try:
if tarinfo.issym() and hasattr(os, "lchown"):
os.lchown(targetpath, u, g)
else:
if sys.platform != "os2emx":
os.chown(targetpath, u, g)
except EnvironmentError, e:
raise ExtractError("could not change owner") def chmod(self, tarinfo, targetpath):
"""Set file permissions of targetpath according to tarinfo.
"""
if hasattr(os, 'chmod'):
try:
os.chmod(targetpath, tarinfo.mode)
except EnvironmentError, e:
raise ExtractError("could not change mode") def utime(self, tarinfo, targetpath):
"""Set modification time of targetpath according to tarinfo.
"""
if not hasattr(os, 'utime'):
return
try:
os.utime(targetpath, (tarinfo.mtime, tarinfo.mtime))
except EnvironmentError, e:
raise ExtractError("could not change modification time") #--------------------------------------------------------------------------
def next(self):
"""Return the next member of the archive as a TarInfo object, when
TarFile is opened for reading. Return None if there is no more
available.
"""
self._check("ra")
if self.firstmember is not None:
m = self.firstmember
self.firstmember = None
return m # Read the next block.
self.fileobj.seek(self.offset)
tarinfo = None
while True:
try:
tarinfo = self.tarinfo.fromtarfile(self)
except EOFHeaderError, e:
if self.ignore_zeros:
self._dbg(2, "0x%X: %s" % (self.offset, e))
self.offset += BLOCKSIZE
continue
except InvalidHeaderError, e:
if self.ignore_zeros:
self._dbg(2, "0x%X: %s" % (self.offset, e))
self.offset += BLOCKSIZE
continue
elif self.offset == 0:
raise ReadError(str(e))
except EmptyHeaderError:
if self.offset == 0:
raise ReadError("empty file")
except TruncatedHeaderError, e:
if self.offset == 0:
raise ReadError(str(e))
except SubsequentHeaderError, e:
raise ReadError(str(e))
break if tarinfo is not None:
self.members.append(tarinfo)
else:
self._loaded = True return tarinfo #--------------------------------------------------------------------------
# Little helper methods: def _getmember(self, name, tarinfo=None, normalize=False):
"""Find an archive member by name from bottom to top.
If tarinfo is given, it is used as the starting point.
"""
# Ensure that all members have been loaded.
members = self.getmembers() # Limit the member search list up to tarinfo.
if tarinfo is not None:
members = members[:members.index(tarinfo)] if normalize:
name = os.path.normpath(name) for member in reversed(members):
if normalize:
member_name = os.path.normpath(member.name)
else:
member_name = member.name if name == member_name:
return member def _load(self):
"""Read through the entire archive file and look for readable
members.
"""
while True:
tarinfo = self.next()
if tarinfo is None:
break
self._loaded = True def _check(self, mode=None):
"""Check if TarFile is still open, and if the operation's mode
corresponds to TarFile's mode.
"""
if self.closed:
raise IOError("%s is closed" % self.__class__.__name__)
if mode is not None and self.mode not in mode:
raise IOError("bad operation for mode %r" % self.mode) def _find_link_target(self, tarinfo):
"""Find the target member of a symlink or hardlink member in the
archive.
"""
if tarinfo.issym():
# Always search the entire archive.
linkname = "/".join(filter(None, (os.path.dirname(tarinfo.name), tarinfo.linkname)))
limit = None
else:
# Search the archive before the link, because a hard link is
# just a reference to an already archived file.
linkname = tarinfo.linkname
limit = tarinfo member = self._getmember(linkname, tarinfo=limit, normalize=True)
if member is None:
raise KeyError("linkname %r not found" % linkname)
return member def __iter__(self):
"""Provide an iterator object.
"""
if self._loaded:
return iter(self.members)
else:
return TarIter(self) def _dbg(self, level, msg):
"""Write debugging output to sys.stderr.
"""
if level <= self.debug:
print >> sys.stderr, msg def __enter__(self):
self._check()
return self def __exit__(self, type, value, traceback):
if type is None:
self.close()
else:
# An exception occurred. We must not call close() because
# it would try to write end-of-archive blocks and padding.
if not self._extfileobj:
self.fileobj.close()
self.closed = True
# class TarFile TarFile
tarfile的源代码
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import tarfile
tar = tarfile.open("yinzhengjie.tar","w")
tar.add(r"D:\python\daima\DAY5\day5.zip",arcname="day5.zip")
tar.add(r"D:\python\daima\DAY3\学生信息.xlsx",arcname="test_11") #相当于将“学生信息.xlsx”这个文件重命名为"test_11"并添加到压缩文件“yinzhengjie.tar”中
tar.close()
tarfile压缩的用法
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import tarfile
tar = tarfile.open("yinzhengjie.tar","r") #注意解压的时候，打开的模式应该是"r"，而不是"w"哟!
tar.extractall(path=r"D:\python\daima\DAY6\test") #其中的path表示设置的解压后的存放地址
tar.close()
tarfile解压的用法
8.shelve模块
它是对pickle的上层封装，比pickle用起来更简单，shelve模块是一个简单的k,v将内存数据通过文件持久化的模块，可以持久化任何pickle可支持的python数据格式。
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import shelve
f = shelve.open('shelve_test') # 打开一个文件
class Test(object):
def __init__(self, n):
self.n = n
def func():
print("my name is yinzhengjie!")
t = Test(123)
name = ["yinzhengjie", "lijing", "test"]
f["first"] = name # 序列化列表
f["second"] = t # 序列化类，注意只是序列化的类名，如果想要在新的操作系统中调用的话，需要导入哟！
f["third"] = func #序列化函数，注意只是序列化的函数名，如果想要在新的操作系统中调用的话，需要导入哟！
f.close()
序列化数据
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import shelve
f = shelve.open('shelve_test')
for i in f.keys():
print(i)
print(f.get("first")) #相当于load，取得里面的数据，但是如果要读取类或者函数的话需要导入相应的模块哟！ #以上代码执行结果如下：
first
second
third
['yinzhengjie', 'lijing', 'test']
反序列化数据
9.xml模块
xml是实现不同语言或程序之间进行数据交换的协议，跟json差不多，但json使用起来更简单，不过，古时候，在json还没诞生的黑暗年代，大家只能选择用xml呀，至今很多传统公司如金融行业的很多系统的接口还主要是xml。
xml的格式如下，就是通过<>节点来区别数据结构的:
<?xml version="1.0"?>
<data>
<country name="Liechtenstein">
<rank updated="yes">2</rank>
<year>2008</year>
<gdppc>141100</gdppc>
<neighbor name="Austria" direction="E"/>
<neighbor name="Switzerland" direction="W"/>
<hotspots>
<test1>维也纳</test1>
<test2>萨尔斯堡</test2>
</hotspots>
</country>
<country name="Singapore">
<rank updated="yes">5</rank>
<year>2011</year>
<gdppc>59900</gdppc>
<neighbor name="Malaysia" direction="N"/>
</country>
<country name="Panama">
<rank updated="yes">69</rank>
<year>2011</year>
<gdppc>13600</gdppc>
<neighbor name="Costa Rica" direction="W"/>
<neighbor name="Colombia" direction="E"/>
</country>
</data>
xmltest.xml
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import xml.etree.ElementTree as ET
tree = ET.parse("xmltest.xml") #解析（可以理解为打开一个文件）
root = tree.getroot() #找到根节点
print(root.tag) #打印根节点的名字
# 遍历xml文档
print("*"*50,"我是分割线","*"*50)
for child in root:
print(child.tag, child.attrib) #打印前者子节点的名字“country”，并打印其属性。
for i in child:
print(i.tag, i.text,i.attrib) #打印子节点的参数名，已经其所对应的值,最后打印其属性。
# 只遍历year 节点
print("*" * 50, "我是分割线", "*" * 50)
for node in root.iter('year'):
print(node.tag, node.text) # 只遍历country节点
print("*" * 50, "我是分割线", "*" * 50)
for node in root.iter('country'):
print(node.tag, node.text,node.attrib) print("*" * 50, "我是分割线", "*" * 50)
for node in root.iter('test1'):
print(node.tag, node.text,node.attrib) #以上代码执行结果如下：
data
************************************************** 我是分割线 **************************************************
country {'name': 'Liechtenstein'}
rank 2 {'updated': 'yes'}
year 2008 {}
gdppc 141100 {}
neighbor None {'direction': 'E', 'name': 'Austria'}
neighbor None {'direction': 'W', 'name': 'Switzerland'}
hotspots
{}
country {'name': 'Singapore'}
rank 5 {'updated': 'yes'}
year 2011 {}
gdppc 59900 {}
neighbor None {'direction': 'N', 'name': 'Malaysia'}
country {'name': 'Panama'}
rank 69 {'updated': 'yes'}
year 2011 {}
gdppc 13600 {}
neighbor None {'direction': 'W', 'name': 'Costa Rica'}
neighbor None {'direction': 'E', 'name': 'Colombia'}
************************************************** 我是分割线 **************************************************
year 2008
year 2011
year 2011
************************************************** 我是分割线 **************************************************
country
{'name': 'Liechtenstein'}
country
{'name': 'Singapore'}
country
{'name': 'Panama'}
************************************************** 我是分割线 **************************************************
test1 维也纳 {}
查看xml的方法
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import xml.etree.ElementTree as ET
tree = ET.parse("xmltest.xml")
root = tree.getroot()
# 修改
for node in root.iter('year'): #表示操作的对象是“year”字段
new_year = int(node.text) + 1 #将年份加“1”
node.text = str(new_year) #把数字转换成字符串
node.set("updated", "yes") #set是更改属性，增加了一个属性updated="yes"
tree.write("xmltest2.xml",encoding="utf-8") #将修改后的内容写入xml文件中。
修改xml文件的属性
<data>
<country name="Liechtenstein">
<rank updated="yes">2</rank>
<year updated="yes">2009</year>
<gdppc>141100</gdppc>
<neighbor direction="E" name="Austria" />
<neighbor direction="W" name="Switzerland" />
<hotspots>
<test1>维也纳</test1>
<test2>萨尔斯堡</test2>
</hotspots>
</country>
<country name="Singapore">
<rank updated="yes">5</rank>
<year updated="yes">2012</year>
<gdppc>59900</gdppc>
<neighbor direction="N" name="Malaysia" />
</country>
<country name="Panama">
<rank updated="yes">69</rank>
<year updated="yes">2012</year>
<gdppc>13600</gdppc>
<neighbor direction="W" name="Costa Rica" />
<neighbor direction="E" name="Colombia" />
</country>
</data>
xmltest2.xml
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import xml.etree.ElementTree as ET
tree = ET.parse("xmltest.xml")
root = tree.getroot()
# 删除node
for country in root.findall('country'):
rank = int(country.find('rank').text)
if rank > 50: #如果国家排名大于50就删除这个国家
root.remove(country)
tree.write('output.xml',encoding="utf-8")
删除xml文件的方法
<data>
<country name="Liechtenstein">
<rank updated="yes">2</rank>
<year>2008</year>
<gdppc>141100</gdppc>
<neighbor direction="E" name="Austria" />
<neighbor direction="W" name="Switzerland" />
<hotspots>
<test1>维也纳</test1>
<test2>萨尔斯堡</test2>
</hotspots>
</country>
<country name="Singapore">
<rank updated="yes">5</rank>
<year>2011</year>
<gdppc>59900</gdppc>
<neighbor direction="N" name="Malaysia" />
</country>
</data>
output.xml
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import xml.etree.ElementTree as ET
new_xml = ET.Element("namelist") #生成一个根节点
name = ET.SubElement(new_xml, "name", attrib={"enrolled": "yes"}) #SubElement创建了一个子节点，其中名字叫“name”，属性是{"enrolled": "yes"}
age = ET.SubElement(name, "age", attrib={"checked": "no"}) #在name下又创建了一个子节点
sex = ET.SubElement(name, "sex")
sex.text = '' name2 = ET.SubElement(new_xml, "name", attrib={"enrolled": "no"})
age = ET.SubElement(name2, "age")
age.text = ''
et = ET.ElementTree(new_xml) # 生成文档对象
et.write("test.xml", encoding="utf-8", xml_declaration=True)
# ET.dump(new_xml) # 打印生成的格式
创建xml文件的方法
<?xml version='1.0' encoding='utf-8'?>
<namelist>
<name enrolled="yes">
<age checked="no" />
<sex>33</sex></name>
<name enrolled="no">
<age>19</age>
</name>
</namelist>
test.xml
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import xml.etree.ElementTree as ET
tree = ET.parse("xmltest.xml")
root = tree.getroot()
for country in root.findall('country'):
rank = int(country.find('rank').text)
if rank > 50:
root.remove(country)
else:
sub_ele = ET.SubElement(country,"population",attrib={"enrolled":"yes"}) #创建一个新的“population”
sub_ele.text = str(100000000000)
tree.write('output2.xml')
在原xml的基础上新增文件内容
<data>
<country name="Liechtenstein">
<rank updated="yes">2</rank>
<year>2008</year>
<gdppc>141100</gdppc>
<neighbor direction="E" name="Austria" />
<neighbor direction="W" name="Switzerland" />
<hotspots>
<test1>维也纳</test1>
<test2>萨尔斯堡</test2>
</hotspots>
<population enrolled="yes">100000000000</population></country>
<country name="Singapore">
<rank updated="yes">5</rank>
<year>2011</year>
<gdppc>59900</gdppc>
<neighbor direction="N" name="Malaysia" />
<population enrolled="yes">100000000000</population></country>
</data>
output2.xml
10.PyYAML模块
请参考官网：http://pyyaml.org/wiki/PyYAMLDocumentation
saltstack自动化运维工具的配置文件就是要ongodb这种格式写的。
11.ConfigParser模块
用于生成和修改常见配置文档，当前模块的名称在 python 3.x 版本中变更为 configparser。类似于apache和mysql的配置文件就用这个模块生成的。
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import configparser
config = configparser.ConfigParser() #生成一个根节点实例
config["DEFAULT"] = {'ServerAliveInterval': '',
'Compression': 'yes',
'CompressionLevel': ''}
config['bitbucket.org'] = {} #生成一个空的字典·
config['bitbucket.org']['User'] = 'yinzhengjie' #往这个空字典中添加信息
config['topsecret.server.com'] = {} #通过config实例生成一个叫“topsecret.server.com”的空节点
topsecret = config['topsecret.server.com'] #找到上面那个空节点然后赋值给一个变量，之后可以往里面赋值参数
topsecret['Host Port'] = '' # mutates the parser
topsecret['ForwardX11'] = 'no' # same here
config['DEFAULT']['ForwardX11'] = 'yes' #给“DEFAULT”字段赋值
with open('example.ini', 'w') as configfile: #最终将数据写入到“example.ini”文件中.
config.write(configfile)
自动生成一个configparser文件
[DEFAULT]
compressionlevel = 9
serveraliveinterval = 45
compression = yes
forwardx11 = yes [bitbucket.org]
user = yinzhengjie [topsecret.server.com]
host port = 50022
forwardx11 = no
example.ini
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import configparser
config = configparser.ConfigParser() #生成一个实例
print(config.sections()) #打印出有多少个子配置实例
print("*"*50,"我是分割线","*"*50)
print(config.read('example.ini')) #读取这个文件
print("*"*50,"我是分割线","*"*50)
print(config.sections()) #打印出有多少个子配置实例，当read到数据之后，就可以看到相关的子实例，并且默认不打印“[DEFAULT]”这个子实例。
print("*"*50,"我是分割线","*"*50)
print('bitbucket.org' in config) #取布尔值，判断bitbucket.org是否在config这个实例中。
print('bytebong.com' in config)
print("*"*50,"我是分割线","*"*50)
print(config['bitbucket.org']['User']) #取“config['bitbucket.org']"子实例中的'User'变量所对应的值
print("*"*50,"我是分割线","*"*50)
print(config['DEFAULT']['Compression'])
topsecret = config['topsecret.server.com']
print(topsecret['ForwardX11'])
print("*"*50,"我是分割线","*"*50)
for key in config['bitbucket.org']: #遍历出'bitbucket.org'子实例中的变量
print(key)
print(config['bitbucket.org']['ForwardX11'])
configparser查询用法实例
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import configparser
config = configparser.ConfigParser() #生成一个实例
config.read('example.ini') #读取这个文件，注意，如果缺少这个步骤，无法进行修改或者读取哟！
config.set("bitbucket.org","name","Yinzhengjie") #在bitbucket.org的sections添加"name"的值为"Yinzhengjie"
config.set("bitbucket.org","user","尹正杰") #在bitbucket.org的sections修改"user"的值为"尹正杰"
config.write(open("test_1.cfg","w",encoding="utf-8"))
configparser修改和新增的实例
[DEFAULT]
compressionlevel = 9
serveraliveinterval = 45
compression = yes
forwardx11 = yes [bitbucket.org]
user = 尹正杰
name = Yinzhengjie [topsecret.server.com]
host port = 50022
forwardx11 = no
test_1.cfg
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import configparser
config = configparser.ConfigParser() #生成一个实例
config.read('example.ini')
config.remove_section("bitbucket.org") #删除整个section！
config.remove_option("DEFAULT","forwardx11") #删除某个section的一个子项，
config.write(open("test_rm.ini","w",encoding="utf-8")) #如果有汉子的话需要制定编码格式才能正常打印汉子哟~
configparser删除的实例
[DEFAULT]
compressionlevel = 9
serveraliveinterval = 45
compression = yes [topsecret.server.com]
host port = 50022
forwardx11 = no
test_rm.ini
12.hashlib模块
用于加密相关的操作，3.x里代替了md5模块和sha模块，主要提供 SHA1, SHA224, SHA256, SHA384, SHA512 ，MD5 算法.
1>.MD5算法参数详解：
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import hashlib
m = hashlib.md5()
m.update(b"hello") #字节格式输入
print(m.hexdigest()) #用十六进制输出一段md5值,注意，只要输入的值不变，这个值就不会变的！
m.update(b"my name is yinzhengjie")
print(m.hexdigest()) #注意，将上面两个字段拼接起来，其中的MD5值也是会发生变化的
m2 = hashlib.md5()
m2.update(b"hello my name is yinzhengjie")
print(m2.hexdigest()) '''
扩展：
MD5值是无法被反解的，网上有人说能破解是骗人的，之所以能破解，是因为他们已经将算好的md5值存入到数据库中，然后跟你你输入的MD5值给你返回一个明文的字符串。
''' #以上代码执行结果如下：
5d41402abc4b2a76b9719d911017c592
1c7bdaafeb36ea7e3236d01afeee39cf
1d19d8f2d5037b0f3e9a2d020930ba91
十六进制md5算法实例展示
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import hashlib
m = hashlib.md5()
m.update(b"hello") #字节格式输入
print(m.digest()) #用十六进制输出一段md5值,注意，只要输入的值不变，这个值就不会变的！
m.update(b"my name is yinzhengjie")
print(m.digest()) #注意，将上面两个字段拼接起来，其中的MD5值也是会发生变化的
m2 = hashlib.md5()
m2.update(b"hello my name is yinzhengjie")
print(m2.digest()) '''
扩展：
MD5值是无法被反解的，网上有人说能破解是骗人的，之所以能破解，是因为他们已经将算好的md5值存入到数据库中，然后跟你你输入的MD5值给你返回一个明文的字符串。
''' #以上代码执行结果如下：
b']A@*\xbcK*v\xb9q\x9d\x91\x10\x17\xc5\x92'
b'\x1c{\xda\xaf\xeb6\xea~26\xd0\x1a\xfe\xee9\xcf'
b'\x1d\x19\xd8\xf2\xd5\x03{\x0f>\x9a-\x02\t0\xba\x91'
二进制md5算法实例展示
2>.sha1算法参数详解：
Google已经将改算法破解了，只是还没有公布，目前很少人用这种算法了！
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import hashlib
m = hashlib.sha1()
m.update(b"hello") #字节格式输入
print(m.digest()) #用十六进制输出一段md5值,注意，只要输入的值不变，这个值就不会变的！
m.update(b"my name is yinzhengjie")
print(m.digest()) #注意，将上面两个字段拼接起来，其中的MD5值也是会发生变化的
m2 = hashlib.sha1()
m2.update(b"hello my name is yinzhengjie")
print(m2.digest()) '''
扩展：
MD5值是无法被反解的，网上有人说能破解是骗人的，之所以能破解，是因为他们已经将算好的md5值存入到数据库中，然后跟你你输入的MD5值给你返回一个明文的字符串。
''' #以上代码执行结果如下：
b'\xaa\xf4\xc6\x1d\xdc\xc5\xe8\xa2\xda\xbe\xde\x0f;H,\xd9\xae\xa9CM'
b'p\xff\xe5<\x08\xb9D?\xabJ\xcdC2f\x84\xa07\xd6\xc2c'
b'\xad\x06\x8b\x91)\x1c \x99\x82*6D^\xb2DA\x12_3\xa6'
二进制sha1算法实例展示
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import hashlib
m = hashlib.sha1()
m.update(b"hello") #字节格式输入
print(m.hexdigest()) #用十六进制输出一段md5值,注意，只要输入的值不变，这个值就不会变的！
m.update(b"my name is yinzhengjie")
print(m.hexdigest()) #注意，将上面两个字段拼接起来，其中的MD5值也是会发生变化的
m2 = hashlib.sha1()
m2.update(b"hello my name is yinzhengjie")
print(m2.hexdigest()) '''
扩展：
MD5值是无法被反解的，网上有人说能破解是骗人的，之所以能破解，是因为他们已经将算好的md5值存入到数据库中，然后跟你你输入的MD5值给你返回一个明文的字符串。
''' #以上代码执行结果如下：
aaf4c61ddcc5e8a2dabede0f3b482cd9aea9434d
70ffe53c08b9443fab4acd43326684a037d6c263
ad068b91291c2099822a36445eb24441125f33a6
十六进制sha1算法实例展示
3>.sha256算法参数详解：
这个是没有被破解的，连谷歌破解的仅仅是sha1,而256加密后的明显字节变成啊，有木有。
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import hashlib
m = hashlib.sha256()
m.update(b"hello") #字节格式输入
print(m.digest()) #用十六进制输出一段md5值,注意，只要输入的值不变，这个值就不会变的！
m.update(b"my name is yinzhengjie")
print(m.digest()) #注意，将上面两个字段拼接起来，其中的MD5值也是会发生变化的
m2 = hashlib.sha256()
m2.update(b"hello my name is yinzhengjie")
print(m2.digest()) #以上代码执行结果如下：
b',\xf2M\xba_\xb0\xa3\x0e&\xe8;*\xc5\xb9\xe2\x9e\x1b\x16\x1e\\\x1f\xa7B^s\x043b\x93\x8b\x98$'
b'\xec\xf6\x8e\x01\x17\xac!:\xb9<\xe4\xab\xee\x13\x03\xcc\xe4r\xb0\xdc\xfb\xcbm\xd4\xec\xa2\xc9P\x02\xfdi\xb7'
b'\xf4\x9d\xe7o\xe3\x01A\xf28\xd0\xc1b4\xa0\xbf\x01\x88\xbf\x9a4\xb4\xe8\xdd\xb6\\P\x8c&\xd5\xb1\xaf\x06'
二进制sha256算法实例展示
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import hashlib
m = hashlib.sha256()
m.update(b"hello") #字节格式输入
print(m.hexdigest()) #用十六进制输出一段md5值,注意，只要输入的值不变，这个值就不会变的！
m.update(b"my name is yinzhengjie")
print(m.hexdigest()) #注意，将上面两个字段拼接起来，其中的MD5值也是会发生变化的
m2 = hashlib.sha256()
m2.update(b"hello my name is yinzhengjie")
print(m2.hexdigest()) #以上代码执行结果如下：
2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824
ecf68e0117ac213ab93ce4abee1303cce472b0dcfbcb6dd4eca2c95002fd69b7
f49de76fe30141f238d0c16234a0bf0188bf9a34b4e8ddb65c508c26d5b1af06
十六进制sha256算法实例展示
4>.sha512算法参数详解：
这个也没有破解，明显的效果就是加密后的值变的比256还要长呢!
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import hashlib
m = hashlib.sha512()
m.update(b"hello") #字节格式输入
print(m.digest()) #用十六进制输出一段md5值,注意，只要输入的值不变，这个值就不会变的！
m.update(b"my name is yinzhengjie")
print(m.digest()) #注意，将上面两个字段拼接起来，其中的MD5值也是会发生变化的
m2 = hashlib.sha512()
m2.update(b"hello my name is yinzhengjie")
print(m2.digest()) #以上代码执行结果如下：
b'\x9bq\xd2$\xbdb\xf3x]\x96\xd4j\xd3\xea=s1\x9b\xfb\xc2\x89\x0c\xaa\xda\xe2\xdf\xf7%\x19g<\xa7##\xc3\xd9\x9b\xa5\xc1\x1d|z\xccn\x14\xb8\xc5\xda\x0cFcG\\.\\:\xde\xf4os\xbc\xde\xc0C'
b"7\x8fb\xe6'\x11\xcc\xa8I\x9b\x89=\xcf\xac\x06\xdc\xbc\xb7GyG\x96\xd9=\xfc\xa7r\xc6\xba\x9ep\x96\xd7X\x05\x82\xbd\x87\xae\x94\x90UD\xdd\xdf\x94-\xa5\xcd\xf9o\x89\xdc\xcf\x85pr\x9ekvE\x12\xcc\x0f"
b'\xea\x1b\xda\xce3r>\x83\x98\x94\xd7\x7fp\xad}\x84w\xb3o\xd2\xf4ZMB\xb6\xb9c|t]\xa5\xf7]*\xb2v\xf10\xa8&\x19\xeb\xc7\xe5;\x9d0\x92o\x9b\xa8\x91v\xc5\x03\xd4\x82Z\xb3;\xea[\x01h'
二进制sha512算法实例展示
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import hashlib
m = hashlib.sha512()
m.update(b"hello") #字节格式输入
print(m.hexdigest()) #用十六进制输出一段md5值,注意，只要输入的值不变，这个值就不会变的！
m.update(b"my name is yinzhengjie")
print(m.hexdigest()) #注意，将上面两个字段拼接起来，其中的MD5值也是会发生变化的
m2 = hashlib.sha512()
m2.update(b"hello my name is yinzhengjie")
print(m2.hexdigest()) #以上代码执行结果如下：
9b71d224bd62f3785d96d46ad3ea3d73319bfbc2890caadae2dff72519673ca72323c3d99ba5c11d7c7acc6e14b8c5da0c4663475c2e5c3adef46f73bcdec043
378f62e62711cca8499b893dcfac06dcbcb747794796d93dfca772c6ba9e7096d7580582bd87ae94905544dddf942da5cdf96f89dccf8570729e6b764512cc0f
ea1bdace33723e839894d77f70ad7d8477b36fd2f45a4d42b6b9637c745da5f75d2ab276f130a82619ebc7e53b9d30926f9ba89176c503d4825ab33bea5b0168
十六进制sha512算法实例展示
5>.hmac模块
如果你觉得以上的加密方法还是不够安全~厉害了，你的安全感可真低啊，看来是伤的不轻，一定是一个有故事的人，不过针对你这种人呢~还有一种算法为你特别定制hmac，等你成为了一个开发大神，你可以自己写一个算法，因为你毕竟只相信你自己嘛，哈哈~
散列消息鉴别码，简称HMAC，是一种基于消息鉴别码MAC（Message Authentication Code）的鉴别机制。使用HMAC时,消息通讯的双方，通过验证消息中加入的鉴别密钥K来鉴别消息的真伪；一般用于网络通信中消息加密，前提是双方先要约定好key,就像接头暗号一样，然后消息发送把用key把消息加密，接收方用key ＋ 消息明文再加密，拿加密后的值 跟 发送者的相对比是否相等，这样就能验证消息的真实性，及发送者的合法性了。（它内部对我们创建 key 和 内容 再进行处理然后再加密）
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import hmac
h = hmac.new("我本有心向明月".encode("utf-8"), "奈何明月照沟渠".encode("utf-8"),) #"我本有心向明月"这就好比双方(A,B)已经约定好了的key，（类似于我们第一次登录linux服务器，使用ssh登陆方式都会提示你，让你输入"yes"后才能输入密码。），接受者(A)接收到了"奈何明月照沟渠"这个明文消息和加密后的字符“489f9932949514ab24894559150088c0”，然后用定义好的key和去加密"奈何明月照沟渠"这个明文字符，如果加密后生成的字符是“489f9932949514ab24894559150088c0”就证明这个消息是发送者(B)发送过来的数据，它只能验证消息的合法来源。如果中间人截获了明文消息加以修改的，就会被发现！
print(h.hexdigest()) #以上代码执行结果如下：
489f9932949514ab24894559150088c0
hmac算法实例展示
13.subproces模块
subproces基本上就是为了取代os.system和os.spawn*模块的。
1>.subprocess.run调用shell命令，只能保存执行后的状态，不能保存命令的执行结果！
#不含参数的调用linux命令的方法
>>> a = subprocess.run("df")
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda2 8854456 4170968 4227040 50% /
tmpfs 502172 228 501944 1% /dev/shm
/dev/sda1 289293 28463 245470 11% /boot #含参数的调用linux命令的方法
>>> a = subprocess.run(["df","-h"])
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 8.5G 3.8G 4.3G 48% /
tmpfs 491M 228K 491M 1% /dev/shm
/dev/sda1 283M 28M 240M 11% /boot
>>> #调用复杂的linux命令的方法，需要加“shell=True”，表示将前面引号的内容放在一个终端(terminal)去执行，需要注意的是这个不能保存命令输出的结果，而是保存命令执行的结果哟！一般非“0”就表示命令没有执行成功，而结果是“0”表示执行命令实成功的，但是命令的输出结果是无法保存的！切记！
>>> a = subprocess.run("df -h | grep /dev/sda1",shell=True)
/dev/sda1 283M 28M 240M 11% /boot
>>> a.returncode
0
subprocess.run实例
2>.执行命令，返回命令的执行状态,“0” or 非 “0”
#执行命令，返回命令执行状态 ， 0 or 非0
>>> retcode = subprocess.call(["ls", "-l"])
total 96
-rw-------. 1 root root 3321 Oct 13 10:26 anaconda-ks.cfg
drwxr-xr-x. 2 root root 4096 Oct 13 22:03 Desktop
drwxr-xr-x. 2 root root 4096 Oct 13 22:03 Documents
drwxr-xr-x. 2 root root 4096 Oct 13 22:03 Downloads
-rw-r--r--. 1 root root 41433 Oct 13 10:26 install.log
-rw-r--r--. 1 root root 9154 Oct 13 10:24 install.log.syslog
drwxr-xr-x. 2 root root 4096 Oct 13 22:03 Music
drwxr-xr-x. 2 root root 4096 Oct 13 22:03 Pictures
drwxr-xr-x. 2 root root 4096 Oct 13 22:03 Public
drwxr-xr-x. 2 root root 4096 Oct 13 22:03 Templates
drwxr-xr-x. 2 root root 4096 Oct 13 22:03 Videos
>>> #执行命令，如果命令结果为0，就正常返回，否则抛异常
>>> subprocess.check_call(["ls", "-l"])
total 96
-rw-------. 1 root root 3321 Oct 13 10:26 anaconda-ks.cfg
drwxr-xr-x. 2 root root 4096 Oct 13 22:03 Desktop
drwxr-xr-x. 2 root root 4096 Oct 13 22:03 Documents
drwxr-xr-x. 2 root root 4096 Oct 13 22:03 Downloads
-rw-r--r--. 1 root root 41433 Oct 13 10:26 install.log
-rw-r--r--. 1 root root 9154 Oct 13 10:24 install.log.syslog
drwxr-xr-x. 2 root root 4096 Oct 13 22:03 Music
drwxr-xr-x. 2 root root 4096 Oct 13 22:03 Pictures
drwxr-xr-x. 2 root root 4096 Oct 13 22:03 Public
drwxr-xr-x. 2 root root 4096 Oct 13 22:03 Templates
drwxr-xr-x. 2 root root 4096 Oct 13 22:03 Videos
0
>>>
subprocess.call与subprocess.check_call函数的调用方法
3>.接收字符串格式命令，返回元组形式，第1个元素是执行状态，第2个是命令结果
>>> subprocess.getstatusoutput('ls /bin/pwd')
(0, '/bin/pwd')
>>>
subprocess.getstatusoutput函数的调用方法
4>.接收字符串格式命令，并返回结果
>>> subprocess.getoutput('ifconfig | grep eth0')
'eth0 Link encap:Ethernet HWaddr 00:0C:29:D4:DB:87 '
>>>
subprocess.getoutput函数调用方法
5>.执行命令，并返回结果，注意是返回结果，不是打印，下例结果返回给res
>>> res=subprocess.check_output(['pwd'])
>>> res
b'/root\n'
>>>
subprocess.check_output函数调用方法
6>.上面那些方法，底层都是封装的subprocess.Popen
>>> p = subprocess.Popen("df -h|grep /dev/sda1",stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.PIPE, shell=True)
>>> p.stdout.read()
b'/dev/sda1 283M 28M 240M 11% /boot\n'
>>>
“”“
注意：
我们来对第一行的进行讲解一下
subprocess.Popen表示打开一个终端（只是启动一个进程），stdin=subprocess.PIPE表示输入通过subprocess.PIPE这个管道传输，stdout=subprocess.PIPE表示输出也通过subprocess.PIPE这个管道传输，stderr=subprocess.PIPE同理。
”“”
subprocess.Popen函数用法
7>.检查命令是否执行完毕
>>> p = subprocess.Popen("top -bn 5",stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.PIPE, shell=True)
>>> p.poll()
>>> p.poll()
>>> p.poll()
>>> p.poll()
>>> p.poll()
>>> p.poll()
0
>>> p.poll()
0
>>>
‘’‘
poll()
Check if child process has terminated. Returns returncode
’‘’
poll()方法调用案例【不需要等】
>>> p = subprocess.Popen("top -bn 5",stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.PIPE, shell=True)
>>> p.wait()
0
>>>
'''
wait()
Wait for child process to terminate. Returns returncode attribute.
'''
wait()方法调用案例【需要等】返回执行状态
>>> p = subprocess.Popen("top -bn 5",stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.PIPE, shell=True)
>>> p.poll()
>>> p.terminate()
>>> p.poll()
143
'''
terminate() 杀掉所启动进程,此时p.poll返回值应该是非“0”，因为不是正常结束的！没有执行完毕就被杀掉了。
'''
terminate（）方法调用案例，直接杀掉启动进程
>>> p = subprocess.Popen("df -h;sleep 100",stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.PIPE, shell=True)
>>> p.poll()
>>> p.poll()
>>> p.communicate(timeout=2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.5/subprocess.py", line 1068, in communicate
stdout, stderr = self._communicate(input, endtime, timeout)
File "/usr/local/lib/python3.5/subprocess.py", line 1699, in _communicate
self._check_timeout(endtime, orig_timeout)
File "/usr/local/lib/python3.5/subprocess.py", line 1094, in _check_timeout
raise TimeoutExpired(self.args, orig_timeout)
subprocess.TimeoutExpired: Command 'df -h;sleep 100' timed out after 2 seconds
>>>
‘’‘
communicate() 等待任务结束，我们需要在里面添加一个参数，默认单位是“s”，如果程序执行时间超过指定的时间就会抛出一个“TimeoutExpired”的字样哟，不过我们可以用异常处理来吧这个错误解决掉！
’‘’
communicate（）函数调用方法
>>> def name():
... print("my name is yinzhengjie!")
...
>>> p = subprocess.Popen("pwd",stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.PIPE, preexec_fn=name) >>> p.stdout.read()
b'my name is yinzhengjie!\n/root\n'
>>> ‘’‘
preexec_fn：只在Unix平台下有效，用于指定一个可执行对象（callable object），它将在子进程运行之前被调用，运行结果见上例。
’‘’
preexec_fn参数调用案例
>>> p = subprocess.Popen("pwd",cwd="/usr/local",stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
>>> p.stdout.read()
b'/usr/local\n'
>>>
'''
cwd：用于设置子进程的当前目录
'''
cwd参数调用案例
>>> p = subprocess.Popen("echo $name_str",cwd="/usr/local",shell=True,stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.PIPE,env={"name_str":"yinzhengjie"})
>>> p.stdout.read()
b'yinzhengjie\n'
>>>
‘’‘
提示：
env：用于指定子进程的环境变量。如果env = None，子进程的环境变量将从父进程中继承。
’‘’
env参数调用案例
14.re模块
正在表达式主要是用于模糊匹配，Python中的正则用法比较简单，但是规则比较麻烦，你可以根据你的需求配置不同的规则
常用正则表达式符号：
'\A' 只从字符开头匹配，re.search("\Yin","yinzhengjie") 是匹配不到的
'\Z' 匹配字符结尾，同$
'\d' 匹配数字0-9
'\D' 匹配非数字
'\w' 匹配[A-Za-z0-9]
'\W' 匹配非[A-Za-z0-9]
's' 匹配空白字符、\t、\n、\r , re.search("\s+","ab\tc1\n3").group() 结果 '\t'
re.match 从头开始匹配
re.search 匹配包含
re.findall 把所有匹配到的字符放到以列表中的元素返回
re.splitall 以匹配到的字符当做列表分隔符
re.sub 匹配字符并替换
最常用的匹配语法
1>.'.' 默认匹配除\n之外的任意一个字符，若指定flag DOTALL,则匹配任意字符，包括换行
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import re
print(re.match("yin","yinzhengjie")) #精确匹配，前面是条件，后面是内容
print(re.match(".","yinzhengjie")) #模糊匹配单个字符
print(re.match("...","yinzhengjie"))
print(re.match("....","yin\nzhengjie")) #从开头开始模糊匹配4个字符，无法识别换行符"\n", #以上代码执行结果如下：
<_sre.SRE_Match object; span=(0, 3), match='yin'>
<_sre.SRE_Match object; span=(0, 1), match='y'>
<_sre.SRE_Match object; span=(0, 3), match='yin'>
None
'.' 的用法展示
2>.'^' 匹配字符开头，若指定flags MULTILINE,这种也可以匹配上(r"^a","\nabc\neee",flags=re.MULTILINE)
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import re
print(re.match("^Y","Yinzhengjie"))
print(re.match("jie","Yinzhengjie")) #表示从头开始匹配
print(re.search("jie","yinzhengjie")) #表示匹配包含 #以上代码执行结果如下：
<_sre.SRE_Match object; span=(0, 1), match='Y'>
None
<_sre.SRE_Match object; span=(8, 11), match='jie'>
'^'的用法展示
3>.'$' 匹配字符结尾，或e.search("foo$","bfoo\nsdfsf",flags=re.MULTILINE).group()也可以
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import re
print(re.search("jie$","Yinzhengjie")) #匹配包含，不从开头开始匹配哟 #以上代码执行结果如下：
<_sre.SRE_Match object; span=(8, 11), match='jie'>
$'用法展示
4>.'+' 匹配前一个字符1次或多次，re.findall("ab+","ab+cd+abb+bba") 结果['ab', 'abb']
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import re
print(re.match(".+","Yinzhengjie")) #表示从头开始匹配
print(re.match("^.+","Yinzhengjie"))
print(re.match(".+","Yinzheng\njie")) #表示匹配包含 #以上代码执行结果如下：
<_sre.SRE_Match object; span=(0, 11), match='Yinzhengjie'>
<_sre.SRE_Match object; span=(0, 11), match='Yinzhengjie'>
<_sre.SRE_Match object; span=(0, 8), match='Yinzheng'>
'+' 的用法展示
5>.'*' 匹配*号前的字符0次或多次，re.findall("ab*","cabb3abcbbac") 结果为['abb', 'ab', 'a']
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import re
print(re.search("Y*","Yinzhengjie"))
print(re.search("jie*","Yinzhengjie"))
print(re.search("e*","Yinzhengjie")) #以上代码执行结果如下：
<_sre.SRE_Match object; span=(0, 1), match='Y'>
<_sre.SRE_Match object; span=(8, 11), match='jie'>
<_sre.SRE_Match object; span=(0, 0), match=''>
'*'用法展示
6>.'?' 匹配前一个字符1次或0次
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import re
print(re.search("Y?","Yinzhengjie"))
print(re.search("jie?","Yinzhengjie"))
print(re.search("e?","Yinzhengjieee")) #以上代码执行结果如下:
<_sre.SRE_Match object; span=(0, 1), match='Y'>
<_sre.SRE_Match object; span=(8, 11), match='jie'>
<_sre.SRE_Match object; span=(0, 0), match=''>
'?'用法展示
7>.'{m}' 匹配前一个字符m次
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import re
print(re.search("jie{1}","Yinzhengjiejie"))
print(re.search("(jie){2}","Yinzhengjiejie"))
print(re.search("e{3}","Yinzhengjieee")) #以上代码执行结果如下：
<_sre.SRE_Match object; span=(8, 11), match='jie'>
<_sre.SRE_Match object; span=(8, 14), match='jiejie'>
<_sre.SRE_Match object; span=(10, 13), match='eee'>
'{m}' 用法展示
8>.'{n,m}' 匹配前一个字符n到m次，re.findall("ab{1,3}","abb abc abbcbbb") 结果'abb', 'ab', 'abb']
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import re
print(re.search("e{1,3}","Yinzhengjieee"))
print(re.search("e{2,3}","Yinzhengjieee"))
print(re.search("e{3,10}","Yinzhengjieee")) #以上代码执行结果如下：
<_sre.SRE_Match object; span=(5, 6), match='e'>
<_sre.SRE_Match object; span=(10, 13), match='eee'>
<_sre.SRE_Match object; span=(10, 13), match='eee'>
'{n,m}'用法展示
9>.'|' 匹配|左或|右的字符，re.search("abc|ABC","ABCBabcCD").group() 结果'ABC'
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import re
print(re.search("e|E","YinzhengjiE"))
print(re.search("e|E","YinzhEngjie")) #以上代码执行结果如下：
<_sre.SRE_Match object; span=(5, 6), match='e'>
<_sre.SRE_Match object; span=(5, 6), match='E'>
'|' 用法展示
10>.'(...)' 分组匹配，re.search("(abc){2}a(123|456)c", "abcabca456c").group() 结果 abcabca456c
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import re
print(re.search("\d","")) #匹配一个数字
print(re.search("\d+","")) #匹配数字一次或者多次，一般是贪婪匹配，会尽量把匹配最长的数字给你打印出来
print(re.search("\d...","")) #匹配一个数字和3个任意字符
print(re.search("\d{4}","")) #匹配4个数字
print(re.search("[0-9]{4}","")) #[0-9]其实就是等效于\d
print(re.search("(\d{3})(\d{3})(\d...(3))","")) #这个有点绕，我当时弄了好久才研究明白，给大家分享一下如何去看，前2个“(\d{3})”表示匹配三个数字类型的字符，也就是要匹配6个数字，“(\d...(3))”这个的意思是：匹配一个数字和三个任意字符，而且最后结尾标识符必须是数字“3”。
print(re.search("(\d{3})(\d{3})(\d(7))","")) #表示匹配以数字“7”结尾前面加七个数字[(\d{3})(\d{3})是6个数字后面还有个\d所以总共是7个数字，后面的(3)我们可以理解是以“3”结尾即可。如果你看懂我上面的例子下面这个就不是事情啦！]
print(re.search("(\d{3})(\d{3})(\d...(3))","").group()) #取到值的结果
print(re.search("(\d{3})(\d{3})(\d...(3))","").groups()) #将取到的结果进行分组，这样大家就能看明白上面我举的那个例子啦
number = re.search("(\d{3})(\d{3})(\d...(3))","").groups() #我们也可以将分组的信息赋值给一个变量，便于取值，如下：
print(number[0])
print(number[3]) #以上代码执行结果如下：
<_sre.SRE_Match object; span=(0, 1), match=''>
<_sre.SRE_Match object; span=(0, 18), match=''>
<_sre.SRE_Match object; span=(0, 4), match=''>
<_sre.SRE_Match object; span=(0, 4), match=''>
<_sre.SRE_Match object; span=(0, 4), match=''>
<_sre.SRE_Match object; span=(3, 14), match=''>
<_sre.SRE_Match object; span=(4, 12), match=''>
40119960723
('', '', '', '')
401
3
'(...)' 分组匹配案例展示
11>.'\d','\D','\w','\W' 用法：
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import re
print(re.search("\D","！Yinzhengjie612401199607237057jie!!!!")) #匹配一个非数字
print(re.search("\D+","Yinzhengjie612401199607237057jie!!!!")) #匹配多个非数字，
print(re.search("\D+\d+","Yinzhengjie612401199607237057jie!!!!")) #匹配非数字和数字（注意，不会匹配到“jie!!!!”，因为它不是数字）
print(re.search("\D+\d+\D+","Yinzhengjie612401199607237057jie!!!!")) #都会被匹配到哟
print(re.search("\w+","Yinzhengjie612401199607237057jie!!!!")) #匹配[A-Za-z0-9]
print(re.search("\W+","Yinzhengjie612401199607237057jie!!!!")) #匹配非字母大小写和数字，即不匹配[A-Za-z0-9]，用于匹配特殊字符 #以上代码执行结果如下：
<_sre.SRE_Match object; span=(0, 1), match='！'>
<_sre.SRE_Match object; span=(0, 11), match='Yinzhengjie'>
<_sre.SRE_Match object; span=(0, 29), match='Yinzhengjie612401199607237057'>
<_sre.SRE_Match object; span=(0, 36), match='Yinzhengjie612401199607237057jie!!!!'>
<_sre.SRE_Match object; span=(0, 32), match='Yinzhengjie612401199607237057jie'>
<_sre.SRE_Match object; span=(32, 36), match='!!!!'>
用法展示
12>.'s' 与'(?P<name>...)' 分组匹配用法：
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import re
print(re.search("\s+","！Yinzhengjie612401199607237057 !!!!")) #可以匹配空格
print(re.search("\s+","！Yinzhengjie612401199607237057 \t !!!!")) #可以匹配\t
print(re.search("\s+","！Yinzhengjie612401199607237057 \n !!!!")) #可以匹配\n
print(re.search("\s+","！Yinzhengjie612401199607237057 \r !!!!")) #可以匹配\r
print(re.search("\s+","！Yinzhengjie612401199607237057 \r\n !!!!")) #可以匹配\r\n
print(re.search("(?P<province>[0-9]{4})(?P<city>[0-9]{2})(?P<birthday>[0-9]{4})","").groupdict() ) #这个看起来很复杂，其实挺简单的，这个比'(...)'分组匹配[是将结果放入一个tuple中]要简单的多，而咱们这个是将其变成一个字典，我们先看第一部分(?P<province>[0-9]{4})，就是让“province”为key,让“[0-9]{4}”为value，后面2个(?P<city>[0-9]{2})与(?P<birthday>[0-9]{4})也是一样的道理
a = re.search("(?P<province>[0-9]{4})(?P<city>[0-9]{2})(?P<birthday>[0-9]{4})","").groupdict() #同意，我们分组后可以对其进行取值
print(a.get("birthday"))
print(a.get("province"))
'(?P...)' 分组匹配详解
13>.findall的用法：
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import re
print(re.findall("[a-z]","Yinzhengjie612401199607237057jie!!!!\n6666\r@@@\t")) #只匹配小写字母并将结果返回为一个列表
print(re.findall("[A-z]","Yinzhengjie612401199607237057jie!!!!\n6666\r@@@\t")) #只匹配大写和小写的字母
print(re.findall("[0-9]","Yinzhengjie612401199607237057jie!!!!\n6666\r@@@\t")) #只匹配数字
print(re.findall("\w","Yinzhengjie612401199607237057jie!!!!\n6666\r@@@\t")) #只匹配数字和字母
print(re.findall("\W","Yinzhengjie612401199607237057jie!!!!\n6666\r@@@\t")) #只匹配特殊字符 #以上代码执行结果如下：
['i', 'n', 'z', 'h', 'e', 'n', 'g', 'j', 'i', 'e', 'j', 'i', 'e']
['Y', 'i', 'n', 'z', 'h', 'e', 'n', 'g', 'j', 'i', 'e', 'j', 'i', 'e']
['', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '']
['Y', 'i', 'n', 'z', 'h', 'e', 'n', 'g', 'j', 'i', 'e', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', 'j', 'i', 'e', '', '', '', '']
['!', '!', '!', '!', '\n', '\r', '@', '@', '@', '\t']
re.findall用法展示
14>.split用法
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import re
print(re.split("\W","172.30.1.2")) #取以特殊字符为分隔符取值
print(re.split("\W","192.168@2!24")) #同上 #以上代码执行结果如下：
['', '', '', '']
['', '', '', '']
re.split用法展示
15>.sub用法：
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import re
print(re.sub("\d{4}","","I was born in 1991-05-19,alex was born in 1937-9-15",count=1)) #表示对"I was born in 1991-05-19"这个字符串进行匹配"\d{4}"前4个数字，然后将匹配的到的数据用"2017"去替换掉,注意：count=1表示只匹配1次，如果不指定的话会默认匹配所有的连续的数字哟！
print(re.sub("\d{4}","","I was born in 1991-05-19,alex was born in 1937-9-15",count=2)) #默认匹配2次
print(re.sub("\d{4}","","I was born in 1991-05-19,alex was born in 1937-9-15",)) #默认全部匹配 #以上代码执行结果如下：
I was born in 2017-05-19,alex was born in 1937-9-15
I was born in 2017-05-19,alex was born in 2017-9-15
I was born in 2017-05-19,alex was born in 2017-9-15
re.sub用法展示
16>.反斜杠的困扰
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import re
'''
与大多数编程语言相同，正则表达式里使用"\"作为转义字符，这就可能造成反斜杠困扰。假如你需要匹配文本中的字符"\"，那么使用编程语言表示的正则表达式里将需要4个反斜杠"\\\\"：前两个和后两个分别用于在编程语言里转义成反斜杠，转换成两个反斜杠后再在正则表达式里转义成一个反斜杠。Python里的原生字符串很好地解决了这个问题，这个例子中的正则表达式可以使用r"\\"表示。同样，匹配一个数字的"\\d"可以写成r"\d"。有了原生字符串，你再也不用担心是不是漏写了反斜杠，写出来的表达式也更直观。
''' print(re.search("\\d","\database")) #表示匹配数字
print(re.search(r"\\d","\database")) #表示匹配字母“d”(可以把一些特殊的符号匹配出来如：”\D“，”\Z“等等)，法一
print(re.search("\\\\d","\database")) #表示匹配字母“d”，法二 #以上代码执行结果如下：
None
<_sre.SRE_Match object; span=(0, 2), match='\\d'>
<_sre.SRE_Match object; span=(0, 2), match='\\d'>
特殊字母如何转义成为普通字符
17>.仅需了解几个匹配模式
#！/usr/bin/env python
#_*_coding:utf-8_*_
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie/tag/python%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4%E4%B9%8B%E8%B7%AF/
#EMAIL:y1053419035@qq.com
import re
print(re.search("[a-z]{2}","My Name Is Yinzhengjie",flags=re.I)) #忽略大小写
print(re.search(".+","My Name Is\r\n Yinzhengjie",flags=re.S)) #点任意匹配模式，改变'.'的行为，可以匹配"\n","\r","\r\n"等等。
print(re.search("^M","\nMy Name Is Yinzhengjie",flags=re.I))
print(re.search("^M","\nMy Name Is Yinzhengjie",flags=re.M)) #匹配以M开头，可以过滤掉换行符，用“flags=re.M”实现。 #以上代码执行结果如下：
<_sre.SRE_Match object; span=(0, 2), match='My'>
<_sre.SRE_Match object; span=(0, 24), match='My Name Is\r\n Yinzhengjie'>
None
<_sre.SRE_Match object; span=(1, 2), match='M'>
扩充Tips