前言
很多地方都要用到多线程,这是我经常用的多线程代码,放在博客园记录下。
代码
from multiprocessing.pool import ThreadPool
thread = 10
item_list = [1,3,4,5,6,7,8,9,10]
def sub(item):
#do somthing...
print(item)
if __name__ == '__main__':
pool = ThreadPool(processes = thread)
pool.map(sub, item_list)
pool.close()
pool.join()
例子
举个例子,比如我现在要多线程爬取某个网页,网站为xxx.com/?page=x
,那么代码可以这样写
from multiprocessing.pool import ThreadPool
thread = 50
page_list = [1,3,4,5,6,7,8,9,10] # 爬取10页
def sub(page):
request.get("xxx.com/?page="+str(page), timeout=5)
#do somthing...
if __name__ == '__main__':
pool = ThreadPool(processes = thread)
pool.map(sub, page_list st)
pool.close()
pool.join()
结语
反正用的场景数不胜数,当然如果你有更好的多线程代码,可以在留言处分享~