D09——C语言基础学PYTHON

C语言基础学习PYTHON——基础学习D09

20180903内容纲要：

　　　线程、进程

　　1、paramiko

　　2、线程、进程初识

　　3、线程

　　　　　（1）线程的调用方式

　　　　　（2）join

　　　　　（3）线程锁、递归锁、信号量

　　　　　（4）Timer

　　　　　（5）Event

　　　　　（6）Queue队列

　　4、小结

　　5、练习：简单主机批量管理工具

1、paramiko

paramiko模块提供了ssh及sft进行远程登录服务器执行命令和上传下载文件的功能。这是一个第三方的软件包，使用之前需要安装。

我个人觉得，这个在Windows上不太好用。在windows上python3需要安装vs2010，但我装了之后还是不行，可能是我自己的原因，在python2.7上好像可以，没试过。

这有链接：https://blog.csdn.net/songfreeman/article/details/50920767

很详细！

2 线程、进程初识

什么是线程(thread)？

线程是操作系统能够进行运算调度的最小单位。它被包含在进程之中，是进程中的实际运作单位。

一条线程指的是进程中一个单一顺序的控制流，一个进程中可以并发多个线程，每条线程并行执行不同的任务

A thread is an execution context, which is all the information a CPU needs to execute a stream of instructions.

Suppose you're reading a book, and you want to take a break right now, but you want to be able to come back and resume reading from the exact point where you stopped. One way to achieve that is by jotting down the page number, line number, and word number. So your execution context for reading a book is these 3 numbers.

If you have a roommate, and she's using the same technique, she can take the book while you're not using it, and resume reading from where she stopped. Then you can take it back, and resume it from where you were.

Threads work in the same way. A CPU is giving you the illusion that it's doing multiple computations at the same time. It does that by spending a bit of time on each computation. It can do that because it has an execution context for each computation. Just like you can share a book with your friend, many tasks can share a CPU.

On a more technical level, an execution context (therefore a thread) consists of the values of the CPU's registers.

Last: threads are different from processes. A thread is a context of execution, while a process is a bunch of resources associated with a computation. A process can have one or many threads.

Clarification: the resources associated with a process include memory pages (all the threads in a process have the same view of the memory), file descriptors (e.g., open sockets), and security credentials (e.g., the ID of the user who started the process).

英文解释

什么是进程(process)？

程序并不能单独运行，只有将程序装载到内存中，系统为它分配资源才能运行，而这种执行的程序就称之为进程。

程序和进程的区别就在于：程序是指令的集合，它是进程运行的静态描述文本；进程是程序的一次执行活动，属于动态概念。

An executing instance of a program is called a process.

Each process provides the resources needed to execute a program. A process has a virtual address space, executable code, open handles to system objects, a security context, a unique process identifier, environment variables, a priority class, minimum and maximum working set sizes, and at least one thread of execution. Each process is started with a single thread, often called the primary thread, but can create additional threads from any of its threads.

英文解释

那么他们的区别呢？

进程与线程的区别？

线程：是操作系统最小的调度单位。是一串指令的集合。

进程：要操作cpu必须先创建一个线程

所有在同一个进程里的线程是共享一块内存空间

 1 1、Threads share the address space of the process that created it; processes have their own address space.
 2 1、线程共享内存空间，进程的内存是独立的
 3 
 4 2、Threads have direct access to the data segment of its process; processes have their own copy of the data segment of the parent process.
 5 
 6 3、Threads can directly communicate with other threads of its process; processes must use interprocess communication to communicate with sibling processes.
 7 3、两个进程之间可以直接交流，两个进程进行通讯，必须通过一个中间代理来实现。
 8 
 9 4、New threads are easily created; new processes require duplication of the parent process.
10 4、创建新线程很简单。创建新进程需要对其父进程进行一次克隆
11 
12 5、Threads can exercise considerable control over threads of the same process; processes can only exercise control over child processes.
13 5、一个线程可以控制和操作同意进程里的其他线程，但是进程只能操作子进程
14 
15 6、Changes to the main thread (cancellation, priority change, etc.) may affect the behavior of the other threads of the process; changes to the parent process does not affect child processes.
16

进程与线程的区别

3 线程

（1）线程的调用方式

直接调用：

 1 #Author：ZhangKanghui
 2 
 3 import threading
 4 import time
 5 
 6 def run(n):
 7     print("task",n,threading.current_thread())
 8     time.sleep(2)
 9 '''#这里是只有两个线程，那么更多线程呢？
10 t1 = threading.Thread(target=run,args=("t1",))
11 t2 = threading.Thread(target=run,args=("t2",))
12 t1.start()
13 t2.start()
14 '''
15 #这是一个多线程，那么我们来看一下他整个程序的执行时间为什么不是2s
16 start_time = time.time()
17 for i in range(50):
18     t = threading.Thread(target=run,args=("t-%s" %i,))   #这个地方args必须有，。。因为这是默认一个元组
19     t.start()
20     #t.join()       #这样可以把并行变成串行
21 
22 print("mian threading has finished....",threading.current_thread(),threading.active_count())
23 print("cost:",time.time()-start_time)
24 #这是因为主线程和子线程没关系。主线程不会等子线程执行完毕才计算时间
25 #那接下来如果想要等所有的线程结束然后打印程序执行时间可以这么干
26 '''
27 
28 import threading
29 import time
30 
31 def run(n):
32     print("task",n)
33     time.sleep(2)
34     
35 
36 start_time = time.time()
37 t_obj =[]       #存线程实例
38 for i in range(50):
39     t = threading.Thread(target=run,args=("t-%s" %i,))   #这个地方args必须有，。。因为这是默认一个元组
40     #t.setDaemon(True)              #把当前线程设置成守护线程，必须在启动strat之前
41     t.start()
42     t_obj.append(t) #为了不阻塞后面线程的启动，不在这里加join，先放到一个列表里
43 for i in t_obj:        #循环线程实例列表，等待所有线程执行完毕
44     t.join()
45     
46 print("cost:",time.time()-start_time)
47 
48 '''

直接调用

继承式调用

 1 #Author：ZhangKanghui
 2 
 3 import threading
 4 import time
 5 
 6 class MyThread(threading.Thread):
 7     def __init__(self,n,sleep_time):
 8         super(MyThread,self).__init__()
 9         self.n = n
10         self.sleep_time = sleep_time
11 
12     def run(self):
13         print("running task",self.n)
14         time.sleep(self.sleep_time)
15         print("task done",self.n)
16 
17 t1 =MyThread("t1",2)
18 t2 =MyThread("t2",4)
19 
20 t1.start()
21 t2.start()
22 
23 #t1.join()
24 #t2.join()
25 
26 print("mian threading....")

继承式调用

（2）join&Daemon

其实join就是wait。Daemon就是守护线程。

Some threads do background tasks, like sending keepalive packets, or performing periodic garbage collection, or whatever. These are only useful when the main program is running, and it's okay to kill them off once the other, non-daemon, threads have exited.

Without daemon threads, you'd have to keep track of them, and tell them to exit, before your program can completely quit. By setting them as daemon threads, you can let them run and forget about them, and when your program quits, any daemon threads are killed automatically.

 1 #Author：ZhangKanghui
 2 import time
 3 import threading
 4 
 5 
 6 def run(n):
 7     print('[%s]------running----\n' % n)
 8     time.sleep(2)
 9     print('--done--')
10 
11 
12 def main():
13     for i in range(5):
14         t = threading.Thread(target=run, args=[i, ])
15         t.start()
16         t.join(1)
17         print('starting thread', t.getName())
18 
19 
20 m = threading.Thread(target=main, args=[])
21 m.setDaemon(True)  # 将main线程设置为Daemon线程,它做为程序主线程的守护线程,当主线程退出时,m线程也会退出,由m启动的其它子线程会同时退出,不管是否执行完任务
22 m.start()
23 m.join()
24 # m.join(timeout=2)
25 print("---main thread done----")

join&Daemon

 1 #Author：ZhangKanghui
 2 
 3 import threading
 4 import time
 5 
 6 class MyThread(threading.Thread):
 7     def __init__(self,n,sleep_time):
 8         super(MyThread,self).__init__()
 9         self.n = n
10         self.sleep_time = sleep_time
11 
12     def run(self):
13         print("running task",self.n)
14         time.sleep(self.sleep_time)
15         print("task done",self.n)
16 
17 t1 =MyThread("t1",2)
18 t2 =MyThread("t2",4)
19 
20 t1.start()
21 t2.start()
22 
23 t1.join()
24 t2.join()
25 
26 print("mian threading....")

join1

 1 import threading
 2 import time
 3 
 4 def run(n):
 5     print("task",n)
 6     time.sleep(2)
 7     
 8 start_time = time.time()
 9 t_obj =[]       #存线程实例
10 for i in range(50):
11     t = threading.Thread(target=run,args=("t-%s" %i,))   #这个地方args必须有，。。因为这是默认一个元组
12     #t.setDaemon(True)              #把当前线程设置成守护线程，必须在启动strat之前
13     t.start()
14     t_obj.append(t) #为了不阻塞后面线程的启动，不在这里加join，先放到一个列表里
15 for i in t_obj:        #循环线程实例列表，等待所有线程执行完毕
16     t.join()
17     
18 print("cost:",time.time()-start_time)

join2

（3）线程锁/递归锁/信号量

线程锁

一个进程下可以启动多个线程，多个线程共享父进程的内存空间，也就意味着每个线程可以访问同一份数据，此时，如果2个线程同时要修改同一份数据，会出现什么状况？

 1 import time
 2 import threading
 3  
 4 def addNum():
 5     global num #在每个线程中都获取这个全局变量
 6     print('--get num:',num )
 7     time.sleep(1)
 8     lock.acquire() #修改数据前加锁
 9     num  -=1 #对此公共变量进行-1操作
10     lock.release() #修改后释放
11  
12 num = 100  #设定一个共享变量
13 thread_list = []
14 lock = threading.Lock() #生成全局锁
15 for i in range(100):
16     t = threading.Thread(target=addNum)
17     t.start()
18     thread_list.append(t)
19  
20 for t in thread_list: #等待所有线程执行完毕
21     t.join()
22  
23 print('final num:', num )

线程锁

递归锁

就是在一个大锁中还要再包含子锁

 1 import threading,time
 2  
 3 def run1():
 4     print("grab the first part data")
 5     lock.acquire()
 6     global num
 7     num +=1
 8     lock.release()
 9     return num
10 def run2():
11     print("grab the second part data")
12     lock.acquire()
13     global  num2
14     num2+=1
15     lock.release()
16     return num2
17 def run3():
18     lock.acquire()
19     res = run1()
20     print('--------between run1 and run2-----')
21     res2 = run2()
22     lock.release()
23     print(res,res2)
24  
25  
26 if __name__ == '__main__':
27  
28     num,num2 = 0,0
29     lock = threading.RLock()
30     for i in range(10):
31         t = threading.Thread(target=run3)
32         t.start()
33  
34 while threading.active_count() != 1:
35     print(threading.active_count())
36 else:
37     print('----all threads done---')
38     print(num,num2)

递归锁

信号量（Semaphore）

互斥锁同时只允许一个线程更改数据，而Semaphore是同时允许一定数量的线程更改数据。

比如厕所有3个坑，那最多只允许3个人上厕所，后面的人只能等里面有人出来了才能再进去。

 1 import threading,time
 2  
 3 def run(n):
 4     semaphore.acquire()
 5     time.sleep(1)
 6     print("run the thread: %s\n" %n)
 7     semaphore.release()
 8  
 9 if __name__ == '__main__':
10  
11     num= 0
12     semaphore  = threading.BoundedSemaphore(5) #最多允许5个线程同时运行
13     for i in range(20):
14         t = threading.Thread(target=run,args=(i,))
15         t.start()
16  
17 while threading.active_count() != 1:
18     pass #print threading.active_count()
19 else:
20     print('----all threads done---')
21     print(num)

信号量

（4）Timer

This class represents an action that should be run only after a certain amount of time has passed。

1 #Author：ZhangKanghui
2 import threading
3 def hello():
4     print("hello, world")
5 
6 t = threading.Thread(target=hello)
7 t = threading.Timer(30.0, hello)
8 t.start()  # after 30 seconds, "hello, world" will be printed

Timer

（5）Event

An event is a simple synchronization object;通过Event来实现两个或多个线程间的交互。

the event represents an internal flag, and threads can wait for the flag to be set, or set or clear the flag themselves.

event.wait()

event.set()

event.clear()

下面是一个红绿灯的例子，即起动一个线程做交通指挥灯，生成几个线程做车辆，车辆行驶按红灯停，绿灯行的规则。

 1 import threading,time
 2 import random
 3 def light():
 4     if not event.isSet():
 5         event.set() #wait就不阻塞 #绿灯状态
 6     count = 0
 7     while True:
 8         if count < 10:
 9             print('\033[42;1m--green light on---\033[0m')
10         elif count <13:
11             print('\033[43;1m--yellow light on---\033[0m')
12         elif count <20:
13             if event.isSet():
14                 event.clear()
15             print('\033[41;1m--red light on---\033[0m')
16         else:
17             count = 0
18             event.set() #打开绿灯
19         time.sleep(1)
20         count +=1
21 def car(n):
22     while 1:
23         time.sleep(random.randrange(10))
24         if  event.isSet(): #绿灯
25             print("car [%s] is running.." % n)
26         else:
27             print("car [%s] is waiting for the red light.." %n)
28 if __name__ == '__main__':
29     event = threading.Event()
30     Light = threading.Thread(target=light)
31     Light.start()
32     for i in range(3):
33         t = threading.Thread(target=car,args=(i,))
34         t.start()

Event实例之红绿灯

这里还有一个event使用的例子，员工进公司门要刷卡，我们这里设置一个线程是“门”，再设置几个线程为“员工”，员工看到门没打开，就刷卡，刷完卡，门开了，员工就可以通过。

 1 #_*_coding:utf-8_*_
 2 __author__ = 'Alex Li'
 3 import threading
 4 import time
 5 import random
 6 
 7 def door():
 8     door_open_time_counter = 0
 9     while True:
10         if door_swiping_event.is_set():
11             print("\033[32;1mdoor opening....\033[0m")
12             door_open_time_counter +=1
13 
14         else:
15             print("\033[31;1mdoor closed...., swipe to open.\033[0m")
16             door_open_time_counter = 0 #清空计时器
17             door_swiping_event.wait()
18 
19 
20         if door_open_time_counter > 3:#门开了已经3s了,该关了
21             door_swiping_event.clear()
22 
23         time.sleep(0.5)
24 
25 
26 def staff(n):
27 
28     print("staff [%s] is comming..." % n )
29     while True:
30         if door_swiping_event.is_set():
31             print("\033[34;1mdoor is opened, passing.....\033[0m")
32             break
33         else:
34             print("staff [%s] sees door got closed, swipping the card....." % n)
35             print(door_swiping_event.set())
36             door_swiping_event.set()
37             print("after set ",door_swiping_event.set())
38         time.sleep(0.5)
39 door_swiping_event  = threading.Event() #设置事件
40 
41 
42 door_thread = threading.Thread(target=door)
43 door_thread.start()
44 
45 
46 
47 for i in range(5):
48     p = threading.Thread(target=staff,args=(i,))
49     time.sleep(random.randrange(3))
50     p.start()

event实例之刷卡

（6）queue队列

queue is especially useful in threaded programming when information must be exchanged safely between multiple threads.

class queue.Queue(maxsize=0) #先入先出
class queue.LifoQueue(maxsize=0) #last in fisrt out
class queue.PriorityQueue(maxsize=0) #存储数据时可设置优先级的队列

Queue.qsize()
Queue.empty() #return True if empty
Queue.full() # return True if full
Queue.get(block=True, timeout=None)
Queue.put(item, block=True, timeout=None)
Queue.put_nowait(item)

Queue.get_nowait()
Queue.join() block直到queue被消费完毕

a、先入先出

 1 #Author：ZhangKanghui
 2 
 3 import queue
 4 q =queue.Queue()
 5 
 6 q.put(1)
 7 q.put(2)
 8 q.put(3)
 9 
10 print(q.qsize())
11 print(q.get())
12 print(q.get())
13 print(q.get())
14 #先入先出，这个时候如果在获取呢？就会挂起卡死
15 print(q.get())
16 #在这里看不出来什么效果。去命令行试试

先入先出

存在一个问题就是，当把所有的数据都get到之后，程序就会挂起卡死。那这个时候怎么办呢？

q.get_nowait（）或者通过get（block=True，Timeout）参数进行设置

会出现先一个queue Empty的异常，这个时候可以通过捕获异常使程序正常进行。

b、后进先出

 1 #Author：ZhangKanghui
 2 
 3 import queue
 4 q =queue.LifoQueue()
 5 q.put(1)
 6 q.put(2)
 7 q.put(3)
 8 
 9 print(q.qsize())
10 print(q.get())
11 print(q.get())
12 print(q.get())

后进先出LifoQueue

c、设置优先级

 1 #Author：ZhangKanghui
 2 
 3 import queue
 4 
 5 q =queue.PriorityQueue()
 6 q.put((1,"erha"))
 7 q.put((5,"hashiqi"))
 8 q.put((3,"taidi"))
 9 print(q.get())
10 print(q.get())
11 print(q.get())

优先级队列

下面以一个生产者消费者的模型实例来深刻体会一下队列。

import threading
import queue
 
def producer():
    for i in range(10):
        q.put("baozi %s" % i )
 
    print("开始等待所有的包子被取走...")
    q.join()
    print("所有的包子被取完了...")
 
 
def consumer(n):
 
    while q.qsize() >0:
 
        print("%s 取到" %n  , q.get())
        q.task_done() #告知这个任务执行完了
 
 
q = queue.Queue()
 
 
 
p = threading.Thread(target=producer,)
p.start()
 
c1 = consumer("二哈")

生产者消费者模型1

 1 #Author：ZhangKanghui
 2 
 3 import threading,time
 4 import queue
 5 
 6 q = queue.Queue()
 7 def Producer(namne):
 8     count = 0
 9     for i in range(10):
10         q.put("骨头%s"%count)
11         print("生产了骨头",count)
12         count +=1
13         #time.sleep(2)
14         time.sleep(1)
15         #time.sleep(0.5)
16 
17 def Consumer(name):
18     while True:
19         print("[%s] 取到 [%s] 并吃了它..."%(name,q.get()))
20 
21 p =threading.Thread(target=Producer,args=("Kanghui",))
22 c =threading.Thread(target=Consumer,args=("erha",))
23 c1 =threading.Thread(target=Consumer,args=("taidi",))
24 
25 p.start()
26 c.start()
27 c1.start()

生产者消费者模型2

4 小结

志不坚者智不达。

撑不住的时候可以对自己说声“我好累”，但永远不要再心里承认“我不行”。

没有鸡汤就没有生活。

5 练习

需求:

主机分组
登录后显示主机分组，选择分组后查看主机列表
可批量执行命令、发送文件，结果实时返回
主机用户名密码可以不同

流程图：

 1 ### 作者介绍：
 2 * author：lzl
 3 ### 博客地址：
 4 * http://www.cnblogs.com/lianzhilei/p/5881434.html
 5 
 6 ### 功能实现
 7 题目:简单主机批量管理工具
 8 
 9     需求:
10     主机分组
11     登录后显示主机分组，选择分组后查看主机列表
12     可批量执行命令、发送文件，结果实时返回
13     主机用户名密码可以不同
14 
15 ### 目录结构：
16     Host-Manage
17     │
18     ├── ftpclient #客户端程序
19             ├── README.txt
20             ├── management.py #服务端入口程序
21             ├── database #数据库
22             ├── test.py #修改数据库
23 
24 
25 ### 注释
26     可批量执行命令、发送文件
27     上传命令格式： put database /tmp/db
28 
29 
30 ### 运行环境
31     windows系统
32     python3.0+
33 
34 README

Readme

 1 import json
 2 import paramiko
 3 import threading
 4  
 5 class Remotehost(object):
 6     #远程操作主机
 7     def __init__(self,host,port,username,password,cmd):
 8         self.host = host
 9         self.port = port
10         self.username = username
11         self.password = password
12         self.cmd = cmd
13  
14     def command(self):
15         #获取命令
16         ssh = paramiko.SSHClient()
17         ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())   # 允许连接不在know_hosts文件中的主机
18         ssh.connect(hostname=self.host, port=self.port, username=self.username, password=self.password)   # 连接服务器
19         stdin, stdout, stderr = ssh.exec_command(self.cmd)              # 获取命令结果
20         res ,err = stdout.read(),stderr.read()              # 三元运算
21         result = res if res else err
22         print("[%s]".center(50,"-")%self.host)
23         print(result.decode())                                      # 打印输出
24         ssh.close()
25  
26     def put(self):
27         #上传
28         try:
29             transport = paramiko.Transport((self.host, self.port))
30             transport.connect(username=self.username, password=self.password)
31             sftp = paramiko.SFTPClient.from_transport(transport)
32             sftp.put(self.cmd.split()[1], self.cmd.split()[2])              # 上传文件
33             transport.close()
34             print("\033[32;0m【%s】 上传 文件【%s】 成功....\033[0m"%(self.host,self.cmd.split()[2]))
35         except Exception as error:                                # 抓住异常
36             print("\033[31;0m错误:【%s】【%s】\033[0m"%(self.host,error))
37  
38     def run(self):
39         #反射
40         cmd_str = self.cmd.split()[0]
41         if hasattr(self,cmd_str):
42             getattr(self,cmd_str)()
43         else:
44             setattr(self,cmd_str,self.command)
45             getattr(self,cmd_str)()
46  
47  
48 if __name__ == "__main__":
49     #主程序
50     with open("database","r") as file:
51         data_dict = json.loads(file.read())     #获取数据库信息
52     for k in data_dict:                        #打印地址组
53         print(k)
54  
55     group_choice = input("输入要操作的组名：").strip()
56     if data_dict.get(group_choice):
57         host_dict = data_dict[group_choice]     #定义主机字典
58         for k in host_dict:                    #打印所选地址组所有的主机名
59             print(k)
60         while True:
61             cmd = input("选择进行的操作的命令：").strip()
62             thread_list=[]
63             if cmd:                                 #命令不为空
64                 for k in host_dict:
65                     host, port, username, password=k,host_dict[k]["port"],host_dict[k]["username"],host_dict[k]["password"]
66                     func = Remotehost(host,port,username,password,cmd)      #实例化类
67                     t = threading.Thread(target=func.run)                   #创建线程
68                     t.start()
69                     thread_list.append(t)
70                 for t in thread_list:
71                     t.join()                                                #等待线程执行结果
72     else:
73         print("\033[31;0m操作组不存在\033[0m")

简单主机批量管理工具主程序

我是尾巴~

这次推荐：摄影app合集　　https://mp.weixin.qq.com/s/3N3m7otgKIZXGyUpGKbTpw

虽不才，才要坚持。

秒客网

D09——C语言基础学PYTHON

什么是线程(thread)？

什么是进程(process)？

进程与线程的区别？

相关文章