Python并发编程eventlet

时间:2021-09-24 21:01:48

1 多进程、多线程和协程

python并发编程之多进程、多线程、异步和协程:http://www.cnblogs.com/tyomcat/p/5486827.html

1.1 总结

1)多进程能够利用多核优势,但是进程间通信比较麻烦,另外进程数目的增加会使性能下降,进程切换的成本较高。程序流程复杂度相对I/O多路复用要低。
2)I/O多路复用是在一个进程内部处理多个逻辑流程,不用进行进程切换,性能较高,另外流程间共享信息简单。但是无法利用多核优势,另外,程序流程被事件处理切割成一个个小块,程序比较复杂,难于理解。
3)线程运行在一个进程内部,由操作系统调度,切换成本,另外,他们共享进程的虚拟地址空间,线程间共享信息简单。但是线程安全问题导致线程学习曲线陡峭,而且易出错。
4)协程有编程语言提供,由程序员控制进行切换,所以没有线程安全问题,可以用来处理状态机,并发请求等。但是无法利用多核优势。

1.2 协程

介绍:http://blog.csdn.net/k_zombie/article/details/50839961

协程不同于线程的地方在于协程不是操作系统进行切换,而是由程序员编码进行切换的,也就是说切换是由程序员控制的,这样就没有了线程所谓的安全问题。


协程的缺点:无法利用多核优势,不过,这个可以通过协程+进程的方式来解决。如果你的任务类型不是那种cpu密集的,那选用协程是个好选择。


1.3 多线程

参考:http://blog.csdn.net/k_zombie/article/details/50769128


2 python yield和generator

参考:http://blog.csdn.net/k_zombie/article/details/50815256


3 greenlet

The greenlet package is a spin-off of Stackless, a version of CPython that supports micro-threads called “tasklets”. Tasklets run pseudo-concurrently (typically in a single or a few OS-level threads) and are synchronized with data exchanges on “channels”.

A “greenlet”, on the other hand, is a still more primitive notion of micro-thread with no implicit scheduling; coroutines, in other words. This is useful when you want to control exactly when your code runs. You can build custom scheduled micro-threads on top of greenlet; however, it seems that greenlets are useful on their own as a way to make advanced control flow structures. For example, we can recreate generators; the difference with Python’s own generators is that our generators can call nested functions and the nested functions can yield values too. Additionally, you don’t need a “yield” keyword. See the example in tests/test_generator.py.

Greenlets are provided as a C extension module for the regular unmodified interpreter.

Greenlets are lightweight coroutines for in-process concurrent programming.

Who is using Greenlet?

There are several libraries that use Greenlet as a more flexible alternative to Python’s built in coroutine support:

Getting Greenlet

The easiest way to get Greenlet is to install it with pip or easy_install:

pip install greenlet
easy_install greenlet

Source code archives and windows installers are available on the python package index at https://pypi.python.org/pypi/greenlet

The source code repository is hosted on github: https://github.com/python-greenlet/greenlet

Documentation is available on readthedocs.org: https://greenlet.readthedocs.io


2.2 源码分析

python协程的实现(greenlet源码分析):http://blog.csdn.net/fjslovejhl/article/details/38824963

python协程入门(greenlet):http://blog.csdn.net/fjslovejhl/article/details/38821673

python greenlet背景介绍与实现机制:http://blog.csdn.net/offbye/article/details/39368781


2.3 


3 Eventlet

官网:http://eventlet.net/

Eventlet is a concurrent networking library for Python that allows you to change how you run your code, not how you write it.

  • It uses epoll or kqueue or libevent for highly scalable non-blocking I/O.
  • Coroutines ensure that the developer uses a blocking style of programming that is similar to threading, but provide the benefits of non-blocking I/O.
  • The event dispatch is implicit, which means you can easily use Eventlet from the Python interpreter, or as a small part of a larger application.

eventlet是一个用来处理和网络相关的Python库函数,而且可以通过协程来实现并发,在eventlet里,把“协程”叫做greenthread(绿色线程)。



参考: openstack nova 基础知识——eventlet:http://blog.csdn.net/xiangmin2587/article/details/8182775
Openstack Eventlet分析(1):http://blog.csdn.net/gaoxingnengjisuan/article/details/12913275
Openstack Eventlet分析(2):http://blog.csdn.net/gaoxingnengjisuan/article/details/12914831