So I am doing a lot of high performance network programming using Boost::Asio (or just Asio if you will), and have a pretty solid grasp of the essentials of both TCP and UDP protocols. I am wondering though, because I still don't consider myself an expert in networking despite my knowledge, what is a good way to frame the essentials of what networking programmers should know, especially for those trying to push the performance of their large networking based applications?
因此,我正在使用Boost :: Asio(或者只是Asio)进行大量高性能网络编程,并且非常可靠地掌握TCP和UDP协议的基本要素。我很想知道,因为尽管我知道,我仍然不认为自己是网络专家,什么是构建网络程序员应该知道的基本要素的好方法,特别是那些试图推动其大型网络性能的人应用程序?
There is a great essay on programmers and what they should know about memory (see below), so I'm wondering if someone has put together something similar for networking.
关于程序员以及他们应该了解的内存(见下文),有一篇很棒的文章,所以我想知道是否有人为网络做了类似的事情。
What every programmer should know about memory
每个程序员应该了解的内存
1 个解决方案
#1
14
Some bullet points off the top of my head of things you should know:
一些子弹指出我应该知道的事情:
- How and why TCP works... 3-way handshakes, acknowledgement, delayed ack, nagling, sliding window protocol. There's a concrete reason for every one of those features... and they can all destroy your application's performance if handled improperly.
- UDP multicast... even if you never think you'll use it, you need to know why it exists so you can make educated decisions when designing systems.
- IP fragmentation, and the impact of MTU.
- Binary serialization and network byte ordering (even if you're just going to use Google proto buffers, it's nice to understand why they are efficient).
- Ascii serialization and message framing (what does
\r\n\r\nmean in HTTP?) - Different I/O dispatch models: Apache-style preforking, thread-per-connection, event-based single-threaded, event-based with worker threads, etc.
- The impact of buffer-overflow vulnerabilities in a networked app
- Protocol-based design, as opposed to API- or library-based design
- asynchronous vs synchronous protocols. Many high-performance systems are asynchronous. HTTP is synchronous unless you use pipelining, and even then, there are many restrictions on what is possible... no out-of-order responses, for example.
TCP的工作原理和原因... 3路握手,确认,延迟确认,唠叨,滑动窗口协议。每个功能都有一个具体原因......如果处理不当,它们都会破坏应用程序的性能。
UDP多播...即使你从未想过你会使用它,你需要知道它为什么存在,以便你在设计系统时做出明智的决定。
IP分片,以及MTU的影响。
二进制序列化和网络字节排序(即使你只是要使用谷歌原型缓冲区,很高兴理解为什么它们是有效的)。
Ascii序列化和消息框架(HTTP中的\ r \ n \ r \ n是什么意思?)
不同的I / O调度模型:Apache样式的preforking,每个连接的线程,基于事件的单线程,基于事件的工作线程等。
网络应用程序中缓冲区溢出漏洞的影响
基于协议的设计,而不是基于API或基于库的设计
异步与同步协议。许多高性能系统都是异步的。 HTTP是同步的,除非你使用流水线技术,即便如此,对可能的内容有很多限制......例如,没有无序响应。
Update: What does protocol-based design mean?
更新:基于协议的设计意味着什么?
Consider HTTP, the protocol of the web. Apache, IIS, Lighttpd, Firefox, Opera, WebKit, etc... All of these pieces of software speak HTTP. It's quite possible that none of them are sharing the code to do so. The downside, of course, is the increased likelihood of bugs due to the net volume of code. There are numerous upsides:
考虑HTTP,即网络协议。 Apache,IIS,Lighttpd,Firefox,Opera,WebKit等......所有这些软件都讲HTTP。很可能他们都没有共享代码来这样做。当然,缺点是由于代码净容量导致错误的可能性增加。有许多好处:
- Any program can communicate via HTTP, regardless of implementation language
- Lightweight/embedded environments can pick and choose a subset of the protocol, rather than using the whole thing
- It's possible to optimize a protocol handler for particular situations. It's not possible to optimize a library without sacrificing generality.
- A variety of different implementations forces library providers to address bugs (rather than just blowing them off because, well, everyone uses the same library).
- There is no organizational or contractual burden on users of HTTP, no licensing fees.
无论实现语言如何,任何程序都可以通过HTTP进行通信
轻量级/嵌入式环境可以选择协议的一个子集,而不是使用整个协议
可以针对特定情况优化协议处理程序。在不牺牲一般性的情况下优化库是不可能的。
各种不同的实现强制库提供程序解决错误(而不是仅仅将它们删除,因为每个人都使用相同的库)。
HTTP用户没有组织或合同负担,没有许可费用。
When you design a network protocol, you can build yourself several APIs, each tailored towards specific use-cases. Or you can build one, it's up to you. Networked software components can be upgraded independent of each other. Basically, everything you hear that's good about Java/C# Interfaces and C++ abstract classes, but applied at the network layer rather than the programming language layer.
在设计网络协议时,您可以自己构建几个API,每个API都针对特定的用例进行定制。或者你可以建立一个,这取决于你。网络化软件组件可以相互独立升级。基本上,您听到的所有内容都与Java / C#接口和C ++抽象类有关,但应用于网络层而不是编程语言层。
#1
14
Some bullet points off the top of my head of things you should know:
一些子弹指出我应该知道的事情:
- How and why TCP works... 3-way handshakes, acknowledgement, delayed ack, nagling, sliding window protocol. There's a concrete reason for every one of those features... and they can all destroy your application's performance if handled improperly.
- UDP multicast... even if you never think you'll use it, you need to know why it exists so you can make educated decisions when designing systems.
- IP fragmentation, and the impact of MTU.
- Binary serialization and network byte ordering (even if you're just going to use Google proto buffers, it's nice to understand why they are efficient).
- Ascii serialization and message framing (what does
\r\n\r\nmean in HTTP?) - Different I/O dispatch models: Apache-style preforking, thread-per-connection, event-based single-threaded, event-based with worker threads, etc.
- The impact of buffer-overflow vulnerabilities in a networked app
- Protocol-based design, as opposed to API- or library-based design
- asynchronous vs synchronous protocols. Many high-performance systems are asynchronous. HTTP is synchronous unless you use pipelining, and even then, there are many restrictions on what is possible... no out-of-order responses, for example.
TCP的工作原理和原因... 3路握手,确认,延迟确认,唠叨,滑动窗口协议。每个功能都有一个具体原因......如果处理不当,它们都会破坏应用程序的性能。
UDP多播...即使你从未想过你会使用它,你需要知道它为什么存在,以便你在设计系统时做出明智的决定。
IP分片,以及MTU的影响。
二进制序列化和网络字节排序(即使你只是要使用谷歌原型缓冲区,很高兴理解为什么它们是有效的)。
Ascii序列化和消息框架(HTTP中的\ r \ n \ r \ n是什么意思?)
不同的I / O调度模型:Apache样式的preforking,每个连接的线程,基于事件的单线程,基于事件的工作线程等。
网络应用程序中缓冲区溢出漏洞的影响
基于协议的设计,而不是基于API或基于库的设计
异步与同步协议。许多高性能系统都是异步的。 HTTP是同步的,除非你使用流水线技术,即便如此,对可能的内容有很多限制......例如,没有无序响应。
Update: What does protocol-based design mean?
更新:基于协议的设计意味着什么?
Consider HTTP, the protocol of the web. Apache, IIS, Lighttpd, Firefox, Opera, WebKit, etc... All of these pieces of software speak HTTP. It's quite possible that none of them are sharing the code to do so. The downside, of course, is the increased likelihood of bugs due to the net volume of code. There are numerous upsides:
考虑HTTP,即网络协议。 Apache,IIS,Lighttpd,Firefox,Opera,WebKit等......所有这些软件都讲HTTP。很可能他们都没有共享代码来这样做。当然,缺点是由于代码净容量导致错误的可能性增加。有许多好处:
- Any program can communicate via HTTP, regardless of implementation language
- Lightweight/embedded environments can pick and choose a subset of the protocol, rather than using the whole thing
- It's possible to optimize a protocol handler for particular situations. It's not possible to optimize a library without sacrificing generality.
- A variety of different implementations forces library providers to address bugs (rather than just blowing them off because, well, everyone uses the same library).
- There is no organizational or contractual burden on users of HTTP, no licensing fees.
无论实现语言如何,任何程序都可以通过HTTP进行通信
轻量级/嵌入式环境可以选择协议的一个子集,而不是使用整个协议
可以针对特定情况优化协议处理程序。在不牺牲一般性的情况下优化库是不可能的。
各种不同的实现强制库提供程序解决错误(而不是仅仅将它们删除,因为每个人都使用相同的库)。
HTTP用户没有组织或合同负担,没有许可费用。
When you design a network protocol, you can build yourself several APIs, each tailored towards specific use-cases. Or you can build one, it's up to you. Networked software components can be upgraded independent of each other. Basically, everything you hear that's good about Java/C# Interfaces and C++ abstract classes, but applied at the network layer rather than the programming language layer.
在设计网络协议时,您可以自己构建几个API,每个API都针对特定的用例进行定制。或者你可以建立一个,这取决于你。网络化软件组件可以相互独立升级。基本上,您听到的所有内容都与Java / C#接口和C ++抽象类有关,但应用于网络层而不是编程语言层。