欢迎关注:1,欢迎关注本博客,你可点击右手边的【QQ邮件订阅】订阅本博客!2,本博客推出江湖救急计划,主要为工作中遇到疑难杂症的兄弟提供远程技术支持和分析,如有需要,请在江湖救急计划页面给我留言!

TCP 的PUSH标志位

作者:易隐者 发布于:2012-8-31 15:32 Friday 分类:网络分析

       发送方使用该标志通知接收方将所收到的数据全部提交给接收进程。这里的数据包括接收方已经接收放在接收缓存的数据和刚刚收到的PUSH位置一的TCP报文中封装的应用数据。
还是看一个简单明了的图示吧:

点击查看原图

关于PUSH位的一些讨论和分析

       Long_323(龙的传人)兄弟曾跟我讨论:在无法解码应用层报文时,能否根据TCP  PUSH位来分析应用响应时间?

       我的回答是:不可以,我个人不会这么做,因为不够科学。

       所谓应用响应时间,是指客户端有发出请求,服务器端会针对该请求作出相应的响应,我们才可以分析计算其响应时间。如果我们无法解码应用层,那么我们就无法确定其到底是不是一个请求报文,如果不是请求报文(例如,我的《TCP交互交互式应用》一文中提到的互动游戏的报文、TCP keepalive保活的报文等)但是其TCP PUSH位是置一的,那么我们怎么能根据这一点就计算出应用的响应时间呢?

       也许有经常关注我的兄弟会问:你说不可以,那怎么在你写的文章《关于“client push”应用响应时间测量方法的讨论》中,建议说“从客户端带有PUSH标志位的数据包开始计算,至服务器端响应应用层数据停止计算”呢?如果真有这么问的兄弟,我会非常欣慰,因为能问出这个问题的人,说明他有认真仔细的研读过我的文章,那么他肯定会从中获得一些帮助和成长,我会为此而心生满足。言归正传,在我的《关于“client push”应用响应时间测量方法的讨论》一文里,全文的基础和基调都已经定在为“client push”的应用场景了,看标题你就会明白,不需要我多说了吧。

       言已至此,我想大家也会明白,我说的不可以并不是绝对不可以,而只是站在我个人的角度而说的,仅代表我的个人主观选择。

       在有些情况下,可以作为一个参考分析思路。那么到底在哪些情况下能这么做呢?我个人认为主要有以下几点:
1, 可以明确我们要分析的应用类型为请求-响应类的应用
2, 虽不能解码,但是我们在做分析时,需要将具体的请求操作和数据报文结合起来分析,以提高分析的科学性和准确性
3, 对分析的结果要求不是非常高、非常严谨的

       还有兄弟认为如果所有带有应用字段的TCP报文PUSH位都置一的话,会对服务器造成较为严重的性能影响。这个所谓的影响如果真的存在的话,我想应该是在服务器的内存、CPU等资源严重不足或应用程序本身处理存在严重性能问题时才会出现,是因为本来就有问题了,导致PUSH位都置一的报文加重了这种影响,而不是因为PUSH位置一的报文导致了服务器的性能问题的出现。

关于PUSH位的应用

        PUSH位就是用来通告接收方立即将收到的报文连同TCP接收缓存里的数据递交应用进程处理。一般会出现在发送方封装最后一个应用字段的TCP报文中,针对TCP交互式应用,则只要封装有应用字段的TCP报文,均会将PUSH位置一,当然,应用程序的开发者,可以根据需要,在某个应用功能模块或某个应用操作时,将所有封装应用字段的TCP报文PUSH位置一,以提高交互双方的处理效率,这在理论上应该也是可行的。

  

阅读全文>>

标签: TCP 应用字段 响应时间 应用响应时间 ART 保活 push 交互式应用 client push 接收缓存 keepalive

评论(4) 引用(0) 浏览(24428)

TCP交互式应用

作者:易隐者 发布于:2012-8-30 21:29 Thursday 分类:网络分析

       TCP交互式应用由于其交互的特性,一般对时延的要求较高,因此一旦有应用数据时便立即向对端发送并要求对端能够及时处理,其交互的报文基本上都是小包(小于MSS),常见的TCP交互式应用主要有telnet、rlogin、互动游戏等,我们先来看一个TCP交互应用的报文分析:

点击查看原图

其具有以下三个明显特点:

1,客户端与服务器端的数据传输是交互式进行的;

2,应用字段长度均较小;

3,带有应用字段的TCP报文PUSH位均置一。

TCP交互式应用的场景(以游戏为例):

       对于实时网络游戏这种交互应用,如果TCP接收方不立即将应用数据提交应用程序处理的话,很可能会导致游戏应用出现卡、客户端与服务器端不同步故障,严重影响游戏应用的正常运行。你想啊,你在玩CS,扮演警察,突然一个悍匪跟你正面冲突,悍匪那边快速操作已经移动位置了,但是悍匪移位这个应用信息却被你的TCP放在接收缓存里,你机器显示器上看到的悍匪还在原来的位置,然后你自然向悍匪原来的位置开枪,结果呢,肯定没伤到悍匪,却被早就移位的悍匪爆头击毙,这时候你会爽吗?肯定骂死这个游戏应用了,呵呵。
        那么交互式应用如何保证应用字段能够被立即处理呢答案就是将带有应用字段的TCP报文(不论应用字段大小)的PUSH位全部置一,虽是小包,但TCP一旦接收到便立即将其交付应用程序处理。

       我在08年做内部《TCPIP详解卷一》培训在讲到第19章《TCP的交互数据流》时,曾今专门捕获了WAR 3的局域网游戏的交互报文,用于讲解TCP交互式应用的特性,现在把这个报文一并放在此供各位参考。

WAR 3的局域网游戏的报文:

点击查看原图

        交互式应用并不是非常常见,因此经常被大家忽略,导致有时在遇到交互应用场景的分析时,将其当作HTTP等常见应用的特性来进行分析,可想而知,如此将走向错误的分析方向,望各位兄弟姐妹引以为鉴。

        这里提到了TCP报文的一个非常重要的标志位——PUSH,我下一篇文章将会针对PUSH位做一个详尽的讨论和分析,望大家关注。

阅读全文>>

标签: 同步 TCP 应用字段 故障 push 交互式应用 交互

评论(0) 引用(0) 浏览(90988)

蚂蚁与TCP

作者:易隐者 发布于:2012-8-29 8:59 Wednesday 分类:其 他

       今天在上看到一帖,很有意思,说是斯坦福大学的研究者发现蚂蚁的行为跟TCP相关特性非常相像,其觅食行为跟TCP的拥塞避免、慢启动、超时机制等暗合,我们先一起来看一下原文吧。

【原文】:

链接:http://jandan.net/2012/08/27/ants-internet.html

人类能做到的一些超炫的事情,其实早就存在了!比如,蚂蚁觅食,就跟网络传送协议(TCP/Internet's Transmission Control Protocol)的方法基本差不多,蚂蚁们一直就这样收集着食物。假如先前的蚂蚁出去好久了都还没带食物回的话,它们就不再外派找吃的了;当初级数据包显示小带宽时,TCP就会节流数据传输。

蚂蚁们也使用TCP的慢启动技术,它们发出派出一小波觅食的蚂蚁,来计算出食物的相对数量(带宽),再调节派出蚂蚁的数目。相似的是,若网络链接超时,源头就会停止传送数据包;若过了20分钟蚂蚁们还没回来,外派停止。

这项发现的研究者之一 Balaji Prabhakar 说,假如蚂蚁的这个行为在网络之前就被发现的话,可能会影响网络的设计。这种觅食方式久经时间考验,可能有很多值得我们学习的地方。鬼知道呢,说不定此刻也还有其它的算法早就存在着,正默默地等着被发现。

【英文原文】:

链接:http://gizmodo.com/5937981/ants-have-been-using-internet-algorithms-for-millions-of-years

Mankind has been able to accomplish some pretty impressive things, but some of them were around long before we figured them out. Ants, for instance, hunt for food in a way that's basically the same as the Internet's Transmission Control Protocol (TCP), and they were doing it long before the Internet was around.

It all has to do with how harvester ants gather their food. The same way that TCP will throttle data transmission if initial packets indicate little bandwidth, harvester ants will send less foragers out for food if the initial ones take too long to come back with grub.

From Stanford News:

[The] rate at which harvester ants – which forage for seeds as individuals – leave the nest to search for food corresponds to food availability.

A forager won't return to the nest until it finds food. If seeds are plentiful, foragers return faster, and more ants leave the nest to forage. If, however, ants begin returning empty handed, the search is slowed, and perhaps called off.

And that's not where the similarities end either. Ants also use TCP's slow start technique, by sending out a wave of foragers (packets) to figure out the relative amount of food (bandwidth) before scaling their numbers up or down. Likewise, the same way a connection will time out if the source stops sending packets, the ants will stop sending out new foragers if none return for 20 minutes.

Balaji Prabhakar, one of the researchers behind the discovery, says that if this behavior had been uncovered pre-Internet, it might have influenced its design. Even so, this foraging process has been seriously time-tested, and there still might be things we can learn from it. In the meantime, who knows what other algorithms might already be out there, quietly waiting to be discovered.

      我顺便把斯坦福大学的研究报道找出来了,链接如下:

http://news.stanford.edu/news/2012/august/ants-mimic-internet-082312.html

斯坦福研究者发现“蚂蚁网”-Stanford researchers discover the 'anternet'

A collaboration between a Stanford ant biologist and a computer scientist has revealed that the behavior of harvester ants as they forage for food mirrors the protocols that control traffic on the Internet.

Katherine Decktar Harvester ant foragers waiting inside the nest

Harvester ant foragers waiting inside the nest.

On the surface, ants and the Internet don't seem to have much in common. But two Stanford researchers have discovered that a species of harvester ants determine how many foragers to send out of the nest in much the same way that Internet protocols discover how much bandwidth is available for the transfer of data. The researchers are calling it the "anternet."

Deborah Gordon, a biology professor at Stanford, has been studying ants for more than 20 years. When she figured out how the harvester ant colonies she had been observing in Arizona decided when to send out more ants to get food, she called across campus to Balaji Prabhakar, a professor of computer science at Stanford and an expert on how files are transferred on a computer network. At first he didn't see any overlap between his and Gordon's work, but inspiration would soon strike.

"The next day it occurred to me, 'Oh wait, this is almost the same as how [Internet] protocols discover how much bandwidth is available for transferring a file!'" Prabhakar said. "The algorithm the ants were using to discover how much food there is available is essentially the same as that used in the Transmission Control Protocol."

Transmission Control Protocol, or TCP, is an algorithm that manages data congestion on the Internet, and as such was integral in allowing the early web to scale up from a few dozen nodes to the billions in use today. Here's how it works: As a source, A, transfers a file to a destination, B, the file is broken into numbered packets. When B receives each packet, it sends an acknowledgment, or an ack, to A, that the packet arrived.

This feedback loop allows TCP to run congestion avoidance: If acks return at a slower rate than the data was sent out, that indicates that there is little bandwidth available, and the source throttles data transmission down accordingly. If acks return quickly, the source boosts its transmission speed. The process determines how much bandwidth is available and throttles data transmission accordingly.

L.A. CiceroDeborah Gordon

Biologist Deborah Gordon has been studying ants for more than 20 years.

It turns out that harvester ants (Pogonomyrmex barbatus) behave nearly the same way when searching for food. Gordon has found that the rate at which harvester ants – which forage for seeds as individuals – leave the nest to search for food corresponds to food availability.

A forager won't return to the nest until it finds food. If seeds are plentiful, foragers return faster, and more ants leave the nest to forage. If, however, ants begin returning empty handed, the search is slowed, and perhaps called off.

Prabhakar wrote an ant algorithm to predict foraging behavior depending on the amount of food – i.e., bandwidth – available. Gordon's experiments manipulate the rate of forager return. Working with Stanford student Katie Dektar, they found that the TCP-influenced algorithm almost exactly matched the ant behavior found in Gordon's experiments.

"Ants have discovered an algorithm that we know well, and they've been doing it for millions of years," Prabhakar said.

They also found that the ants followed two other phases of TCP. One phase is known as slow start, which describes how a source sends out a large wave of packets at the beginning of a transmission to gauge bandwidth; similarly, when the harvester ants begin foraging, they send out foragers to scope out food availability before scaling up or down the rate of outgoing foragers.

Another protocol, called time-out, occurs when a data transfer link breaks or is disrupted, and the source stops sending packets. Similarly, when foragers are prevented from returning to the nest for more than 20 minutes, no more foragers leave the nest.

Prabhakar said that had this discovery been made in the 1970s, before TCP was written, harvester ants very well could have influenced the design of the Internet.

Gordon thinks that scientists have just scratched the surface for how ant colony behavior could help us in the design of networked systems.

There are 11,000 species of ants, living in every habitat and dealing with every type of ecological problem, Gordon said. "Ants have evolved ways of doing things that we haven't thought up, but could apply in computer systems. Computationally speaking, each ant has limited capabilities, but the collective can perform complex tasks.

"So ant algorithms have to be simple, distributed and scalable – the very qualities that we need in large engineered distributed systems," she said. "I think as we start understanding more about how species of ants regulate their behavior, we'll find many more useful applications for network algorithms."

The work is published in the Aug. 23 issue of PLoS Computational Biology.

       看完这些,你或许会说:“难怪你的这个个人博客叫蚂蚁网。”,我想我又找到了一个取名为“蚂蚁网”的强有力的佐证,但当时取名“蚂蚁网”的时候,我并不知道这个研究成果,我的初衷等哪天得闲想写的时候,我会将其写到《关于》里面。

       大自然总会有很多难以解释清楚的暗合现象,以前我们怎能想到蚂蚁的行为跟TCP的工作机制有关联呢?我又怎能想到我将我的博客取名为“蚂蚁网”后不足半年的时间,竟然因这个研究让其名字跟博客的主要内容关联起来了呢?

       冥冥之中,总有很对东西是难以用理性理解和解释的,而我则相信这些都是注定了的。

阅读全文>>

标签: TCP 蚂蚁 蚂蚁网

评论(0) 引用(0) 浏览(10378)

基于UDP的应用如何保证应用数据的可靠性

作者:易隐者 发布于:2012-8-28 22:58 Tuesday 分类:网络分析

       UDP的传输效率比TCP更高一些,其在网络中交互时具有更小的延时,因为其一,发送方只要把应用字段全部交给网络层处理就行了,接收方不需要确认发送方发送的数据,发送方也没有等待接收方确认报文这些过程;其二,TCP报头一般20个字节,UDP报头只有8字节,因此UDP跟TCP相比,在同长度的IP报文中能够承载更多的有效应用数据。

       很多应用设计者和开发者正是看中了UDP的这种特性,将他们的应用程序利用UDP来实现端对端的数据交互,但是UDP不可靠,它没有TCP的确认、重传、窗口、保活等机制,那么基于UDP的应用如何保证其应用数据在交互过程的可靠性呢?答案是由基于UDP的应用程序来完成

    为此我花费了点时间找了一些基于UDP 的应用报文解码,我们一起逐一来看看吧。 

TFTP(应用层确认):  

点击查看原图

某UDP应用(应用层保活):

点击查看原图

RTP(应用层排序): 

点击查看原图

DHCP:(应用层标识) 

点击查看原图

       上次有个网友问我QQ传大文件的时候,为什么没抓到分片包文?我的回答是:应用层做了分片。虽然我并不清楚QQ文件传输协议的具体工作机制,但是其实跟我上面分析讨论的一样,QQ利用UDP传输文件,抓包的时候也看到的确是UDP的连接,但是没看到分片报文,那么原因只能是QQ应用程序为了保障传输文件的可靠性,避免在网络层被分片,应用程序在将大的应用字段交付给UDP之前,针对大的应用字段做了分片处理,这个过程跟TCP的MSS协商后做分段处理有点类似。

       现实工作环境下常见的UDP应用主要有P2P下载、视频会议、在线视频、VOIP语音、在线游戏等,有兴趣的兄弟姐妹,可以找一些工作环境中的这些应用的报文,来验证或完善一下我上述所言,如果找到的话,记得给我留言,顺便把报文给我一份。

阅读全文>>

标签: UDP 基于UDP的应用 可靠性 DHCP RTP TFTP 保活

评论(0) 引用(0) 浏览(13370)

基于UDP组播实施分片攻击的可能性

作者:易隐者 发布于:2012-8-27 18:36 Monday 分类:网络安全

        UDP是无连接的,这种特性决定了其可以通过组播或广播来进行业务数据的交付,同时我们也知道UDP不会像TCP那样将应用程序交付的大应用字段划分为适合网络传输的数据段(segment),UDP会将大应用字段直接交付于网络层处理,那么在网络层必然会面对分片的问题。

关于应用程序、UDP、IP之间交互的过程,我做了一个图示:

点击查看原图

应用层、UDP以及IP间的交互情况图示

       大的应用字段会被网络层分片后,在网络中传输,这些分片报文会在接收端的网络层重组后,提交给UDP和应用程序。

       我们接下来再来看一下协议栈在各层对数据帧/数据报的过滤过程:

点击查看原图

协议栈在各层对数据帧/数据报的过滤过程图示

       这个是根据《TCP/IP详解卷一》中的第十二章《广播和多播》总结出来的,在此不针对此图做详细描述,有兴趣的兄弟姐妹可以自行参考详解卷一第十二章相关的内容。

       我们接着来看一个实际用户工作环境下抓取的UDP 组播分片报文: 

点击查看原图

用户实际环境中的UDP多播分片报文

         因此,基于UDP组播实施分片攻击的可能性是存在

遭受这种攻击的假设场景:

       用户的某个业务(视频会议、语音、IPTV等)使用组播传输数据报文,黑客只需要构造一些永远无法重组的IP分片报文,以该业务用到的组播地址发送出去,那么所有属于这个多播组的业务主机都将会接收到这些精心构造的攻击报文,那么,可想而知这些多播组内的业务主机在处理的时候,将会出现资源耗尽以至于死机的情况,黑客从而达到拒绝服务的攻击效果。


 

阅读全文>>

标签: UDP 组播 分片攻击 多播 协议栈 拒绝服务攻击

评论(0) 引用(0) 浏览(7805)

Powered by 易隐者 基于emlog 皖ICP备12002343号-1