Microsecond TCP timestamp

TCP Time accuracy is still ms level , This makes TCP It's hard to adapt to IDC such RTT stay 50 us Short range network of magnitude , In this network RTO It should also be controlled in us Level .

Besides , And early TCP Of burst The sending mode is different , In today's low latency demand scenarios ,burst yes bufferbloat The culprit of , So the general transition to pacing Mode send .ms Level time precision support pacing Can only deal with MBps Class bandwidth .

Linux TCP pacing timer use hrtimer Has supported ns Level timer , but TCP The timestamp option is not connected us Not even level , Still in ms level , This prevents the timestamp option from being used to participate pacing Collection and calculation of .

I modified Linux Kernel code , Make it TCP The timestamp option precision is improved to us level . At the sending end , Make the following changes ：

-static inline u32 tcp_skb_timestamp(const struct sk_buff *skb)
-{
    
-       return div_u64(skb->skb_mstamp_ns, NSEC_PER_SEC / TCP_TS_HZ);
-}
-
 /* provide the departure time in us unit */
 static inline u64 tcp_skb_timestamp_us(const struct sk_buff *skb)
 {
    
        return div_u64(skb->skb_mstamp_ns, NSEC_PER_USEC);
 }

+static inline u32 tcp_skb_timestamp(const struct sk_buff *skb)
+{
    
+       return tcp_skb_timestamp_us(skb);
+}

This code is not all , Just an example , It means to put all ms The timestamps of level are changed to us Level .

For the receiver , To avoid ms Grade PAWS Check , Do the following trick ：

#!/usr/local/bin/stap -g

%{
    
#include <net/tcp.h>
%}

function disable_ts(opt_rx:long, skbp:long)
%{
    
	struct tcp_options_received *opt = (struct tcp_options_received *)STAP_ARG_opt_rx;
	struct sk_buff *skb = (struct sk_buff *)STAP_ARG_skbp;
	struct sock *sk = skb->sk;

	if (opt->saw_tstamp && opt->rcv_tsecr && sk) {
    
		struct tcp_sock *tp = tcp_sk(sk);
		opt->rcv_tsecr -= tp->tsoffset;
	}
	opt->saw_tstamp = 0;
%}

function enable_ts(skp:long)
%{
    
	struct sock *sk = (struct sock *)STAP_ARG_skp;
	struct tcp_sock *tp = tcp_sk(sk);

	if (tp && tp->rx_opt.rcv_tsval) {
    
		tp->rx_opt.saw_tstamp = 1;
	}
%}

probe kernel.function("tcp_validate_incoming").return
{
    
	enable_ts($sk);
}

probe kernel.function("tcp_parse_options").return
{
    
	disable_ts($opt_rx, $skb);
}

Above stap Code to avoid tcp_paws_discard Check , Look at the notes below ：

static bool tcp_validate_incoming(struct sock *sk, struct sk_buff *skb,
				  const struct tcphdr *th, int syn_inerr)
{
    
	struct tcp_sock *tp = tcp_sk(sk);
	bool rst_seq_match = false;

	/* RFC1323: H1. Apply PAWS check first. */
	if (tcp_fast_parse_options(sock_net(sk), skb, th, tp) &&
	    tp->rx_opt.saw_tstamp && //  Need to put  saw_tstamp  Set to 0, Last in  validate  Put it back before returning  1.
	    tcp_paws_discard(sk, skb)) {
      
		if (!th->rst) {
    
			NET_INC_STATS(sock_net(sk), LINUX_MIB_PAWSESTABREJECTED);
			if (!tcp_oow_rate_limited(sock_net(sk), skb,
						  LINUX_MIB_TCPACKSKIPPEDPAWS,
						  &tp->last_oow_ack_time))
				tcp_send_dupack(sk, skb);
			goto discard;
		}
		/* Reset is accepted even if it did not pass PAWS. */
	}
    ....

Such a thing can be done . Then you can find two adjacent packets skb Of tsval It's not the same anymore .

Before you change 1ms The timestamp of the packets sent in the is the same , Now at least the accuracy has been improved 1000 times , Although still unable to cope with pacing Pre marking , But deal with RTO Enough calculations .

Promoted TCP Time precision except to make TCP Yes IDC The network is more sensitive and adaptable pacing outside , It can also assist the receiver to synchronize pacing, For video stream relay , This mechanism effectively keeps sending pacing , Play a plastic effect . By looking at the timestamp option tsval Field , It is possible to know exactly what to send pacing , And then decide to relay pacing.

meanwhile ,us Level time precision can make the sampling calculation at the sending end more accurate , So as to improve the resolution of congestion control algorithm .

Why at first TCP Time is not us Precision ？ No matter what RTO Neither the timestamp option .

In addition to historical reasons , There are also reasons to maintain the closeness and completeness of the state machine , To be specific, see RFC1323 Of ‘4.2.2 Timestamp Clock’：

That's the answer .

But now these reasons have weakened , In my opinion, with RST Closing the connection may be more direct . There is no need to take care of MSL, Time stamps wrap around these things .

in fact , although RFC1323 It is recommended to use the timestamp option to “ Optimize ” Connect , but Linux TCP It's basically not used to calculate RTT, Instead, additional continuous marking is used to collect .

It is difficult to make a private agreement , The simplest solution is to take the existing TCP modify , That is to say, we have completed the bilateral affairs . The first step is to improve the time accuracy .

Zhejiang Wenzhou leather shoes wet , It's not fat when it's raining .