webrt分析六（nack）

nack是什么

丢包重传(NACK)是抵抗网络错误的重要手段。NACK在接收端检测到数据丢包后，发送NACK报文到发送端；发送端根据NACK报文中的序列号，在发送缓冲区找到对应的数据包，重新发送到接收端。NACK需要发送端，发送缓冲区的支持。

nack流程

发送端发送rtp，到达接收端时，发现丢包，接收端发送nack请求，发送端会从历史队列中取出数据重发。

nack协议

在rfc4585协议中定义可重传未到达数据的类型有二种：

1 RTPFB：rtp报文丢失重传（丢包重传）。

2 PSFB：指定净荷重传，指定净荷重传里面又分如下三种（请求关键帧）：

(1) PLI(Picture Loss Indication)视频帧丢失重传。

(2) SLI(Slice Loss Indication)slice丢失重转。

(3)RPSI(Reference Picture Selection Indication)参考帧丢失重传。

RFC 4585: Feedback format.

Common packet format:

//    0                   1                   2                   3
//    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
//   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
//   |V=2|P|   FMT   |       PT      |          length               |
//   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
// 0 |                  SSRC of packet sender                        |
//   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
// 4 |                  SSRC of media source                         |
//   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
//   :            Feedback Control Information (FCI)                 :
//   :                                                               :
//
// Generic NACK (RFC 4585).
//
// FCI:
//    0                   1                   2                   3
//    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
//   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
//   |            PID                |             BLP               |
//   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

V： 2bit 目前固定为2
P： 1bit padding
FMT：5bit Feedback message type。
RTP FP模式下定义值为：
0:unassigned
1:Generic NACK
2-30:unassigned
31:reserved for future expansion of the identifier number space

PS FP模式下定义值为：
0: unassigned
1: Picture Loss Indication (PLI)
2: Slice Loss Indication (SLI)
3: Reference Picture Selection Indication (RPSI)
4-14: unassigned
15: Application layer FB (AFB) message
16-30: unassigned
31: reserved for future expansion of the sequence number space

PT: 8bit Payload type。

FCI：变长 Feedback Control Information。

RTPFB

Packet identifier(PID)即为丢失RTP数据包的序列号，Bitmap of Lost Packets(BLP)指示从PID开始接下来16个RTP数据包的丢失情况。一个NACK报文可以携带多个RTP序列号，NACK接收端对这些序列号逐个处理。

webrtc代码走读

webrtc中nack的发送流程：

1.DeliveryStatus Call::DeliverPacket
2.PacketReceiver::DeliveryStatus Call::DeliverRtp
3.RtpStreamReceiverController::OnRtpPacket
4.RtpDemuxer::OnRtpPacket
5.RtpVideoStreamReceiver2::OnRtpPacket
6.RtpVideoStreamReceiver2::ReceivePacket
7.RtpVideoStreamReceiver2::OnReceivedPayloadData
8. NackModule2::OnReceivedPacket

前面就是接受数据，把收到数据的序号传给nack模块。我们从 NackModule2::OnReceivedPacket开始看

int NackModule2::OnReceivedPacket(uint16_t seq_num,
                                  bool is_keyframe,
                                  bool is_recovered) {
  RTC_DCHECK_RUN_ON(worker_thread_);
  // TODO(philipel): When the packet includes information whether it is
  //                 retransmitted or not, use that value instead. For
  //                 now set it to true, which will cause the reordering
  //                 statistics to never be updated.
  bool is_retransmitted = true;

  if (!initialized_) {
    newest_seq_num_ = seq_num;
    if (is_keyframe)
      keyframe_list_.insert(seq_num);
    initialized_ = true;
    return 0;
  }

  // Since the |newest_seq_num_| is a packet we have actually received we know
  // that packet has never been Nacked.
  if (seq_num == newest_seq_num_)
    return 0;
  //if newest_seq_num_> seq_num  
  //是不是乱序包，是乱序的包，也有可能是nack请求要回来的
  //要把之前加进nack的删掉
  if (AheadOf(newest_seq_num_, seq_num)) {
    // An out of order packet has been received.
    auto nack_list_it = nack_list_.find(seq_num);
    int nacks_sent_for_packet = 0;
    if (nack_list_it != nack_list_.end()) {
      nacks_sent_for_packet = nack_list_it->second.retries;
      nack_list_.erase(nack_list_it);
    }
    if (!is_retransmitted)
      UpdateReorderingStatistics(seq_num);
    return nacks_sent_for_packet;
  }

  // Keep track of new keyframes.
  if (is_keyframe)
    keyframe_list_.insert(seq_num);

  // And remove old ones so we don't accumulate keyframes.
  auto it = keyframe_list_.lower_bound(seq_num - kMaxPacketAge);
  if (it != keyframe_list_.begin())
    keyframe_list_.erase(keyframe_list_.begin(), it);

  if (is_recovered) {
    recovered_list_.insert(seq_num);

    // Remove old ones so we don't accumulate recovered packets.
    auto it = recovered_list_.lower_bound(seq_num - kMaxPacketAge);
    if (it != recovered_list_.begin())
      recovered_list_.erase(recovered_list_.begin(), it);

    // Do not send nack for packets recovered by FEC or RTX.
    return 0;
  }

  AddPacketsToNack(newest_seq_num_ + 1, seq_num);
  newest_seq_num_ = seq_num;

  // Are there any nacks that are waiting for this seq_num.
  std::vector<uint16_t> nack_batch = GetNackBatch(kSeqNumOnly);
  if (!nack_batch.empty()) {
    // This batch of NACKs is triggered externally; the initiator can
    // batch them with other feedback messages.
    nack_sender_->SendNack(nack_batch, /*buffering_allowed=*/true);
  }

  return 0;
}

这部分逻辑主要是收到包，查一下是不是乱序的，可能是网络造成乱序，也可能是重发过来的，收到了就把nack list里面的记录删掉
接下来看
AddPacketsToNack 和 GetNackBatch

void NackModule2::AddPacketsToNack(uint16_t seq_num_start,
                                   uint16_t seq_num_end) {
  // Called on worker_thread_.
  // Remove old packets.
  //包间隔最大kMaxPacketAge=1000
  auto it = nack_list_.lower_bound(seq_num_end - kMaxPacketAge);
  nack_list_.erase(nack_list_.begin(), it);

  // If the nack list is too large, remove packets from the nack list until
  // the latest first packet of a keyframe. If the list is still too large,
  // clear it and request a keyframe.
  uint16_t num_new_nacks = ForwardDiff(seq_num_start, seq_num_end);
  if (nack_list_.size() + num_new_nacks > kMaxNackPackets) {
    while (RemovePacketsUntilKeyFrame() &&
           nack_list_.size() + num_new_nacks > kMaxNackPackets) {
    }

    if (nack_list_.size() + num_new_nacks > kMaxNackPackets) {
      nack_list_.clear();
      RTC_LOG(LS_WARNING) << "NACK list full, clearing NACK"
                             " list and requesting keyframe.";
      keyframe_request_sender_->RequestKeyFrame();
      return;
    }
  }

  for (uint16_t seq_num = seq_num_start; seq_num != seq_num_end; ++seq_num) {
    // Do not send nack for packets that are already recovered by FEC or RTX
    if (recovered_list_.find(seq_num) != recovered_list_.end())
      continue;
    NackInfo nack_info(seq_num, seq_num + WaitNumberOfPackets(0.5),
                       clock_->TimeInMilliseconds());
    RTC_DCHECK(nack_list_.find(seq_num) == nack_list_.end());
    nack_list_[seq_num] = nack_info;
  }
}

我们可以看到AddPacketsToNack()函数主要实现了：
nack_list 的最大容量为 kMaxNackPackets = 1000, 如果满了会删除最后一个 KeyFrame 之前的所有nacked 序号, 如果删除之后还是满的那么清空 nack_list 并请求KeyFrame。

std::vector<uint16_t> NackModule2::GetNackBatch(NackFilterOptions options) {
  // Called on worker_thread_.

  bool consider_seq_num = options != kTimeOnly;
  bool consider_timestamp = options != kSeqNumOnly;
  Timestamp now = clock_->CurrentTime();
  std::vector<uint16_t> nack_batch;
  auto it = nack_list_.begin();
  while (it != nack_list_.end()) {
    //请求延时rtt
    TimeDelta resend_delay = TimeDelta::Millis(rtt_ms_);
    if (backoff_settings_) {
        //延时要大于min_retry_interval，默认5ms
      resend_delay =
          std::max(resend_delay, backoff_settings_->min_retry_interval);
      //如果请求次数大于1，每次延时增大25%，1.25的n次幂
      if (it->second.retries > 1) {
        TimeDelta exponential_backoff =
            std::min(TimeDelta::Millis(rtt_ms_), backoff_settings_->max_rtt) *
            std::pow(backoff_settings_->base, it->second.retries - 1);
        resend_delay = std::max(resend_delay, exponential_backoff);
      }
    }

    bool delay_timed_out =
        now.ms() - it->second.created_at_time >= send_nack_delay_ms_;
    bool nack_on_rtt_passed =
        now.ms() - it->second.sent_at_time >= resend_delay.ms();
    bool nack_on_seq_num_passed =
        it->second.sent_at_time == -1 &&
        AheadOrAt(newest_seq_num_, it->second.send_at_seq_num);
    if (delay_timed_out && ((consider_seq_num && nack_on_seq_num_passed) ||
                            (consider_timestamp && nack_on_rtt_passed))) {
      nack_batch.emplace_back(it->second.seq_num);
      ++it->second.retries;
      it->second.sent_at_time = now.ms();
      //大于请求次数10次，不在请求
      if (it->second.retries >= kMaxNackRetries) {
        RTC_LOG(LS_WARNING) << "Sequence number " << it->second.seq_num
                            << " removed from NACK list due to max retries.";
        it = nack_list_.erase(it);
      } else {
        ++it;
      }
      continue;
    }
    ++it;
  }
  return nack_batch;
}

1、delay_timed_out ：加入nacklist的时间大于要发送nack的延时
2、nack_on_rtt_passed ：该序号上次发送NACK的时间到当前时间要超过前面计算出来的延时。
3：nack_on_seq_num_passed ：确定有最新的包序号比这个大，是被丢失的

下面是默认值的定义

absl::optional<NackModule2::BackoffSettings>
NackModule2::BackoffSettings::ParseFromFieldTrials() {
  // Matches magic number in RTPSender::OnReceivedNack().
  const TimeDelta kDefaultMinRetryInterval = TimeDelta::Millis(5);
  // Upper bound on link-delay considered for exponential backoff.
  // Selected so that cumulative delay with 1.25 base and 10 retries ends up
  // below 3s, since above that there will be a FIR generated instead.
  const TimeDelta kDefaultMaxRtt = TimeDelta::Millis(160);
  // Default base for exponential backoff, adds 25% RTT delay for each retry.
  const double kDefaultBase = 1.25;

webrtc中nack的请求处理流程：

解析nack

void RTCPReceiver::HandleNack(const CommonHeader& rtcp_block,
                              PacketInformation* packet_information) {
  rtcp::Nack nack;
  if (!nack.Parse(rtcp_block)) {
    ++num_skipped_packets_;
    return;
  }

  if (receiver_only_ || main_ssrc_ != nack.media_ssrc())  // Not to us.
    return;

  packet_information->nack_sequence_numbers.insert(
      packet_information->nack_sequence_numbers.end(),
      nack.packet_ids().begin(), nack.packet_ids().end());
  for (uint16_t packet_id : nack.packet_ids())
    nack_stats_.ReportRequest(packet_id);

  if (!nack.packet_ids().empty()) {
    packet_information->packet_type_flags |= kRtcpNack;
    ++packet_type_counter_.nack_packets;
    packet_type_counter_.nack_requests = nack_stats_.requests();
    packet_type_counter_.unique_nack_requests = nack_stats_.unique_requests();
  }
}

继续调用

void ModuleRtpRtcpImpl2::OnReceivedNack(
    const std::vector<uint16_t>& nack_sequence_numbers) {
  if (!rtp_sender_)
    return;

  if (!StorePackets() || nack_sequence_numbers.empty()) {
    return;
  }
  // Use RTT from RtcpRttStats class if provided.
  int64_t rtt = rtt_ms();
  if (rtt == 0) {
    rtcp_receiver_.RTT(rtcp_receiver_.RemoteSSRC(), NULL, &rtt, NULL, NULL);
  }
  //取得rtt，把请求和rtt时间调用rtp补报
  rtp_sender_->packet_generator.OnReceivedNack(nack_sequence_numbers, rtt);
}

rtpSender直接就重新发送了， 后续还有码率，rtt等的比较计算

void RTPSender::OnReceivedNack(
    const std::vector<uint16_t>& nack_sequence_numbers,
    int64_t avg_rtt) {
  packet_history_->SetRtt(5 + avg_rtt);
  for (uint16_t seq_no : nack_sequence_numbers) {
    const int32_t bytes_sent = ReSendPacket(seq_no);
    if (bytes_sent < 0) {
      // Failed to send one Sequence number. Give up the rest in this nack.
      RTC_LOG(LS_WARNING) << "Failed resending RTP packet " << seq_no
                          << ", Discard rest of packets.";
      break;
    }
  }
}

总结：视频接收端发现丢包，计算发送nack的时机（rtt检测，重发次数），发送nack请求
视频发送端收到nack请求，查看可用带宽，间隔发送是否大于rtt，判断重发包