This is a G.hn link that usually sustains 50Mbps but occasionally drops to sub 1Mbps (so more like 1st gen DSL speeds than dialup speeds); it only drops that low for a fairly short period of time, but it takes a long time to recover if the buffers are full.
That's not a buffering issue (as others mentioned before, 20s would be quite a buffer). G.hn made the same mistake as dial-up modem long time ago: implementing forward error correction. That's a great feature on a space probe leaving the solar system, but not for peers on a LAN talking TCP, as that does performs its own error correction (as is well known). If some link layer hiccup occurs, the forward error correction of the link layer causes delays potentially causing re-transmissions on the TCP layer (check with `netstat -s`).
Does it really take 20s to perform FEC on a packet? I assumed it was retransmitting at the link-level.
Either way, the packet is buffered in the sense that it is stored in a buffer on the switch; otherwise the packets would be dropped, not eventually make it through.
Sorry, brainfart, no excuses, I had my coffee already. FEC ought to add only minimal latency, even if correction is necessary. I too believe the long latencies are due to retransmissions on the link level (what modems suffered from too), which, if on-going for long enough, causes retransmissions by TCP.