Players Disconnect but character stays for 30 seconds.

We noticed the issue of players not correctly disconnecting.
Then we tested the current version (when you do --rev 1.16.4) git-Spigot-628435a-b5e5adc and latest version (–rev latest) on buildtools.
In either version the player disconnects and the character stays for 30 seconds, you can hit and kill it, but it has no knockback or physics.

This does not happen when connected from the local network.
The only console output is timed out for everyone except me.

I tried it with and without plugins, with proxy and with direct connect.
The issue lies either on spigot or my linux server.