I've noticed that attributes modified by plugins stay modified even when the plugin is not loaded in. For example, if you set the GENERIC_ATTACK_SPEED attribute to 16 in AttributeDemo and then remove the plugin, any player online while the plugin was loaded and modifying attributes had that attribute modifier stick to them. Players could have no GENERIC_ATTACK_SPEED cooldowns even when the plugin AttributeDemo was no loaded in the server.

I also have a private plugin which multiplies the GENERIC_ARMOR attribute by 1.2 which no longer works, it only works for a short period of time after I reset all the player data in the world directory (which i something I don't plan to constantly do!).