Created a villager who trades an unbreakable flower for a mushroom.

summon Villager ~ ~1 ~ {Profession:0,CustomName:"Frank",CustomNameVisible:1,Career:1,CareerLevel:42,CanPickUpLoot:0,Invulnerable:1,Offers:{Recipes:[{buy:{id:"red_flower",Count:1,tag:{Unbreakable:1,HideFlags:4}},maxUses:1,sell:{id:"red_mushroom",Count:1,tag:{Unbreakable:1,HideFlags:4}},rewardExp:false}]}}

When first opening the trade window, everything looks fine. After inserting the buy item (the red flower) and clicking on the sell item (the mushroom) the action will be cancelled and the trade will be locked, with a Trade something else to unlock! message

This only seems to affect the buy item. If I add tags to the sell item, there don't appear to be any issues. I have also tested this on a singleplayer world, and found no issues.