[SPIGOT-3349] Snakeyaml Created: 17/Jun/17  Updated: 01/Jul/17  Resolved: 17/Jun/17

Status: Resolved
Project: Spigot
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: Zrips Assignee: Unassigned
Resolution: Invalid Votes: 0
Labels: 1.12


 Description   

For some strange reason, big (~46MB) YAML files are not being loaded and server dont seem to crash on 1.12 and loads file forever, while its fine on 1.11.2 and previous, and just takes around 25sec.

Did something was changed or this is an unintentional bug?



 Comments   
Comment by Nathan Wolf [ 01/Jul/17 ]

Ah, heh, yeah Yaml's not the best format for data storage but a lot of us abuse it for that

So my issue was actually a bug, I worked up a test case for them and they fixed it:

https://bitbucket.org/asomov/snakeyaml/issues/386/duplicate-keys-cause-issues

I'm not sure how to get this into Spigot, though. Can we build against 1.19-SNAPSHOT? Should I ask them if they would make a point release off of the 1.18 branch for us?

Comment by Zrips [ 01/Jul/17 ]

NathanWolf

Dint contacted as the issue was in messed up line, was hard to find in file with 2million lines... For some reason, one key was recorded in Byte format with over 2million symbols. While older version would still pass throw it after a couple seconds of processing, new one would just hang on that part forever. So it seems that new version really don't like some funky data in file and its even more picky than YAML format was already  It's time to think about new data saving file system i think 

Comment by Nathan Wolf [ 30/Jun/17 ]

For what it's worth, I've narrowed down my problem. Nothing to do with large files, but duplicate keys. The file had several top-level duplicate keys. When I removed them, the issues went away.

The strange thing was that having duplicate keys caused other sections to go missing, ones that were not duplicated.

This makes me suspicious of this PR:

https://bitbucket.org/asomov/snakeyaml/pull-requests/11/dupkeys/diff

But I don't really have any evidence to support that. If I can find the time I will try to work up a standalone repro case, now that I know the root problem.

In the meantime at least I've got a workaround, and it sounds probably unrelated to the original JIRA issue I'm commenting on so I'm going to stop doing that now.

Comment by Nathan Wolf [ 30/Jun/17 ]

There are quite a lot of changes, and they use Mercurial which I'm not very familiar with. Just scanning the repo on Bitbucket I can't really identify a suspicious commit, though.

I'll maybe see what I can do when I get some time, this bug is really breaking some people using my plugin. The alternative is for me to break up large Yaml config files before sending them to YamlConfiguration, but this seems inefficient and hacky and I don't really know if it'll fix the problem. I've only seen it with large files, but don't know if that's just coincidental.

Comment by Nathan Wolf [ 30/Jun/17 ]

Are they? I went and looked and there was a year between 1.17 and 1.18 .. I'll check the commit history, but it looked like a lot of changes.

Comment by md_5 [ 30/Jun/17 ]

The snakeyaml changes are small, just bisect between the two versions and file a bug report upstream after finding the issue.

Comment by Nathan Wolf [ 30/Jun/17 ]

I reverted SnakeYaml to 1.17 to verify that is definitely the problem.

Unfortunately it seems like rolling back officially is not an option, since the set syntax used for ItemFlag serialization isn't supported in 1.17.

Oddly, I also tried the latest 1.19 snapshot and found that the issue is still there, but surprisingly the set support doesn't seem to work in that version, either. (Just judging by errors thrown when deserializing items with ItemFlags)

Comment by Nathan Wolf [ 30/Jun/17 ]

Did you ever report the issue? I'm curious if you got a response.

I'm having an issue that's related to the snakeyaml update, too. Semi-large Yaml config (~6000 lines) loads ok, no hangs- but some sections are randomly ignored. 

I can't seem to get the issue to happen with a smaller file, so I'm having a hard time seeing how I can present them with sample data. Especially since these are some server's private configs.

Comment by md_5 [ 17/Jun/17 ]

Snakeyaml was updated, you should report this to them with sample data and code

Generated at Tue Apr 08 03:01:47 UTC 2025 using Jira 10.3.3#10030003-sha1:d220e3fefc8dfc6d47f522d3b9a20c1455e12b7b.