Input/output error?

[ThatBum]

Ohai. I'm getting an intermittent error:

Code: Select all

2012-04-18 11:04:19 [SEVERE] null
java.io.IOException: Input/output error
	at java.io.FileInputStream.readBytes(Native Method)
	at java.io.FileInputStream.read(Unknown Source)
	at java.io.BufferedInputStream.fill(Unknown Source)
	at java.io.BufferedInputStream.read(Unknown Source)
	at jline.Terminal.readCharacter(Terminal.java:99)
	at jline.UnixTerminal.readVirtualKey(UnixTerminal.java:128)
	at jline.ConsoleReader.readVirtualKey(ConsoleReader.java:1453)
	at jline.ConsoleReader.readBinding(ConsoleReader.java:654)
	at jline.ConsoleReader.readLine(ConsoleReader.java:494)
	at net.minecraft.server.ThreadCommandReader.run(ThreadCommandReader.java:31)

After which, it zombies, the clients say it times out and are unable to reconnect, and needs a manual restarting. The Control Panel says it's up, but when I try to put any command in, it makes a left angle bracket like it's going to give some output, but nothing happens.

It seems to be totally random. It happens any number of times a day, regardless of the time or whether there's players on or not. Upon restarting, the world loses the changes made to it before the crash, so it doesn't write to disk before shutting down/being killed.

I/O error made me think of a corrupted world, so I downloaded it and ran it past MinecraftRegionFixer (Chunkster is no longer effective by the way, it's not updated for Minecraft's new Anvil filesystem). It said all >40,000 chunks were OK.

Maybe it's bad RAM or harddisk, or non-sane java environment? I already checked my Bukkit jar, the checksum matches up with a fresh copy.

Possibly having to do with the streaming output thing of the Control Panel? It has JLine in the stack trace, JLine is some kind of enhanced console format thing added by the Bukkit team. It can be disabled with -nojline in the commandline parameters, after the jar is called, not where the memory and stuff is.

I downloaded the whole server to my local machine and ran it for about a day...was unable to reproduce the error. I did a bit a musical chairs with the plugins to try and see if it was one of them, but I couldn't find any evidence.

My plugins:

Code: Select all

pl
17:01:48 [INFO] Plugins (34): LagMeter, SuperCauldrons, GroupManager, Minequery, WorldEdit, MorePhysics, Vault, Multiverse-Core, MobBountyReloaded, WorldGuard, iConomy, ReportRTS, Permissions, HawkEye, LimitedCreative, TreeCutter, CraftBukkitUpToDate, mcMMO, MobArena, FoundDiamonds, Essentials, CreeperHeal, EssentialsProtect, DynamicEconomy, EssentialsSpawn, Multiverse-Portals, EssentialsGeoIP, Multiverse-NetherPortals, LWC, dynmap, DeathChest, EssentialsChat, WorldBorder, Multiverse-Adventure
>

Kind of a lot, but they're all pretty lightweight, except for Dynmap, and maybe Essentials. They're all updated and confirmed to work with 1.2.5...no [SEVERE] or [WARNING] lines in the log, other than this one random error.

Any insights?

EDIT: Found something else from looking in the logs:

Fatal bug! Process <a number> is without parent

This has to do with Linux's cpulimit program, yes?

[Edge100x]

Is this a standard shared server or one on a VDS?

That error suggests that it's having problems with reading/writing from/to the PTY, such as might happen if you were watching the server through the console tool on the "Server control" page and thousands of errors were printed at once. Was it printing many lines of output in a very short period of time immediately prior to this?

It could also be a plugin problem of some sort, certainly, as it's a fairly generic error. Running without plugins as a troubleshooting step is always a safe thing to do.

The cpulimit program would not cause this, no.

[ThatBum]

It's on a regular shared machine. 10 players/512MB, node la-dodec306-2.

As stated, there is no errors. Just player normal chat and player commands, and some occasional automated stuff, like one line of Hawkeye deleting old logging data in the MySQL database.

I guess I'll go to the ol' standby of trial-and-error with the plugins to see which one's doing it, if any. Might take a long time. Got to get back to a real computer first, this is a phone.

[Edge100x]

Have you been using the console tool on the "Server control" page at all? If so, I also recommend testing without that to see if it makes a difference. The console tool captures and sends output from the server to our webserver.

[ThatBum]

Nope. I can literally wake up and find that it's zombied.

I have it running without plugins now, since about midnight last night, and it seems to be stable...will give it a bit more time and add some back.

Somewhat worryingly, there was a corrupted chunk right near the spawn area that's been replaced with a newly generated chunk, and it took out a road and part of a building. It must have been caused by the unsafe shutdowns, it wasn't there before.

I guess I'll keep this thread updated. I'm kind of thinking about asking for that command line switch.

[Edge100x]

If you'd like it added, please let us know in a support request and we should be able to do it for you.

[.=QUACK=.Major.Pain]

Unless they changed things, minecraft doesn't save the world as you go, so a hard crash would cause some loss. Plugins can be installed to save progress every 5 mins or more depending on your preference. So startup would revert to the world as of the latest save.

There was an issue with some control panels also losing new changes to maps if server was shutdown. Developers had to change their shutdown to save the world moments before shutdown.

[Edge100x]

I've gone ahead and added -nojline to every server now, since it looks like jline is useless with our servers anyway (we don't pass through colors and have our own console input system).

Pain, Minecraft itself saves periodically, something like every 10 minutes. We also perform a save every hour as part of the snapshotting process. If a server is shut down gracefully in our system (i.e., it doesn't crash), that's done through the "stop" command provided by Minecraft, which also performs a save. More information here: http://www.nfoservers.com/forums/viewto ... =69&t=6598.

[ThatBum]

Excellent. I'll put the plugins back and see if that was the problem.