I will create a separate post about this since your crashing issue sounds like mine, ie the crashing while wandering, minus the crashing when trying to log back in, I thought I would post what I have been seeing so you can test to see if the issue is the same.
I have been noticing, from some very basic testing, that the better dungeons mod with Hexxit 1.0.5 causes an internal server error every time it tries to spawn a better dungeons castle or even the nether villages when I have my Hexxit server running on multicraft.jjservers.com, yet no one else is kicked out (3 people including myself) since they are in areas that we have already explored. Now what is interesting is when I run the same Hexxit server on my local machine running Windows I can wander all day long and not get a crash and the better dungeons structures are generated. The question is what is the multicraft server doing that is causing the generation issue? The multicraft server has 3 GB of RAM allocated to it too and is only using up 25% of the RAM as well.
If you go in to creative mode and give yourself a castles.prop and then try to spawn it. I suspect you will crash out, but if you take the same server and start it on a Windows computer then try to spawn a castles.prop, your Minecraft will not give an internal server error.
FYI - The server I am on is running a newer version of Java - INFO Java is Java HotSpot 64-Bit Server VM, version 1.7.0_21, running on Linux:amd64:2.6.32-358.0.1.el6.x86_64, installed at /usr/java/jdk1.7.0_21/jre