Modpack Testing

[plowmanplow]

I do a fair amount of modpack testing and debugging, typically in threads posted to this forum. I thought I could demystify the process by sharing some of the steps and tools I use during an evaulation.

 

Testing Environment for Monolithic packs:

• I keep a "custompacks" folder on one of my drives and made a link to it in Windows Explorer's Favorites to make it easy to navigate into.
• When someone requests assistance with a pack I copy their API URL (this is why I always ask for it) and paste it into a custom page I maintain for displaying pack metadata (feel free to use it or see what yours looks like: http://www.circlecraft.info/platform ).
• If the pack download link is valid I save it to custompacks in a subfolder titled as the username of the person requesting assistance. I name the ZIP file the same as the platform (i.e. slug-version.zip). Keeping the version in the filename is important because it permits me to notice when someone is changing their pack contents but not the version number.
• I then look through the pack to make sure everything looks "normal". Does the pack have the correct modpack.jar file (version, name, contents, etc.)? Does the pack have config files? Are the mods named "correctly" (not renamed, don't look like they came from content scraping sites, etc.)? Do the well known standards (ChickenBones, CoFH, Immibis', etc.) have all their dependencies? Are the mod versions current?
• I use my BareBonesPack (BBP) ( http://www.technicpack.net/modpack/barebonespack.271497 ) for testing monolithic modpacks (packs which are one big ZIP file download). This allows me to easily deploy a base platform for testing which I know has a working Forge environment. For a new test I will reset the BBP (using this script: http://www.circlecraft.info/bbp/ ) and select the correct version of Forge. I will then copy in the pack's required folders (config, mods, etc.).
• I then try to launch the BBP.
• If the pack crashes I will look at the crash report and ForgeModLoader logs (FML logs are in the root folder of the pack for MC <= 1.6.4 and in the /logs/ subfolder for MC >= 1.7.2) to get an idea about what may be causing the crash. This is sometimes quite clear (missing dependency mods, ID conflicts) but sometimes requires reading through Java errors/exceptions. Unless there are dozens of block ID conflicts, or the mods folder is a hot steaming mess I will usually do what is required to get the pack started (update mods, fix dependencies, fix conflicts, etc.) so I can report on that.
• If the pack launches successfully the first step is to go through the entire FML log and look for glaring issues. Things like mods which completely fail to load (errors reported), item ID conflicts (if MC <= 1.6.4), errors other than missing textures and failures to contact update checkers.
• I will then try to create an SSP world with the correct world type (like Biomes O' Plenty, etc.) and do some quick checks. Does NEI/Waila/etc. show up? Are there any obviously missing item textures (look through NEI pages)? Do the cornerstone mods allow block placement? Can I successfully change dimensions? Does the pack use an expected amount of RAM/CPU?
• At this point I will usually write up my findings. Many of you have probably seen them, but here's an example:

Testing Environment for Solder packs:

• It is usually easiest to just add packs to my Launcher when they are managed by Solder. However, I do look around a bit first.
• My Platform metadata page will provide a link to the direct API URL for the Solder modpack. I will follow this to make sure that things look "normal". I will append the latest version number (i.e. if the latest version is 3.2 I would append "/3.2" without quotes) and make sure that a full mod list shows up. I will usually hit one of the direct mod URLs to make sure the web server is feeding things correctly.
• Once the pack finishes installing to the Launcher I will examine the pack folder to ensure all the folders look correct (that the modpack.jar in /bin/ is correct, that the config files exist, that the contents of the /cache/modname-version.zip look correct) and perform most of the steps from the Monolithic section above.
• If everything looks good I will attemp to launch the pack. The analysis steps from here on mirror those from the Monolithic section above except that I would be working in the current pack's folder instead of inside the BBP.

Testing Environment for Servers:

• The first step for server assistance requests is always to do an analysis of the client modpack. If the above steps yield a fully working pack I will continue with the server.
• I keep a Servers folder in my custompacks folder with pre-built servers (both Forge and MCPC+/Cauldron) for all versions of MC. I will make a new folder for the test server, copy in the required top level folders from the pack (/config, /mods, etc.) then copy in one of these pre-built servers.
• I then make a new folder in the server folder called client_only and move all the client only mods (minimap, visual animations, etc.) from the server's /mods/ folder. Which mods these are is mostly learned by research and trial/error. However, most (decent) mod authors will state that their mods are client only when relevant (like Damage Indicators).
• I then launch the server using the Forge JAR as the base. If it starts correctly I will then connect from the client and verify that things are working as expected. If the person requesting assistance has provided crash reports or a test which makes the server crash I will attempt to duplicate that and analyze the results if it does indeed crash.
• Once everything is working as expected I will report my findings about the client pack, which client only mods must be removed from the server (sometimes mods should work in SMP but don't because they are poorly coded), and any other "gotchas" required to get the server up and running.

[The_Doctors_Life]

Oh wow, u do a lot of steps

[plowmanplow]

Yeah, once you go through that process a few hundred times it goes pretty quickly. Also, the "sanity check the /mods/ folder" step catches a lot of problems before I even try to launch the pack, so that can save a lot of time.