So this is happening in multiplayer too? On the SMP server I play on, I fixed up one of those "ruined" battle towers (the six-floor ones made from a mix of cobble and mossy cobble). I have a twilight portal on a chunk adjacent to the chunk the tower spawned on and a nether portal four chunks away. I also have several Secret Rooms blocks in and near the chunk the tower spawned on, but even after using the nether and twilight portals a lot, I've never had any chunk respawn on me.
Edit: However, no Secret Rooms blocks happen to be in a chunk containing either a twilight or nether portal. Maybe that has something to do with it?