How About a little imagination or Maker ingenuity?
Of course I don't know what your piece of equipment is but...perhaps a setup like the alarm bypass trick in the movies might work. Could you connect up a stabilized power supply connection to the battery holder to keep power "on" and then a power switch inside the case so you can then disconnect the battery you have to replace without power related worries and then put in the new battery? If it won't fit inside, then perhaps an addon box or dongle with the stuff in it. If it is that important to you, there must be some way to overcome the (purposeful?) fragility of the design. I've done this sort of thing since I was a teenager and couldn't afford to buy a new device each time something happened. Just a suggestion.