I'm a big fan of CR2032 "coin cell" batteries. They provide just over 3 volts of electricity in an very compact size. You can plug one into a small holder, then connect the leads as needed.
But what if you need more than three volts? You could connect several holders in series, but the size adds up, negating one of the important benefits of these cells. My solution? Shrink wrap!
You'll need a coin cell, wire, and rather large shrink wrap. I used this package from Amazon* which worked quite well, but certainly there are others as well.
*Affiliate link