So I have an original iPod and I am wondering if I should open it and remove the original lithium ion battery. I currently am not using it. It's just on display. I worry that the original battery might damage it over time, especially if it decides to expand? Am I worried about nothing?

I would. I cant imagine the existing oem battery still holds a decent charge after all these years and the swap is chgeap and pretty easy to do. There's a nice ifixit tutorial on the process for a gen1. I just replaced the battery and hdd with a CF card on my Ipod classic gen3. The battery was a simple swap. Reformatting the CF card was quite the ... adventure let's say but I did finally get it done LOL :D

Iirc @XodiumLabs has done lots of these.

The only damage that can really occur is if the battery swells up, other than that it's whatever to leave it in. I usually just charge my iPods every so often and give them a once over to make sure.

From my experience, the original batteries are of pretty decent quality. I would leave it inside if there is no sign of damage.

I have yet to discover a third-party battery that is worth swapping in. More than often these "easy to come by" solutions are of bad quality and these tend to expand. Also they lose capacity so fast, in a matter of months it is below the old original battery. So I try to keep that alive as good as possible instead.

I find Cameron Sino batteries to be of sufficient quality. Obviously nothing beats OEM but OEM doesn't really exist anymore unless you get lucky (my touches are all on their original batteries and still get great runtime).

But I've run third party batteries in my mini and 4G mono and they both have held up just fine to this day. It's a roll of the dice, sure, but again, OEM doesn't really exist anymore, only trusted sellers who (I'd hope) vet their suppliers.