Without batteries our high-tech world would stop. Scientists and engineers have come a long way since the zinc-carbon batteries that we put into flashlights and transistor radios 50 years ago. Other batteries we used in those days were nickel-cadmium rechargables and Duracells, a brand name for alkaline dry cells. Batteries today have much better energy density, much more favorable recharge characteristics, and far less weight than those. Also, there are international regimes known as RoHS and WEEE to manage the use of problematic substances in batteries. That's important -- cadmium, mercury (originally used in Duracells), and lead and arsenic (found in automobile batteries) are highly toxic.
Back to lithium, which is ideal for many batteries because
- lithium is lightweight,
- is available throughout the world (although half of the known deposits are in Chile),
- is affordable,
- is sufficiently reactive chemically that it can release a prodigious amount of electricity,
- is not toxic at low concentrations (some medications use lithium),
- has a reversible chemistry for recharging, and
- can be used at ordinary temperatures.
By the way, solar panels are either composed of toxic substances (e.g. cadmium again) or require toxic substances (e.g. pure or halogenated silane) in the manufacturing process.