Yes, my bad, there was already a hospital in the base. Curious that it disappears from the list after building one, unlike any other structure.
I checked that it's like with all other structures in that it's possible to build a hospital without a power plant, it's just not operational until there's a working powerplant in the base.