We investigate sensor transmission power control for remote state estimation. Instead of using a conventional sensor, a sensor equipped with an energy harvester which can obtain energy from the external environment is utilized. We formulate power control of the energy harvesting sensor into an infinite time-horizon Markov decision process (MDP). To deal with the computation complexity associated with this multi-dimensional MDP, a continuous-time approach and perturbation analysis are used and a closed-form approximate value function is derived. Based on the approximation, we obtain a closed-form optimal power control solution which has a threshold-based structure. A numerical example is provided to evaluate the estimation performance of the optimal solution compared with other power scheduling schemes.