Estimation theoretic secure transmission of a scalar random parameter is investigated in the presence of an eavesdropper. The aim is to minimize the estimation error at the receiver under a secrecy constraint at the eavesdropper; or, alternatively, to maximize the estimation error at the eavesdropper for a given estimation accuracy limit at the receiver. In the considered setting, the encoder at the transmitter is allowed to use a randomized mapping between two one-to-one and continuous functions and the eavesdropper is fully aware of the encoding strategy at the transmitter. For small numbers of observations, both the eavesdropper and the receiver are modeled to employ linear minimum mean-squared error (LMMSE) estimators, and for large numbers of observations, the expectation of the conditional Cramér-Rao bound (ECRB) metric is employed for both the receiver and the eavesdropper. Optimization problems are formulated and various theoretical results are provided in order to obtain the optimal solutions and to analyze the effects of encoder randomization. In addition, numerical examples are presented to corroborate the theoretical results. It is observed that stochastic encoding can bring significant performance gains for estimation theoretic secrecy problems.