It should stay at rest in any position you put it in.

It should come to a rest at random locations as it spins down. Manually putting it horizontal, and observing that "it doesn't move" is pretty close, but is dependent on the friction of your balancing method. Ditto for any other position.

For precision static balancing, you can--with sufficient skill--balance to about 1/2 of the friction of the balancer. All balancers have friction. Some applications are much more demanding than balancing a model airplane propeller, which is mounted to a single cylinder engine which can never be fully dynamically balanced.

When static balancing, always note which way the prop hangs from multiple trials. It won't likely be just along the span of the blade. It will generally have a cross-axis component. If you think of the true center of gravity as concentrating all of the mass at a point, that point will be inside the hub and straight down. Now imagine balancing a 3-blade prop. You can see that a correct mental picture of what is going on becomes more important to success.

There is a video out there of an experienced modeler telling folks how to balance a prop. He invents his own terminology based on the way he choses to think about it. But at best, it is highly misleading. He talks about "hub balancing" when he orients the prop vertically. Since the blades and the hub are integral, there is no point in talking about true hub balance, such as may done with a full-scale constant speed propeller hub prior to assembly. He really has no idea if the blades have a chordwise imbalance, or if the entire prop is off-center from the hole, which is your datum for balancing.

Dave