As I understand it, stability is a measure of a planes tendency to return to it's previous flight direction when dislodged, in this case, pitched. You need some of that tendency in order to control the plane smoothly - unless you have a computer and appropriate sensors for it to react to divergent behavior instantaneously and proportionately. Neutral stability is still not good enough for us, since that would demand that we have incredible reflexes and placement of the handle. All pitch directions would be entirely our doing by simple hand placement, with no help from the plane. We really need a restoring force to work against. You're right, unreasonable stability would keep us from controlling the plane at all too.
The Wright brothers "Flyer" was deliberately "control configured" as a reaction to what happened to Lillienthal, when lack of control authority cost him his life. So they deliberately made it only marginally stable. The Flyer was a real hand-full to control, and the Wrights taught themselves not only to be the first pilots but to be expert pilots. From what I've read, Bob Baron must have excelled at flying closer to the edge too.
We all have our preferences in static "stability," how "lively" the plane is at the end of the lines. Canards and conventional aft-tail designs can each have the stability one desires. The problem with canards is that in order for them to have their best efficiencies and maximum lift coefficients, they must be about neutrally "stable," which just won't do in CL or anywhere else without computers. In fact, CL planes typically have significantly greater static margins (distance between c.g. and aerodynamic center) than R/C and full-sized planes. The FF guys have the furthest aft c.g.'s of all So we will have to stunt with canards that just cannot produce the maximum performance of the best conventional stunters.