On the ground when you start the engine, can you get an accurate control of the rpm up and down by turning the needle?
I am asking this because it is very common for the fuel start foaming with vibration when using a low pitch prop and high rpms in a profile. When fuel foaming starts it is impossible to adjust the mixture and control de rpms, the engine peaks out and stay at very high rpms no matter how much you open your needle, until you open way to much and suddenly the rpms drops dramatically and the mixture get extremely rich. If you try to close the needle to get the proper mixture and rpm it will go back to peak out, so the engine works to lean and fast or too rich and slow.
You can observe the fuel running in the fuel tubing that feeds the engine, a few bubbles, although not desired, it is ok. But when you get many bubbles or foaming in the fuel line, then the engine will not run properly.
So before anything I would like to know for sure you are not having this problem.
Martin