We borrowed an EVI-D70 and used it for one stake conference webcast and found a similar "jerky" movement using the IR control, especially when zoomed in. The pan/tilt IS proportional to the amount of zoom dialed in. In other words, the pans/tilt motors run slower the more it's zoomed in. However (IMO), the pan/tilt speed is not slow enough when zoomed in, making it jerky and easy to overshoot your target, even with a single tap on the remote. We didn't use the serial interface, so maybe that has the ability for a finer control, I don't know.
Our solution was to setup and use the available presets. We had one set on the podium, one on the conductor and one zoomed back to take in the choir. Occasionally a rather tall speaker would require a manual tap on the remote to tilt the camera up and the best way we learned to do that was to zoom out a bit, then tap the up button, then zoom back in. That minimized the jerkiness and avoided overshooting his height.
Also, we found the range of the IR to be a disappointing 20 feet or so. I would think it should do at least twice that, so make sure you are close enough to avoid those kinds of control problems.
Maybe others can add their experiences, but from what you are describing, yours is most likely working just like ours did.
So we can better help you, please edit your Profile to include your general location.