You can edit the tracker channels directly on the image instead of editing them in the Channel Editor. Note that the curve on the image shows the X and Y components together and not separately like in the Channel Editor. The tracking path on the image shows the actual displacement of the tracker box.

To edit the curve directly in the image window, Path must be enabled in the Setup menu. When you select a keyframe on the curve, the timeline goes to the frame that corresponds to the keyframe. When you select the reference box, the timeline goes to the frame where you set the reference.

When you edit the tracking path in the image window, all the options in the Edit Mode box are available except Rect Zoom, Zoom, and Pan.