How to make a moving ball with mouse motion listener

Question

 addMouseMotionListener(new MouseAdapter() {
        public void mouseMoved(MouseEvent e) {
            relativeX = e.getX();
            relativeY = e.getY();
            System.out.println(relativeX + "," + relativeY);
            System.out.println(x + "," + y);
            x = x + speedX;
            y = y + speedY;
            repaint();
            if (x > relativeX) {
                speedX = speed * -1;
            } else if (x < relativeX) {
                speedX = speed;
            }
            if (y > relativeY) {
                speedY = speed * -1;
            } else if (y < relativeY) {
                speedY = speed;
            }
        }
    });

Hi everyone, I created one mouse motion listener and the ball will follow the direction where my mouse moved to. However, the ball will stop moving once I stop moving my mouse. Even though, there is a quite a distance between my mouse cursor and the ball's location, the ball just refuse to move to the location of my mouse. I think it is because of my motion listener stopped working, since I am no longer moving the mouse. Does anyone have any ideas of how to force the ball to move to the exact position of my mouse.(first time asking on stack overflow hope everyone bare with my grammar)

 private void doDrawing(Graphics g) {
    radius = 20;
    Graphics2D g2d = (Graphics2D) g;
    g2d.setColor(red);
    g2d.fillOval((int) x, (int) y, radius, radius);
}

And this is the moving ball.