How do you stop pdf from rebuilding in flutter?

Question

I have been trying to work on this pdf view which tracks the user activity on the pdf for how far they have reached. The pdf is allowed to be rotated and on every page change I am trying to save the page number using provider. This pdf viewer comes in one of pages rendered using pageView.builder, since I am getting all the page details from the api.

The problem comes when the user tries to rotate the screen or go to previous page the pdf rebuilds and resets it to first page. I have managed to save the highest page they have attained so that when they come back to the page where pdf is rendered, it'll redirect to the page where they left from. But this comes with lot of screen flickers as the UI needs to rebuild again and again. How do I avoid this?

Below is the code I am using for it:

 return Scaffold(
  body: Column(
    children: [
      Expanded(
        child: Container(
          child: Stack(
            children: [
              PDFView(
                onPageError: (page, error) => const CircularProgressIndicator(),
                pageSnap: true,
                fitPolicy: FitPolicy.WIDTH,
                fitEachPage: true,
                filePath: snapshot.data!,
                enableSwipe: true,
                swipeHorizontal: true,
                onRender: (pages) {
                  debugPrint("previous page = ${controller.page!}");
                  if (pages! > 1) {
                    if (controller.page! > previousPage) {
                      debugPrint("page completely swiped");
                      // To avoid clash of pdf pages and pageView builder horizontal scroll
                      if (!_pdfLoaded) {
                        setState(() {
                          _isScrollable = false;
                          _pdfLoaded = true;
                        });
                      }
                    } else {
                      debugPrint("page incomplete swipe");
                    }
                  }
                },
                onViewCreated: (pdfcontroller) {
                  debugPrint("page = $_pageTrack");
                  int gotoPage = _pageTrack.fold(0, math.max);
                  Future.delayed(const Duration(seconds: 1), () async {
                    await pdfcontroller.setPage(gotoPage);
                  });
                },
                onPageChanged: (page, total) {
                  debugPrint("page Changed");
                  if (page == total! - 1) {
                    Future.delayed(const Duration(milliseconds: 100), () {
                      if (!_isScrollable) {
                        setState(() {
                          _isScrollable = true;
                          _latestPdfPage = page!;
                        });
                      }
                    });
                  }
                  if (page == 2 &&
                      Provider.of<UserProvider>(context, listen: false)
                              .getPageTrack ==
                          1) {
                    // Do something
                  }
                  _pageTrack.add(page!);

                  Provider.of<UserProvider>(context, listen: false)
                      .setPageTrack(_pageTrack, total);
                },
              ),
              // To hide/show app bar
              IgnorePointer(
                ignoring: false,
                child: GestureDetector(
                  onTap: () {
                    context.read<TabProvider>().setShowAppBar =
                        !context.read<TabProvider>().showAppBar;
                  },
                ),
              ),
            ],
          ),
        ),
      ),
    ],
  ),
);

I have tried using AutomaticKeepAliveClientMixin and adding
@override bool get wantKeepAlive => true;

but this solution isn't working for me.