CMake always rebuilding included file even if it didn't change

Question

In my project I am including 2 header files InitializingValues.hpp and PrimitivePolynoms.hpp (with respective cpp files) into a file called SobolGenerator.cpp. The content of the included files is always constant, so there is nothing changing dynamically. Both consist of a single class with only constants and some functions to access those constants.
For building the project I use CMake. Those 3 files are located in the same directory and are built using a submodule building process for the full project. This submodule building looks like this:

# Constants for the UQ library especially for the sobol generation
set(constant_src
    generators/InitializingValues.cpp
    generators/PrimitivePolynoms.cpp
)
add_library(LIB_CONSTANTS STATIC ${constant_src})

set(uq_src
        generators/Random.cpp
        generators/CirculantEmbedding.cpp
        generators/SymmetricCovariance.cpp
        generators/KLExpansionGenerator.cpp
        generators/HybridFluxGenerator.cpp
        generators/SobolGenerator.cpp

        problems/StochasticReactionProblem.cpp
        problems/StochasticTransportProblems.cpp
        problems/StochasticAcousticProblems.cpp

        estimators/SingleLevelEstimator.cpp
        estimators/MultiLevelEstimator.cpp
        estimators/EstimatorMap.cpp
        estimators/WelfordAggregate.cpp
        estimators/MultiSampleFEM.cpp
#        sgd/StochasticGradientDescent.cpp
)

if (USE_TASMANIAN)
    set(uq_src ${uq_src}
            SparseGridGenerator.cpp
    )
endif ()

if (USE_SPACETIME)
    set(uq_src ${uq_src}
            problems/StochasticSTAcousticProblems.cpp
            problems/StochasticSTTransportProblems.cpp
    )
endif ()

add_library(LIB_UQ STATIC ${uq_src})
target_link_libraries(LIB_UQ LIB_APP sprng LIB_CONSTANTS)

if (USE_TASMANIAN)
    target_link_libraries(LIB_UQ Tasmanian_libsparsegrid)
endif ()

#add_subdirectory(smc)

I split up the 2 constant files and the SobolGenerator.cpp into seperate libraries s.t. those constants won't get rebuilt if I only change something in SobolGenerator.cpp to save some time (the 2 constant files contain about 20000 LOC which otherwise would take years to rebuilt every time). For some reason those 2 constant files are rebuilt when only changing SobolGenerator.cpp even with this CMake config. Am I doing something wrong or is there a better way to prevent those constant files to be recompiled every time even if they didn't change?