We have a single stage dockerfile building a single image.
Due to many logical considerations, we want to separate this build to many different stages (e.g. production, development, testing etc..). We plan to use BuildKit enabled for all its benefits.
My question is: Is there a way to know in advance which targets need rebuilding when the dockerfile changes? We really don't want to rebuild all images every time the dockerfile changed. That would be a huge waste of time for the end user, especially since there is a good chance the change is not applicable to that specific end user.
Separating the dockerfile to different files is not an option. There are many shared components between the different stages, some take a lot of storage. We're relying on docker layering to save up the storage space by reusing those layers between stages.
Here's a simplified example of what the dockerfile will look like. I don't want to rebuild the "testing" stage if I only changed the toolchain, or added some package to the "developer" stage. Also, toolchain stage will be shared between many stages so can't just embed it inside "developer" stage.
FROM SomeImage as toolchain
RUN wget https://path.to/compiler.tar.gz
RUN tar zxfv compiler.tar.gz /path/to/compiler
RUN rm compiler.tar.gz
FROM SomeImage as deployment
# Base packages
RUN apt-get install -y openssl which python3 python3-pip ...
CMD /bin/bash
FROM deployment as testing
RUN wget https://path.to/testing.suite.tar.gz
RUN tar zxfv testing.suite.tar.gz testing/
RUN cd testing && ./install_testing_suite
RUN rm -rf testing.tar.gz testing/
FROM deployment as developer
COPY --from=toolchain /path/to/compiler /path/to/compiler
.....
Note: We use docker 19.3