building docker images for a complex repository and dependencies to several repositories

Question

I'm trying to build multi docker image from a GitHub repository which has dependencies to other repositories (it's a dotnet solution) using buildx. I have several docker file for each service.

org:
  - framework
    - multi .csproj but a .sln
  - modules
    - multi .csproj & multi .sln per module but no dockerfile
  - project:
    - services:
      - path/to/service1/Dockerfile
      - path/to/service2/Dockerfile
    - apps:
      - path/to/app1/Dockerfile
      - path/to/app2/Dockerfile
    - Dockerfile.base    // base image for shared layers
    - project.sln

here is a sample dependency from project repository:

<ProjectReference Include="../../../../../framework/src/Caching.StackExchangeRedis/Caching.StackExchangeRedis.csproj" />
<ProjectReference Include="../../../../../modules/theme/src/AspNetCore.Mvc.UI.Theme/AspNetCore.Mvc.UI.Theme.csproj" />

Currently, I am doing this build by cloning the rest of the repositories and building dockerfiles from org context. Is there an option to spend less CI time at docker build?

I tried this locally:

- name: Build and push Docker image
  uses: docker/build-push-action@v4
  with:
    build-contexts: |
      framework=https://github.com/userName/framework.git#master
      modules=https://github.com/userName/modules.git#dev
    context: "{{defaultContext}}"
    file: ${{ matrix.dockerfile }}
    push: ${{ github.event_name != 'pull_request' }}
    tags: ${{ steps.meta.outputs.tags }}
    labels: ${{ steps.meta.outputs.labels }}
    secrets: |
      GIT_AUTH_TOKEN=${{ secrets.GH_AUTH_TOKEN }}

The problem is that if one build fails, all the other images will be upgraded, so we have a problem with the automatic version release and one or more images are not updated.