JGit and blobless clone: unable to fetch blobs later

Question

Unable to use JGit to do some operations on a blobless clone that I can do via Git on the command line. Specifically unable to get the blobs later. For example, consider the following sequence of Git operations on the command line:

# Clone the repository but don't checkout any blobs
git clone --no-checkout --filter=blob:none <GIT-URL> .
# Checkout a branch that is *not* the main branch on the remote
git fetch origin develop:develop
# Move to the new branch
git symbolic-ref HEAD refs/heads/develop
git checkout develop <DIRECTORY-OR-FILE>
...

I tried to translate this to JGit calls as follows:

    FilterSpec filter = FilterSpec.fromFilterLine("blob:none");
    try (Git git = Git.cloneRepository()
            .setCredentialsProvider(getCredentialsProvider())
            .setDirectory(new File("/tmp/repository"))
            .setURI(<GIT-URL>)
            .setRemote(Constants.DEFAULT_REMOTE_NAME)
            .setTransportConfigCallback(t -> t.setFilterSpec(filter))
            .setNoCheckout(true)
            .call()) {
      git.fetch()
              .setCredentialsProvider(getCredentialsProvider())
              .setRemote(Constants.DEFAULT_REMOTE_NAME)
              .setRefSpecs("refs/heads/develop:refs/heads/develop")
              .call();
      git.getRepository()
              .getRefDatabase()
              .newUpdate(Constants.HEAD, false)
              .link("refs/heads/develop");
      git.checkout()
              .setName("develop")
              .addPath(<DIRECTORY-OR-FILE>)
              .call();
      ...
    }

The checkoutfails with messages like:

org.eclipse.jgit.errors.MissingObjectException: Missing unknown 57a7e7d782f72a1e846b08415c56684f8f42e5bf

It also fails from the command-line on the repository as cloned by JGit, which is not the case if I do everything on the command line.

Answer 1

A command line git checkout ... effectively becomes a git fetch ... when the blobs to checkout are not already available, as in a blobless clone.

In the case of JGit, a CheckoutCommand can't morph into or invoke a FetchCommand - among other things there would be no way to supply a credentials provider to the fetch.

The solution is to fetch all the needed blobs by walking the revision graph as below.

void fetch(Git git, List<String> paths) throws IOException, GitAPIException {
    if (paths == null || paths.isEmpty()) {
      return;
    }

    Repository repository = git.getRepository();
    TreeWalk treeWalk = new TreeWalk(repository);
    RevTree tree = new RevWalk(repository).parseCommit(repository.findRef(HEAD).getObjectId()).getTree();
    treeWalk.addTree(tree);
    treeWalk.setRecursive(false);
    TreeFilter filter = paths.size() == 1 ? PathFilter.create(paths.get(0)) :
            OrTreeFilter.create(paths.stream().map(PathFilter::create).collect(Collectors.toList()));
    treeWalk.setFilter(filter);

    String root = repository.getDirectory().getParentFile().getAbsolutePath();
    Map<ObjectId, String> files = new HashMap<>();
    List<String> directories = new ArrayList<>();
    while(treeWalk.next()) {
      if (treeWalk.isSubtree()) {
        //noinspection ResultOfMethodCallIgnored
        new File(root + File.separator + treeWalk.getPathString()).mkdirs();
        directories.add(treeWalk.getObjectId(0).getName());
        treeWalk.enterSubtree();
      } else {
        files.put(treeWalk.getObjectId(0), treeWalk.getPathString());
      }
    }

    // Fetch all the requested blobs.
    String[] dirsAndFiles = Stream.concat(files.keySet().stream().map(ObjectId::getName), directories.stream()).toArray(String[]::new);
    files.keySet().stream().map(ObjectId::getName).forEach(directories::add);
    git.fetch()
            .setCredentialsProvider(getCredentialsProvider())
            .setRefSpecs(dirsAndFiles)
            .setCheckFetchedObjects(true)
            .call();

    // Write out the files.
    files.forEach((key, value) -> {
      try {
        repository.open(key).copyTo(new FileOutputStream(root + File.separator + value));
      } catch (IOException e) {
        throw new RuntimeException(e + " File = " + value);
      }
    });
  }