MissingObjectException when formatting diff with working directory

[xcq1]

Version

7.5.0

Operating System

MacOS

Bug description

I'm trying to do a diff in a repository from the index with the things that were only changed on the file system.
I have started to write a simple test case like this:

	val repo = createTempDirectory()
    val git = Git.init().setDirectory(repo.toFile()).call()

    repo.resolve("file.txt").createFile().writeText("Hello World")
    git.commit().setMessage("Test").call()
    repo.resolve("file.txt").writeText("Modified")

    val diff = git.diff()
		.setOldTree(DirCacheIterator(git.repository.readDirCache()))
		.setNewTree(FileTreeIterator(git.repository))
		.call()
    diff.forEach { println(it) }

    val diffString = ByteArrayOutputStream().use { baos ->
        DiffFormatter(baos).use { fmt ->
            fmt.setRepository(git.repository)
            fmt.format(diff)
        }

        baos.toString(Charsets.UTF_8)
    }
    println(diffString)

But when executing this, I'm not getting the results I expect. I see only DiffEntry[ADD file.txt] as log output, but no diff is reported due to a MissingObjectException.

Actual behavior

A MissingObjectException is thrown. I assume the DiffFormatter is looking for an object that does not yet exist in the index?

Expected behavior

The code to complete without exception and returning a valid diff. To illustrate, consider the same workflow on the shell:

tmp.Y5n222eC68 on  main [!]
➜ cd $(mktemp -d)

3yl0tbgs25j7gdl2ttgclmsw54gtc3/T/tmp.HRxmal1kE9
➜ git init
Initialized empty Git repository in /private/var/folders/j9/3yl0tbgs25j7gdl2ttgclmsw54gtc3/T/tmp.HRxmal1kE9/.git/

tmp.HRxmal1kE9 on  main
➜ echo "Hello World" > file.txt

tmp.HRxmal1kE9 on  main [?]
➜ git add file.txt

tmp.HRxmal1kE9 on  main [+]
➜ git commit -m "Test"
[main (root-commit) 559d728] Test
 1 file changed, 1 insertion(+)
 create mode 100644 file.txt

tmp.HRxmal1kE9 on  main
➜ echo "Modified" > file.txt

tmp.HRxmal1kE9 on  main [!]
➜ git diff
diff --git a/file.txt b/file.txt
index 557db03..a07e6a9 100644
--- a/file.txt
+++ b/file.txt
@@ -1 +1 @@
-Hello World
+Modified

Relevant log output

org.eclipse.jgit.errors.MissingObjectException: Missing blob da6fda422c3d917e2a85b30d06686b1c0a990785
	at org.eclipse.jgit.internal.storage.file.WindowCursor.open(WindowCursor.java:151)
	at org.eclipse.jgit.diff.ContentSource$ObjectReaderSource.open(ContentSource.java:135)
	at org.eclipse.jgit.diff.ContentSource$Pair.open(ContentSource.java:302)
	at org.eclipse.jgit.diff.DiffFormatter.open(DiffFormatter.java:1156)
	at org.eclipse.jgit.diff.DiffFormatter.createFormatResult(DiffFormatter.java:1084)
	at org.eclipse.jgit.diff.DiffFormatter.format(DiffFormatter.java:707)
	at org.eclipse.jgit.diff.DiffFormatter.format(DiffFormatter.java:694)
	...

Other information

I tried to see if I can compare with a ref instead by using this code:

    val ref = git.repository.exactRef("HEAD")
    val tree = RevWalk(git.repository).use { walk ->
        val commit = walk.parseCommit(ref.objectId)
        walk.parseTree(commit.tree.id)
    }

    val diff = git.diff()
        .setOldTree(CanonicalTreeParser().also { treeParser ->
            treeParser.reset(git.repository.newObjectReader(), tree.id)
        })
        .setNewTree(FileTreeIterator(git.repository))
        .call()

But this results in the same exception, if the ref exists. (If the ref does not exist, then ref is null and I'm getting an NPE)

[tomaswolf]

With C git, you do a git add file.txt before the commit. In the JGit code you've given I don't see that.

[xcq1]

Apologies, of course the equivalent code should include a git.add()....
However, it does not make a difference, I'm seeing the same exception.
I have pushed a minimal repro project with the fixed code here.

[tomaswolf]

See the javadoc of DiffFormatter.format(List<? extends DiffEntry>)

jgit/org.eclipse.jgit/src/org/eclipse/jgit/diff/DiffFormatter.java

Lines 682 to 684 in dcb0ed1

	* Format a patch script from a list of difference entries. Requires
	* {@link #scan(AbstractTreeIterator, AbstractTreeIterator)} to have been
	* called first.

Requires scan(AbstractTreeIterator, AbstractTreeIterator) to have been called first.

You're mixing abstractions here. git.diff() is a high-level DiffCommand, while DiffFormatter is a lower level thing. DiffCommand uses DiffFormatter internally.

To get a list of DiffEntries and the diff chunks, call DiffCommand.setOutputStream():

try (ByteArrayOutputStream buf = new ByteArrayOutputStream()) {
	List<DiffEntry> diffs = git.diff()
			.setOldTree(new DirCacheIterator(git.getRepository().readDirCache()))
			.setNewTree(new FileTreeIterator(git.getRepository()))
			.setOutputStream(buf)
			.call();
	for (DiffEntry diff : diffs) {
		System.out.println(diff.toString());
	}
	buf.flush();
	System.out.println(buf.toString(StandardCharsets.UTF_8));
}

(Java code; I don't have Kotlin.)

If you want to use DiffFormatter directly, do

try (ByteArrayOutputStream buf = new ByteArrayOutputStream();
		DiffFormatter fmt = new DiffFormatter(buf)) {
	fmt.setRepository(git.getRepository());
	List<DiffEntry> diffs = fmt.scan(
			new DirCacheIterator(git.getRepository().readDirCache()),
			new FileTreeIterator(git.getRepository()));
	for (DiffEntry diff : diffs) {
		System.out.println(diff.toString());
	}
	fmt.format(diffs);
	buf.flush();
	System.out.println(buf.toString(StandardCharsets.UTF_8));
}

With that second approach, you can then of course also choose to format only some of the diffs.