How to make clangd background-index work for yet unopened files without using compile_commands.json?

Question

I'm trying to use the clangd extension in Visual Studio Code to view & edit several already existing C projects for embedded ARM. The projects are similar in structure, use virtually only C code and can use the same compiler flags for all files. My goal is to add some settings / config files into a project that are enough to make clangd's features (code suggestions, correct symbol referencing etc.) work, ideally without the need for more extra tools or big files. It's NOT my goal to replace the already used toolchain (Eclipse STM32CubeIDE) for compiling into binaries.

I managed to get CMake to output a compile_commands.json and let clang use that. With this, everything works fine. However the paths in there are all absolute and thus not portable to other projects or computers (and the file tends to get big), so commiting it is not an option. And I don't want to setup CMake for each project every time just to create compile_commands.json at first.

After a lot of trial and error, I have found a combination of settings that works ALMOST perfectly without using compile_commands.json: a few entries for "clangd.arguments" and "clangd.fallbackFlags" in settings.json, and various compiler flags in the .clangd configuration file. Only problem left now is that "Go to definition" only jumps to the declaration in the included .h file, not to the implementation in .c. As soon as I once opened the respective .c file, "Go to definition" correctly jumps to the implementation in .c. But after a VS Code restart, it only opens the .h declaration again.

I read there is a background indexer in clangd that does indexing of project files which are not opened. This looks exactly like the thing that I'm still missing. But it seems the background indexer does not run at all although I explicitly activated it with --background-index. There is an auto-generated directory .cache\clangd\index\ that contains the index information for every source file. But even with all these files present, "Go to definition" still resolves incorrectly before the .c file has been opened in the editor.

So it's not a problem of the information not being there. I suppose due to the lack of compile_commands.json, clangd has simply no information about which files are actually part of the project, and so it doesn't try to use the respective source file's index at all unless it is opened.(?) Is there any way to somehow specify to clangd or background-index which files are part of the project, so that the index of all files is created and also actually used from the beginning on? Or lies the problem somewhere else?

I would be fine with e.g. some sort of "reduced" compile_commands.json that only contains the paths to all used source files, but this is not supported, it seems.

Answer 1

A compile_commands.json file is required for clangd's background index to work, that's where clangd gets the list of files to index.

You are also correct to observe that, since the compile_commands.json file contains absolute paths, you generally don't want to commit it to the repository. Rather, you would typically commit a script that generates the compile_commands.json, and then run that script after checking out the project.

Often this script is part of / built into the build system. A popular tool to use with build systems that don't have dedicated suppport for generating compile_commands.json is Bear. Bear will wrap any build command (for example, if your build command is make, you can run bear -- make), and it will "listen" for the compiler invocations performed during the build and output a compile_commands.json reflecting those compile commands.

The advantage of tying the generation of compile_commands.json to the build is that the compile flags only need to be specified in one place (the build system).

However, there's no requirement for the generation of compile_commands.json to be tied to the build. Nothing prevents you from writing a simple shell script that, say, recursively enumerates all the files in the project directory, and for every file with a .c extension, outputs an entry in the compile_commands.json with some hard-coded flags.