Adding a submodule in a git remote repository

Question

In git, I created a top-level firmware project like this

mkdir firmware
cd firmware
git init --bare

cd ..
mkdir firmware_pci
cd firmware_pci
git init --bare

Now I want to add firmware_pci as a submodule to firmware, so I did this

cd ../firmware
git submodule add ../firmware_pci
fatal: /usr/libexec/git-core/git-submodule cannot be used without a working tree.

Basically, I want to create multiple firmware_xxx projects under firmware, so when I recursively clone firmware, it will clone all the firmware_xxx projects into a directory called firmware. How can I achieve this? I'm able to do this on a local repo but having trouble doing it on a remote repo.

Based on Julio P.C's feedback, I tried this in remote system:

mkdir firmware
cd firmware
git init --bare

On my local system I did this:

git clone path/to/remote_repo
cd firmware
mkdir firmware_pci
cd firmware_pci
git init
touch delme.txt
git add .
git commit -m "first checkin"
cd ..
git submodule add ./firmware_pci
git add .
git commit -m "added submodule"
git push origin master

Then, on my local PC, I did this in a different dir

git clone --recursive path/to/remote_repo
Cloning into 'firmware'...
done.
Submodule 'firmware_pci' (/path/to/firmware/firmware_pci) registered for path 'firmware_pci'
fatal: repository '/path/to/firmware/firmware_pci' does not exist
fatal: clone of '/path/to/firmware/firmware_pci' into submodule path '/home/rash/git_temp2/firmware/firmware_pci' failed
Failed to clone 'firmware_pci'. Retry scheduled
fatal: repository '/path/to/firmware/firmware_pci' does not exist
fatal: clone of '/path/to/firmware/firmware_pci' into submodule path '/home/rash/git_temp2/firmware/firmware_pci' failed
Failed to clone 'firmware_pci' a second time, aborting

TIA

Answer 1

Based on your example:

mkdir firmware
cd firmware
git init --bare

git clone path/to/remote_repo
cd firmware
[...]
git submodule add ./firmware_pci

It seems like you're trying to add a submodule directly to a remote bare repository, rather than working on a development repository that tracks your remote, and then pushing your changes.

Bare repositories shouldn't be used to work directly on them, but rather as an authoritative basis for collaborative development. As a matter of fact, they do not have the notion of current branch, since there is no working directory where to check out a copy of the current branch and make changes, as @Julio P.C. has already said in the comments.

I assume that your work (firmware) is already contained in a development repository and that you just need to add one or more submodules to it, while maintaining said submodules in sync with their respective remote. In my example, I've recreated the repositories firmware and firmware_pci as development repos, along with their remotes, although in your case they should already exist.

#creating a mock firmware repo
mkdir firmware
cd firmware
git init
echo "some work" > firmware.txt
git add .
git commit -m "init commit"

#creating a mock firmware_pci repo
cd ..
mkdir firmware_pci
cd firmware_pci
git init
echo "some work" > firmware_pci.txt
git add .
git commit -m "init commit"

At this point, we can create the bare remote repositories for both firmware and firmware_pci and push their changes to them. Subsequently, firmware_pci's remote will be added as a submodule of firmware, so that both versions of firmware_pci (the actual repo and the submodule) can track and exchange their work with the common authoritative basis (the remote).

#creating firmware's bare remote
cd ..
mkdir firmware_remote
cd firmware_remote
git init --bare

#pushing firmware's changes to its remote
cd ../firmware
git remote add origin ../firmware_remote
git push -u origin master

#creating firmware_pci's bare remote
cd ..
mkdir firmware_pci_remote
cd firmware_pci_remote
git init --bare

#pushing firmware_pci's changes to its remote
cd ../firmware_pci
git remote add origin ../firmware_pci_remote
git push -u origin master

Now, we can add the repository firmware_pci as a submodule of firmware via firmware_pci's remote and push the new submodule to firmware's remote. This time, the push command is issued with the --recurse-submodule option and the on-demand value. This version of the command pushes all the submodules' changes before pushing the repository's modifications; if any submodule push fails, then the whole command fails without pushing the repository's changes. Furthermore, since for the submodule addition we're using local paths instead of URLs, we need to enable the protocol.file.allow configuration:

#enabling file transmission
git config --global protocol.file.allow always

#adding the new submodule
cd ../firmware
git submodule add ../firmware_pci_remote
git add .
git commit -m "add submodule"

#pushing the submodules' changes. If the operations succeed, the repo's changes are pushed too.
git push --recurse-submodules=on-demand

Finally, we can now clone the firmware repository with the --recurse-submodules option. This version of the command not only clones the firmware repository, but it also initializes every submodule with their nested submodules and fetches their data. In fact, before being able to fetch a submodule's content, we first need to initialize it.

cd ..
git clone firmware_remote --recurse-submodules firmware_clone

---

Comment's answer: Detached HEAD state scenario for submodule

Sadly, whenever you're pulling changes into a submodule with a git submodule update --remote, Git always brings the submodule into the detached HEAD state, whether you've checked out a branch or not, aligning the submodule's content to its remote. At this point, once the new changes have been fetched, it's the developer's duty to choose whether those changes should be merged or rebased with a specific branch of the submodule.

In order to maintain your checked-out branch and integrate the remote's new changes, you can either launch a git submodule update --remote --merge or a git submodule update --remote --rebase.

#cd-ing into the submodule
cd firmware/firmware_pci_remote/

#checking out the desired submodule's branch
git checkout master

#cd-ing out of the submodule and into the outer repo
cd ..

#merging the remote's changes into the checked-out branch master...
git submodule update --remote --merge

#...or alternatively, rebasing the remote's changes onto the checked-out branch master
git submodule update --remote --rebase

---

Further Notes on Pulling Submodule Changes

Since pulling changes from a submodule's remote aligns the working directory to the fetched content, the command git submodule update --remote can be quite dangerous. In fact, if we were to have committed some work while being in the detached HEAD state and then pulled some new changes, all the commits created in the detached HEAD state would be lost, as the anonymous branch would be aligned to the remote.

#cd-ing into the submodule

#detached HEAD state

#making some commits

#cd-ing out of the submodule and into the outer repo
cd ..

#pulling the changes from the remote
git submodule update --remote

#the submodule is aligned with the remote's state losing all our commits

The only instance where a pull operation does not align the working directory of a submodule with its remote is when there are uncommitted changes on the checked-out branch. In fact, in this case the fetched changes will be stored on the corresponding tracking branch, and at that point it is the developer's duty to manually merge (or rebase) them.

#cd-ing into the submodule

#checking-out a branch
git checkout master

#making some changes without committing

#cd-ing out of the submodule and into the outer repo
cd ..

#pulling the changes from the remote
git submodule update --remote

#cd-ing into the submodule

#we're still on the master branch and we haven't lost our work

#committing our uncommitted changes for a clean merge

#merging the pulled in changes
git merge origin/master

In order to save our work while fetching new changes, we can use a few approaches (each case works also with a rebase approach):

Case 1

Check-out a branch, making some changes and then merge/rebase the remote's changes

#cd inside the submodule

#checking out a branch
git checkout master

#making some commits

#cd-ing out of the submodule and into the outer repo
cd ..

#merging the remote's changes into the checked-out branch
git submodule update --remote --merge

Case 2

Make some changes in the detached HEAD state, bring those changes to a new branch and then merge/rebase the remote's changes

#cd inside the submodule

#detached HEAD state

#making some commits

#brining the changes to a new branch
git checkout -b temp

#cd-ing out of the submodule and into the outer repo
cd ..

#merging the remote's changes into the checked-out branch
git submodule update --remote --merge

Case 3

Check-out a branch, commit the new work, pull the changes with no --merge or --rebase option and then manually merging/rebasing

#cd-ing into the submodule

#checking the master branch
git checkout master

#cd-ing out of the submodule and into the outer repo
cd ..

#fetching the changes and re-entering the detached HEAD state since no --merge or --rebase option has been specified
git submodule update --remote

#cd-ing into the submodule

#detached HEAD state

#re-checking out the master branch
git checkout master

#merging the changes pulled from the corresponding tracking branch of the remote's default branch (in .gitmodules file)
git merge origin/master