How to update an svn dump

Question

Im using reposurgon which reads svn dump's, created simply by:

svnadmin dump /svnroot/my-repo > svn_dump

However I need to update this file because we are still committing to the repository and re-creating the entire file each time is quite time consuming (its almost 60gig).

So my question is:

Is there a way to append onto this file each time to avoid fully re-creating the dump?

Ideally this would be possible without manually having to record the ranges dumped each time

---

Update

I manged to do this using incremental dumps, but the first 4 lines of each file after the first need to be removed.

svnadmin dump -r0 /svnroot/my-repo --incremental                   > svn_dump
svnadmin dump -r1 /svnroot/my-repo --incremental | sed -n '5,$ p' >> svn_dump
svnadmin dump -r2 /svnroot/my-repo --incremental | sed -n '5,$ p' >> svn_dump
svnadmin dump -r3 /svnroot/my-repo --incremental | sed -n '5,$ p' >> svn_dump
svnadmin dump -r4 /svnroot/my-repo --incremental | sed -n '5,$ p' >> svn_dump

# is the same as:
svnadmin dump -r0:4 /svnroot/my-repo                               > svn_dump

However this seems fairly add-hoc, and for it to be useful Id need to write a more comprehensive script.

Answer 1

Apache Subversion 1.8 added support for incremental svnadmin hotcopy backups. Prior to version 1.8, svnadmin hotcopy was able to create only complete backup, refusing to copy over existing hotcopied repository.

The major benefit of using svnadmin hotcopy instead of svnadmin dump / load is it's performance which is limited to your disk I/O speed only.

Answer 2

You dumpfile is really big. Are you storing a lot of binary files in your repository? If so you may want to consider storing binaries in another location.

Anyway, there is the possibility of an incremental dump. From the svn manual:

Two useful options modify the dump file generator's behavior. The first is the --incremental option, which simply causes that first revision in the dump stream to contain only the files and directories modified in that revision, instead of being presented as the addition of a new tree, and in exactly the same way that every other revision in the dump file is presented. This is useful for generating a relatively small dump file to be loaded into another repository that already has the files and directories that exist in the original repository.

The second useful option is --deltas. This option causes svnadmin dump to, instead of emitting full-text representations of file contents and property lists, emit only deltas of those items against their previous versions. This reduces (in some cases, drastically) the size of the dump file that svnadmin dump creates. There are, however, disadvantages to using this option—deltified dump files are more CPU-intensive to create, cannot be operated on by svndumpfilter, and tend not to compress as well as their nondeltified counterparts when using third-party tools such as gzip and bzip2.

You can have a script that uses this functionality like:

$ svnadmin dump /var/svn/repos -r 21 --incremental > incr.dump
* Dumped revision 21.

You can also use the other useful thing - the deltas! For example you can have a weekly full backup and daily - incremental and deltas backups.

However if you are storing big amounts of binary data be aware that deltas will be very ineffective. The way that svn works internally is to blame here - it is just not meant to store binary files.

The general answer is that you cannot directly do this - i.e. you cannot append to a dump file. That's it.