Skip to main content
Welcome guest. | Register | Login | Post

splitting large files or directories into small tar files

Every once in a while there comes a situation where a file is too big for a CD, or you want to store a file larger than 2GB on a FAT partition, or you want to make a backup of data with UNIX-style permissions (almost a 100% necessity if backing up a $HOME folder with settings) and all you have is a FAT volume somewhere. Here the multi-volume feature of GNU tar comes to play. The ZIP and RAR formats have a similar feature, probably originally for floppy disks, but they don't store UNIX-style permissions. Also, the feature is in GNU tar for a partly historical reason (partly because it is probably still widely used for the original purpose):

TAR stands for Tape Archive and is the way to store data on magnetic tapes, the sort that is still sometimes used for backups. Tapes have a fixed size, and backups can get enourmous, so logically you'll need multiple tapes. However, this feature can be used with files as well: (user-entered text is in italics)

% tar --create --multi-volume --tape-length 200M --file data1.tar --same-permissions file1 file2 ... fileN
Prepare volume #2 for `data1.tar' and hit return: n data2.tar
Prepare volume #3 for `data2.tar' and hit return: n data3.tar
Prepare volume #N for `data2.tar' and hit return: n dataN.tar

the option name --tape-length strongly hints the origin of the feature. when entering n dataN.tar, you're telling GNU tar that the name of the new part should be dataN.tar; if you don't specify a new name, it will write to the same file again. This, again, comes from the tape history of the feature, where you would change tapes and press enter. The same procedure can be used with floppy disks.
It is quite a nuisance to enter the new file name all the time (I should know, I did it with 35 archives before learning about the now-described usage), so you can also specify multiple files at the command line thus: (This time, I am using short options)

% tar -cMp -f data1.tar -f data2.tar -f data3.tar -L 200M file1 file2 file3 ... fileN

Using strange old-style options allows you to use shell expansions here (you need the correct number of 'f's)

% tar cMpLfff 200M data{1,2,3}.tar file1 file2 ... fileN


little more than swapping --create or -c for --get or -x, for example:

% tar xpMfff data{1,2,3}.tar

Though I haven't tried it, you should be able to use -z or -j options for gzip or bzip2 compression, respectively, with multi-volume archives as well.


Good tips. It may come in

Good tips. It may come in handy for my current issue where I need to split a file.