Ubuntu Manpage: fpart — Sort and pack files into partitions

NAME

       fpart — Sort and pack files into partitions

SYNOPSIS

       fpart  [-h]  [-V] -n num | -f files | -s size [-i infile] [-a] [-o outfile] [-0] [-e] [-P] [-v] [-l] [-b]
             [-y pattern] [-Y pattern] [-x pattern] [-X pattern] [-z] [-zz] [-zzz] [-Z]  [-d  depth]  [-D]  [-E]
             [-L] [-S] [-w cmd] [-W cmd] [-R cmd] [-p num] [-q num] [-r num] [FILE or DIR...]

DESCRIPTION

       The fpart utility helps you sort file trees and pack them into bags (called "partitions").

GENERAL OPTIONS

       -h, --help
               Print help

       -V, --version
               Print version

PARTITION CONTROL

       -n num, --parts num
               Create  exactly  num  partitions  and try to generate partitions with the same size and number of
               files.  This option cannot be used in conjunction with -f, -s or -L.

       -f files, --files files
               Create partitions containing at most files files or directories.  This  option  can  be  used  in
               conjunction with -s and -L.

       -s size, --size size
               Create partitions with a maximum size of size bytes.  With this option, a special partition 0 may
               be  used  to  handle files that do not fit in a regular partition, given the provided size limit.
               This option can be used in conjunction with -f and -L.  You can use a human-friendly unit  suffix
               here (k, m, g, t, p).

INPUT CONTROL

       -i infile
               Read file list from infile.  If infile is “-”, then list is read from stdin.

       -a, --arbitrary
               Input contains arbitrary values; just sort them (do not crawl filesystem).  Input must follow the
               “size(blank)path” scheme.  This option is incompatible with crawling-related options.

OUTPUT CONTROL

-o outfile
Output partitions' contents to outfile template. Multiple files will be generated given that
template. Each outfile will get partition number as a suffix. If outfile is “-”, then
partitions will be printed to stdout, with partition number used as a prefix (so you can grep
partitions you are interested in, or do whatever you want).

-0 End filenames with a null (’\0’) character when using option -o.

-e When adding directories (see “DIRECTORY HANDLING” ), add an ending “/” to each directory entry.

-P Add parent directories when closing partitions. That option can be used in conjunction with -zzz
to produce partitions that can be synchronized in parallel whith tools such as cpio(1) or tar(1).
Adding parent directories at the end of each partition ensures that modification times get
reapplied to directories whatever the processing order of partitions is. Directories added that
way are 0-sized and not subject to partition counters (for example, a file limit given using
option -f will be exceeded by n parent directories). Also, they are not subject to
inclusion/exclusion options ( -y, -Y, -x, -X) and they always have an ending “/” even if option
-e has not been used (this simplifies symlinks handling as we always want to add targets here,
never the links themselves). That option may lead to creating duplicate directory entries when
next partition begins with a directory entry that has already been added as a parent when closing
the previous partition. Only intermediate partitions will get parent directories added, not the
very last one which gets its parents through option -zzz when fts(3) crawling finishes. Requires
live mode (option -L)

-v, --verbose
Verbose mode (may be specified more than once).

FILESYSTEM CRAWLING CONTROL

-l Follow symbolic links (default: do not follow).

-b Do not cross filesystem boundaries (default: cross).

-y pattern, --include pattern
Include files or directories matching pattern only (and discard all other files). This option
may be specified several times. Pattern may be a leaf (file or directory) name or a specific
path. Shell pattern matching characters (“[”, “]”, “*”, “?”) may be used. Include patterns are
ignored when computing size of directories.

-Y pattern
Same as -y but case insensitive. This option may not be available on your platform (at least
FreeBSD and GNU/Linux support it, Solaris does not).

-x pattern, --exclude pattern
Exclude files or directories matching pattern. This option can be used in conjunction with -y
and -Y. In this case, exclusion is performed after. This option may be specified several times.
Pattern may be a leaf (file or directory) name or a specific path. Shell pattern matching
characters (“[”, “]”, “*”, “?”) may be used. Exclude patterns also apply when computing size of
directories.

-X pattern
Same as -x but case insensitive. This option may not be available on your platform (at least
FreeBSD and GNU/Linux support it, Solaris does not).

DIRECTORY HANDLING

-z Pack empty directories. By default, fpart will pack files only (except when using the -d or -D
options). This option can be useful for tools such as rsync(1) to be able to recreate a full
file tree when used with fpart (e.g. using rsync's --files-from option). See the -zz option to
also pack un-readable directories.

-zz Treat un-readable or erroneous (partly-read) directories as empty, causing them to be packed
anyway. Partly-read (non-empty) directories can end up being packed while some of their children
have already been packed.

-zzz Pack all directories (as empty). Useful when 3rd party tools need directory entries to update
them (e.g. cpio(1) or tar(1) ).

-Z Pack un-readable/erroneous directories in dedicated partitions. This option helps isolating
erroneous parts of a filesystem. Used in conjunction with FPART_PARTERRNO variable, hooks can
try to handle or work around the error. Requires live mode (option -L) and option -zz (- or
-zzz)

-d depth
After a certain depth, pack directories instead of files (directories themselves will be added to
partitions, instead of their content). You can force a specific file to be packed anyway by
listing it on the command line explicitly.

-D, --leaf-dirs
Implies -z. Pack leaf directories: if a directory contains files only, it will be packed as a
single entry. You can force a specific file to be packed anyway by listing it on the command
line explicitly.

-E, --dirs-only
Implies -D. Pack directories only (work on a per-directory basis): in that mode, no file will be
packed. Instead, each directory will be packed as a single entry with a size being the sum of
all top-level files' sizes. You can force a specific file to be packed anyway by listing it on
the command line explicitly.

LIVE MODE

-L, --live
Live mode (default: disabled). When using this mode, partitions will be generated while crawling
filesystem. This option saves time and memory but will never produce special partition 0 (see
options -s and -S ). As a consequence, it will generate partitions slightly larger than the size
specified with option -s. This option can be used in conjunction with options -f and -s, but not
with option -n.

-S Skip big files (default: disabled). In live mode, no special partition 0 can be produced and big
files are added to the current partition as they are found while crawling the filesystem. That
can lead to huge partitions. That option makes fpart skip files bigger than the specified
maximum partition size (option -s ) and print them to stdout (even when using option -o ) as
belonging to a pseudo-partition S (as in 'S'kipped). It allows a consumer to handle them
immediately through a separate process (no fpart hook will be executed for skipped files). That
option can only be used in Live mode (option -L ), when a maximum partition size has been given
(option -s ).

-w cmd, --pre-part-cmd cmd
When using live mode, execute cmd when starting a new partition (before having opened next output
file, if any). cmd is run in a specific environment that provides several variables describing
the state of the program: FPART_HOOKTYPE ("pre-part", "post-part" or "post-run"),
FPART_PARTFILENAME (current partition's output file name), FPART_PARTNUMBER (current partition
number), FPART_PARTSIZE (current partition's size), FPART_TOTALSIZE (total partitions' size),
FPART_PARTNUMFILES (number of files packed in current partition), FPART_TOTALNUMFILES (total
number of files packed so far), FPART_PARTERRNO (0 if every single partition's entry has been
read without error, else last erroneous entry's errno. For error detection to work properly, you
may need to rebuild fpart using embedded fts(3) library, depending on the version shipped with
your OS), FPART_PID (PID of fpart), FPART_TOTALNUMPARTS (total number of partitions generated so
far). Variables may or may not be defined, depending on requested options and current
partition's state when the hook is triggered. Hooks are executed in a synchronous way while
crawling filesystem, so 1) avoid executing commands that take a long time to return as it slows
down filesystem crawling and 2) do not presume cwd (PWD) is the one fpart has been started in, as
it is regularly changed to speed up crawling (i.e. use absolute paths within hooks). Size-
related variables are affected by preloading, overloading and rounding options.

-W cmd, --post-part-cmd cmd
Same as -w, but executes cmd when finishing a partition (after having closed last output file, if
any).

-R cmd, --post-run-cmd cmd
Same as -w and -W but executes cmd just once before exiting. Note that only FPART_TOTALSIZE and
FPART_TOTALNUMFILES environment variables are available in post-run hooks.

SIZE HANDLING

       -p num  Preload  each  partition with num bytes.  You can use a human-friendly unit suffix here (k, m, g,
               t, p).

       -q num  Overload each file size with num bytes.  You can use a human-friendly unit suffix here (k, m,  g,
               t, p).

       -r num  Round  each file size up to next num bytes multiple.  This option can be used in conjunction with
               overloading, which is done *before* rounding.  You can use a human-friendly unit suffix here  (k,
               m, g, t, p).

EXAMPLES

       Here are some examples:

       fpart -n 3 -o var-parts /var
               Produce  3  partitions,  with  (hopefully)  the same size and number of files.  Three files: var-
               parts.1, var-parts.2 and var-parts.3 are generated as output.

       fpart -s 4724464025 -o music-parts /path/to/music ./*.mp3
               Produce partitions of 4.4 GB, containing music files from /path/to/music as  well  as  MP3  files
               from  current  directory;  with such a partition size, each partition content will be ready to be
               burnt to a DVD.  Files music-parts.0 to music-parts.n, are generated as output.

       find /usr ! -type d | fpart -f 10000 -i - /home | grep '^1 '
               Produce partitions containing 10000 files each by examining /usr first and then /home and display
               only partition 1 on stdout.

       du * | fpart -n 2 -a
               Produce two partitions by using du(1) output.  Fpart will not examine the file system but instead
               use arbitrary values printed by du(1) and sort them.

AUTHOR, AVAILABILITY

       Fpart  has  been  written  by  Ganaël  LAPLANCHE  and   is   available   under   the   BSD   license   on
       http://contribs.martymac.org

BUGS

       No bug known (yet).

Debian                                          November 18, 2011                                       FPART(1)