27 Aug 2015
gfpcopy 1 Gfarm
gfpcopy copy Gfarm files in parallel
`gfpcopy` -nqvdpPU -X regexp -S source-domainname -h source-hostfile -D destination-domainname -H destination-hostfile -j num-of-processes -J num-of-processes -M limit-byte -z minimum-byte -Z maximum-byte -w way-of-scheduling -W kilobytes-for-threshold -s KB/s-to-simulate -F num-for-readahead -b bufsize -f -e -k source-path destination-path

## DESCRIPTION

`gfpcopy` copies files in parallel.

When the source-path is a directory, files under the directory will be copied recursively.

When the destination-path does not exist, the directory is created. When the destination-path exists, a directory of the same name as the source-path is created under the destination-path.

A set of source/destination hosts can be specified by a domain name and/or a hostlist file. When both a domain name and a hostlist file are specified, a set of hosts is determined by both conditions. When a set of source hosts is specified, only files stored on the source hosts are copied.

`gfpcopy` also retrieves the directory entries in parallel.

## SOURCE PATH

source-path must be one of the following formats. Files on HPSS cannot be copied.

path-name

  is a relative path or an absolute path of a local file system. When the path is a mount point on gfarm2fs, files are copied without passing through the gfarm2fs.

gfarm:...

  is a Gfarm URL.

file:...

  is an URL of a local file system.

## DESTINATION PATH

destination-path must be one of the following formats.

path-name

  is a relative path or an absolute path of a local file system. When the path is a mount point on gfarm2fs, files are copied without passing through the gfarm2fs.

gfarm:...

  is a Gfarm URL of a directory.

file:...

  is an URL of a directory on a local file system.

hpss:...

  is an URL of a directory on HPSS. If the same directory as the source-path exists under this directory, the source-path cannot be copied. The differential copy is not supported for HPSS. Relative path such as "hpss:". and "hpss:dir" can be specified.

## GFPCOPY OPTIONS

These are options only for `gfpcopy`.

`-b` bufsize

  Specifies the buffer size in bytes to copy.

  The default value is 64 KiB (64 * 1024).

`-f`

  With the -f option, existing files will be overwritten when the size is different or the modification time (mtime) is different from the source file.

  Without the -f option, existing files will be overwritten when they are older than the corresponding source files in the modification time.

`-e`

  Skips existing files in order to execute gfpcopy simultaneously.

`-k`

  Does not copy symbolic links.

## COMMON OPTIONS

There are several common options with `gfprep`. (see man `gfprep`)

## EXAMPLES

To copy files under the directory recursively.

    $ gfpcopy gfarm:///dir file:///tmp/dir

    $ gfpcopy file:///tmp/dir gfarm:///dir

    $ gfpcopy gfarm:///dir1 gfarm:///dir2

    $ gfpcopy gfarm:///dir hpss:///tmp/dir

    $ cd /mnt/gfarm2fs
    $ gfpcopy dir /tmp/dir

To copy a file.

    $ gfpcopy gfarm:///dir/file file:///dir

    $ cd /mnt/gfarm2fs
    $ gfpcopy file /tmp/dir

## NOTES

To retrieve the directory entries efficiently, it is better to execute `gfpcopy` command near the metadata server. When you need to execute `gfpcopy` command far from the metadata server, increase the parallelism by the -j and -J options.

## SEE ALSO

gfprep 1 , gfreg 1 , gfexport 1 , gfarm2.conf 5