Author: FCM team

# FCM: User Guide: FCM Make

## Introduction

The FCM make system provides a common environment for running the extract system, build system, and other utilities. Simply put, it controls a chain of steps. It is NOT to be confused with the standard Unix utility `make`.

See Annex: FCM Configuration File > FCM Make Configuration for a full list of declarations in an FCM Make configuration file.

## FCM Make Concept

### FCM Make Concept: Command Line Interface

The FCM make system can be invoked using the command line interface:

    fcm make [OPTIONS] [DECLARAION ...]

The FCM make system relies on a configuration file to tell it what to do. It looks for configurations with the following logic:

1. It reads each configuration file specified using the `--config-file=PATH` option in the order they are specified.
2. If the `--config-file=PATH` option is not specified, it will attempt to read `fcm-make.cfg` if it exists.
3. It looks at the current working directory, or the directory specified in the `--directory=PATH` option for configuration files specified as relative paths.
4. If one or more `--config-file-path=PATH` option is specified, it also searches for configuration files under the specified values, in the order the options are specified.
5. Finally, each KEY=VALUE command line argument is considered a configuration declaration line.

For details of the other command line options please see FCM Command Reference > fcm make.

### FCM Make Concept: Configuration File

The FCM make configuration file is expected to be an FCM configuration file. (See Annex: FCM Configuration File.) A typical FCM make configuration file may look like:

    steps = extract build                           # 1

    extract.ns = egg ham bacon                      # 2
    # ... more extract configuration

    build.target = egg.exe ham.exe bacon            # 3
    # ... more build configuration

At point 1, the `steps` declaration tells the system to invoke two steps. Their IDs are `extract` and `build`, both are IDs for built-in systems. (The other 2 being `mirror` and `preprocess`.) Each step has its own declarations, which are prefixed with the step ID and a full stop. The declarations at point 2 are used by the logic for running the `extract` step and the declarations at point 3 are used by the logic for running the `build` step.

There are times when you may need to invoke the same system (e.g. the build system) in slightly different configurations. To do this you need to use the `step.class` declaration to define custom step IDs for each build. E.g.:

    step.class[build-this build-that] = build       # 1
    steps = extract build-this build-that           # 2

    extract.ns = egg ham bacon                      # 3
    # ... more extract configuration

    build-this.prop{fc.defs} = DEFS TO BUILD THIS   # 4
    build-this.target = this.exe
    # ... more build configuration for "build-this"

    build-that.prop{fc.defs} = DEFS TO BUILD THAT   # 5
    build-that.target = that.exe
    # ... more build configuration for "build-that"

At point 1, we define the IDs `build-this` and `build-that` to be an instance of the `build` class. At point 2, we tell the system to run `extract`, then `build-this`, then `build-that`. At point 4 and 5, we define the configurations for `build-this` and `build-that` respectively.

### FCM Make Concept: Inheritance

The FCM Make system allows inheritance of configuration and files from succeeded makes, so the current make does not have to re-do everything that may already be available else where. To do this, specify the inheritance with a `use declaration`, e.g.:

    use = /path/to/succeeded/make/

For more information on how an inheritance behaves, see:

- Extract Inheritance
- Build Inheritance

### FCM Make Concept: Directory Structure

When you run `fcm make`, it will create a directory structure that may look like:

    .fcm-make/...
    extract/...
    build/...
    ...

Each normal sub-directory, such as `extract/` and `build/`, contains the result of each step. The hidden sub-directory `.fcm-make/` is used by `fcm make` as a working area. It may contain the following items:

`.fcm-make/cache/`
  An area for caching non-local items. E.g. the extract system exports source trees from the version control system into this cache. If `fcm make` is invoked with the `--archive` option, contents in this directory will be compressed into TAR-GZIP files, e.g. `.fcm-make/cache/extract/` will become `.fcm-make/cache/extract.tar.gz`.
`fcm-make-as-parsed.cfg -> .fcm-make/config-as-parsed.cfg`
  The configuration file as parsed by the latest `fcm make` command at this destination.
`fcm-make-on-success.cfg -> .fcm-make/config-on-success.cfg`
  The configuration file that can be used to repeat the latest successful `fcm make` command at this destination.
`.fcm-make/ctx.gz`
  A serialised data structure that represents the context of the latest `fcm make` command at this destination. It is a `gzip` file containing data in the Perl Storable format.
`fcm-make.log -> .fcm-make/log`
  A diagnostic log generated by the latest `fcm make` command at this destination. The content should be equivalent to the diagnostic output in STDOUT and STDERR at `-vv` verbosity.

### FCM Make Concept: Context Name

You may sometimes need to run `fcm make` in the same location of the file system. Perhaps you are on a network with a shared file system, but only certain tools are available on different hosts, e.g. you can only extract on one host and build on a different host, you should name the makes so that they don't end up overwriting each other's context, logs and other outputs. To do so, you can use the `--name=NAME` option on the command line, and/or the name declaration in the configuration file. Using `--name=2` as an example, you will get the following:

- The command will search for `fcm-make2.cfg` instead of `fcm-make.cfg`.
- The command will dump context, logs, etc with file names such as `.fcm-make2/*`, `fcm-make2-as-parsed.cfg`, `fcm-make2.log`, etc.
- The command will only allow inheritance from a location where a successful make with the same name exists.

## Extract

The extract system provides an interface between the version control system (currently Subversion) and the build system. Where appropriate, it extracts code and combines changes from the repositories and other user-defined locations to a directory tree suitable for feeding into the build system.

The extract system supports the `--jobs=N` option of the `fcm make` command. It uses N child processes to get source trees information and to export source trees from their repositories in parallel.

### Extract: Basic

The following is an example of how to extract the source trees from a file system path and the trunks of 2 known projects (version controlled in Subversion repositories with a standard FCM layout):

    steps = extract                    # line 1
    extract.ns = ops var um            # line 2
    extract.location[ops] = $HOME/ops  # line 3

Here is an explanation of what each line does:

- line 1: the `steps` declaration tells the make system to invoke the extract system.
- line 2: the `extract.ns` declaration is used to specify a space delimited list of names of the projects to extract.
- line 3: the `extract.location` declaration is used to specify the base source tree of a project (`ops` in this case) to extract. The value of the declaration can be a path in the file system or a location in a version control system. In this case, the declaration specifies the base source tree of the `ops` project to be a path in the file system at `$HOME/ops`. If the base source tree of a specified project name-space is not declared, the system will make an assumption. For a project hosted in a Subversion repository, the system will assume `trunk@HEAD` if the project URL is registered in the keyword configuration file or with the `extract.location{primary}` declaration (see below).

If you save this file as `fcm-make.cfg` and invoke the `fcm make` command you should end up with a source tree in the current working directory that looks like (hidden path not shown):

    extract/ops/...
    extract/um/...
    extract/var/...

The result of the extract can be found in the `extract/` sub-directory. Note: The generated source tree will not contain any symbolic links or hidden files (e.g. file names beginning with a `.`), because the extract system ignores them.

Note - project name-space and location

Whoever installed FCM at your site would have defined the name-spaces and the locations of the common projects at your site using the keyword configuration file at `$FCM/etc/fcm-keyword.cfg` (where `$FCM/bin` is the path at which FCM is installed). Alternatively, users can define their own set of project name-spaces and their locations using `$HOME/.metomi/fcm/keyword.cfg`. For information on keyword configuration files, please refer to System Administration > FCM keywords. If a project name-space is not defined in a keyword configuration, it can be defined in the FCM make configuration using the `extract.location{primary}` declaration. E.g. to define the location of the `foo` project, you can do: `extract.location{primary}[foo] = svn://server/repos/foo`.

Note - incremental mode

Suppose you have already invoked `fcm make` using the above configuration file. At a later time, you have made some changes to some of the files in the source trees. Re-running `fcm make` on the same configuration will trigger the incremental mode. In the incremental mode, the extract system will update only those files that are modified. If the last modified time (or last commit revision) of a source file in the current extract differs from that in the previous extract, the system will attempt a content comparison. The system updates the destination only if the content and/or file access permission of the source differs from that of the destination. To avoid the incremental mode and start afresh, invoke `fcm make` with the `--new` option.

### Extract: Location Types

The extract system currently supports the following location types:

fs
  A readable location in the local file system. E.g. `~/my-project`, `/home/lily/my-project`, etc.
ssh
  A readable location in the file system of a remote host accessible via `ssh` and `rsync`, specified in the form [USER@]HOST:PATH. The `ssh` access must be without a password, and you must be able to run the GNU coreutils version of the `find` and `stat` on the remote host. E.g. `mylinuxbox:my-project`, `holly@hpc:/data/holly/my-project`.
svn
  A Subversion location, which may be a working copy or a valid Subversion URL. Supported URL schemes are: `file`, `http`, `https`, `svn` and `svn+ssh`. E.g. `svn://mysvnhost/my-project/trunk@40778`, `~/my-project@32734`.

See also extract.location.

### Extract: Redefine the Root of the Source Tree of a Project

Consider a project called `foo` with a source tree that looks like:

    doc/...
    src/bar/...
    src/baz/...

Suppose you are only interested in the contents of the `src/` sub-tree. You can specify the root of the extract using the `extract.path-root` declaration. E.g.:

    steps = extract              # line 1
    extract.ns = foo             # line 2
    extract.path-root[foo] = src # line 3

Running `fcm make` with this configuration should give a source tree that looks like (hidden path not shown):

    extract/foo/bar/...
    extract/foo/baz/...

### Extract: Filter the Paths in the Source Tree of a Project

Going back to the above source tree in the `foo` project, imagine the `src/bar/` sub-directory contains:

    src/bar/wine/red.c
    src/bar/wine/rose.c
    src/bar/wine/white.c
    src/bar/...

If you do not want `src/bar/wine/rose.c` in the extract, you can ask for it to be excluded using the `extract.path-excl` declaration. E.g.:

    steps = extract                          # line 1
    extract.ns = foo
    extract.path-root[foo] = src             # line 3
    extract.path-excl[foo] = bar/wine/rose.c # line 4

Note: Because the root is redefined in line 3, the path in the `extract.path-excl` declaration in line 4 is declared from the new root level.

Running `fcm make` with this configuration should give a source tree that looks like (hidden path not shown):

    extract/foo/bar/wine/red.c
    extract/foo/bar/wine/white.c
    extract/foo/bar/...
    extract/foo/baz/...

On the other hand, if you only want `src/bar/wine/rose.c` in the extract, you can ask the system to exclude everything in `src/bar/wine/` but include `src/bar/wine/rose.c` using the `extract.path-incl` declaration. E.g.:

    steps = extract                          # line 1
    extract.ns = foo                         # line 2
    extract.path-root[foo] = src             # line 3
    extract.path-excl[foo] = bar/wine        # line 4
    extract.path-incl[foo] = bar/wine/rose.c # line 5

Running `fcm make` with this configuration should give a source tree that looks like (hidden path not shown):

    extract/foo/bar/wine/rose.c
    extract/foo/bar/...
    extract/foo/baz/...

### Extract: Combining Changes from Multiple Branches of a Project

We have so far only dealt with extracts from a single base source tree in each project. The extract system can also be used to combine changes from different source trees (against a base source tree) of a project.

E.g. consider a project called `food`. In the latest release (`trunk@3739`), the source tree looks like this:

    doc/...
    src/egg/boiled.y
    src/egg/microwave.x
    src/egg/omelette.c
    src/egg/poached.f90
    src/...

Jamie, Gordon and Rick are all developing changes against the latest release of the project.

Jamie has made the following changes (displayed using the notation of `svn status`) in his branch at `branches/dev/jamie/r3739_t381@3984`:

    D      src/egg/boiled.y
    A      src/egg/fried.pl
    M      src/egg/omelette.c

Gordon has made the following changes in his branch at `branches/dev/gordon/r3739_t376@3993`:

    M      src/egg/omelette.c
    M      src/egg/poached.f90

Rick has made the following changes in his working copy at `~rick/food`, but has yet to commit his changes back:

    M      src/egg/boiled.y
    A      src/egg/scrambled.bash

To combine their changes in an extract, the FCM make configuration file should look like:

    steps = extract
    extract.ns = food
    extract.location[food] = trunk@3739
    extract.location{diff}[food] = \
        branches/dev/jamie/r3739_t381@3984 \
        branches/dev/gordon/r3739_t376@3993 \
        ~rick/food

Here we have an `extract.location` declaration like before. This time it is pointing to the latest release of the `food` project. The changes against the base source tree are declared using the `extract.location{diff}` declaration.

Invoking `fcm make` with this configuration file will result in a source tree that looks like (hidden file not shown):

    extract/doc/...
    extract/src/egg/fried.pl
    extract/src/egg/microwave.x
    extract/src/egg/omelette.c
    extract/src/egg/poached.f90
    extract/src/egg/scrambled.bash
    extract/src/...

Note:

- There is no `extract/src/egg/boiled.y` file in the extract tree because it is deleted by Jamie's branch. Even though this file is modified in Rick's working copy, the extract will ignore this and assume that the deletion takes precedence.
- The `extract/src/egg/fried.pl` file comes from Jamie's branch.
- The `extract/src/egg/microwave.x` file comes from the latest release (base).
- The `extract/src/egg/omelette.c` file is the result of a merge of the changes in Jamie's and Gordon's branches. This example assumes that there is no conflict in the merge. If the merge results in a conflict, the extract will fail.
- The `extract/src/egg/poached.f90` file comes from Gordon's branch.
- The `extract/src/egg/scrambled.bash` file comes from Rick's working copy.

### Extract Inheritance

If a previous extract with a similar configuration exists in another location, it can be more efficient to inherit from this previous extract in your current extract. This works like a normal incremental extract, except that your extract will only contain the changes you have specified (compared with the inherited extract) instead of the full directory tree in the destination. This type of incremental extract is useful in several ways. For instance:

- It is fast, because you only have to extract files that you have changed.
- The subsequent build will also be fast, since it will act like an incremental build.
- You do not need write access to the original extract. A system administrator can set up a stable version in a central account, which developers can then inherit from.
- You want an incremental extract, but you need to leave the original extract unmodified.

Consider the previous example. Imagine an extract already exists for the latest release for the `food` project at `/home/food/latest/`, and now you want to test the changes introduced by Jamie, Gordon and Rick. You can `use` the original extract with the changes:

    use = /home/food/latest
    extract.location{diff}[food] = \
        branches/dev/jamie/r3739_t381@3984 \
        branches/dev/gordon/r3739_t376@3993 \
        ~rick/food

Invoking `fcm make` with this configuration file will result in a source tree that looks like (hidden file not shown):

    extract/src/egg/fried.pl
    extract/src/egg/omelette.c
    extract/src/egg/poached.f90
    extract/src/egg/scrambled.bash

The extract will work in an incremental-like mode. The only difference is that the original extract at `/home/food/latest/` will be left untouched, and the new extract will contain only the changes introduced by the diff locations. Note that, although `extract/src/egg/boiled.y` remains in the original extract, it will not be used in any subsequent build step.

Extract inheritance limitation

  Extract inheritance allows you to add more diff locations to a project, but you should not include any other declarations relating to the extract. Doing so is not safe and should trigger an exception.

  In some situations this implies that it will not be possible to use inherited extracts. You should use a new extract if, for instance, a new diff location contains a change which requires the use of source files in a previously excluded name-space.

### Extract Diagnostic

The amount of diagnostic messages generated by the extract system is dependent on the diagnostic verbosity level that can be modified by the `-v` and `-q` options to the `fcm make` command.

The following is a list of diagnostic output at each verbosity level:

-q
  - Exceptions.
default
  - Everything at the -q level.
  - Start time of the extract.
  - The number and location of each source tree in each project. The base source tree is number 0. E.g.:

        [info] location um: 0: svn://fcm2/UM_svn/UM/trunk@11732
        [info] location um: 1: svn://fcm2/UM_svn/UM/branches/dev/Share/VN7.3_hg3_dust_443@11858
        [info] location um: 2: svn://fcm2/UM_svn/UM/branches/dev/Share/VN7.3_hg3_ccw_precip@11857
        [info] location um: 3: svn://fcm2/UM_svn/UM/branches/dev/hadco/VN7.3_HG3_porting_lsp_fixes@12029
        ...

  - The number of targets by their destination status and their source status. E.g.:

        [info]   dest:    3 [A added]
        [info]   dest:  134 [a added, overriding inherited]
        [info]   dest:    6 [d deleted, overriding inherited]
        [info]   dest: 2818 [U unchanged]
        [info] source:    3 [A added by a diff source tree]
        [info] source:    6 [D deleted by a diff source tree]
        [info] source:   16 [G merged from 2+ diff source trees]
        [info] source:  118 [M modified by a diff source tree]
        [info] source: 2818 [U from base]

  - Total time.
-v
  - Everything at the default level.
  - The destination and source status for each modified target. E.g.:

        ...
        [info] aM um:0,13    atmosphere/short_wave_radiation/r2_lwrad3c.F90
        [info] aG um:0,6,13  control/top_level/scm_main.F90
        [info] AA um:-,8     include/constant/cnv_parc_lim.h
        ...

    The 2 letters following `[info]` is the destination status and the source status of the target. This is followed by the name-space of the project, a colon and the source tree number(s) providing the source for the target (in a comma separated list). This is then followed by the name-space of the target path. The source tree number `0` denotes the base source tree, a dash `-` in place of a `0` means that the source only exists in a diff source tree.

-vv
  - Everything at the -v level.
  - Each shell command invoked with elapsed time and return code.

Here is an explanation of each target destination status:

`[A added]`
  Target newly added to the destination.
`[a added, overriding inherited]`
  Target added to the current extract destination, i.e. modified compared with the target with the same name-space in an inherited extract destination.
`[D deleted]`
  Target deleted from the extract destination during an incremental extract.
`[d deleted, overriding inherited]`
  Target deleted from the current extract destination, i.e. a target with the same name-space exists in an inherited extract destination.
`[M modified]`
  Target modified in the extract destination during an incremental extract.
`[U unchanged]`
  Target unchanged compared with the previous (or any inherited) extract.
`[? unknown]`
  Target does not have a destination. This destination status is normally associated with the `[D deleted]` source status.

Here is an explanation of each target source status:

`[A added by a diff source tree]`
  The source of the target comes from a diff source tree, and the base source tree does not have a source in the same name-space.
`[D deleted by a diff source tree]`
  Target is deleted by a diff source tree, (i.e. exists in base source tree, but missing from a diff source tree).
`[G merged from 2+ diff source trees]`
  The source of the target comes from 2 or more diff source trees, (i.e. the actual source is the result of a merge between all the changes).
`[M modified by a diff source tree]`
  The source of the target comes from a diff source tree.
`[U from base]`
  The source of the target comes from the base source tree, (i.e. the source is unchanged by any diff source tree).
`[? unknown]`
  The target has no source. This source status is normally displayed in an incremental extract, where a target in a previous extract is not a target in the current extract, and is normally associated with the `[D deleted]` destination status.

## Mirror

The mirror system provides a way to mirror the results of the make steps to another location, where the FCM make may need to continue. It is typically used after an extract to set up the build on an alternate platform.

### Mirror: Basic

Consider the following example:

    steps = extract mirror                            # 1
    # ... some extract declarations
    mirror.target = user@somewhere:/path/in/somewhere # 2
    mirror.prop{config-file.name} = -at-somewhere     # 3
    mirror.prop{config-file.steps} = preprocess build # 4
    # ... some preprocess declarations
    # ... some build declarations

When the system runs with this configuration, the system will mirror the result of the extract to the mirror.target, i.e. `somewhere:/path/in/somewhere` using `ssh` and `rsync`.

With the config-file.name property set at point 3 and the config-file.steps property set at point 4, it will write a configuration file called `fcm-make-at-somewhere.cfg` with the make name set as `-at-somewhere` in the mirror target so that the `preprocess` and `build` steps can continue there.

### Mirror Diagnostic

The amount of diagnostic messages generated by the mirror system is dependent on the diagnostic verbosity level that can be modified by the `-v` and `-q` options to the `fcm make` command.

The following is a list of diagnostic output at each verbosity level:

-q
  - Exceptions.
default
  - Everything at the -q level.
  - Start time of the mirror.
  - The updated destination and its source.
  - Total time.
-v
  - Everything at the default level.
-vv
  - Everything at the -v level.
  - Each shell command invoked with elapsed time and return code.

## Build

The build system performs actions on a set of source files using its predefined logic and the properties specified in the configuration. For example, it will attempt to create a binary executable for a source file containing a Fortran program.

The build system supports the `--jobs=N` option of the `fcm make` command. It uses N child processes to analyse the source files and to update the targets in parallel.

### Build: Basic

Consider a source tree at `$HOME/my-source-tree/` containing some Fortran source files including at least one with a main program (and maybe other supported types of source files), you can set up an FCM make configuration file to build an executable. E.g.:

    steps = build
    build.target{task} = link
    build.source = $HOME/my-source-tree

In this simple 3 line configuration, the `steps` declaration tells the make system to invoke the build system, the `build.target` declaration tells the system to build any targets which require linking (i.e. any main programs), and the `build.source` declaration specifies the location of the source tree.

If you save this file as `fcm-make.cfg` and invoke the `fcm make` command it should attempt to build the source tree in the current working directory, using the default properties. If the default Fortran compiler `gfortran` is installed, and nothing goes wrong, you will end up with a directory tree that looks like (hidden path not shown):

    build/bin/...
    build/include/...
    build/o/...

The result of the build can be found in the sub-directories of the `build/` sub-directory. Each `build/*/` sub-directory contains a category of targets:

`bin`
  e.g. executable binary and script.
`etc`
  e.g. data files.
`include`
  e.g. include files and Fortran module definition files.
`lib`
  e.g. object archives.
`o`
  e.g. object files

Sub-directories are only created as necessary, so you may not find all of the above in your destination tree.

If `fcm make` is invoked with the `--archive` option, sub-directories in categories containing intermediate build files (or any category specified in the archive-ok-target-category property) will be put into TAR-GZIP files. E.g. with the default setting, `include/` and `o/` will become `include.tar.gz` and `o.tar.gz` respectively.

To use a different compiler and/or compiler options for Fortran/C/C++, you use the build.prop declaration to redefine the build properties. E.g.:

    steps = build
    build.target{task} = link
    build.source = $HOME/my-source-tree

    # Set Fortran compiler/linker
    build.prop{fc} = ifort
    # Set Fortran compiler options
    build.prop{fc.flags} = -i8 -r8 -O3
    # Add include paths to Fortran compiler
    build.prop{fc.include-paths} = /a/path/to/include /more/path/to/include
    # Set link libraries for Fortran executables
    build.prop{fc.lib-paths} = /path/to/my-lib
    build.prop{fc.libs} = mine
    # Set C compiler/linker
    build.prop{cc} = icc
    # Set C compiler options
    build.prop{cc.flags} = -O3
    # Set C++ compiler options
    build.prop{cxx.flags} = -O2
    # Set link libraries for C executables
    build.prop{cc.lib-paths} = /path/to/my-lib /path/to/your-lib
    build.prop{cc.libs} = mine yours
    # Set linker, if compiler cannot be used as linker
    #build.prop{ld} = ld

### Build Source Locations

The build system locates its source files from various places, including:

- Inherited locations. (See Build Inheritance.)
- Usable target locations of previous steps in the make. E.g. targets of an extract step, and `src` category targets of a preprocess step can both be source files of the build system. The `build.prop{no-step-source}` declaration can be used to switch off this behaviour.
- Locations specified by the `build.source` declaration. Note: If you assign a relative path to the `build.source` declaration, the system will assume the path to be relative to the make destination (not the current working directory, unless they happen to be the same).

There are situations when it may not be desirable to consider every source file in a build. You can apply filters by source file name-spaces using the `build.ns-excl` and `build.ns-incl` declarations. E.g.:

    # To include items in "foo" and "bar/baz" only
    build.ns-excl = /              # exclude everything ...
    build.ns-incl = foo bar/baz    # but include items from these name-spaces

### Build Source Name-spaces and Properties

Each source file is assigned a name-space (which is used to fine tune the build properties, such as the compiler flags). If the source file is a target of an extract, the name-space will be the same as the extract target (the relative path of the `extract/` sub-directory in the make destination). If the source file is a file in a source tree specified by `build.source`, the name-space is the relative path to the specified value. The `build.source` declaration also accepts an optional name-space, in which case the name-space of each source file in the tree will be prefixed with the specified name-space. Suppose you have a source tree in `$HOME/food`:

    $HOME/food/egg.c
    $HOME/food/ham.f90

If you specify the source tree with `build.source = $HOME/food`, then `$HOME/food/egg.c` will be given the name-space `egg.c` and `$HOME/food/ham.f90` will be given the name-space `ham.f90`.

On the other hand, if you specify the source tree with `build.source[food] = $HOME`, then `$HOME/food/egg.c` will be given the name-space `food/egg.c` and `$HOME/food/ham.f90` will be given the name-space `food/ham.f90`.

The name-space is organised in a simple hierarchy. For instance, the `foo/bar/egg` name-space belongs to `foo/bar`, which belongs to `foo`, which belongs to the root name-space. (The root name-space is either an empty string or a `/`.)

For instance, you can set the flags of the Fortran compiler using the `build.prop{fc.flags}` declaration at different name-space levels:

    # The global Fortran compiler flags
    build.prop{fc.flags} = -O3

    # The Fortran compiler flags for the "food" name-space
    build.prop{fc.flags}[food] = -O2 -i8 -r8

    # The Fortran compiler flags for a source file
    build.prop{fc.flags}[food/bacon.f90] = -O0 -g -C

### Build Source Types

Before the build system can do anything with its source files, it needs to know what they are. It determines the type of each source file by looking at its file name extension, then the file name itself, and then the `#!` line for a text file. Source files without a type are treated as data files.

The following types are associated with file extensions:

c (C source file)
  .c .i .m .mi
cxx (C++ source file)
  .cc .cp .cxx .cpp .CPP .c++ .C .mm .M .mii
fortran (Fortran source file)
  .F .FOR .FTN .F90 .F95 .f .for .ftn .f90 .f95 .inc
h (Preprocessor header file)
  .h
script (script in various languages)
  (empty)

The `prop{file-ext.type} = extensions` declaration can be used to modify the extensions associated with a type. E.g. if you need to add `.fort` as a file extension for a Fortran source file, you can do:

    build.prop{file-ext.fortran} = .F .FOR .FORT .FTN .F90 .F95 \
                                   .f .for .fort .ftn .f90 .f95 .inc

You can associate file names to some file types using a `prop{file-pat.type} = regular-expression` declaration. E.g. if you have executable scripts in the source tree with no `#!` lines but are recognised by a `*Scr_*` pattern of their file names, you can specify a regular expression to match their file names using the file-pat.script property:

    build.prop{file-pat.script} = (?msx-i:\w+Scr_\w+)

All other text files with a `#!` line are recognised as scripts by the build system.

### Build Source Analysis

Each source file with a known type (that is not ignored) is analysed by the build system for dependencies and other information. Here is a list of what the system looks for in each type of file:

c and cxx

  main program: e.g. `int main()`.

  dependency on include: e.g. `#include "name.h"`.

  dependency on object: e.g. `/* depends on: name.o */` (for legacy support).

fortran

  main program: e.g. `program name`.

  list of symbols: i.e. names of top level program units including blockdata, function, module, program and subroutine.

  dependency on include: e.g. `#include "name.h"` and `include 'name.f90'`.

  dependency on module: e.g. `use name`.

  dependency on object: e.g. `! depends on: name.o` (for legacy support).

h

  dependency on include: e.g. `#include "file-name"` (and `include 'file-name'` for legacy support).

script

  dependency on executable: e.g. `# calls: name` (for legacy support).

Note: The following features are for legacy support.

`DEPENDS ON: x` directives in C/Fortran source files
  The `DEPENDS ON: x` directive can be used to identify dependencies on other compiled objects. However, it is much better to specify this kind of dependency information in the configuration for the build where necessary. In any case, in modern Fortran code almost all dependencies should be identified automatically via the use of modules and/or interface files.
`calls: x` directives in scripts
  The `calls: x` directive can be used to identify a dependency on another executable. However, it is much better to specify this kind of dependency information in the configuration for the build, and leave the source code to concentrate on the run time logic.
`*.h` files as Fortran include files
  `*.h` files are normally identified as C header files. However, they are also being used by some old Fortran programs as include files. Therefore, when the system analyses a `*.h` file, it has to detect the Fortran include syntax, i.e. `include 'file-name'` as well as the regular C preprocessor include syntax.

Note: Dependency Analysis and Fortran OpenMP Sentinels.

The build system recognises statements with Fortran OpenMP sentinels that affect build dependencies. E.g.:

    !$ USE my_omp_mod, ONLY: my_omp_sub
    ! ...
    !$ INCLUDE 'my_omp_logic'

These dependencies are normally ignored. However, if a relevant `build.prop{fc.flag-omp}` property is specified, the build system will treat these statements as normal dependency statements.

There are some situations when it is not possible for the system to identify a dependency. E.g. a Fortran source file may depend on external objects that are not detected by the automatic analysis. Therefore, the system allows you to specify manual dependencies in the configuration file using the `build.prop{dep.type}` and `build.prop{ns-dep.type}` declarations. E.g.:

    # Tell the system that (the object of) food/egg.c depends on chicken.o
    build.prop{dep.o}[food/egg.c] = chicken.o

    # Tell the system that (the object of) meal/big.c depends on all objects in the
    # "food" and "drink" name-spaces
    build.prop{ns-dep.o}[meal/big.c] = food drink

Like all declarations that accept name-spaces, if you specify a name-space in this declaration, the property will apply to all source files in the name-space. If you do not specify a name-space, it applies to the root name-space (i.e. globally to all relevant source files).

The following manual dependency declarations are recognised:

build.prop{dep.bin}
  Specifies a list of dependencies on a script or a binary executable.
build.prop{dep.f.module}
  Specifies a list of Fortran module import dependencies. Note: a dependency on a Fortran module called `module_1` becomes an include dependency on `module_1.mod` when the system turns the source file into its targets.
build.prop{dep.include}
  Specifies a list of include file dependencies.
build.prop{dep.o}
  Specifies a list of link-time object dependencies.
build.prop{dep.o.special}
  Specifies a list of special type of link-time object dependencies. Normally, an object file can be put in an object archive before being linked with the main object. There are special cases when an object file must be specified on the command line of the linker. (E.g. an object file containing a Fortran blockdata program unit.) This special behaviour must be declared using this declaration.
build.prop{ns-dep.o}
  Specifies a list of link-time object dependencies on all objects in the specified name-space.

There are times when you know that your source tree does not contain a particular type of dependency, in which case you can switch off the automatic analysis by using the `build.prop{no-dep.type}` declaration. E.g. if you know that all include files in the `food` name-space are provided outside of the source tree, you can do:

    # Do not check for "include" dependencies
    build.prop{no-dep.include}[food] = *

All the types supported by the `build.prop{dep.type}` declarations are supported by the `build.prop{no-dep.type}`, except that there is no `build.prop{no-dep.o.special}` (because this type of dependency is never automatic).

### Build Targets from Source Files

The system derives the build targets from the source files. E.g. a C source file `egg.c` is turned into a compile target to generate `egg.o`.

The following is a list of what targets are available for each type of file. The title of each item in the list is in the format source type -> target key. The description of each target describes what the target is, and where appropriate, explains how the target keys are named. The task is the action the target needs to perform to get up to date. The category and destination is the sub-directory and destination of the target. The properties are the list of properties that may be used by the task to update the target. The dependencies list the types of dependencies the target may have. The update if is the condition when the target is considered out of date. The pass on information is a list of dependeny types which a target can pass on the status, (see Build Targets Update in Incremental Mode for an explanation of what this means.)

c/cxx -> name

  description: source file as an include file.

  task: install.

  category and destination: include and include/name

  dependencies: include and o (object).

  update if: source file is modified.

  pass on: include, o and o.special.

c/cxx -> name.o

  description: object file. The file is named by mapping the base name of the original source file in lower case characters, with the file extension replaced by the first value of the `file-ext.o` property.

  task: compile (cc).

  category and destination: o and o/name.o

  properties: cc, cc.flags, cc.defs, cc.flag-compile, cc.flag-define, cc.flag-include, cc.include-paths, cc.flag-omp, cc.flag-output

  dependencies: include and o (object).

  update if: source file or any of the required properties are modified, or if any include dependencies are updated.

  pass on: o and o.special.

c/cxx (with main function) -> name.exe

  description: binary executable. The file is named after the base name of the original source file, with the file extension replaced by the first value of the `file-ext.bin` property.

  task: link (cc).

  category and destination: bin and bin/name.exe

  properties: ar, ar.flags, file-ext.a, cc, cc.flags-ld, cc.flag-lib, cc.flag-lib-path, cc.libs, cc.lib-paths, cc.flag-omp, cc.flag-output

  dependencies: name.o and other objects (o and o.special).

  update if: source file or any of the required properties are modified, or if any dependencies are updated.

fortran -> name

  description: source file as an include file.

  task: install.

  category and destination: include and include/name

  dependencies: include and o (object). A source's f.module dependency on a module called `xyz` is turned into an include dependency on the `xyz.mod`.

  update if: source file is modified.

  pass on: include, o and o.special.

fortran (with a valid Fortran program unit) -> unit.o

  description: object file. The file is named by concatenating the lower case characters of the name of the first program unit in the source file and the first value of the `file-ext.o` property.

  task: compile (fc).

  category and destination: o and o/unit.o

  properties: fc, fc.flags, fc.defs, fc.flag-compile, fc.flag-define, fc.flag-include, fc.include-paths, fc.flag-module, fc.flag-omp, fc.flag-output

  dependencies: include and o (object). A source's f.module dependency on a module called `xyz` is turned into an include dependency on the `xyz.mod`.

  update if: source file or any of the required properties are modified, or if any include dependencies are updated.

  pass on: o and o.special.

  remark: trigger `unit.mod` targets if source file contains a Fortran module.

fortran (with function or subroutine) -> name.interface

  description: Fortran interface file. The file is named by concatenating the base name of the source file with the file extension replaced by the `file-ext.f90-interface` property.

  task: ext-iface

  category and destination: include and include/name.interface

  dependencies: unit.o

  update if: source file or unit.o is modified.

  pass on: include, o and o.special.

fortran (each module in source) -> unit.mod

  description: Fortran module definition file. The file is named by concatenating the lower case characters of the name of the module and the first value of the `file-ext.f90-mod` property.

  task: compile+.

  category and destination: include and include/unit.mod

  dependencies: unit.o

  update if: source file or unit.o is modified.

  pass on: o.

fortran (with program) -> name.exe

  description: binary executable. The object file is named after the base name of the original source file, with the file extension replaced by the first value of the `file-ext.bin` property.

  task: link (fc).

  category and destination: bin and bin/name.exe

  properties: ar, ar.flags, fc, fc.flags-ld, fc.flag-lib, fc.flag-lib-path, fc.libs, fc.lib-paths, fc.flag-omp, fc.flag-output

  dependencies: unit.o and other objects (o and o.special).

  update if: source file or any of the required properties are modified, or if any dependencies are updated.

h -> name

  description: a header (include) file.

  task: install.

  category and destination: include and include/name

  dependencies: include and o (object).

  update if: source file is modified.

  pass on: include, o and o.special.

script -> name

  description: an executable script.

  task: install.

  category and destination: bin and bin/name

  dependencies: bin (executable).

  update if: source file is modified.

  pass on: bin.

data -> name-space

  description: a data file.

  task: install.

  category and destination: etc and etc/name-space

  update if: source file is modified.

Here is an explanation of what each build system task does:

archive
  Creates an object archive by invoking an archiver command. (See Build Targets from Name-space.)
compile
  Creates an object file by invoking the C/C++/Fortran compiler on the source file.
compile+
  Copies the Fortran module definition file created by a compile task to the include sub-directory.
ext-iface
  Extracts the calling interfaces of all functions and subroutines in a Fortran source file (free format only) and writes the results in an interface block that can be included by other Fortran source files with an `INCLUDE 'name.interface'` statement. In an incremental build, if you have modified a Fortran source file, its interface file will only be re-generated if the content of the interface has changed. This can make incremental build very efficient, as non-interface changes in a function or subroutine will only trigger a re-link of the executable.
install
  Copies the source file to the destination.
link
  Creates an executable by invoking the archiver to load all required objects into an archive, and then the C/C++/Fortran compiler on the object file previously compiled using a source file containing a main program, with the temporary archive.

### Build Targets and Properties

If you need to specify a property for a specific target, you can either use their source file namespace or the target key. E.g. If the `sausage.o` target is generated from the source file in the `src/food/sausage.f90` namespace, you can specify its Fortran compiler flags `build.prop{fc.flags}` by doing either:

    build.prop{fc.flags}[sausage.o] = -O4
    # would be the same as:
    build.prop{fc.flags}[src/food/sausage.f90] = -O4

This works with most property modifiers, even for dependency related modifiers such as `no-dep.o`. E.g.:

    build.prop{no-dep.include}[sausage.o] = enum.f90
    build.prop{include-paths}[sausage.o] = /path/to/additives

However, the following will not work:

    build.prop{no-dep.f.module}[sausage.o] = pork

This is because an object file target is never dependent on a Fortran module by its name. The following will work, however:

    build.prop{no-dep.include}[sausage.o] = pork.mod
    build.prop{no-dep.o}[sausage.mod] = pork.o
    # would be the same as:
    build.prop{no-dep.f.module}[src/food/sausage.f90] = pork

### Build Targets from Source Files: Fortran Specifics

To ensure that a Fortran application is built automatically, its source code should be designed with the following considerations:

The name of each compilable program unit should be unique in the source tree, bearing in mind that Fortran is NOT case sensitive.

Always supply an interface for functions and subroutines, i.e.:

- Place functions and subroutines in a module, and give them the `PUBLIC` attribute. Import them with the `USE <module>` statement. We recommend adding the `ONLY` clause in a `USE <module>` when importing symbols from a module. This makes it easier to locate the source of each symbol, and avoids unintentional access to other `PUBLIC` symbols within the `MODULE`. If you are importing from an intrinsic module, you should add the `INTRINSIC` clause to the `USE <module>` statement to tell the build system not to look for the module from your source tree.
- Place functions and subroutines in the `CONTAINS` section of a standalone program unit. There are two advantages for this approach. Firstly, the sub-programs will get an automatic interface when the container program unit is compiled. Secondly, it should be easier for the compiler to provide optimisation when the sub-programs are internal to the caller. The disadvantage of this approach is that the sub-programs are local to the caller, and so they cannot be called by other program units. Therefore, this approach is only suitable for small sub-programs local to a particular program unit.
- Use the build system's automatic interface file feature. See below.

For each free format Fortran source file, e.g. `name.f90`, with 1 or more top level function and/or subroutine, the system creates a target with the ext-iface task in the include category, e.g. `name.interface`, to extract the calling interfaces of all functions and subroutines into an interface block. Another Fortran source file, e.g. `caller.f90` that relies on the functions and/or subroutines in `name.f90` can have an `INCLUDE 'name.interface'` statement in its specification section, which serves 2 purposes:

- It allows `caller.f90` to call the functions and/or subroutines in `name.f90` with explicit interfaces.
- It introduces an include dependency for `caller.o` on `name.interface`.

In an incremental build, if you modify `name.f90`, the system will only regenerate `name.interface` if only the calling interfaces of the functions and/or subroutines in `name.f90` have changed. Consequently, non-interface changes in `name.f90` will not trigger the re-compile of `caller.o`, but will only trigger a re-link of the executable.

### Build Targets Selection and Rename

You need to tell the build system what targets to build or it will do nothing. The `build.target` declaration allows you to select targets according to their categories, source name-spaces, tasks and keys. The logic is demonstrated by the following example:

    # Select targets matching these keys
    build.target = egg.bin ham.o bacon.sh

    # Select all targets doing tasks "install" or "link"
    build.target{task} = install link

    # Select targets in name-space "foo" or "bar" doing tasks "link"
    build.target{task}[foo bar] = link

    # Select all targets in the "bin" category
    build.target{category} = bin

    # Select targets in name-space "foo" in the "etc" category
    build.target{category}[foo] = etc

There are times when an automatic target name is not what you want. In which case, you can rename a target using the `build.target-rename` declaration to specify an alternate name. E.g. if the target `bacon.sh` should be called `streaky`, you can do:

    build.target-rename = bacon.sh:streaky

In order for a target to build, all its dependencies must be satisfied. If a target has a dependency that is not available in the list of targets, the build will fail. Normally, you can avoid this by using one of the `build.prop{no-dep.*}` declarations to switch off a non-existent dependency, as described in the Build Source Analysis section. However, there may be times when this is inefficient or insufficient, in which case you can use the property `ignore-missing-dep-ns` to specify a list of source name-spaces, in which targets can ignore missing dependencies. E.g.:

    # Allows targets in the "foo" and "bar/baz"
    # name-spaces to ignore missing dependencies.
    build.prop{ignore-missing-dep-ns} = foo bar/baz

### Build Targets File Extensions

You can rename the file name extension of the targets using `build.prop{file-ext.type}` declaration (provided that the file name extension is supported by your compiler, etc). E.g. if you want your binary executables to have `.bin` extension rather than the default `.exe`, you can do:

    build.prop{file-ext.bin} = .bin

The following file extensions are currently used by the system:

a (object archive)
  .a
bin (binary executable)
  .exe
f90-interface (Fortran free format interface file)
  .interface
f90-mod (Fortran compiler module definition file)
  .mod
o (object file)
  .o

### Build Targets from Name-space

Apart from source file targets, the build system also generates targets for each (directory-level) name-space. One target is for creating an object archive to contain all object files in the name-space. The other target is a convenient shorthand to allow all data files in the name-space to be installed. The following is the full description:

name-space > name-space/libo.a

  description: object archive.

  task: archive.

  category and destination: lib and lib/name-space/libo.a

  dependencies: all o (object) targets in the name-space.

  properties: ar, ar.flags

  update if: any dependencies or properties are modified.

name-space > name-space/.etc

  description: dummy file.

  task: install.

  category and destination: etc and etc/name-space/.etc

  dependencies: all data files in the name-space.

  update if: any dependencies are modified.

### Build Targets Update in Incremental Mode

In incremental mode, a target is only updated if it is marked out of date. A target is considered out of date if:

- the source file's checksum is changed.
- a required property is modified.
- a dependency is marked as modified, and the dependency is a type that the target cannot pass on. E.g. If `object_1.o` depends on `object_2.o`, and `object_2.o` is marked as modified, the system does not need to re-compile `object_1.o` (as long as its source file and properties remain unchanged). However, `object_1.o` will have to pass the information up the dependency tree, so that a target with a `link` task to build an executable (or an `archive` task to build a library) will know that it needs to be updated.
- a dependency is passing on a modified status for a dependency type, which cannot be passed on by the target.
- the target does not exist or its checksum is changed.

If, after an update, the target's checksum is the same as before, the target will be considered unchanged and up to date. In an incremental build, the use of checksum ensures that any targets manually modified by the user after the previous build is rebuilt accordingly. It also prevents unnecessary updates of targets in incremental and inherited builds.

E.g. Consider an incremental build where the only change is the content of a Fortran module `my_mod.f90`. The content change should trigger an update of the `my_mod.o` and `my_mod.mod` targets, and everything depending on them. However, if the source content is modified in such a way that it does not affect the module's public interface, most compilers will generate an identical `my_mod.mod`. The system can detect this by comparing the checksums. If `my_mod.mod` is unchanged, the build system will not need to trigger the re-compile of all targets depending on `my_mod.mod`, and it will only need to re-link the executable. This allows incremental builds to be more efficient.

Note - checksum algorithm

By default, the MD5 algorithm is used to calculate the checksum. This is normally good enough to detect whether a file is modified or not. If this is insufficient for whatever reasons, you can tell the build system to use one of the SHA algorithms supported by the Perl module Digest::SHA, by setting the value of build.prop{checksum-method}.

### Build Inheritance

If a previous build with a similar configuration exists in another location, it can be more efficient to inherit from this previous build in your current build. This works like a normal incremental build, except that your build will only contain the changes you have specified (compared with the inherited build) instead of the full set of targets.

The current build inherits all properties and target settings, as well as sources and targets from the inherited build. While properties and target settings can be overridden with a corresponding declaration, source inheritance can only be prevented by using a `build.prop{no-inherit-source}` declaration. E.g.:

    # Prevents inheritance from some name-spaces:
    build.prop{no-inherit-source} = food/mint drink/soft/cola.c

For multiple inheritance, the last one takes precedence, and any search for source files or targets are recursive and depth first. For instance, if we have the following declarations in the current FCM make configuration:

    use = /path/to/a /path/to/b /path/to/c

and the following in the FCM make configuration of `/path/to/b`:

    use /path/to/d

The relationship looks like:

    /path/to/current
        /path/to/c
        /path/to/b
            /path/to/d
        /path/to/a

Therefore, we would expect the search path to follow the order:

    /path/to/current
    /path/to/c
    /path/to/b
    /path/to/d
    /path/to/a

In its normal setting, the system does not inherit targets in the `bin`, `etc` and `lib` categories. A target in one of these categories is rebuilt in the current destination, whether the inherited target is up to date or not. This allows someone to use the executables of the build by setting the PATH environment variable to point only to `$DEST/build/bin/` (where $DEST is the destination of the current make). If this behaviour is undesirable for whatever reason, it can be altered using the `build.prop{no-inherit-target-category}` declaration.

Build inheritance limitation: handling of include files

  The build system uses the compiler's `-I` option to specify the search path for include files. E.g. it uses this option to specify the `inc/` sub-directories of the current build and its inherited build.

  However, some compilers (e.g. `cpp`) search for include files from the container directory of the source file before searching for the paths specified by the `-I` options. This behaviour may cause the build to behave incorrectly.

  Consider a source file `egg/hen.c` that includes `fried.h`. If the directory structure looks like:

      # Sources in inherited build:
      egg/hen.c
      egg/fried.h

      # Sources in current build:
      egg/fried.h

  The system will correctly identify that `fried.h` is out of date, and trigger a re-compilation of `egg/hen.c`. However, if the compiler searches for the include files from the container directory of the source file first, it will wrongly use the include file in the inherited build instead of the current one.

  If your directory structure does not have any include files in the same directory as the source files that include them then you do not need to worry. If it does then you need to check whether you are affected by this problem before using an inherited build. The situation will vary according to whether the affected code uses Fortran or preprocessor include statements and also whether you are using the preprocess system. Some compilers (e.g. `gfortran`) work fine for Fortran includes but not preprocessor includes. Others (e.g. `ifort`) have options which can be used (e.g. `-assume nosource_include`) to get the desired behaviour. The FCM distribution includes some simple test code to help you test how your chosen compilers behave. If you cannot ensure the correct behaviour then it is safer not to use inherited builds.

### Build Diagnostic

The amount of diagnostic messages generated by the build system is dependent on the diagnostic verbosity level that can be modified by the `-v` and `-q` options to the `fcm make` command.

The following is a list of diagnostic output at each verbosity level:

-q
  - Exceptions.
default
  - Everything at the -q level.
  - Start time of the build.
  - The summary of source analysis.
  - The summary of targets. Each row except the last reports the number of modified and unchanged targets with a given type of task, and the total time spent to perform the tasks. The last row reports the total number of modified and unchanged targets, and the actual elapsed time. It is worth noting that the elapsed time in a multi-process build should be significant shorter than the sum of the total time for each type of task. E.g.:

        [info] compile   targets: modified=5, unchanged=0, total-time=0.5s
        [info] compile+  targets: modified=1, unchanged=0, total-time=0.0s
        [info] ext-iface targets: modified=2, unchanged=0, total-time=0.0s
        [info] install   targets: modified=1, unchanged=0, total-time=0.0s
        [info] link      targets: modified=1, unchanged=0, total-time=0.1s
        [info] TOTAL     targets: modified=10, unchanged=0, elapsed-time=0.7s

  - Total time.
-v
  - Everything at the default level.
  - Elapsed time and name-space for each analysed source.
  - Task name, elapsed time, target status (`M` for modified or `U` for unchanged), target key, and source name-space for each modified target. E.g.:

        [info] compile    0.0 M hello_func.o         <- lib/function/hello_func.f90
        [info] ext-iface  0.0 M hello_func.interface <- lib/function/hello_func.f90
        [info] install    0.0 M hello_inc.f90        <- include/hello_inc.f90
        [info] compile    0.0 M hello_1.o            <- bin/hello_1.f90
        [info] link       0.1 M hello_1.exe          <- bin/hello_1.f90

-vv
  - Everything at the -v level.
  - A list of dependencies (type and name) of each analysed source. E.g.

        [info] analyse  0.0 bin/hello_1.f90
        [info]              -> (  include) hello_inc.f90
        [info]              -> (        o) hello_void.o
        [info]              -> ( f.module) hello_mod

  - A list of the available targets from the sources. Each row contains the source namespace, the target task, the target category and the key of the target. E.g.:

        [info] source->target / -> (archive) lib/ libo.a
        [info] source->target hello.f90 -> (link) bin/ hello.exe
        [info] source->target hello.f90 -> (install) include/ hello.f90
        [info] source->target hello.f90 -> (compile) o/ hello.o
        [info] source->target world.f90 -> (install) include/ world.f90
        [info] source->target world.f90 -> (compile+) include/ world.mod
        [info] source->target world.f90 -> (compile) o/ world.o

  - A list of the required targets for this build. Each row contains the task, the category and the key of the target. E.g.:

        [info] required-target: link      bin     hello_1.exe

  - The dependency tree of all the required targets. (N.B. A `(n-deps=N)` at the end of a line means that the target has already appeared earlier and that it has `N` direct dependencies, which will not be reported again.) E.g.:

        [info] target hello_1.exe
        [info] target  - hello_void.o
        [info] target  - hello_1.o
        [info] target  -  - hello_mod.mod
        [info] target  -  -  - hello_mod.o
        [info] target  -  - hello_void.o
        [info] target  -  - hello_inc.f90
        [info] target  -  -  - hello_sub.interface
        [info] target  -  -  -  - hello_sub.o
        [info] target  -  -  -  -  - hello_func.interface
        [info] target  -  -  -  -  -  - hello_func.o
        [info] target  - hello_2.o
        [info] target  -  - hello_mod.mod (n-deps=1)

  - Each shell command invoked with elapsed time and return code.
  - STDOUT and STDERR from shell commands invoked by the build tasks, e.g. diagnostic output from compilers and linkers.

## Preprocess

As most modern compilers can handle preprocessing, you should normally leave preprocessing to the compiler. However, it is recognised that some code is written with preprocessor directives that can alter the calling interface of the procedure and/or their dependencies. If a source file requires preprocessing in such a way, we have to preprocess it before feeding it to the build system. The preprocess system can be used to do this. It is typically run as a step before build.

However, using a separate preprocess step is not the best way of working, as it adds an overhead to the build process. If your code requires preprocessing, you should try to design it to avoid changes in the above.

In practice, the only reasonable use of a preprocessor with Fortran is for code selection. For example, preprocessing is useful for isolating machine specific libraries or instructions, where it may be appropriate to use inline alternatives for small sections of code. Another example is when multiple versions of the same procedure exist in the source tree and you need to use the preprocessor to select the correct version for your build.

Avoid using the a preprocessor for code inclusion, as you should be able to do the same via the Fortran `INCLUDE` statement. You should also avoid embedding preprocessor macros within the continuations of a Fortran statement, as it can make your code very confusing.

The preprocess system works using the same logic as the build system, but is configured primarily to preprocess C/C++ and Fortran source files. We shall document only the main differences to the build system in the remainder of this section.

### Preprocess: Basic

A typical usage of the preprocess system may look like:

    steps = extract preprocess build

    # ... some extract configuration

    # Switch off preprocessing for all
    preprocess.target{task} =
    # Only preprocess source files in these name-spaces
    preprocess.target{task}[foo/bar egg/fried.F90] = process
    # Specifies the macro definitions for the Fortran preprocessor
    preprocess.prop{fpp.defs} = THING=stuff HIGH=tall
    # Specifies the macro definitions for the C preprocessor
    preprocess.prop{cpp.defs} = LOWER=lower UNDER LINUX

    # ... some build configuration

The result of the preprocess can be found in the sub-directories of the `preprocess/` sub-directory. There are only 2 target categories:

`include`
  e.g. include files.
`src`
  e.g. preprocessed source files

### Preprocess Source Types

Only files in the following types (with the given file extensions) are recognised by the preprocess system:

cpp (C/C++ source file)
  .c .m .cc .cp .cxx .cpp .CPP .c++ .C .mm .M
fpp (Fortran source file requiring preprocessing)
  .F .FOR .FTN .F90 .F95
h (Preprocessor header file)
  .h

### Preprocess Source Analysis

The preprocess system only looks for include dependencies using the pattern `#include "name.h"`. Macros using the angle brackets syntax (e.g. `#include <name.h>`) are ignored.

### Preprocess Targets from Source Files

The preprocess system only generates targets from source files, (i.e. targets are not generated by name-spaces). Targets of the preprocess system perform one of the following tasks:

install
  Copies the source file to the destination.
process
  Creates a new source file by invoking the C/Fortran preprocessor on the original source file.

By default, it attempts to build all targets with a `process` task, i.e.:

    preprocess.target =
    preprocess.target{task} = process
    preprocess.target{category} =

Here is a list of what targets are available for each type of file:

cpp -> name-space

  description: the preprocessed version of the original file.

  task: process (cpp).

  category and destination: src and src/name-space

  properties: cpp, cpp.flags, cpp.defs, cpp.flag-define, cpp.flag-include, cpp.include-paths

  dependencies: include.

  update if: source file or any of the required properties are modified, or if any include dependencies are updated.

fpp -> name-space

  description: the preprocessed version of the original file.

  task: process (fpp).

  category and destination: src and src/name-space

  properties: fpp, fpp.flags, fpp.defs, fpp.flag-define, fpp.flag-include, fpp.include-paths

  dependencies: include.

  update if: source file or any of the required properties are modified, or if any include dependencies are updated.

h -> name

  description: a header (include) file.

  task: install.

  category and destination: include and include/name

  dependencies: include.

  update if: source file is modified.