FCM 2-0 Release Notes
Author: FCM team

# FCM 2-0 Release Notes 11 March 2011

These are the release notes for FCM 2-0. You can use this release of FCM freely under the terms of the FCM LICENSE, which you should receive with the distribution of this release.

FCM is maintained by the FCM team at the Met Office. Please feedback any bug reports or feature requests to us by e-mail.

## What's New?

`fcm make`: new command
  New extract/build system. Deprecates `fcm extract` and `fcm build`. See User Guide > FCM Make for detail on how to use the new system. See also fcm make: key differences compared with fcm extract/build.
`fcm branch-create`: new command
  Deprecates `fcm branch --create`.
`fcm branch-delete`: new command
  Deprecates `fcm branch --delete`.
`fcm branch-diff`: new command
  Deprecates `fcm diff --branch`.
`fcm branch-info`: new command
  Deprecates `fcm branch --info`.
`fcm branch-list`: new command
  Deprecates `fcm branch --list`.
`fcm browse`
  This is now the preferred name of `fcm trac` or `fcm www`.
`fcm cfg-print`
  This is now the preferred name of `fcm cfg`.
`fcm export-items`: new command
  Deprecates `fcm_update_version_dir.pl`.
Configuration for keywords
  New syntax and location. See User Guide > Annex: FCM Configuration File > FCM Keyword Configuration for detail. Equivalent settings in `$FCM/etc/fcm.cfg` will no longer work. Those in `$HOME/.fcm` should continue to work. We would encourage users to migrate to the new syntax and location.
Configuration for external commands
  New syntax and location. See User Guide > Annex: FCM Configuration File > FCM External Configuration for detail. Equivalent settings in `$FCM/etc/fcm.cfg` and `$HOME/.fcm` will no longer work.

### fcm make: key differences compared with fcm extract/build

Single command and framework of a configurable chain of steps, e.g. extract, mirror, preprocess, build, etc.

- as opposed to 2 separate commands with fixed steps.
- possible to set up multiple builds with different configurations from the same extract.

New configuration file format, with more powerful syntax and declarations. E.g.:

- Improved support for specifying name-spaces (previously package or sub-package) for a declaration.
- Improved support for declaring and referencing variables.
- Improved support for space and other meta-characters.

extract: automatically associates location keywords to source tree locations. E.g.:

- `fcm extract` configuration requires repeated declarations:

      cfg::type ext
      repos::foo::base fcm:foo/trunk
      expsrc::foo::base
      repos::bar::base fcm:bar/trunk
      expsrc::bar::base

- `fcm make` configuration is much simpler:

      steps = extract
      extract.ns = foo bar

extract: clearly distinguishes a base source tree from the diff source trees for each project. E.g.:

- `fcm extract` configuration requires arbitrary IDs for each source tree, separate revision declarations, and assumes that the first declared tree for a project is the base:

      cfg::type ext
      repos::foo::base fcm:foo/trunk
      revision::foo::base 1234
      expsrc::foo::base
      repos::foo::b1 fcm:foo/branches/dev/fred/r1234_b1
      revision::foo::b1 2345
      repos::foo::b2 fcm:foo/branches/dev/bob/r1234_b2
      repos::foo::b3 fcm:foo/branches/dev/alice/r1234_b3

- `fcm make` configuration uses different declarations for the location of the base source tree and the locations of the diff source trees:

      steps = extract
      extract.ns = foo
      extract.location[foo] = trunk@1234
      extract.location{diff}[foo] = \
          branches/dev/fred/r1234_b1@2345 \
          branches/dev/bob/r1234_b2 \
          branches/dev/alice/r1234_b3

extract: can easily filter parts of a project source tree, and/or change the root of the extract tree. E.g.:

- `fcm extract` configuration can cause confusion:

      cfg::type ext
      repos::um::base fcm:um/trunk/src
      revision::foo::base vn7.7
      expsrc::um::base
      repos::um::b1 fcm:um/branches/dev/fred/vn7.7_b1/src
      repos::um::b2 fcm:um/branches/dev/bob/vn7.7_b2/src
      repos::um::b3 fcm:um/branches/dev/alice/vn7.7_b3/src

- `fcm make` configuration is clearer and has more features:

      steps = extract
      extract.ns = um
      extract.path-root[um] = src
      extract.path-excl[um] = configs scm
      extract.location[um] = trunk@vn7.7
      extract.location{diff}[um] = \
          branches/dev/fred/vn7.7_b1 \
          branches/dev/bob/vn7.7_b2 \
          branches/dev/alice/vn7.7_b3

extract: works with project source trees as opposed to individual source directories.

- fewer calls to the version control system servers.
- deleted directories are now handled correctly.

extract: can use multiple processes to retrieve source trees information and to export source tree files from the version control system.

- extract of multiple projects and/or with multiple source trees can be much faster.

mirror: is now an independent step.

- can set up multiple mirror steps to mirror an extract to alternate destinations.

build: can use multiple processes to analyse the source files for dependencies and other information.

- multi-process build is much faster.

build: uses an internal task manager and runner - more efficient logic possible:

- no longer requires GNU make.
- no longer requires dummy files such as `*.done *.flags`.
- uses MD5 checksums to determine whether sources and targets are out of date - as opposed to time stamps.
- fails the build if duplicated targets are detected if those targets are required by the build.

build: has improved the logic for building Fortran program units.

- detects correctly multiple top program units in the same source file.
- sets up a module usage as an include dependency on the `*.mod` file instead of the `*.o` file - reduces the chance of module compile cascades in incremental mode.
- only generates interface files on demand.

build: has improved facilities for sources and targets selection.

- can now select targets by name-space, category and task.
- has better documentation on the relationship between source files and build targets.
- note that target declarations are not cumulative and that targets are inherited by default (unlike with `fcm build`).

build: automatically uses the Fortran compiler to link Fortran executables and the C compiler to link C executables.

build: has more diagnostics, e.g. on source dependencies, target build tree, etc.

preprocess: is now an independent step, but shares all the logic of the build system, e.g.:

- preprocessing dependency analysis and target update can be performed in multiple processes.
- note that file extensions are not modified by the preprocess step unlike with `fcm build` which changed, for example `.F90` extensions to `.f90`.

Other notable changes:

- By default, `fcm make` will always rebuild link targets and re-install scripts in inherited builds. Therefore, to use the executables from a build all you need to do is set your PATH environment variable to point to `$DEST/build/bin/` (where $DEST is the destination of the make). Note that there is no `fcm_env.sh` file produced by `fcm make`.
- `fcm extract` has the ability to fail if the declared revision of a branch does not correspond to a changeset of that branch. Furthermore, it can output the latest revision of a branch if the declared revision is not the latest. `fcm make` does not support this.
- There is no equivalent of `fcm cmp-ext-cfg` for FCM make configurations.
- `fcm make` does not support defining a separate linker - it always uses the compiler of the source file containing the main program (which is the default with `fcm build`).
- `fcm make` does not recognise existing binaries as install targets (unlike `fcm build`). This feature is currently used to allow the Met Office's Suite Control System (SCS) to "build" suites but is no longer considered the best method. SCS will continue to use `fcm build` until such time as a better method is adopted or the system is retired.
- `fcm make` recognises data files as install targets in a similar way to `fcm build`. However, the default destination of such targets is now the full name-space under the `etc/` sub-directory.
- `fcm make` supports the building of libraries but does not generate the relevant exclude dependency configuration as is done by `fcm build`.
- `fcm make` has no equivalent of the `--archive` nor the `--targets` options provided by `fcm build`.
- `fcm build` recognises a file name without its extension as a sub-package name. This is not true with `fcm make` which only recognises the full file name as a namespace.

In addition to the differences noted above, `fcm make` fixes various subtle problems which can occur with `fcm extract` and `fcm build` as a result of limitations in the internal design. Overall, `fcm make` is much better and we recommend that all users migrate to it. `fcm extract` and `fcm build` will continue to be maintained for legacy systems but will not be developed further.

## Minor Changes and Bug Fixes

`fcm build`

  Handle directory names with a dot extension.

  Correct search path for inherited configuration file.

  Always export OBJECTS in generated `Makefile`.

`fcm cfg`

  Now an alias of `fcm cfg-print`.

  The default behaviour is to parse FCM 2 configuration files. To parse FCM 1 configuration files, use the `--fcm1` option.

  The values in the output will no longer be lined up.

`fcm extract`

  Fix double slashes in cache of extract with project root level files.

  Correct search path for inherited configuration file.

  Fix incremental mode behaviour of targets with deleted, overriding inherited status.

`fcm keyword-print`
  Change in output format to match the new configuration file format.
`fcm mkpatch`

  Don't use patch file if PDF file detected.

  Handle property changes to directories.

  Handle copies within new directories.

  Handle replaced directories.

  Fix handling of symbolic links.

  Fix pattern match used when checking for excluded or copied paths.

  Use `--no-backup-if-mismatch` option to patch command to ensure backup files not created if patch does not match exactly.

`fcm` direct wrappers to `svn` commands
  No longer prints `=> svn ...` on STDOUT.
Misc fixes
  Misc fixes related to changes in Perl 5.10 and Subversion 1.6.

## Known Issues

Build inheritance limitation: handling of include files
  See the User Guide > FCM Make > Build > Build Inheritance for detail.

## System Requirements

### Perl

The core part of FCM is a set of Perl scripts and modules. At the Met Office, FCM runs on:

Perl 5.8.2 on AIX 5.3

  `Text::ParseWords` (core Perl module) is upgraded to version 3.22.

  Met Office users do not use the code management commands and the extract system on this platform.

Perl 5.8.5 on RHEL 4

  HTTP::Date in libwww-perl is required by `fcm extract` and the extract system in `fcm make`. (libwww-perl 5.79 installed.)

  XML::DOM in libxml-enno is required by the code management commands. (libxml-enno 1.02 installed.)

  Tk is required by the `fcm gui` command. (Tk 804.027 installed.)

### Subversion

To use the code management commands (and relevant parts of the extract system) you need to have Subversion installed.

FCM requires Subversion 1.4.x or above. At the Met Office we are currently using Subversion 1.4.3.

Note: you can use the extract system to mirror code to a remote platform for building. Therefore it is only necessary to have Subversion installed on the platform where you do your code development. If you use other platforms purely for building and running then you do not need to have Subversion installed on these platforms.

### Trac

The use of Trac is entirely optional (although highly recommended if you are using Subversion).

At the Met Office we are currently using Trac 0.11.7. Note:

- The `--trac` and `--wiki` options to the `fcm diff --branch` command allow you to view branch differences using Trac. This requires Trac 0.10 or above.
- Some of the example scripts in the `examples/` directory require Trac 0.11.

### Other Requirements

The `fcm conflicts` command requires xxdiff. At the Met Office we are currently using version 3.1. The `fcm diff --graphical` command also uses xxdiff by default although other graphical diff tools can also be used.

The `fcm make` command uses `gzip`. At the Met Office we are currently using gzip 1.2.4 on AIX 5.3 and gzip 1.3.3 on RHEL 4.

The extract system uses `diff3`, (which is part of GNU diffutils), to merge together changes where the same file is modified by two different branches (compared with the base branch). At the Met Office we are currently using version 2.8.1.

The mirror system uses rsync to mirror source file to another machine. At the Met Office we are currently using version 2.6.3

The deprecated `fcm build` requires GNU make. At the Met Office we are currently using version 3.80.

FCM is intended to run on a Unix/Linux system. It is currently used at the Met Office on Linux (RHEL 4.8) and AIX 5.3.

## Installation

FCM is distributed in the form of a compressed tar file. Un-pack the tar file into an appropriate location on your system. Add the `bin/` directory into your PATH environment variable. Once you have done this you should now have full access to the FCM system, assuming that you have met the requirements described in the previous section.

You should find the following contents in the distribution:

README
  The README file contains the internal revision number of the release.
COPYRIGHT.txt
LICENSE.html
  The FCM license and other copyright information.
bin/
  Contains the `fcm` command and other utilities.
doc/
  System documentation.
doc/release_notes/
  Contains these release notes. It also contains the release notes for all previous versions which may be useful if you have skipped any versions.
doc/user_guide/
  Contains the FCM User Guide.
doc/standards/
  Contains the FCM Perl and Fortran coding standards. The Perl standard describes the standards followed by the FCM code. The Fortran standard contains some specific advice on the best way of writing Fortran code for use with FCM as well as more general advice on good practice.
doc/collaboration/
  Contains the External Distribution & Collaboration for FCM Projects document which discusses how projects configured under FCM can be distributed externally.
etc/
  Miscellaneous items, including the `fcm/keyword.cfg.eg` file. If you wish to define keywords for your site you will need to create the `etc/fcm/keyword.cfg` file. An example file, `fcm/keyword.cfg.eg`, is provided which is a copy of the file currently used at the Met Office. For further details please refer to the section FCM keywords in the System Admin chapter of the User Guide.
examples/
  Contains various example scripts which you may find useful. Note that these scripts are all specific to the Met Office and may contain hard coded paths and email addresses. They are provided in the hope that you may find them useful as examples for setting up similar scripts of your own. However, they should only be used after careful review to adapt them to your environment.
examples/etc/regular-update.eg
  An example of how you might set up a cron job to make use of the `<repos>.latest` file (see `examples/svn-hooks/post-commit-background`).
examples/lib/
  Contains the `FCM::Admin::*` Perl library, which implements the functionalities of the FCM admin utility commands.
examples/sbin/
  Contains a selection of useful admin utility commands.
examples/svn-hooks/pre-commit
  This script restricts write-access to the repository by checking the following:
  - It executes the Subversion utility `svnperms.py` if it, and the associated `svnperms.conf` file, exist. This utility checks whether the author of the current transaction has enough permission to write to particular paths in the repository.
  - It checks the disk space required by the current transaction. It fails the commit if it requires more than 5Mb of disk space.
examples/svn-hooks/post-commit
  A simple post-commit hook script which runs the script `post-commit-background` in the background.
examples/svn-hooks/post-commit-background
  This script runs in the background after each commit.
  - It updates a `<repos>.latest` file with the latest revision number.
  - It creates a dump of the new revision.
  - It calls `post-commit-background-custom` if it exists.
examples/svn-hooks/pre-revprop-change
  A simple pre-revprop-change hook script which runs the script `pre-revprop-change.pl`.
examples/svn-hooks/pre-revprop-change.pl
  If a user attempts to modify the log message of a changeset and he/she is not the original author of the changeset, this script will e-mail the original author. You can also set up a watch facility to monitor changes of log messages that affect particular paths in the repository. For further details please refer to the section Watching changes in log messages in the System Admin chapter of the User Guide.
examples/svn-hooks/post-revprop-change
  A simple post-revprop-change hook script which invokes the `trac-admin` command to `resync` the revision property cache stored in the corresponding Trac environment.
lib/
  Contains the Perl library of FCM.
man/
  Contains a basic manual page for `fcm`.
t/
  Contains unit test for FCM.
test/
  Contains regression tests for FCM.
test/test_include/
  Contains simple test code to check how your chosen compilers handle include files (see Known Issues).
tutorial/
  Contains the files necessary to set up a Subversion repository for the FCM tutorial. This will allow you to follow the tutorial section in the User Guide. See `tutorial/README` on how to set it up.