CBFlib

| IUCr Home Page | CIF Home Page | CBF/imgCIF | CBFlib |
| NOTICE | doc/GPL | doc/LGPL | imgCIF dictionary | SourceForge CBFlib site |
| Click Here to Make a Donation |

## README

### Information for CBFlib 0.9.6 release of 4 December 2018

by
Paul J. Ellis
Stanford Synchrotron Radiation Laboratory

and
Herbert J. Bernstein
Bernstein + Sons

© Copyright 2006 -- 2018 Herbert J. Bernstein

YOU MAY REDISTRIBUTE THE CBFLIB PACKAGE UNDER THE TERMS OF THE GPL.

ALTERNATIVELY YOU MAY REDISTRIBUTE THE CBFLIB API UNDER THE TERMS OF THE LGPL.

All functions in the src, include and examples directories are included in the term "API" unless explicitly placed under a diferent license in the header comments of that particular source code.

### Before using this software, please read the
for important disclaimers and the IUCr Policy on the Use of the Crystallographic Information File (CIF) and other important information.

Work on imgCIF and CBFlib supported in part by the U. S. Department of Energy (DOE) under grants ER63601-1021466-0009501 and ER64212-1027708-0011962, by the U. S. National Science Foundation (NSF) under grants DBI-0610407, DBI-0315281 and EF-0312612, the U. S. National Institutes of Health (NIH) under grants 1R15GM078077 from NIGMS and 1R13RR023192 from NCRR and funding from the International Union for Crystallography (IUCr) and Dectris, Ltd. The content is solely the responsibility of the authors and does not necessarily represent the official views of DOE, NSF, NIH, NIGMS, NCRR, IUCr or Dectris. Recent work on integration among CBF, HDF5 and NeXus supported in part by Pandata ODI (EU 7th Framework Programme)

CBFlib 0.9.6 is the full release supporting the integration of CBF and NeXus, including support for the NeXus NXpdb embedding of CIF files in NeXus files. *** IMPORTANT: Because of the requirements of dynamic loading to support the compressions used by the Dectris NeXus/HDF5 format, all applications require setting of library and plugin paths. See the initialization file cbflib.ini, which should be sourced before running any applications ***. The primary development source is at https://github.com/yayahjb/cbflib. The branch pdb_in_nexus, which was used for much of this development has been merged, is now deprecated, and will be removed in the near future.

CBFlib 0.9.5 is the full release resulting from the interim effort on 0.9.4, reflecting significant and unfortunately somewhat disruptive changes resulting from changes to the agreed CBF-NeXus mapping. Axis data on the NeXus side is now in a new tranformations:NXtransformations group. Polarization is now handled with Stokes vectors with optional esds. Further changes are expected for full support of mapping multi-module detectors such as FELs where module positioning may be refined, but this mapping should work for unitary detectors.

CBFlib 0.9.4.1 has further refactoring by Jonathan Sloan, and partial documentation with an updated dictionary. This is an interim release, expected to be become a full release with 0.9.4.2.

CBFlib 0.9.4 provides a reasonably functional mapping of CBF files to NeXus files that carries all essential information for monchromatic MX processing in the NeXus file, and extends the cmake build capabilities and refactors a lot of the mapping code. Our thanks to Jonathan Sloan for this effort.

CBFlib 0.9.3.3 added examples of the description of a multi-module FEL detector, thanks to Aaron Brewster of LCLS. New convenience accessor functions were been added to expose the details of axes field by field. Corrections were been made to the handling of _axis.rotation and _axis.rotation_axis. This was an interim release. The API version and dictionary were to be updated on the next release.

CBFlib 0.9.3.2 has a revised API version, and, otherwise, the same code as CBFlib 0.9.3.1, but with further update to the imgCIF dictionary to avoid issues with the PDB validation software. Because of the revised API version, many of the data files changed to update the version in the file comments.

CBFlib 0.9.3.1 has the same code as CBFlib 0.9.3, but with an update to the imgCIF dictionary reflecting a major technical cleanup by John Westbrook. An explanation of the choices of regex versions has been added to the README.

CBFlib 0.9.3 is the formal base release integrating CBF, HDF5 and NeXus. It also includes changes to support the description of FEL detectors. More details on the integration and the new dctionary including both NeXus information and the new FEL detector tags, _axis.rotation_axis and _axis.rotation are avialable at https://sites.google.com/site/nexuscbf/. The commulative changes in releases 0.9.1, 0.9.2, and 0.9.3 since CBFlib 0.9.0 were:

- Temporary removal of default PyCifRW support for compliance with Fedora license requirements.
- Addition of a new tiff2cbf example program.
- Update pycbf python wraapper for CBFlib.
- Padding options added to adscimg2cbf by C. Nielsen.
- System and gnu versions of getopt replaced by cbf_getopt.
- Code to handle CIF2 bracketed constructs and quoted strings added.
- System to log errors and warnings added.
- Java wrapper by Peter Chang added.
- Dectris template code by E. Eikenberry added.
- Addition of nibble offset compression trial code.
- Addition of support for full axis poise calculations.
- Incorporation of hdf5-1.8.11 and the first phase of code in CBFlib to support NeXus, including mincbf2nexus and cbf2nexus by Jonathan Sloan of DLS
- Cleanup and fixes to Makefiles for more reliable builds, and a partial preliminary CMakeLists.txt
- Fixes to support newer DECTRIS minicbf headers in convertminicbf
- The first phase of code to support

The changes for FEL support involve major changes to the positioner logic used in computing goniometer and detector positions. This code should be used with caution.

The changes for the CBF-HDF5 mappings are extensive and should be used with caution.

Please report all difficulties you may encounter with this release to

Our thanks to Jonathan Sloan, Tobias Richter and Graeme Winter of Diamond Light Source and Robert M. Sweet of Brookhaven National Laboratory for their major contributions to this release. Without their efforts it would not have happened.

CBFlib 0.9.2.5 in October 2012 through CBFlib 0.9.2.12 in June 2013 were an accumulation of minor revisions to the CBFlib 0.9.2.4 release, plus the testbed for HDF5/NeXus support and preliminary Eiger support, now released in CBFlib 0.9.3.

CBFlib 0.9.2.4 was a minor revision to the CBFlib 0.9.2.3 release to support tiff2cbf for short in tiffs and to fix problems with 64 bit long integers uncovered on Mac OSX 10.6 and 10.7 for 64 compilers.

CBFlib 0.9.2.3 was a minor revision to the CBFlib 0.9.2.2 release to allow use of NO_CBF_REGEX to suppress use of regex for convenience in doing visual studio builds for cctbx as requested by Ralf Ralf Grosse-Kunstleve for cctbx builds.

CBFlib 0.9.2.2 was a minor revision to the CBFlib 0.9.2.1 release in July 2011 to update doc/cif_img.dic to the 1.6.4 revision.

CBFlib 0.9.2.1 was a minor revision to the CBFlib 0.9.2 release in June 2011 to upgrade the setup script for the pycbf Python bindings to simplify using pycbf outside the context of the CBFlib pycbf directory.

CBFlib 0.9.2 was the recommended release of CBFlib of February 2011.

CBFlib 0.9.1 included a correction to CBFlib 0.9.0 to make axis the cbf_simple routines apply axis rotations correctly for detectors and to pick up corrections for byte offet compression incorporated into the upcoming CBFlib 0.9.1 release. The earlier version had failed to apply the rotations to the accumulated displacements. Our thanks to Joerg Kaercher of Bruker-AXS for identifying the rotation problem.

CBFlib 0.9.0 was a partial pre-release of CBFlib version 0.8 needed to support changes in RasMol. This release was incomplete and used were advided to use it with caution, but it has proven to be a reliable, stable release for 2 years. There have been significant changes in the input/output logic and in validation. For a ChangeLog consult the SVN of the CBFlib project on sourceforge.

CBFLIB is a library of ANSI-C functions providing a simple mechanism for accessing Crystallographic Binary Files (CBF files) and Image-supporting CIF (imgCIF) files. The CBFLIB API is loosely based on the CIFPARSE API for mmCIF files. Starting with this release, CBFLIB performs validation checks on reading of a CBF. If a dictionary is provided, values will be validated against dictionary ranges and enumerations. Tags missing under parent-child relationships or category key requirements will be reported. CBFlib provides functions to create, read, modify and write CBF binary data files and imgCIF ASCII data files.

### Installation

CBFLIB should be built on a disk with at least 500 megabytes of free space, for a full installation with complete tests. Read the instructions below carefully, if space is a problem.

You may download clone the git repository with
git clone https://github.com/yayahjb/cbflib.git
or a gizpped tarball of this release is available on sourceforge at

http://downloads.sf.net/cbflib/CBFlib-0.9.6.tar.gz

In addition, http://downloads.sf.net/cbflib/CBFlib_0.9.6_Data_Files_Input.tar.gz (13 MB) is a "gzipped" tar of the input data files needed to test the API, http://downloads.sf.net/cbflib/CBFlib_0.9.6_Data_Files_Output.tar.gz (34 MB) is a "gzipped" tar of the output data files needed to test the API, and, if space is at a premium, http://downloads.sf.net/cbflib/CBFlib_0.9.6_Data_Files_Output_Sigs_Only.tar.gz (1KB) is a "gzipped" tar of only the MD5 signatures of the output data files needed to test the API. Place the CBFlib_0.9.6.tar.gz file in the directory that is intended to contain up to 4 new directories, named CBFlib_0.9.6 (the "top-level" directory), CBFlib_0.9.6_Data_Files_Input and either CBFlib_0.9.6_Data_Files_Output or CBFlib_0.9.6_Data_Files_Output_Sigs_Only. If you have wget on your machine, you only need to download the source tarball. If you do not have wget, you will need to download all the tarballs into the same directory

Uncompress CBFlib_0.9.6.tar.gz with gunzip and unpack it with tar:

         gunzip CBFlib_0.9.6.tar.gz
         tar xvf CBFLIB_0.9.6.tar

To run the test programs, you will also need Paul Ellis's sample MAR345 image, example.mar2300, Chris Nielsen's sample ADSC Quantum 315 image, mb_LP_1_001.img, and Eric Eikenberry's SLS sample Pilatus 6m image, insulin_pilatus6m, as sample data. In addition there are is a PDB mmCIF file, 9ins.cif, and 3 special test files testflatin.cbf, testflatpackedin.cbf and testrealin.cbf, and several files related to NeXus and FEL testing. All these files will be dowloaded and extracted by the Makefile from CBFlib_0.9.6_Data_Files_Input. Do not download copies into the top level directory.

After unpacking the archives, the top-level directory should contain a makefile and a cmake file.:

Makefile | Makefile for Linux
CMakeLists.txt | Top level data file for cmake builds

and several alternate makefiles for other systems

and the subdirectories:

doc/ | Documentation
src/ | CBFLIB source files
include/ | CBFLIB header files
external_packages/ | For additional support kits
bin/ | Executable example programs
examples/ | Example program source files
html_images/ | JPEG images used in rendering the HTML files
lib/ | Compiled CBFLIB (libcbf.a) and FCBLIB (libfcb.a) libraries
m4/ | CBFLIB m4 macro files (used to build .f90 files)
mswin/ | An MS Windows CodeWarrior project file
pycbf/ | Jon Wright's Python bindings
ply-3.2/ | A support kit needed for dREL code
drel-ply/ | A support kit needed for dREL code
dREL-ply-0.5/ | A support kit needed for dREL code

All the makefiles are created from m4/Makefile.m4. Edit the closest approximation to your system, and then copy that variant to Makefile.

In building this release there is an enviroment variable you may need to set if you are building on an older machine with a broken dlfnc that cannot support the HDF5 filter plugin mechanism:

HDF5REGISTER

which should be set to the string "--register manual" to disable the automatic plugin search.

For instructions on compiling and testing the library, go to the top-level directory and type:

         make

Once you have a properly configure Makefile, compile and test the package with

         make tests

or, if space is at a premium, with

         make tests_sigs_only

### regex

CBFlib makes can optionally use a system regular expression library to assist in validation of CIFS. The feature is controlled by two compile-time defines. The possible choices are

- Not to use regex. If CBF_NO_REGEX is defined, no regex will be used.
- To used the tradition unix regex. If CBF_REGEXLIB_REGEX is defined, regex.h is used.
- To use the newer and more popular POSIX pcre regex. If neither CBF_NO_REGEX nor CBF_REGEXLIB_REGEX is defined, pcreposix.h is used.

### Cmake

A partial preliminary CMakeLists.txt has been included in this kit. It is not yet complete, but on machines for which the Makefiles are not appropriate, it is worth a try. To use in, create a directory named CBFlib-0.9.6-build on the same level as CBFlib-0.9.6, and in the new, empty build directory, try

        cmake ../CBFlib-0.9.6
        make all
        make test

Work on improving cmake support will continue, and comments, corrections and suggestions would be appreciated.

Depending on your system, some tests may fail mysteriously under cmake. The most likely cause is a failure to link example programs to the HDF5 shared library or other shared libraries when tests are run prior to those libraries having been installed in locations known to your system's loader. This is handled automatically in the Makefiles, but we do not yet have a reliable way to do so for cmake ctest tests. You may need to set various system-dependent environment variables such as DYLD_LIBRARY_PATH, LD_LIBRARY_PATH or PATH. The bash script in the kit cbflib.ini should be sourced before testing.

Please refer to the manual doc/CBFlib.html for more detailed information.

Updated 4 December 2018.