Installation and customization — APSW 3.46.0.1 documentation

# Installation and customization

## PyPI (recommended)

APSW is on PyPI at https://pypi.org/project/apsw/

It can be installed in the same way as other packages:

    python3 -m pip install apsw

When you install from PyPI:

- The corresponding SQLite version is embedded privately inside and not affected by or visible to the rest of the machine or even the rest of the process.

  This means other modules and libraries will continue using whatever SQLite they would have before. For example Core Data on MacOS uses SQLite, but will not know of or be affected by the SQLite inside APSW.

- All extensions are enabled, except ICU.

- SQLITE_ENABLE_COLUMN_METADATA is enabled, providing `Cursor.description_full`

The PyPI releases include pre-built binaries for common platforms. If yours is not covered, then pip will download the source release and automatically compile with the same settings. It will require a C compiler and the Python development header files.

## Linux/BSD provided

Most Linux & BSD distributions have packaged APSW which may trail the SQLite and APSW releases by a year, or more. The distribution provided APSW uses the system wide SQLite library.

Debian | Install python3-apsw
Fedora | Install python3-apsw
Ubuntu | Install python3-apsw
Gentoo | Install dev-python/apsw
Arch | Install python-apsw
FreeBSD | databases/py-apsw in Ports

There is a full list (150+) of distributions, the package name for APSW, and what APSW version they are currently on.

## Source

It is recommended you get the source from Github releases. If you get the source from PyPi then ensure you edit the `setup.apsw` file inside.

- apsw-3.46.0.1.zip (Source, includes this HTML Help)

- apsw-3.46.0.1.cosign-bundle cosign signature

### Verifying your download

Github source releases are digitally signed so you can verify they have not been tampered with, and were produced by the project maintainer.

Sigstore is used via the cosign tool. Download the corresponding cosign bundle which contains the signature.

Verify

> Install cosign if you don’t have it already. It is available for a wide variety of platforms including Linux, MacOS, and Windows.

> Checking the signature needs to provide the source release, the cosign bundle, the maintainer id, and issuer. The command is all one line shown here across multiple lines for clarity.

>     $ cosign verify-blob apsw-3.46.0.1.zip                           \
>         --bundle apsw-3.46.0.1.cosign-bundle                         \
>         --certificate-identity=rogerb@rogerbinns.com                 \
>         --certificate-oidc-issuer=https://github.com/login/oauth
>     Verified OK

> Check for a success exit code, and verified message.

## Building and customization

APSW is configured for standard building (PEP 518)

    $ python3 -m build

You will need to update the MANIFEST first if you are providing your own SQLite, or if you are providing a `setup.apsw` with custom configuration. setuptools is used to compile the extension. You can use it directly instead by invoking `setup.py`.

### Build process

A series of commands and options are given to `setup.py` in this pattern:

    python setup.py cmdone --option --option value cmdtwo --option \
       cmdthree --option --option value

The only necessary command is build. You can get help by –help:

    python setup.py build --help

Each command takes options which can be specified on the command line, or in a configuration file named `setup.cfg` or `setup.apsw`. The leading double dash on options is omitted, and dashes inside should become underscores.

    # This is used with pypi source and binary builds

    [build]
    # download corresponding sqlite release
    fetch = True
    # all extensions included
    enable_all_extensions = True
    # ... except icu (not abi stable)
    omit = icu
    # for Cursor.description_full
    enable = COLUMN_METADATA

### SQLite options

It is important to understand SQLite’s compile time options. They provide control over functionality and APIs included or excluded from SQLite.

APSW needs to know the options chosen so it can adapt. For example if extension loading is omitted from SQLite then APSW also needs to omit the same functionality, otherwise compilation or linking will fail.

### Finding SQLite

APSW can fetch SQLite as detailed below, and places it in a `sqlite3/` subdirectory. You can place your own SQLite in that directory. If there is a `sqlite3.c` (ie the amalgamation) then it will be statically included inside APSW. A compiled SQLite will be picked up if present. If none of that is present, then the standard compiler locations are used (eg `/usr/include` on Unix).

If `sqlite3/sqlite3config.h` is present it is included before `sqlite3/sqlite3.c`. It is a good location to put platform configuration which APSW’s fetch does automatically by running `configure`.

### setup.py commands and their options

These are the relevant `setup.py` commands and their relevant options.

#### build

Does the complete build. This will invoke build_ext - use only one of build or build_ext.

`--fetch` | Fetches the corresponding SQLite version
`--enable-all-extensions` | Enables all the standard extensions
`--enable` | A comma separated list of options to enable that are normally off omitting the `SQLITE_ENABLE` prefix. They will be uppercased. eg `--enable column_metadata,fts5`
`--omit` | A comma separated list of options to omit that are normally enabled omitting the `SQLITE_OMIT` prefix. They will be uppercased. eg `--omit automatic_index`

#### fetch

This provides more fine grained control over what is fetched.

`--version` | Specify an explicit version of SQLite to fetch
`--fetch-sqlite` | Downloads the SQLite amalgamation
`--all` | Downloads all SQLite components other than the amalgamation. Over time this has included additional extensions and SQLite functions, but currently is nothing.
`--missing-checksum-ok` | APSW includes checksums of SQLite releases and will fail a fetch if you specify a version for which no checksum is known. This allows proceeding.

#### build_ext

This performs the compilation of the C code, and provides more control than build.

`--use-system-sqlite-config` | Uses `ctypes` to determine the system wide SQLite library compilation options
`--definevalues` | Additional #defines separated by commas. eg `--definevalues SQLITE_MAX_ATTACHED=37,SQLITE_EXTRA_INIT=mycore_init`
`--enable-all-extensions` | Enables all the standard extensions
`--enable` | A comma separated list of options to enable that are normally off omitting the `SQLITE_ENABLE` prefix. They will be uppercased. eg `--enable column_metadata,fts5`
`--omit` | A comma separated list of options to omit that are normally enabled omitting the `SQLITE_OMIT` prefix. They will be uppercased. eg `--omit automatic_index`
`--apsw-no-old-names` | Excludes old non PEP 8 complaint name aliases from the extension and type stubs.

## Testing

SQLite itself is extensively tested. It has considerably more code dedicated to testing than makes up the actual database functionality.

APSW includes tests which use the standard Python testing modules to verify correct operation. New code is developed alongside the tests. Reported issues also have test cases to ensure the issue doesn’t happen or doesn’t happen again.:

                    Python  /usr/bin/python3 sys.version_info(major=3, minor=11, micro=6, releaselevel='final', serial=0) 64bit ELF
    Testing with APSW file  /space/apsw/apsw/__init__.cpython-311-x86_64-linux-gnu.so
              APSW version  3.44.0.0
        SQLite lib version  3.44.0
    SQLite headers version  3044000
        Using amalgamation  True
    .............................................................A message due to RecursionError is possible, and what is being tested
    object address  : 0x7fea3c94ada0
    object refcount : 1
    object type     : 0x9de7a0
    object type name: TypeError
    object repr     :
    lost sys.stderr
    object address  : 0x7fea3c94ad40
    object refcount : 1
    object type     : 0x9de7a0
    object type name: TypeError
    object repr     :
    lost sys.stderr
    ................................................................
    ----------------------------------------------------------------------
    Ran 125 tests in 26.783s

    OK

The tests also ensure that as much APSW code as possible is executed including alternate paths through the code. 95.5% of the APSW code is executed by the tests. In the source, there is a script that enables extra code that deliberately induces extra conditions such as memory allocation failures, SQLite returning error codes, Python APIs erroring etc. That brings coverage up to 99.6% of the code.

A memory checker Valgrind and compiler sanitizer options are also used for further validation.

To ensure compatibility with the various Python versions, a script downloads and compiles all supported Python versions in both debug and release configurations (and 32 and 64 bit) against the APSW and SQLite supported versions running the tests.

In short both SQLite and APSW have a lot of testing!