Hoel: Hoel

Hoel C Database abstraction library with json based language

Loading...
Searching...
No Matches
Hoel

Database abstraction library written in C.

Simple and easy to use database access library. Works with SQLite 3, MariaDB and PostgreSQL databases. Uses a JSON-based language with `jansson` to execute simple queries based on one table.

# Online documentation

See the online documentation for a doxygen format of the API documentation.

# Installation

## Prerequisites

### Jansson

Install Jansson library for JSON manipulation, minimum version 2.4. On a Debian-based platform, run the following command:

$ sudo apt-get install libjansson-dev

### Database libraries

Install Hoel database dependencies based on your requirements:

- SQLite3: Install the package `libsqlite3-dev`
- MariaDB: Install the package `libmariadb-dev`
- PostgreSQL: Install the package `libpq-dev`

### Distribution packages

Hoel is available in multiple distributions as official package. Check out your distribution documentation to install the package automatically.

$ # Example for Debian testing
$ sudo apt install libhoel-dev

### Pre-compiled packages

You can install Hoel with a pre-compiled package available in the release pages. `jansson`, `sqlite3`, `libmariadb-client` and `libpq` development files packages are required to install Hoel. The packages files `hoel-dev-full_*` contain the libraries `orcania`, `yder` and `hoel`. It's also compiled with all the database backends.

For example, to install Hoel with the `hoel-dev-full_2.3.0_Debian_stretch_x86_64.tar.gz` package downloaded on the `releases` page, you must execute the following commands:

$ sudo apt install -y libjansson-dev libmariadb-dev libsqlite3-dev libpq-dev libsystemd-dev
$ wget https://github.com/babelouest/hoel/releases/download/v1.4.0/hoel-dev-full_1.4.0_Debian_stretch_x86_64.tar.gz
$ tar xf hoel-dev-full_1.4.0_Debian_stretch_x86_64.tar.gz
$ sudo dpkg -i liborcania-dev_1.2.0_Debian_stretch_x86_64.deb
$ sudo dpkg -i libyder-dev_1.2.0_Debian_stretch_x86_64.deb
$ sudo dpkg -i libhoel-dev_1.4.0_Debian_stretch_x86_64.deb

If there's no package available for your distribution, you can recompile it manually using `CMake` or `Makefile`.

## Install from the source

### CMake - Multi architecture

CMake minimum 3.5 is required.

Last Hoel release: https://github.com/babelouest/hoel/releases/latest/

Run the CMake script in a sub-directory, example:

$ cd <hoel_source>
$ mkdir build
$ cd build
$ cmake ..
$ make && sudo make install

The available options for CMake are:

- `-DWITH_SQLITE3=[on|off]` (default `on`): Enable/disable SQLite3 database backend
- `-DWITH_MARIADB=[on|off]` (default `on`): Enable/disable MariaDB database backend
- `-DWITH_PGSQL=[on|off]` (default `on`): Enable/disable PostgreSQL database backend
- `-DWITH_JOURNALD=[on|off]` (default `on`): Build with journald (SystemD) support for logging
- `-DBUILD_STATIC=[on|off]` (default `off`): Build the static archive in addition to the shared library
- `-DBUILD_HOEL_TESTING=[on|off]` (default `off`): Build unit tests
- `-DBUILD_HOEL_DOCUMENTATION=[on|off]` (default `off`): Build the documentation, doxygen is required
- `-DINSTALL_HEADER=[on|off]` (default `on`): Install header file `hoel.h`
- `-DBUILD_RPM=[on|off]` (default `off`): Build RPM package when running `make package`
- `-DCMAKE_BUILD_TYPE=[Debug|Release]` (default `Release`): Compile with debugging symbols or not

### Good ol' Makefile

Clone, compile and install Orcania and Yder libraries.

### Orcania (Miscellaneous functions)

Last Orcania release: https://github.com/babelouest/orcania/releases/latest/

$ cd orcania/src
$ make && sudo make install

### Yder (simple logs library)

Last Yder release: https://github.com/babelouest/yder/releases/latest/

$ cd yder/src
$ make
$ sudo make install

### Hoel

Last Hoel release: https://github.com/babelouest/hoel/releases/latest/

$ cd hoel/src
$ make
$ sudo make install

By default, Hoel is compiled with the 3 databases support. If you don't need one or more database, follow these instructions

#### SQLite 3

Add DISABLE_SQLITE=1 to the `make` command:

$ cd hoel/src
$ make DISABLE_SQLITE=1
$ sudo make install

#### MariaDB

Add DISABLE_MARIADB=1 to the `make` command:

$ cd hoel/src
$ make DISABLE_MARIADB=1
$ sudo make install

#### Postgre SQL

Add DISABLE_POSTGRESQL=1 to the `make` command:

$ cd hoel/src
$ make DISABLE_POSTGRESQL=1
$ sudo make install

### Disable 2 backends

You can disable 2 databases backends to keep just one, simply add both parameters to the `make` command:

$ cd hoel/src
$ make DISABLE_MARIADB=1 DISABLE_POSTGRESQL=1
$ sudo make install

### Installation folder

By default, the shared library and the header file will be installed in the `/usr/local` location. To change this setting, you can modify the `DESTDIR` value in the `src/Makefile`.

# API Documentation

## Header files and compilation

To use Hoel in your code, include the file `hoel.h`.

#include <hoel.h>
hoel.h
Hoel database abstraction library.

Use the flag `-lhoel` to include Hoel library in the linking process.

### Return values

When specified, some functions return `H_OK` on success, and other values otherwise. `H_OK` is 0, other values are non-0 values. The defined errors list is the following:

#define H_OK 0 // No error
#define H_ERROR 1 // Generic error
#define H_ERROR_PARAMS 2 // Error in input parameters
#define H_ERROR_CONNECTION 3 // Error in database connection
#define H_ERROR_DISABLED 4 // Database connection is disabled
#define H_ERROR_QUERY 5 // Error executing query
#define H_ERROR_MEMORY 99 // Error allocating memory

### Memory allocation

Some function return allocated values. When the value is not a structure, you must use the function `h_free` to clean it. Otherwise, use the dedicated functions.

void h_free(void * data);
h_free
void h_free(void *data)
Definition hoel.c:32

### Initialization

To create a connection to a database, use its dedicated function

struct _h_connection * h_connect_sqlite(const char * db_path);
struct _h_connection * h_connect_mariadb(const char * host, const char * user, const char * passwd, const char * db, const unsigned int port, const char * unix_socket);
struct _h_connection * h_connect_pgsql(char * conninfo);
h_connect_pgsql
struct _h_connection * h_connect_pgsql(const char *conninfo)
Definition hoel-pgsql.c:410
h_connect_sqlite
struct _h_connection * h_connect_sqlite(const char *db_path)
Definition hoel-sqlite.c:324
h_connect_mariadb
struct _h_connection * h_connect_mariadb(const char *host, const char *user, const char *passwd, const char *db, const unsigned int port, const char *unix_socket)
Definition hoel-mariadb.c:437
_h_connection
Definition hoel.h:87

All these functions return a struct _h_connection * on success. This pointer will be needed on every call to Hoel functions.

When you no longer need your connection, close it using the function `h_close_db`. This will close the connection to the database and free the memory allocated by the connection.

int h_close_db(struct _h_connection * conn);
h_close_db
int h_close_db(struct _h_connection *conn)
Definition hoel.c:40

The connection must be cleaned when it's no longer needed.

int h_clean_connection(struct _h_connection * conn);
h_clean_connection
int h_clean_connection(struct _h_connection *conn)
Definition hoel.c:633

### Escape string

If you need to escape parameters, you can use the functions `h_escape_string`, the returned value must be h_free'd after use.

The meaning of existence of the function `h_escape_string_with_quotes` is because of PostgreSQL ways to escape an unsafe string.

TL;DR:

- Use `h_escape_string_with_quotes` only if you use a PostgreSQL database.
- You can use both `h_escape_string_with_quotes` and `h_escape_string` if you use only SQLite3 and/or MariaDB databases.

If the unsafe string contains backslashes `"\"</tt>, the escaped string will have the pattern &lsquo;E&rsquo;<safe>'<tt>, otherwise it will have the pattern</tt>'<safe>'<tt>. Since the function</tt>h_escape_string<tt>must return the safe string only, without surroundings quotes</tt>'<tt>, the format</tt>E'<safe>'` is incompatible with Hoel API. Therefore, using <tt>h_escape_string</tt> with PostgreSQL could lead to undefined behaviour and more importantly exploitable bugs. To avoid this, on a PostgreSQL database, an escaped string using <tt>h_escape_string</tt> that should return the format &lsquo;E&rsquo;<safe>'<tt>will return</tt>NULL<tt>, and an escaped string using</tt>h_escape_string_with_quotes<tt>will return the format</tt>E'<safe>'`. @icode{c} /** * h_escape_string * Escapes a string * returned value must be h_free'd after use */ char * h_escape_string(const struct _h_connection * conn, const char * unsafe); /** * h_escape_string_with_quotes * Escapes a string and returns it ready to be inserted in the query * returned value must be h_h_free'd after use */ char * h_escape_string_with_quotes(const struct _h_connection * conn, const char * unsafe); @endicode @subsubsection autotoc_md25 Build a more complicated where clause When you need to run a query with a where clause using multiple parameters, such as &lsquo;WHERE col1='a&rsquo; AND (col2='b' OR col3=5) AND col4=42.3<tt>, you can use the operator</tt>raw`: @icode{JSON} { "table": "table1", "columns": ["col1", "col2"] "where": { " ": { "operator", "raw", "value", "col1='a' AND (col2='b' or col3='c') AND col4=5" } } } @endicode In some cases, you may need to build the where clause with multiple variables. In hoel 1.4.27, the function <tt>h_build_where_clause</tt> was introduced to help that. Please note that this function is still in Beta. @icode{C} /** * h_build_where_clause * Generates a where clause based on the pattern and the values given * @param conn the connection to the database * @param pattern the pattern to build the where clause * the pattern variables available are the following: * - %s: a string value to escape with quotes * - %S: a string value to escape without quotes * - %c: a string value not to escape with quotes * - %C: a string value not to escape without quotes * - %d: an integer value in json_int_t format * - %f: a double value * - %j: a json_t value, the value must be of the type JSON_INTEGER, JSON_REAL or JSON_STRING, string values will be escaped with quotes * - %%: the value '%' * @return a heap-allocated string * returned value must be h_free'd after use */ char * h_build_where_clause(const struct _h_connection * conn, const char * pattern, ...); @endicode Then, to build the where clause above using <tt>h_build_where_clause</tt>, you can use the following code: @icode{C} const char col1[] = "a", col2[] = "b"; json_int_t col3 = 5; double col4 = 42.3; char * where_clause = h_build_where_clause("col1=s AND (col2='S' OR col3=d) AND col4=f", col1, col2, col3, col4); json_t * j_query = json_pack("{sss[ss]s{s{ssss}}}", "table", "table1", "columns", "col1", "col2", "where", " ", "operator", "raw", "value", where_clause); h_free(where_clause); // Execute j_query @endicode Note that if you use constant litteral for integer or double values, you should cast them first: @icode{C} const char col1[] = "a", col2[] = "b"; char * where_clause = h_build_where_clause("col1=s AND (col2='S' OR col3=d) AND col4=f", col1, col2, (json_int_t)5, (double)42.3); @endicode @subsubsection autotoc_md26 Execute a SQL query To execute a SQL query, you can use the function <tt>h_execute_query</tt> which will run the query in the database specified by the parameter <tt>conn</tt>. If a <tt>result</tt> parameter is specified, the result of the query (if any) will be stored in the <tt>result</tt> structure. @icode{c} /** * h_execute_query * Execute a query, set the result structure with the returned values if available * if result is NULL, the query is executed but no value will be returned * options available * H_OPTION_NONE (0): no option * H_OPTION_SELECT: Execute a prepare statement (sqlite only) * H_OPTION_EXEC: Execute an exec statement (sqlite only) * return H_OK on success */ int h_execute_query(const struct _h_connection * conn, const char * query, struct _h_result * result, int options); @endicode @subsubsection autotoc_md27 Result structure The <tt>struct _h_result</tt> is a structure containing the values returned by a query. The definition of the structure is: @icode{c} /** * sql result structure */ struct _h_result { unsigned int nb_rows; unsigned int nb_columns; struct _h_data ** data; }; @endicode The data value is a 2 dimensional array with <tt>struct _h_data</tt> variables. A <tt>struct _h_data</tt> is defined as: @icode{c} /** * sql data container */ struct _h_data { int type; void * t_data; }; @endicode where <tt>type</tt> can be the following values: @icode{c} #define HOEL_COL_TYPE_INT 0 #define HOEL_COL_TYPE_DOUBLE 1 #define HOEL_COL_TYPE_TEXT 2 #define HOEL_COL_TYPE_DATE 3 #define HOEL_COL_TYPE_BLOB 4 #define HOEL_COL_TYPE_NULL 5 @endicode <tt>t_data</tt> will point to a <tt>struct _h_type_*</tt> corresponding to the type. The <tt>struct _h_type_*</tt> available are: @icode{c} /** * sql value integer type */ struct _h_type_int { int value; }; /** * sql value double type */ struct _h_type_double { double value; }; /** * sql value date/time type */ struct _h_type_datetime { struct tm value; }; /** * sql value string type */ struct _h_type_text { char * value; }; /** * sql value blob type */ struct _h_type_blob { size_t length; void * value; }; @endicode @subsubsection autotoc_md28 Clean results or data To clean a result or a data structure, you can use its dedicated functions: @icode{c} /** * h_clean_result * Free all the memory allocated by the struct _h_result * return H_OK on success */ int h_clean_result(struct _h_result * result); /** * h_clean_data * Free memory allocated by the struct _h_data * return H_OK on success */ int h_clean_data(struct _h_data * data); @endicode @subsubsection autotoc_md29 Get last id inserted If you need the last id generated after an insert query, you can use the following function: @icode{c} /** * h_query_last_insert_id * return the id of the last inserted value * return a pointer to `struct _h_data *` on success, NULL otherwise. */ struct _h_data * h_query_last_insert_id(const struct _h_connection * conn); @endicode @subsubsection autotoc_md30 Additional query functions You can use additional functions for specific needs. All these function will use <tt>h_execute_query</tt> but check input parameters before. @icode{c} /** * h_query_insert * Execute an insert query * return H_OK on success */ int h_query_insert(const struct _h_connection * conn, const char * query); /** * h_query_update * Execute an update query * return H_OK on success */ int h_query_update(const struct _h_connection * conn, const char * query); /** * h_query_delete * Execute an delete query * return H_OK on success */ int h_query_delete(const struct _h_connection * conn, const char * query); /** * h_execute_query * Execute a select query, set the result structure with the returned values * return H_OK on success */ int h_query_select(const struct _h_connection * conn, const char * query, struct _h_result * result); @endicode @subsubsection autotoc_md31 Simple JSON queries Hoel allows to use JSON objects for simple queries with <tt>jansson</tt> library. In the simple JSON queries, a JSON object called <tt>json_t * j_query</tt> is used to generate the query. All <tt>json_t *</tt> returned and updated values must be free after use. A <tt>j_query</tt> has the following form: @icode{javascript} * { * "table": "table_name" // String, mandatory, the table name where the query is executed * "columns": ["col1", "col2"] // Array of strings, available for h_select, optional. If not specified, * will be used, the columns values are not escaped by the library * "order_by": "col_name [asc|desc]" // String, available for h_select, specify the order by clause, optional, the value is not escaped by the library * "limit": integer_value // Integer, available for h_select, specify the limit value, optional * "offset" // Integer, available for h_select, specify the limit value, optional but available only if limit is set * "values": [{ // json object or json array of json objects, available for h_insert, mandatory, specify the values to update * "col1": "value1", // Generates col1='value1' for an update query * "col2": value_integer, // Generates col2=value_integer for an update query * "col3", "value3", // Generates col3='value3' for an update query * "col4", null // Generates col4=NULL for an update query * }] * "set": { // json object, available for h_update, mandatory, specify the values to update * "col1": "value1", // Generates col1='value1' for an update query * "col2": value_integer, // Generates col2=value_integer for an update query * "col3", "value3", // Generates col3='value3' for an update query * "col4", null // Generates col4=NULL for an update query * } * "where": { // json object, available for h_select, h_update and h_delete, mandatory, specify the where clause. All clauses are separated with an AND operator * "col1": "value1", // Generates col1='value1' * "col2": value_integer, // Generates col2=value_integer * "col3": null, // Generates col3=NULL * "col4", { // Generates col4<12 * "operator": "<", * "value": 12 * }, * "col5", { // Generates col5 IS NOT NULL * "operator": "NOT NULL" * }, * "col6", { // Generates col6 LIKE '%value6%' * "operator": "raw", * "value": "LIKE 'value6'" * } * } * } @endicode @paragraph autotoc_md32 Where clause construction A <tt>where</tt> clause is a JSON object containing a series of clauses. A clause can have 2 different forms: - <tt>col_name: value</tt> - <tt>col_name: {operator: "operator_value", value: value}</tt> In the first case, <tt>col_name: value</tt>, the clause becomes <tt>col_name = value</tt>. Value is always escaped. In the second case, <tt>col_name: {operator: "operator_value", value: value}</tt>, depending on the <tt>operator</tt> value, the clause can have different forms: - <tt>operator: "NOT NULL"</tt>, the clause becomes <tt>col_name IS NOT NULL</tt> - <tt>operator: "raw"</tt>, the <tt>value</tt> value becomes the clause itself, not escaped, for example in &lsquo;{ "operator": "raw", "value": "LIKE ’value6'" }<tt>, the clause becomes</tt>col6 LIKE '%value6%'<tt> - otherwise, the clause becomes</tt>col_name operator value`, value is escaped All clauses are separated by an <tt>AND</tt> operator. As en example, here is a JSON object and its generated where clause: JSON object: @icode{javascript} { "col1": "value1", "col2": 42, "col3": { "operator": ">=", "value": 55.5 }, "col4": { "operator": "raw", "value": "LIKE 'alu'" } }`

`SQL Where clause:`

WHERE col1 = 'value1'
AND col2 = 42
AND col3 >= 55.5
AND col4 LIKE '%alu%'

`If you need less simple clauses, you can build it on your own and use the `h_execute_query` or the `h_execute_query_json` functions.`

`The simple JSON queries functions are:`

``

int h_select(const struct _h_connection * conn, const json_t * j_query, json_t ** j_result, char ** generated_query);
int h_insert(const struct _h_connection * conn, const json_t * j_query, char ** generated_query);
json_t * h_last_insert_id(const struct _h_connection * conn);
int h_update(const struct _h_connection * conn, const json_t * j_query, char ** generated_query);
int h_delete(const struct _h_connection * conn, const json_t * j_query, char ** generated_query);
h_delete
int h_delete(const struct _h_connection *conn, const json_t *j_query, char **generated_query)
Definition hoel-simple-json.c:762
h_last_insert_id
json_t * h_last_insert_id(const struct _h_connection *conn)
Definition hoel-simple-json.c:674
h_select
int h_select(const struct _h_connection *conn, const json_t *j_query, json_t **j_result, char **generated_query)
Definition hoel-simple-json.c:458
h_insert
int h_insert(const struct _h_connection *conn, const json_t *j_query, char **generated_query)
Definition hoel-simple-json.c:604
h_update
int h_update(const struct _h_connection *conn, const json_t *j_query, char **generated_query)
Definition hoel-simple-json.c:712

``

#### JSON last insert id

``

`The function `h_last_insert_id` returns the last inserted id in a `json_t *` format.`

``

json_t * h_last_insert_id(const struct _h_connection * conn);

``

### Example source code

``

`See `examples` folder for detailed sample source codes.`