7.16.17. language_model_vectorize — Groonga v16.0.8 documentation

Skip to main content
`Ctrl`+`K`

# 7.16.17. `language_model_vectorize`

Added in version 14.1.0.

Note

This is an experimental feature. Currently, this feature is still not stable.

## 7.16.17.1. Summary

`language_model_vectorize` generates a normalized embedding from the given text.

See also Language model how to prepare a language model.

You can use Generated column to automate embeddings generation.

To enable this function, register `functions/language_model` plugin by the following command:

    plugin_register functions/language_model

## 7.16.17.2. Syntax

`language_model_vectorize` requires two parameters and has optional parameters:

    language_model_vectorize(model_name, text)
    language_model_vectorize(model_name, text, options)

`model_name` is the name of language mode to be used. It’s associated with file name. If `${PREFIX}/share/groonga/language_models/mistral-7b-v0.1.Q4_K_M.gguf` exists, you can refer it by `mistral-7b-v0.1.Q4_K_M`. It’s computed by removing directory and `.gguf` extension.

Added in version 15.1.8: You can also specify a Hugging Face URI for `model_name`.

`text` is the input text.

Added in version 16.0.1: Added `options`

`options` uses the following format. All of the key-value pairs are optional:

      {
        "prefix": prefix_string
      }

## 7.16.17.3. Requirements

You need llama.cpp enabled Groonga. The official packages enable it.

You need enough CPU/memory resources to use this feature. Language model related features require more resources than other features.

You can use GPU in the feature.

## 7.16.17.4. Usage

You need to register `functions/language_model` plugin at first:

Execution example:

    plugin_register functions/language_model
    # [[0,1337566253.89858,0.000355720520019531],true]

Here is a schema definition and sample data.

Sample schema:

Execution example:

    table_create --name Memos --flags TABLE_NO_KEY
    # [[0,1337566253.89858,0.000355720520019531],true]
    column_create \
      --table Memos \
      --name content \
      --flags COLUMN_SCALAR \
      --type ShortText
    # [[0,1337566253.89858,0.000355720520019531],true]

Sample data:

Execution example:

    load --table Memos
    [
    {"content": "Groonga is fast and embeddable full text search engine."},
    {"content": "PGroonga is a PostgreSQL extension that uses Groonga."},
    {"content": "PostgreSQL is a RDBMS."}
    ]
    # [[0,1337566253.89858,0.000355720520019531],3]

Here is a schema that creates a Generated column that generates embeddings of `Memos.content` automatically:

Execution example:

    column_create \
      --table Memos \
      --name content_embedding \
      --flags COLUMN_VECTOR \
      --type Float32 \
      --source content \
      --generator 'language_model_vectorize("mistral-7b-v0.1.Q4_K_M", content)'
    # [[0,1337566253.89858,0.000355720520019531],true]

You can re-rank matched records by using `distance_inner_product()` not `distance_cosine()` because `language_model_vectorize()` returns a normalized embedding. The following example uses all records instead of filtered records to show this usage simply:

Execution example:

    select \
      --table Memos \
      --columns[similarity].stage filtered \
      --columns[similarity].flags COLUMN_SCALAR \
      --columns[similarity].types Float32 \
      --columns[similarity].value 'distance_inner_product(content_embedding, language_model_vectorize("mistral-7b-v0.1.Q4_K_M", "high performance FTS"))' \
      --output_columns content,similarity \
      --sort_keys -similarity
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         3
    #       ],
    #       [
    #         [
    #           "content",
    #           "ShortText"
    #         ],
    #         [
    #           "similarity",
    #           "Text"
    #         ]
    #       ],
    #       [
    #         "Groonga is fast and embeddable full text search engine.",
    #         "0.6581704020500183"
    #       ],
    #       [
    #         "PGroonga is a PostgreSQL extension that uses Groonga.",
    #         "0.6540993452072144"
    #       ],
    #       [
    #         "PostgreSQL is a RDBMS.",
    #         "0.6449499130249023"
    #       ]
    #     ]
    #   ]
    # ]

## 7.16.17.5. Parameters

There are two required parameters and optional parameters.

### 7.16.17.5.1. Required parameters

#### 7.16.17.5.1.1. `model_name`

`model_name` is the name of language mode to be used. It’s associated with file name. If `${PREFIX}/share/groonga/language_models/mistral-7b-v0.1.Q4_K_M.gguf` exists, you can refer it by `mistral-7b-v0.1.Q4_K_M`. It’s computed by removing directory and `.gguf` extension.

Added in version 15.1.8: You can specify a Hugging Face URI for `model_name`.

When you specify a Hugging Face URI, the model will be automatically downloaded.

The model is downloaded to the directory where the Groonga database files are located.

Example of URI: `hf:///groonga/bge-m3-Q4_K_M-GGUF` for `https://huggingface.co/groonga/bge-m3-Q4_K_M-GGUF`.

Example of function execution:

    language_model_vectorize("hf:///groonga/bge-m3-Q4_K_M-GGUF", content)

#### 7.16.17.5.1.2. `text`

`text` is the input text.

### 7.16.17.5.2. Optional parameters

#### 7.16.17.5.2.1. `options`

Added in version 16.0.1.

`options` can be passed as a JSON object.

##### 7.16.17.5.2.1.1. `prefix`

Added in version 16.0.1.

`prefix` specifies the prefix for `text`.

Some models such as `multilingual-e5` require a prefix for search target text and query text:

- Use `"passage: "` when vectorizing search target text.

- Use `"query: "` when vectorizing query text.

Examples:

    language_model_vectorize("hf:///groonga/multilingual-e5-base-Q4_K_M-GGUF", \
                             content, \
                             {"prefix": "passage: "})

    language_model_vectorize("hf:///groonga/multilingual-e5-base-Q4_K_M-GGUF", \
                             "male child", \
                             {"prefix": "query: "})

## 7.16.17.6. Return value

`language_model_vectorize` returns `Float32` vector which as a normalized embedding.