Advanced filtering
ColiVara offers different lookup methods to extract highly relevant data
ColiVara combines the power of Colpali RAG model to retrieve admissible contextual information. An LLM of your choice can then be used to generate answers to your query as a structured data (JSON) format. Additionally, ColiVara offers built-in lookup methods to improve your query.
Colivara offers fine-grained control over data retrieval by implementing various Query Filters, which boosts precision in response generation by filtering by document's metadata (or collection's metadata)
Queries running on document can be filtered by:
Direct Matching (
key_lookup
,has_key
,has_keys
,has_any_keys
): Allows for exact matches, making it ideal for queries where the exact value matters (e.g., retrieving specific data where exact metrics are critical).Partial Containment (
contains
,contained_by
): These enable partial matches, so if a user asks for a general topic (like "budget plans"), the model can pull relevant documents even if they do not perfectly match but contain related content.
Advanced filters can be used on metadata from a single document, or from the entire collection
This leads to higher relevance, as users get nuanced responses tailored to both broad and specific information requests.
How to utilize query filter
To use Advanced Filtering on your documents or collection, pass in a query_filter
parameter to the search
function call.
Available Filters
1. key_lookup
key_lookup
Description: Performs an exact match between a specified metadata key and a value. Useful when a precise key-value pair match is required.
Parameters:
key (str)
: Metadata key to match. Must be a single string.value(str, int, float, bool)
: Exact value for key. Required.
Example Usage:
2. contains
contains
Description: checks if a specified metadata key contains a particular substring or element. Useful for partial matches.
Parameters:
key (str)
: Metadata key in which to search for the substring.value (str)
: Substring to find within the key’s value. Required.
Example Usage:
3. contained_by
contained_by
Description: verifies if a metadata value is fully contained within a provided structure, such as a string or list. Useful for partial matches.
Parameters:
key (str)
: Metadata key to evaluate.value (str, List[str])
: A master string, or a list of string, of which the key's value is a substring. Required.
Example Usage:
4. has_key
has_key
Description: checks if a specified key exists within the metadata, without verifying its value. Useful for filtering documents or collections based on the presence of a particular metadata field.
Parameters:
key (str)
: Metadata key to check for existenceMust not contain a
value
parameter
Validation:
key
must be a string.Must not contain a
value
parameter
Example Usage:
5. has_keys
has_keys
Description: ensures that all specified keys in a list exist within the metadata. Useful for filtering documents or collections based on the presence of a list of metadata fields.
Parameters:
key (List[str])
: Must be a list of strings.Must not contain a
value
parameter
Example Usage:
6. has_any_keys
has_any_keys
Description: checks if at least one of the keys in a specified list exists in the metadata
Parameters:
key (List[str])
: Must be a list of strings.Must not contain a
value
parameter
Example Usage:
Last updated