We are adding more integrations regularly. If you are looking for a specific integration, you can request one via our .
In this section, we’ll guide you through integrating ColiVara with LLM frameworks. This integration will allow you to seamlessly connect to other moving parts in your RAG project.
2. Set up ColiVara RAG client and Process your documents
1
Initialize your Colivara RAG client
import base64 # for converting docs binaries to base64
from pathlib import Path
import os
import requests
from colivara_py import ColiVara
rag_client = ColiVara(
base_url="https://api.colivara.com",
api_key=YOUR_COLIVARA_KEY
)
2
(Optional) Download your documents
You can skip this step if your documents are already downloaded into a folder. Here we are downloading documents using their URLs, and save them in the docs/folder
# List of filenames and their URLS
files = [
{
"url": "https://github.com/tjmlabs/colivara-demo/raw/main/docs/Work-From-Home%20Guidance.pdf",
"filename": "docs/Work-From-Home-Guidance.pdf",
},
{
"url": "https://github.com/tjmlabs/colivara-demo/raw/main/docs/StaffVendorPolicy-Jan2019.pdf",
"filename": "docs/StaffVendorPolicy-Jan2019.pdf",
},
]
# Downloading the files into the docs/ folder
def download_file(url, local_filename):
response = requests.get(url)
if response.status_code == 200:
os.makedirs("docs", exist_ok=True)
with open(local_filename, "wb") as f:
f.write(response.content)
print(f"Successfully downloaded: {local_filename}")
else:
print(f"Failed to download: {url}")
for file in files:
download_file(file["url"], file["filename"])
3
Upload and Process the documents
def sync_documents():
documents_dir = Path("docs")
files = list(documents_dir.glob("**/*"))
for file in files:
with open(file, "rb") as f:
file_content = f.read()
encoded_content = base64.b64encode(file_content).decode("utf-8")
rag_client.upsert_document(
name=file.name,
document_base64=encoded_content,
collection_name="openrouter-demo",
wait=True,
)
print(f"Upserted: {file.name}")
sync_documents()
3. Create the Context
For simplicity, we are skipping Query transformation step. In RAG, query transformation converts user queries into the format the model needs. Instead, we are using the query directly.
def get_context(query):
results = rag_client.search(query=query, collection_name="openrouter-demo", top_k=3)
results = results.results
context = []
for result in results:
base64 = result.img_base64
if "data:image" not in base64:
base64 = f"data:image/png;base64,{base64}"
context.append(base64)
return context
query = "What is the work from home policy?"
context = get_context(query)
4. Create a Client with your integrated LLM Framework
Llamaindex
from llama_index.multi_modal_llms.openai import OpenAIMultiModal
from llama_index.core.schema import ImageNode
openai_api_key = YOUR_OPENAI_API_KEY
gpt_4o = OpenAIMultiModal(model="gpt-4o", max_new_tokens=500, api_key=openai_api_key, temperature=0)
Langchain
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI
model = ChatOpenAI(model="gpt-4o", temperature=0)
OpenRouter
No need to create a Client as we will be calling OpenRouter via an HTTP request