Fund Financial Compliance Procedures - Germany

How is the financial data indexed and embeddings created?

• Data is sources from German Commercial Code, Investment Code, Federal Financial Supervisory Authority (Bafin), Expert Commentary and Best-Practise Cases
• The data is preprocessed by normalizing numerical values and identifying key trends and patterns.
• Data is then indexed using LangChain’s RecursiveCharacterTextSplitter with chunk_size as 1000, chunk_overlap as 100 and length_function as len.
• Embeddings are created using OpenAI text-embedding-ada-002 model and have a dimension of 1536.
• These embeddings are then safely stored in a database optimized for high-dimensional data, ensuring efficient storage and retrieval.

How do we keep the data updated?

Our system is integrated with major legal text and blog crawler streams. Every week, a process runs to check for new data or updates to existing data. New and updated data is preprocessed, indexed, and embeddings are generated before it’s stored in our vector database. This ensures our AI embeddings always have the most recent financial market data.

How to use

• Python
• Curl

Python Code

You can access the embedding using EmbedElite python package or curl request.

# this returns a list of embeddings
from embedelite import load_embedding

embeddings = load_embedding("fund-financial-compliance-procedures-germany")
print(len(embeddings))
print(embeddings[1])

# this returns an object which can be directly inserted into Qdrant
result = load_embedding("fund-financial-compliance-procedures-germany", embed_for="qdrant")
# result is {"embeddings": [], "documents": [] "ids": []}
print(result["embeddings"])
print(result["documents"])
print(result["ids"])

Curl

Request

curl -X POST -H "Content-Type: application/json" -d '{
"doc_id": "fund-financial-compliance-procedures-germany"
}' https://api.embedelite.com/v1/embeddings/download/

Response

{
    "mappings": {
        "properties": {
        "doc_source": {"type": "keyword"},
        "sentence": { "type": "text" },
        "embeddings": { "type": "dense_vector", "dims": 1536, "index": False },
        "doc_source": {"type": "keyword"}
        }
    }