Rudolf Šimo

Posted on Apr 25

Implementing Simple RAG in local environment /w .NET (C#).

#rag #tutorial #ollama

This tutorial demonstrates how to build a simple Retrieval-Augmented Generation (RAG) solution using:

Ollama LLM (Gemma 3)
mxbai large embedding model
PostgreSQL with the pgvector extension
Pokémon dataset

Overview

This solution demonstrates how to:

Generate embeddings for Pokémon data using the mxbai large embedding model.
Store embeddings in a PostgreSQL database with pgvector.
Perform similarity searches to retrieve relevant Pokémon data.
Use Ollama LLM to generate responses based on user input and retrieved data.

Prerequisites

Environment Setup:
- Install .NET 8 SDK.
- Install PostgreSQL with the pgvector extension enabled.
- Install Ollama from: https://ollama.com/
- Install LLM model through Ollama, I used gemma3 model: https://ollama.com/library/gemma3
- Install Embedding model through Ollama, I used mxbai-embed-large: https://ollama.com/library/mxbai-embed-large
Dependencies:

Add the following NuGet packages to your project:
- Microsoft.SemanticKernel
- Microsoft.SemanticKernel.Connectors.Postgres
- IdeaTech.SemanticKernel.Connectors.Ollama
- Npgsql
Database Setup:

Ensure your PostgreSQL database is running and has the pgvector extension installed. Use the connection string:

   Host=localhost;Port=5433;Username=postgres;Password=admin;Database=postgres;

Step 1: Define Pokémon Data

Create a Data class to hold Pokémon records and generate sample data.

internal class Data
{
    private static List<Pokemon> myRecords;

    public Data()
    {
        myRecords = new List<Pokemon>();
        CreateRecords();
    }

    private void CreateRecords()
    {
        myRecords = new List<Pokemon>
        {
            new Pokemon { Id = 1, Name = "Bulbasaur", Description = "A Grass/Poison type Pokémon." },
            new Pokemon { Id = 2, Name = "Charmander", Description = "A Fire type Pokémon." },
            new Pokemon { Id = 3, Name = "Squirtle", Description = "A Water type Pokémon." },
            // Add more Pokémon as needed
        };
    }

    public List<Pokemon> RetrieveRecords()
    {
        return myRecords;
    }
}

internal class Pokemon : IEmbeddable
{
    [VectorStoreRecordKey(StoragePropertyName = "id")]
    public long Id { get; set; }

    [VectorStoreRecordData(StoragePropertyName = "name")]
    public string Name { get; set; }

    [VectorStoreRecordData(StoragePropertyName = "description")]
    public string Description { get; set; }

    [VectorStoreRecordVector(Dimensions: 1024, DistanceFunction.CosineDistance, StoragePropertyName = "descriptionembedding")]
    public ReadOnlyMemory<float> DescriptionEmbedding { get; set; }
}

Step 2: Initialize the Database

Create a DatabaseInitializer class to set up the PostgreSQL database.

For the embedding VECTOR(1024) size, in this case 1024, can be found as embedding_length, embedding_size, vector size, output_dim, hidden size...

For mxbai-embed-large Embedding Model, the dimensions were found here:

internal class DatabaseInitializer
{
    private readonly string _connectionString;

    public DatabaseInitializer(string connectionString)
    {
        _connectionString = connectionString;
    }

    public async Task InitializeDatabaseAsync<T>() where T : IEmbeddable
    {
        using var connection = new NpgsqlConnection(_connectionString);
        await connection.OpenAsync();

        string createTableQuery = @"
            CREATE TABLE IF NOT EXISTS pokemons (
                id BIGINT PRIMARY KEY,
                name TEXT,
                description TEXT,
                descriptionembedding VECTOR(1024)
            );";

        using var command = new NpgsqlCommand(createTableQuery, connection);
        await command.ExecuteNonQueryAsync();
    }
}

Step 3: Generate and Store Embeddings

Use the TextEmbedding class to generate embeddings for Pokémon descriptions and store them in the database.

internal class TextEmbedding
{
    public async Task GenerateEmbeddingAndUpsertAsync<T>(
        ITextEmbeddingGenerationService textEmbeddingGenerationService,
        IVectorStoreRecordCollection<long, T> vectorStoreRecordCollection,
        T objectToEmbed) where T : IEmbeddable
    {
        string description = objectToEmbed.Description;

        ReadOnlyMemory<float> searchEmbedding = await textEmbeddingGenerationService.GenerateEmbeddingAsync(description);
        objectToEmbed.DescriptionEmbedding = searchEmbedding;

        await vectorStoreRecordCollection.UpsertAsync(objectToEmbed);
    }
}

Step 4: Perform Similarity Search

Retrieve the most relevant Pokémon data based on user input.

public async Task<string> SearchAsync<T>(
    ITextEmbeddingGenerationService textEmbeddingGenerationService,
    IVectorStoreRecordCollection<long, T> vectorStoreRecordCollection,
    string query) where T : IEmbeddable
{
    ReadOnlyMemory<float> queryEmbedding = await textEmbeddingGenerationService.GenerateEmbeddingAsync(query);
    VectorSearchResults<T> results = await vectorStoreRecordCollection.VectorizedSearchAsync(queryEmbedding);
    var resultItems = await results.Results.ToListAsync();

    var topResults = resultItems.Take(3).ToList();

    StringBuilder resultString = new StringBuilder();
    foreach (var result in topResults)
    {
        if (result.Record is Pokemon pokemon)
        {
            resultString.AppendLine($"{pokemon.Name}: {pokemon.Description}");
        }
    }

    return resultString.ToString();
}

Step 5: Integrate Ollama LLM

Use Ollama LLM to generate responses based on user input and retrieved Pokémon data.

internal class LLMCommunicator
{
    private TextEmbedding myTextEmbedding;
    private Kernel myKernel;
    private PostgresVectorStoreRecordCollection<long, Pokemon> myVectorStoreRecordCollection;

    public LLMCommunicator()
    {
        myKernel = InitializeKernel();
        myTextEmbedding = new TextEmbedding();
    }

    public Kernel InitializeKernel()
    {
        IKernelBuilder kernelBuilder = Kernel.CreateBuilder();
        kernelBuilder.Services
            .AddOllamaChatCompletion("gemma3", new Uri("http://localhost:11434"))
            .AddOllamaTextEmbeddingGeneration("mxbai-embed-large", new Uri("http://localhost:11434"));

        return kernelBuilder.Build();
    }

    public async Task AddRecords()
    {
        var ollamaEmbedding = myKernel.Services.GetService<ITextEmbeddingGenerationService>();
        Data data = new Data();

        var databaseInitializer = new DatabaseInitializer("Host=localhost;Port=5433;Username=postgres;Password=admin;Database=postgres;");
        await databaseInitializer.InitializeDatabaseAsync<Pokemon>();

        NpgsqlDataSourceBuilder dataSourceBuilder = new("Host=localhost;Port=5433;Username=postgres;Password=admin;Database=postgres;");
        dataSourceBuilder.UseVector();
        var dataSource = dataSourceBuilder.Build();
        myVectorStoreRecordCollection = new PostgresVectorStoreRecordCollection<long, Pokemon>(dataSource, "pokemons");

        foreach (var record in data.RetrieveRecords())
        {
            await myTextEmbedding.GenerateEmbeddingAndUpsertAsync(ollamaEmbedding, myVectorStoreRecordCollection, record);
        }
    }

    public async Task Run(string userInput)
    {
        var ollamaEmbedding = myKernel.Services.GetService<ITextEmbeddingGenerationService>();
        string mostSimilarDocument = await myTextEmbedding.SearchAsync(ollamaEmbedding, myVectorStoreRecordCollection, userInput);

        var chatCompletionService = myKernel.Services.GetService<IChatCompletionService>();
        string prompt = $"You are a bot that makes Pokémon recommendations. Recommended Pokémon: {mostSimilarDocument}. User input: {userInput}.";

        var settings = new OllamaPromptExecutionSettings { MaxTokens = 4096 };
        var history = new ChatHistory(prompt);
        var responses = await chatCompletionService.GetChatMessageContentsAsync(history, settings, myKernel);

        foreach (var response in responses)
        {
            Console.WriteLine(response.Content);
        }
    }
}

Step 6: Run the Solution

In your Program.cs, initialize the LLMCommunicator and run the solution.

internal class Program
{
    static async Task Main(string[] args)
    {
        LLMCommunicator communicator = new LLMCommunicator();
        await communicator.AddRecords();

        Console.WriteLine("Enter your query:");
        string userInput = Console.ReadLine();
        await communicator.Run(userInput);
    }
}

Try it

Now try asking: Could you recommend me a Fire type Pokemon? (For the best results, try to create a Non Existing pokémon, e.g. NonExistant /w Description "A Non Existing Pokemon". Then try: Could you recommend me a Non Existing Pokemon?

Conclusion

This tutorial demonstrates how to build a simple RAG solution using Ollama LLM, mxbai embeddings, pgvector, and Pokémon data. You can extend this solution by adding more data, improving the prompt, or integrating additional features.

Sources

Original sample used (Without Embeding model and vector database): https://learnbybuilding.ai/tutorial/rag-from-scratch/

DEV Community