LLM, Tell Me a Story

So I read an article on running LLMs locally and thought I’d give it a try.

I downloaded a pretty reasonably sized one , then a big one I used for the demo below.

My code:

using LLama;

LLamaModel model = new LLamaModel(
    new LLama.Common.ModelParams("E:\\RND\\ggml-model-q8_0.bin", convertEosToNewLine: true, threads: 3)
);

InteractiveExecutor executor = new InteractiveExecutor(model);

ChatSession chatSession = new ChatSession(executor);

ConsoleColor originalConsoleColor = Console.ForegroundColor;
while (true)
{
    Console.ForegroundColor = ConsoleColor.Green;
    Console.Write("> ");
    string? line = Console.ReadLine();
    if (string.IsNullOrWhiteSpace(line) || line == "exit" || line == "EXIT") break;
    Console.ForegroundColor = ConsoleColor.Yellow;
    var responseEnumerator = chatSession.Chat(line + "\n").GetEnumerator();
    int whitespaceAlready = 0;
    while (responseEnumerator.MoveNext())
    {
        if (string.IsNullOrWhiteSpace(responseEnumerator.Current))
        {
            if (whitespaceAlready == 2) break;
            else whitespaceAlready++;
        }
        else
        {
            whitespaceAlready = 0;
        }
        Console.Write(responseEnumerator.Current);
    }
    Console.WriteLine();
}
Console.ForegroundColor = originalConsoleColor;

After about 3 minutes to load the data, I asked it to tell me a story.  It took about 25 minutes to render but just amazing this story is in the model somewhere.  Here’s the story: