AccessViolationException #654
Labels
bug
Something isn't working
Comments
|
What model file are you trying to use? |
|
Open LLama 3b |
|
Depending on the exact version of llama.cpp you used to covnert it, you may be using a slightly incompatible file format. LLamaSharp is always a bit behind mainline llama.cpp. Just to confirm everything else is working I'd suggest trying one of the models linked in the readme (e.g. Phi2). |
|
Did trying another model work? |
|
I suspect the backend binaries do not match the version of the llamasharp library used. I just had the same error message. Version 0.11.02 finally worked together with the binaries you can download here. https://github.com/SciSharp/LLamaSharp/actions/runs/8118890586 |
|
@hswlab If I'm not misunderstanding it, you got an error when using LLamaSharp v0.11.2 and corresponding backend package, but everything goes well with LLamaSharp v0.11.2 and the downloaded binaries? Could you please post your log of native library loading here? To enable the full logs, please add the following code to the very beginning of your code. var showLLamaCppLogs = true;
NativeLibraryConfig
.Instance
.WithLogCallback((level, message) =>
{
if (showLLamaCppLogs)
Console.WriteLine($"[llama {level}]: {message.TrimEnd('\n')}");
}); |
|
I have the same issue on my first start program. Code below: using LLama.Common;
using LLama;
using LLama.Native;
var showLLamaCppLogs = true;
NativeLibraryConfig
.Instance
.WithLogCallback((level, message) =>
{
if (showLLamaCppLogs)
Console.WriteLine($"[llama {level}]: {message.TrimEnd('\n')}");
});
string modelPath = @"C:\Users\1004975\Downloads\Phi-3-mini-4k-instruct-q4.gguf"; // change it to your own model path.
var parameters = new ModelParams(modelPath)
{
ContextSize = 1024, // The longest length of chat as memory.
GpuLayerCount = 1 // How many layers to offload to GPU. Please adjust it according to your GPU memory.
};
using var model = LLamaWeights.LoadFromFile(parameters);
using var context = model.CreateContext(parameters);
var executor = new InteractiveExecutor(context);
// Add chat histories as prompt to tell AI how to act.
var chatHistory = new ChatHistory();
chatHistory.AddMessage(AuthorRole.System, "Transcript of a dialog, where the User interacts with an Assistant named Bob. Bob is helpful, kind, honest, good at writing, and never fails to answer the User's requests immediately and with precision.");
chatHistory.AddMessage(AuthorRole.User, "Hello, Bob.");
chatHistory.AddMessage(AuthorRole.Assistant, "Hello. How may I help you today?");
ChatSession session = new(executor, chatHistory);
InferenceParams inferenceParams = new InferenceParams()
{
MaxTokens = 256, // No more than 256 tokens should appear in answer. Remove it if antiprompt is enough for control.
AntiPrompts = new List<string> { "User:" } // Stop generation once antiprompts appear.
};
Console.ForegroundColor = ConsoleColor.Yellow;
Console.Write("The chat session has started.\nUser: ");
Console.ForegroundColor = ConsoleColor.Green;
string userInput = Console.ReadLine() ?? "";
while (userInput != "exit")
{
await foreach ( // Generate the response streamingly.
var text
in session.ChatAsync(
new ChatHistory.Message(AuthorRole.User, userInput),
inferenceParams))
{
Console.ForegroundColor = ConsoleColor.White;
Console.Write(text);
}
Console.ForegroundColor = ConsoleColor.Green;
userInput = Console.ReadLine() ?? "";
}And the output was: |
|
@AsakusaRinne
|
|
@hswlab So the binary older than 0.11.1 works for you but binary after that doesn't? |
|
I just tried the latest binaries "Update Binaries #37" with v0.11.2 of llamasharp, but with no success. The older binaries "Update Binaries #30" are working fine with v0.11.2 Sometimes I'm not really sure which binaries to choose for which release version of llamasharp in this list. This is usually a try an error for me :) Below is the log I recieved when the following code was called with "Update Binaries #37" |
|
@hswlab What is your CPU on your PC and your windows version? I have no idea about this problem yet but I'll look into that. |
|
I'm Using fully updated Win 10.
Edit: I also tested "Update Binaries #34" just now, this is the last version of the Binaries, which are working fine on my PC. "Update Binaries #35" and upwards are making errors. |
|
Does this problem only appear when you are using cuda backend, or with all backends? |
To answer this specific question, you want to look at this section of the readme. It tells you the exact llama.cpp commit hash to use. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I only copied the code from the ReadMe, I installed the LLama NuGet package with the CPU-Only backend, and it always returns
System.AccessViolationException: "Attempted to read or write protected memory. This is often an indication that other memory is corrupt."
on reading the weights with LLamaWeights.LoadFromFile();
EDIT:
Additional information: Windows 10, .NET 8
The text was updated successfully, but these errors were encountered: