You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi, I try to perform softmax at the last dimension of a tensor. So I need to permute it first, and perform softmax at every n continguous elements. To permute the tensor, I think the expected code in ggml, should be like auto _tensor = ggml_cont(ctx, ggml_permute(ctx,input, 1,2,0,3));
But I print the content of content of _tensor via float* data = ggml_get_data_f32(input); , the content of _tensor is all zero, later I use ggml_cont_inplace instead of ggml_cont, the data looks the same as in "input", which is not what I expected.
In pytorch, when using contiguous() after permutation, the raw data layout would be changed, that is what I expected. In ggml, while the raw data would not be changed using ggml_cont... So further explaination is wanted about the design intention of ggml_cont,... Furthermore, could someone show how to perform softmax. at a specific dimension of a tensor in ggml ? like in following code dense = np.exp(input) # Softmax. dense = dense / (np.sum(dense, axis=0)+.00001) # Should sum to 1.
The text was updated successfully, but these errors were encountered:
BayRanger
changed the title
Does ggml_cont really works?
Does ggml_cont really work?
Feb 17, 2024
Hi, I try to perform softmax at the last dimension of a tensor. So I need to permute it first, and perform softmax at every n continguous elements. To permute the tensor, I think the expected code in ggml, should be like
auto _tensor = ggml_cont(ctx, ggml_permute(ctx,input, 1,2,0,3));
But I print the content of content of _tensor via
float* data = ggml_get_data_f32(input);
, the content of _tensor is all zero, later I use ggml_cont_inplace instead of ggml_cont, the data looks the same as in "input", which is not what I expected.In pytorch, when using contiguous() after permutation, the raw data layout would be changed, that is what I expected. In ggml, while the raw data would not be changed using ggml_cont... So further explaination is wanted about the design intention of ggml_cont,... Furthermore, could someone show how to perform softmax. at a specific dimension of a tensor in ggml ? like in following code
dense = np.exp(input) # Softmax. dense = dense / (np.sum(dense, axis=0)+.00001) # Should sum to 1.
The text was updated successfully, but these errors were encountered: