Eval bug: Segmentation fault qwen3 next rocm/hip gfx906

[fuutott]

Name and Version

Model loads fine and warms up, prompt is processed but after few tokens of response segentation fault and core is dumped
works fine on cpu, tried different batch sizes but no change

$ llama-cli --version
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon Graphics, gfx906:sramecc+:xnack- (0x906), VMM: no, Wave Size: 64
Device 1: AMD Radeon Graphics, gfx906:sramecc+:xnack- (0x906), VMM: no, Wave Size: 64
version: 7193 (d82b7a7)
built with cc (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0 for x86_64-linux-gnu

HIP version: 7.1.25424-4179531dcd

Operating systems

Linux

GGML backends

HIP

Hardware

xeon 4216r + 2x amd MI50 32gb

Models

qwen3 next unsloth q_4_xl

Problem description & steps to reproduce

run it, ask it something, dies
cat chatqn.sh
HIP_VISIBLE_DEVICES=0,1 llama-cli -hf unsloth/Qwen3-Next-80B-A3B-Instruct-GGUF:Q4_0
--gpu-layers 999
-t 8
-b 256
-ub 128
--ctx-size 32000
--flash-attn on
--jinja
--no-mmap
--cache-type-k q8_0
--cache-type-v q8_0

First Bad Commit

No response

Relevant log output

x@noisia:~$ ./chatqn.sh
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
  Device 0: AMD Radeon Graphics, gfx906:sramecc+:xnack- (0x906), VMM: no, Wave Size: 64
  Device 1: AMD Radeon Graphics, gfx906:sramecc+:xnack- (0x906), VMM: no, Wave Size: 64
* Host huggingface.co:443 was resolved.
* IPv6: 2600:9000:28fd:8a00:17:b174:6d00:93a1, 2600:9000:28fd:de00:17:b174:6d00:93a1, 2600:9000:28fd:d800:17:b174:6d00:93a1, 2600:9000:28fd:1400:17:b174:6d00:93a1, 2600:9000:28fd:4200:17:b174:6d00:93a1, 2600:9000:28fd:8400:17:b174:6d00:93a1, 2600:9000:28fd:fa00:17:b174:6d00:93a1, 2600:9000:28fd:dc00:17:b174:6d00:93a1
* IPv4: 3.174.141.117, 3.174.141.51, 3.174.141.63, 3.174.141.96
*   Trying [2600:9000:28fd:8a00:17:b174:6d00:93a1]:443...
* Connected to huggingface.co (2600:9000:28fd:8a00:17:b174:6d00:93a1) port 443
* ALPN: curl offers h2,http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256 / X25519 / RSASSA-PSS
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=huggingface.co
*  start date: Apr 13 00:00:00 2025 GMT
*  expire date: May 12 23:59:59 2026 GMT
*  subjectAltName: host "huggingface.co" matched cert's "huggingface.co"
*  issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M02
*  SSL certificate verify ok.
*   Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://huggingface.co/v2/unsloth/Qwen3-Next-80B-A3B-Instruct-GGUF/manifests/Q4_0
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: huggingface.co]
* [HTTP/2] [1] [:path: /v2/unsloth/Qwen3-Next-80B-A3B-Instruct-GGUF/manifests/Q4_0]
* [HTTP/2] [1] [user-agent: llama-cpp]
* [HTTP/2] [1] [accept: application/json]
> GET /v2/unsloth/Qwen3-Next-80B-A3B-Instruct-GGUF/manifests/Q4_0 HTTP/2
Host: huggingface.co
User-Agent: llama-cpp
Accept: application/json

< HTTP/2 200
< content-type: application/json; charset=utf-8
< content-length: 998
< date: Sat, 29 Nov 2025 00:41:14 GMT
< etag: W/"3e6-Dpob+BD5jYUUsRGniI5cv5ntgDA"
< x-powered-by: huggingface-moon
< x-request-id: Root=1-692a4129-389342695341d06100692441
< ratelimit: "pages";r=98;t=175
< ratelimit-policy: "fixed window";"pages";q=100;w=300
< cross-origin-opener-policy: same-origin
< referrer-policy: strict-origin-when-cross-origin
< access-control-max-age: 86400
< access-control-allow-origin: https://huggingface.co
< vary: Origin
< access-control-expose-headers: X-Repo-Commit,X-Request-Id,X-Error-Code,X-Error-Message,X-Total-Count,ETag,Link,Accept-Ranges,Content-Range,X-Linked-Size,X-Linked-ETag,X-Xet-Hash
< x-cache: Miss from cloudfront
< via: 1.1 d34a2a1d2fc10c8d442572df271a2cc2.cloudfront.net (CloudFront)
< x-amz-cf-pop: LHR3-P3
< x-amz-cf-id: 6Skwbe6-Rf9qTBSufUEtJjxsJMpjIfNiRz4dtRCGuK66CdrDa_Kktg==
<
* Connection #0 to host huggingface.co left intact
common_download_file_single_online: using cached file: /home/[---]/.cache/llama.cpp/unsloth_Qwen3-Next-80B-A3B-Instruct-GGUF_Qwen3-Next-80B-A3B-Instruct-Q4_0.gguf
build: 7193 (d82b7a7c1) with cc (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0 for x86_64-linux-gnu
main: llama backend init
main: load the model and apply lora adapter, if any
llama_model_load_from_file_impl: using device ROCm0 (AMD Radeon Graphics) (0000:67:00.0) - 32730 MiB free
llama_model_load_from_file_impl: using device ROCm1 (AMD Radeon Graphics) (0000:b5:00.0) - 32730 MiB free
llama_model_loader: loaded meta data with 49 key-value pairs and 807 tensors from /home/[---]/.cache/llama.cpp/unsloth_Qwen3-Next-80B-A3B-Instruct-GGUF_Qwen3-Next-80B-A3B-Instruct-Q4_0.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = qwen3next
llama_model_loader: - kv   1:                               general.type str              = model
llama_model_loader: - kv   2:                     general.sampling.top_k i32              = 20
llama_model_loader: - kv   3:                     general.sampling.top_p f32              = 0.800000
llama_model_loader: - kv   4:                      general.sampling.temp f32              = 0.700000
llama_model_loader: - kv   5:                               general.name str              = Qwen3-Next-80B-A3B-Instruct
llama_model_loader: - kv   6:                           general.finetune str              = Instruct
llama_model_loader: - kv   7:                           general.basename str              = Qwen3-Next-80B-A3B-Instruct
llama_model_loader: - kv   8:                       general.quantized_by str              = Unsloth
llama_model_loader: - kv   9:                         general.size_label str              = 80B-A3B
llama_model_loader: - kv  10:                            general.license str              = apache-2.0
llama_model_loader: - kv  11:                       general.license.link str              = https://huggingface.co/Qwen/Qwen3-Nex...
llama_model_loader: - kv  12:                           general.repo_url str              = https://huggingface.co/unsloth
llama_model_loader: - kv  13:                   general.base_model.count u32              = 1
llama_model_loader: - kv  14:                  general.base_model.0.name str              = Qwen3 Next 80B A3B Instruct
llama_model_loader: - kv  15:          general.base_model.0.organization str              = Qwen
llama_model_loader: - kv  16:              general.base_model.0.repo_url str              = https://huggingface.co/Qwen/Qwen3-Nex...
llama_model_loader: - kv  17:                               general.tags arr[str,2]       = ["unsloth", "text-generation"]
llama_model_loader: - kv  18:                      qwen3next.block_count u32              = 48
llama_model_loader: - kv  19:                   qwen3next.context_length u32              = 262144
llama_model_loader: - kv  20:                 qwen3next.embedding_length u32              = 2048
llama_model_loader: - kv  21:              qwen3next.feed_forward_length u32              = 5120
llama_model_loader: - kv  22:             qwen3next.attention.head_count u32              = 16
llama_model_loader: - kv  23:          qwen3next.attention.head_count_kv u32              = 2
llama_model_loader: - kv  24:                   qwen3next.rope.freq_base f32              = 10000000.000000
llama_model_loader: - kv  25: qwen3next.attention.layer_norm_rms_epsilon f32              = 0.000001
llama_model_loader: - kv  26:                qwen3next.expert_used_count u32              = 10
llama_model_loader: - kv  27:             qwen3next.attention.key_length u32              = 256
llama_model_loader: - kv  28:           qwen3next.attention.value_length u32              = 256
llama_model_loader: - kv  29:                     qwen3next.expert_count u32              = 512
llama_model_loader: - kv  30:       qwen3next.expert_feed_forward_length u32              = 512
llama_model_loader: - kv  31: qwen3next.expert_shared_feed_forward_length u32              = 512
llama_model_loader: - kv  32:                  qwen3next.ssm.conv_kernel u32              = 4
llama_model_loader: - kv  33:                   qwen3next.ssm.state_size u32              = 128
llama_model_loader: - kv  34:                  qwen3next.ssm.group_count u32              = 16
llama_model_loader: - kv  35:               qwen3next.ssm.time_step_rank u32              = 32
llama_model_loader: - kv  36:                   qwen3next.ssm.inner_size u32              = 4096
llama_model_loader: - kv  37:             qwen3next.rope.dimension_count u32              = 64
llama_model_loader: - kv  38:                       tokenizer.ggml.model str              = gpt2
llama_model_loader: - kv  39:                         tokenizer.ggml.pre str              = qwen2
llama_model_loader: - kv  40:                      tokenizer.ggml.tokens arr[str,151936]  = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv  41:                  tokenizer.ggml.token_type arr[i32,151936]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv  42:                      tokenizer.ggml.merges arr[str,151387]  = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
llama_model_loader: - kv  43:                tokenizer.ggml.eos_token_id u32              = 151645
llama_model_loader: - kv  44:            tokenizer.ggml.padding_token_id u32              = 151654
llama_model_loader: - kv  45:               tokenizer.ggml.add_bos_token bool             = false
llama_model_loader: - kv  46:                    tokenizer.chat_template str              = {%- if tools %}\n    {{- '<|im_start|>...
llama_model_loader: - kv  47:               general.quantization_version u32              = 2
llama_model_loader: - kv  48:                          general.file_type u32              = 2
llama_model_loader: - type  f32:  313 tensors
llama_model_loader: - type q4_0:  301 tensors
llama_model_loader: - type q5_K:   96 tensors
llama_model_loader: - type q6_K:   49 tensors
llama_model_loader: - type bf16:   48 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type   = Q4_0
print_info: file size   = 42.00 GiB (4.53 BPW)
load: printing all EOG tokens:
load:   - 151643 ('<|endoftext|>')
load:   - 151645 ('<|im_end|>')
load:   - 151662 ('<|fim_pad|>')
load:   - 151663 ('<|repo_name|>')
load:   - 151664 ('<|file_sep|>')
load: special tokens cache size = 26
load: token to piece cache size = 0.9311 MB
print_info: arch             = qwen3next
print_info: vocab_only       = 0
print_info: n_ctx_train      = 262144
print_info: n_embd           = 2048
print_info: n_embd_inp       = 2048
print_info: n_layer          = 48
print_info: n_head           = 16
print_info: n_head_kv        = 2
print_info: n_rot            = 64
print_info: n_swa            = 0
print_info: is_swa_any       = 0
print_info: n_embd_head_k    = 256
print_info: n_embd_head_v    = 256
print_info: n_gqa            = 8
print_info: n_embd_k_gqa     = 512
print_info: n_embd_v_gqa     = 512
print_info: f_norm_eps       = 0.0e+00
print_info: f_norm_rms_eps   = 1.0e-06
print_info: f_clamp_kqv      = 0.0e+00
print_info: f_max_alibi_bias = 0.0e+00
print_info: f_logit_scale    = 0.0e+00
print_info: f_attn_scale     = 0.0e+00
print_info: n_ff             = 5120
print_info: n_expert         = 512
print_info: n_expert_used    = 10
print_info: n_expert_groups  = 0
print_info: n_group_used     = 0
print_info: causal attn      = 1
print_info: pooling type     = 0
print_info: rope type        = 2
print_info: rope scaling     = linear
print_info: freq_base_train  = 10000000.0
print_info: freq_scale_train = 1
print_info: n_ctx_orig_yarn  = 262144
print_info: rope_finetuned   = unknown
print_info: ssm_d_conv       = 4
print_info: ssm_d_inner      = 4096
print_info: ssm_d_state      = 128
print_info: ssm_dt_rank      = 32
print_info: ssm_n_group      = 16
print_info: ssm_dt_b_c_rms   = 0
print_info: model type       = ?B
print_info: model params     = 79.67 B
print_info: general.name     = Qwen3-Next-80B-A3B-Instruct
print_info: vocab type       = BPE
print_info: n_vocab          = 151936
print_info: n_merges         = 151387
print_info: BOS token        = 11 ','
print_info: EOS token        = 151645 '<|im_end|>'
print_info: EOT token        = 151645 '<|im_end|>'
print_info: PAD token        = 151654 '<|vision_pad|>'
print_info: LF token         = 198 'Ċ'
print_info: FIM PRE token    = 151659 '<|fim_prefix|>'
print_info: FIM SUF token    = 151661 '<|fim_suffix|>'
print_info: FIM MID token    = 151660 '<|fim_middle|>'
print_info: FIM PAD token    = 151662 '<|fim_pad|>'
print_info: FIM REP token    = 151663 '<|repo_name|>'
print_info: FIM SEP token    = 151664 '<|file_sep|>'
print_info: EOG token        = 151643 '<|endoftext|>'
print_info: EOG token        = 151645 '<|im_end|>'
print_info: EOG token        = 151662 '<|fim_pad|>'
print_info: EOG token        = 151663 '<|repo_name|>'
print_info: EOG token        = 151664 '<|file_sep|>'
print_info: max token length = 256
load_tensors: loading model tensors, this can take a while... (mmap = false)
load_tensors: offloading 48 repeating layers to GPU
load_tensors: offloading output layer to GPU
load_tensors: offloaded 49/49 layers to GPU
load_tensors:          CPU model buffer size =     0.00 MiB
load_tensors:        ROCm0 model buffer size = 22188.86 MiB
load_tensors:        ROCm1 model buffer size = 20655.47 MiB
load_tensors:    ROCm_Host model buffer size =   166.92 MiB
....................................................................................................
llama_context: constructing llama_context
llama_context: n_seq_max     = 1
llama_context: n_ctx         = 32000
llama_context: n_ctx_seq     = 32000
llama_context: n_batch       = 256
llama_context: n_ubatch      = 128
llama_context: causal_attn   = 1
llama_context: flash_attn    = enabled
llama_context: kv_unified    = false
llama_context: freq_base     = 10000000.0
llama_context: freq_scale    = 1
llama_context: n_ctx_seq (32000) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
llama_context:  ROCm_Host  output buffer size =     0.58 MiB
llama_kv_cache:      ROCm0 KV buffer size =   199.22 MiB
llama_kv_cache:      ROCm1 KV buffer size =   199.22 MiB
llama_kv_cache: size =  398.44 MiB ( 32000 cells,  12 layers,  1/1 seqs), K (q8_0):  199.22 MiB, V (q8_0):  199.22 MiB
llama_memory_recurrent:      ROCm0 RS buffer size =    39.78 MiB
llama_memory_recurrent:      ROCm1 RS buffer size =    35.59 MiB
llama_memory_recurrent: size =   75.38 MiB (     1 cells,  48 layers,  1 seqs), R (f32):    3.38 MiB, S (f32):   72.00 MiB
llama_context: pipeline parallelism enabled (n_copies=4)
llama_context:      ROCm0 compute buffer size =  1288.61 MiB
llama_context:      ROCm1 compute buffer size =   782.39 MiB
llama_context:  ROCm_Host compute buffer size =   353.90 MiB
llama_context: graph nodes  = 11401 (with bs=128), 8449 (with bs=1)
llama_context: graph splits = 204 (with bs=128), 151 (with bs=1)
common_init_from_params: added <|endoftext|> logit bias = -inf
common_init_from_params: added <|im_end|> logit bias = -inf
common_init_from_params: added <|fim_pad|> logit bias = -inf
common_init_from_params: added <|repo_name|> logit bias = -inf
common_init_from_params: added <|file_sep|> logit bias = -inf
common_init_from_params: setting dry_penalty_last_n to ctx_size = 32000
common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
main: llama threadpool init, n_threads = 8
main: chat template is available, enabling conversation mode (disable it with -no-cnv)
main: chat template example:
<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
Hello<|im_end|>
<|im_start|>assistant
Hi there<|im_end|>
<|im_start|>user
How are you?<|im_end|>
<|im_start|>assistant


system_info: n_threads = 8 (n_threads_batch = 8) / 16 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VNNI = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |

main: interactive mode on.
sampler seed: 419957296
sampler params:
        repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
        dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 32000
        top_k = 20, top_p = 0.800, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.700
        mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000
sampler chain: logits -> logit-bias -> penalties -> dry -> top-n-sigma -> top-k -> typical -> top-p -> min-p -> xtc -> temp-ext -> dist
generate: n_ctx = 32000, n_batch = 256, n_predict = -1, n_keep = 0

== Running in interactive mode. ==
 - Press Ctrl+C to interject at any time.
 - Press Return to return control to the AI.
 - To return control without starting a new line, end your input with '/'.
 - If you want to submit another line, end your input with '\'.
 - Not using system message. To change it, set a different value via -sys PROMPT


> Who was copernicus?
**./chatqn.sh: line 11:  7672 Segmentation fault      (core dumped) HIP_VISIBLE_DEVICES=0,1 llama-cli -hf unsloth/Qwen3-Next-80B-A3B-Instruct-GGUF:Q4_0 --gpu-layers 999 -t 8 -b 256 -ub 128 --ctx-size 32000 --flash-attn on --jinja --no-mmap --cache-type-k q8_0 --cache-type-v q8_0

[pwilkin]

Please recompile with debug mode and provide a stacktrace of the crash.

[fuutott]

[---]@[...]:~$ cat qwendbg.sh
#!/usr/bin/env bash
set -e

HIP_VISIBLE_DEVICES=0,1 gdb --args \
  llama-cli -hf unsloth/Qwen3-Next-80B-A3B-Instruct-GGUF:Q4_0 \
    --gpu-layers 999 \
    -t 8 \
    -b 256 \
    -ub 128 \
    --ctx-size 32000 \
    --flash-attn on \
    --jinja \
    --no-mmap \
    --cache-type-k q8_0 \
    --cache-type-v q8_0
[---]@[...]:~$ ./qwendbg.sh
GNU gdb (Ubuntu 15.0.50.20240403-0ubuntu1) 15.0.50.20240403-git
Copyright (C) 2024 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from llama-cli...
(gdb) run
Starting program: /usr/local/bin/llama-cli -hf unsloth/Qwen3-Next-80B-A3B-Instruct-GGUF:Q4_0 --gpu-layers 999 -t 8 -b 256 -ub 128 --ctx-size 32000 --flash-attn on --jinja --no-mmap --cache-type-k q8_0 --cache-type-v q8_0

This GDB supports auto-downloading debuginfo from the following URLs:
  <https://debuginfod.ubuntu.com>
Enable debuginfod for this session? (y or [n]) y
Debuginfod has been enabled.
To make this setting permanent, add 'set debuginfod enabled on' to .gdbinit.
Downloading separate debug info for system-supplied DSO at 0x7ffff7fc3000
Downloading separate debug info for /lib/x86_64-linux-gnu/libcurl.so.4
Downloading separate debug info for /lib/x86_64-linux-gnu/libstdc++.so.6
Downloading separate debug info for /lib/x86_64-linux-gnu/libgcc_s.so.1
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Downloading separate debug info for /lib/x86_64-linux-gnu/libgomp.so.1
Downloading separate debug info for /lib/x86_64-linux-gnu/libnghttp2.so.14
Downloading separate debug info for /lib/x86_64-linux-gnu/libidn2.so.0
Downloading separate debug info for /lib/x86_64-linux-gnu/librtmp.so.1
Downloading separate debug info for /lib/x86_64-linux-gnu/libssh.so.4
Downloading separate debug info for /lib/x86_64-linux-gnu/libpsl.so.5
Downloading separate debug info for /lib/x86_64-linux-gnu/libssl.so.3
Downloading separate debug info for /lib/x86_64-linux-gnu/libcrypto.so.3
Downloading separate debug info for /lib/x86_64-linux-gnu/libgssapi_krb5.so.2
Downloading separate debug info for /lib/x86_64-linux-gnu/libldap.so.2
Downloading separate debug info for /lib/x86_64-linux-gnu/liblber.so.2
Downloading separate debug info for /lib/x86_64-linux-gnu/liblber.so.2
Downloading separate debug info for /lib/x86_64-linux-gnu/libzstd.so.1
Downloading separate debug info for /lib/x86_64-linux-gnu/libbrotlidec.so.1
Downloading separate debug info for /lib/x86_64-linux-gnu/libbrotlidec.so.1
Downloading separate debug info for /lib/x86_64-linux-gnu/libz.so.1
Downloading separate debug info for /opt/rocm-7.1.0/lib/libhipblas.so.3
Downloading separate debug info for /opt/rocm-7.1.0/lib/librocblas.so.5
Downloading separate debug info for /opt/rocm-7.1.0/lib/libamdhip64.so.7
Downloading separate debug info for /lib/x86_64-linux-gnu/libunistring.so.5
Downloading separate debug info for /lib/x86_64-linux-gnu/libgnutls.so.30
Downloading separate debug info for /lib/x86_64-linux-gnu/libhogweed.so.6
Downloading separate debug info for /lib/x86_64-linux-gnu/libnettle.so.8
Downloading separate debug info for /lib/x86_64-linux-gnu/libgmp.so.10
Downloading separate debug info for /lib/x86_64-linux-gnu/libkrb5.so.3
Downloading separate debug info for /lib/x86_64-linux-gnu/libk5crypto.so.3
Downloading separate debug info for /lib/x86_64-linux-gnu/libcom_err.so.2
Downloading separate debug info for /lib/x86_64-linux-gnu/libkrb5support.so.0
Downloading separate debug info for /lib/x86_64-linux-gnu/libsasl2.so.2
Downloading separate debug info for /lib/x86_64-linux-gnu/libbrotlicommon.so.1
Downloading separate debug info for /opt/rocm-7.1.0/lib/librocsolver.so.0
Downloading separate debug info for /opt/rocm-7.1.0/lib/libhipblaslt.so.1
Downloading separate debug info for /opt/rocm-7.1.0/lib/librocprofiler-register.so.0
Downloading separate debug info for /opt/rocm-7.1.0/lib/libhsa-runtime64.so.1
Downloading separate debug info for /lib/x86_64-linux-gnu/libnuma.so.1
Downloading separate debug info for /lib/x86_64-linux-gnu/libp11-kit.so.0
Downloading separate debug info for /lib/x86_64-linux-gnu/libtasn1.so.6
Downloading separate debug info for /lib/x86_64-linux-gnu/libkeyutils.so.1
Downloading separate debug info for /opt/rocm-7.1.0/lib/libroctx64.so.4
Downloading separate debug info for /lib/x86_64-linux-gnu/libelf.so.1
Downloading separate debug info for /opt/amdgpu/lib/x86_64-linux-gnu/libdrm.so.2
Downloading separate debug info for /opt/amdgpu/lib/x86_64-linux-gnu/libdrm_amdgpu.so.1
Downloading separate debug info for /lib/x86_64-linux-gnu/libffi.so.8
Downloading separate debug info for /opt/rocm-7.1.0/lib/../lib/libamd_comgr.so.3
[New Thread 0x7fffb1bff6c0 (LWP 15637)]
[New Thread 0x7fffb13fe6c0 (LWP 15638)]
[Thread 0x7fffb13fe6c0 (LWP 15638) exited]
Downloading separate debug info for /opt/rocm-7.1.0/lib/libhsa-amd-aqlprofile64.so
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
  Device 0: AMD Radeon Graphics, gfx906:sramecc+:xnack- (0x906), VMM: no, Wave Size: 64
  Device 1: AMD Radeon Graphics, gfx906:sramecc+:xnack- (0x906), VMM: no, Wave Size: 64
register_backend: registered backend ROCm (2 devices)
register_device: registered device ROCm0 (AMD Radeon Graphics)
register_device: registered device ROCm1 (AMD Radeon Graphics)
register_backend: registered backend CPU (1 devices)
register_device: registered device CPU (Intel(R) Xeon(R) Silver 4215R CPU @ 3.20GHz)
load_backend: failed to find ggml_backend_init in /usr/local/bin/libggml-hip.so
Downloading separate debug info for /usr/local/bin/libggml-cpu.so
load_backend: failed to find ggml_backend_init in /usr/local/bin/libggml-cpu.so
[New Thread 0x7ffdabbff6c0 (LWP 15639)]
* Host huggingface.co:443 was resolved.
* IPv6: 2600:9000:28fd:4a00:17:b174:6d00:93a1, 2600:9000:28fd:7200:17:b174:6d00:93a1, 2600:9000:28fd:7400:17:b174:6d00:93a1, 2600:9000:28fd:7600:17:b174:6d00:93a1, 2600:9000:28fd:4800:17:b174:6d00:93a1, 2600:9000:28fd:a800:17:b174:6d00:93a1, 2600:9000:28fd:8a00:17:b174:6d00:93a1, 2600:9000:28fd:4c00:17:b174:6d00:93a1
* IPv4: 3.174.141.96, 3.174.141.63, 3.174.141.51, 3.174.141.117
*   Trying [2600:9000:28fd:4a00:17:b174:6d00:93a1]:443...
[Thread 0x7ffdabbff6c0 (LWP 15639) exited]
* Connected to huggingface.co (2600:9000:28fd:4a00:17:b174:6d00:93a1) port 443
* ALPN: curl offers h2,http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256 / X25519 / RSASSA-PSS
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=huggingface.co
*  start date: Apr 13 00:00:00 2025 GMT
*  expire date: May 12 23:59:59 2026 GMT
*  subjectAltName: host "huggingface.co" matched cert's "huggingface.co"
*  issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M02
*  SSL certificate verify ok.
*   Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://huggingface.co/v2/unsloth/Qwen3-Next-80B-A3B-Instruct-GGUF/manifests/Q4_0
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: huggingface.co]
* [HTTP/2] [1] [:path: /v2/unsloth/Qwen3-Next-80B-A3B-Instruct-GGUF/manifests/Q4_0]
* [HTTP/2] [1] [user-agent: llama-cpp]
* [HTTP/2] [1] [accept: application/json]
> GET /v2/unsloth/Qwen3-Next-80B-A3B-Instruct-GGUF/manifests/Q4_0 HTTP/2
Host: huggingface.co
User-Agent: llama-cpp
Accept: application/json

< HTTP/2 200
< content-type: application/json; charset=utf-8
< content-length: 998
< date: Sat, 29 Nov 2025 13:01:15 GMT
< etag: W/"3e6-Dpob+BD5jYUUsRGniI5cv5ntgDA"
< x-powered-by: huggingface-moon
< x-request-id: Root=1-692aee9b-12b118a56c5b9014797da449
< ratelimit: "pages";r=99;t=173
< ratelimit-policy: "fixed window";"pages";q=100;w=300
< cross-origin-opener-policy: same-origin
< referrer-policy: strict-origin-when-cross-origin
< access-control-max-age: 86400
< access-control-allow-origin: https://huggingface.co
< vary: Origin
< access-control-expose-headers: X-Repo-Commit,X-Request-Id,X-Error-Code,X-Error-Message,X-Total-Count,ETag,Link,Accept-Ranges,Content-Range,X-Linked-Size,X-Linked-ETag,X-Xet-Hash
< x-cache: Miss from cloudfront
< via: 1.1 d34a2a1d2fc10c8d442572df271a2cc2.cloudfront.net (CloudFront)
< x-amz-cf-pop: LHR3-P3
< x-amz-cf-id: nCr5xU4e-V9ASX2vAWgDObi_z1jOMlVgt5KkWvfyyDR_JIqV54PDTw==
<
* Connection #0 to host huggingface.co left intact
[New Thread 0x7ffdabbff6c0 (LWP 15640)]
[Thread 0x7ffdabbff6c0 (LWP 15640) exited]
[New Thread 0x7ffdabbff6c0 (LWP 15641)]
[Thread 0x7ffdabbff6c0 (LWP 15641) exited]
[New Thread 0x7ffdabbff6c0 (LWP 15642)]
[Thread 0x7ffdabbff6c0 (LWP 15642) exited]
[New Thread 0x7ffdabbff6c0 (LWP 15643)]
common_download_file_single_online: using cached file: /home/[---]/.cache/llama.cpp/unsloth_Qwen3-Next-80B-A3B-Instruct-GGUF_Qwen3-Next-80B-A3B-Instruct-Q4_0.gguf
build: 7195 (47a268ea5) with cc (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0 for x86_64-linux-gnu (debug)
main: llama backend init
main: load the model and apply lora adapter, if any
llama_model_load_from_file_impl: using device ROCm0 (AMD Radeon Graphics) (0000:67:00.0) - 32730 MiB free
llama_model_load_from_file_impl: using device ROCm1 (AMD Radeon Graphics) (0000:b5:00.0) - 32730 MiB free
llama_model_loader: loaded meta data with 49 key-value pairs and 807 tensors from /home/[---]/.cache/llama.cpp/unsloth_Qwen3-Next-80B-A3B-Instruct-GGUF_Qwen3-Next-80B-A3B-Instruct-Q4_0.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = qwen3next
llama_model_loader: - kv   1:                               general.type str              = model
llama_model_loader: - kv   2:                     general.sampling.top_k i32              = 20
llama_model_loader: - kv   3:                     general.sampling.top_p f32              = 0.800000
llama_model_loader: - kv   4:                      general.sampling.temp f32              = 0.700000
llama_model_loader: - kv   5:                               general.name str              = Qwen3-Next-80B-A3B-Instruct
llama_model_loader: - kv   6:                           general.finetune str              = Instruct
llama_model_loader: - kv   7:                           general.basename str              = Qwen3-Next-80B-A3B-Instruct
llama_model_loader: - kv   8:                       general.quantized_by str              = Unsloth
llama_model_loader: - kv   9:                         general.size_label str              = 80B-A3B
llama_model_loader: - kv  10:                            general.license str              = apache-2.0
llama_model_loader: - kv  11:                       general.license.link str              = https://huggingface.co/Qwen/Qwen3-Nex...
llama_model_loader: - kv  12:                           general.repo_url str              = https://huggingface.co/unsloth
llama_model_loader: - kv  13:                   general.base_model.count u32              = 1
llama_model_loader: - kv  14:                  general.base_model.0.name str              = Qwen3 Next 80B A3B Instruct
llama_model_loader: - kv  15:          general.base_model.0.organization str              = Qwen
llama_model_loader: - kv  16:              general.base_model.0.repo_url str              = https://huggingface.co/Qwen/Qwen3-Nex...
llama_model_loader: - kv  17:                               general.tags arr[str,2]       = ["unsloth", "text-generation"]
llama_model_loader: - kv  18:                      qwen3next.block_count u32              = 48
llama_model_loader: - kv  19:                   qwen3next.context_length u32              = 262144
llama_model_loader: - kv  20:                 qwen3next.embedding_length u32              = 2048
llama_model_loader: - kv  21:              qwen3next.feed_forward_length u32              = 5120
llama_model_loader: - kv  22:             qwen3next.attention.head_count u32              = 16
llama_model_loader: - kv  23:          qwen3next.attention.head_count_kv u32              = 2
llama_model_loader: - kv  24:                   qwen3next.rope.freq_base f32              = 10000000.000000
llama_model_loader: - kv  25: qwen3next.attention.layer_norm_rms_epsilon f32              = 0.000001
llama_model_loader: - kv  26:                qwen3next.expert_used_count u32              = 10
llama_model_loader: - kv  27:             qwen3next.attention.key_length u32              = 256
llama_model_loader: - kv  28:           qwen3next.attention.value_length u32              = 256
llama_model_loader: - kv  29:                     qwen3next.expert_count u32              = 512
llama_model_loader: - kv  30:       qwen3next.expert_feed_forward_length u32              = 512
llama_model_loader: - kv  31: qwen3next.expert_shared_feed_forward_length u32              = 512
llama_model_loader: - kv  32:                  qwen3next.ssm.conv_kernel u32              = 4
llama_model_loader: - kv  33:                   qwen3next.ssm.state_size u32              = 128
llama_model_loader: - kv  34:                  qwen3next.ssm.group_count u32              = 16
llama_model_loader: - kv  35:               qwen3next.ssm.time_step_rank u32              = 32
llama_model_loader: - kv  36:                   qwen3next.ssm.inner_size u32              = 4096
llama_model_loader: - kv  37:             qwen3next.rope.dimension_count u32              = 64
llama_model_loader: - kv  38:                       tokenizer.ggml.model str              = gpt2
llama_model_loader: - kv  39:                         tokenizer.ggml.pre str              = qwen2
llama_model_loader: - kv  40:                      tokenizer.ggml.tokens arr[str,151936]  = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv  41:                  tokenizer.ggml.token_type arr[i32,151936]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv  42:                      tokenizer.ggml.merges arr[str,151387]  = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
llama_model_loader: - kv  43:                tokenizer.ggml.eos_token_id u32              = 151645
llama_model_loader: - kv  44:            tokenizer.ggml.padding_token_id u32              = 151654
llama_model_loader: - kv  45:               tokenizer.ggml.add_bos_token bool             = false
llama_model_loader: - kv  46:                    tokenizer.chat_template str              = {%- if tools %}\n    {{- '<|im_start|>...
llama_model_loader: - kv  47:               general.quantization_version u32              = 2
llama_model_loader: - kv  48:                          general.file_type u32              = 2
llama_model_loader: - type  f32:  313 tensors
llama_model_loader: - type q4_0:  301 tensors
llama_model_loader: - type q5_K:   96 tensors
llama_model_loader: - type q6_K:   49 tensors
llama_model_loader: - type bf16:   48 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type   = Q4_0
print_info: file size   = 42.00 GiB (4.53 BPW)
load: printing all EOG tokens:
load:   - 151643 ('<|endoftext|>')
load:   - 151645 ('<|im_end|>')
load:   - 151662 ('<|fim_pad|>')
load:   - 151663 ('<|repo_name|>')
load:   - 151664 ('<|file_sep|>')
load: special tokens cache size = 26
load: token to piece cache size = 0.9311 MB
print_info: arch             = qwen3next
print_info: vocab_only       = 0
print_info: n_ctx_train      = 262144
print_info: n_embd           = 2048
print_info: n_embd_inp       = 2048
print_info: n_layer          = 48
print_info: n_head           = 16
print_info: n_head_kv        = 2
print_info: n_rot            = 64
print_info: n_swa            = 0
print_info: is_swa_any       = 0
print_info: n_embd_head_k    = 256
print_info: n_embd_head_v    = 256
print_info: n_gqa            = 8
print_info: n_embd_k_gqa     = 512
print_info: n_embd_v_gqa     = 512
print_info: f_norm_eps       = 0.0e+00
print_info: f_norm_rms_eps   = 1.0e-06
print_info: f_clamp_kqv      = 0.0e+00
print_info: f_max_alibi_bias = 0.0e+00
print_info: f_logit_scale    = 0.0e+00
print_info: f_attn_scale     = 0.0e+00
print_info: n_ff             = 5120
print_info: n_expert         = 512
print_info: n_expert_used    = 10
print_info: n_expert_groups  = 0
print_info: n_group_used     = 0
print_info: causal attn      = 1
print_info: pooling type     = 0
print_info: rope type        = 2
print_info: rope scaling     = linear
print_info: freq_base_train  = 10000000.0
print_info: freq_scale_train = 1
print_info: n_ctx_orig_yarn  = 262144
print_info: rope_finetuned   = unknown
print_info: ssm_d_conv       = 4
print_info: ssm_d_inner      = 4096
print_info: ssm_d_state      = 128
print_info: ssm_dt_rank      = 32
print_info: ssm_n_group      = 16
print_info: ssm_dt_b_c_rms   = 0
print_info: model type       = ?B
print_info: model params     = 79.67 B
print_info: general.name     = Qwen3-Next-80B-A3B-Instruct
print_info: vocab type       = BPE
print_info: n_vocab          = 151936
print_info: n_merges         = 151387
print_info: BOS token        = 11 ','
print_info: EOS token        = 151645 '<|im_end|>'
print_info: EOT token        = 151645 '<|im_end|>'
print_info: PAD token        = 151654 '<|vision_pad|>'
print_info: LF token         = 198 'Ċ'
print_info: FIM PRE token    = 151659 '<|fim_prefix|>'
print_info: FIM SUF token    = 151661 '<|fim_suffix|>'
print_info: FIM MID token    = 151660 '<|fim_middle|>'
print_info: FIM PAD token    = 151662 '<|fim_pad|>'
print_info: FIM REP token    = 151663 '<|repo_name|>'
print_info: FIM SEP token    = 151664 '<|file_sep|>'
print_info: EOG token        = 151643 '<|endoftext|>'
print_info: EOG token        = 151645 '<|im_end|>'
print_info: EOG token        = 151662 '<|fim_pad|>'
print_info: EOG token        = 151663 '<|repo_name|>'
print_info: EOG token        = 151664 '<|file_sep|>'
print_info: max token length = 256
load_tensors: loading model tensors, this can take a while... (mmap = false)
load_tensors: offloading 48 repeating layers to GPU
load_tensors: offloading output layer to GPU
load_tensors: offloaded 49/49 layers to GPU
load_tensors:          CPU model buffer size =     0.00 MiB
load_tensors:        ROCm0 model buffer size = 22188.86 MiB
load_tensors:        ROCm1 model buffer size = 20655.47 MiB
load_tensors:    ROCm_Host model buffer size =   166.92 MiB
[New Thread 0x7ffda97ff6c0 (LWP 15644)]
....................................................................................................
llama_context: constructing llama_context
llama_context: n_seq_max     = 1
llama_context: n_ctx         = 32000
llama_context: n_ctx_seq     = 32000
llama_context: n_batch       = 256
llama_context: n_ubatch      = 128
llama_context: causal_attn   = 1
llama_context: flash_attn    = enabled
llama_context: kv_unified    = false
llama_context: freq_base     = 10000000.0
llama_context: freq_scale    = 1
llama_context: n_ctx_seq (32000) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
llama_context:  ROCm_Host  output buffer size =     0.58 MiB
llama_kv_cache:      ROCm0 KV buffer size =   199.22 MiB
llama_kv_cache:      ROCm1 KV buffer size =   199.22 MiB
llama_kv_cache: size =  398.44 MiB ( 32000 cells,  12 layers,  1/1 seqs), K (q8_0):  199.22 MiB, V (q8_0):  199.22 MiB
llama_memory_recurrent:      ROCm0 RS buffer size =    39.78 MiB
llama_memory_recurrent:      ROCm1 RS buffer size =    35.59 MiB
llama_memory_recurrent: size =   75.38 MiB (     1 cells,  48 layers,  1 seqs), R (f32):    3.38 MiB, S (f32):   72.00 MiB
llama_context: pipeline parallelism enabled (n_copies=4)
llama_context:      ROCm0 compute buffer size =  1288.61 MiB
llama_context:      ROCm1 compute buffer size =   782.39 MiB
llama_context:  ROCm_Host compute buffer size =   353.90 MiB
llama_context: graph nodes  = 11401 (with bs=128), 8449 (with bs=1)
llama_context: graph splits = 204 (with bs=128), 151 (with bs=1)
common_init_from_params: added <|endoftext|> logit bias = -inf
common_init_from_params: added <|im_end|> logit bias = -inf
common_init_from_params: added <|fim_pad|> logit bias = -inf
common_init_from_params: added <|repo_name|> logit bias = -inf
common_init_from_params: added <|file_sep|> logit bias = -inf
common_init_from_params: setting dry_penalty_last_n to ctx_size = 32000
common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
[New Thread 0x7ff23f7ff6c0 (LWP 15645)]
[New Thread 0x7ff23effe6c0 (LWP 15646)]
[New Thread 0x7ff23e7fd6c0 (LWP 15647)]
[New Thread 0x7ff23dffc6c0 (LWP 15648)]
[New Thread 0x7ff23d7fb6c0 (LWP 15649)]
[New Thread 0x7ff23cffa6c0 (LWP 15650)]
[New Thread 0x7ff23c7f96c0 (LWP 15651)]
[New Thread 0x7ff1391ff6c0 (LWP 15652)]
[Thread 0x7ff1391ff6c0 (LWP 15652) exited]
main: llama threadpool init, n_threads = 8
main: chat template is available, enabling conversation mode (disable it with -no-cnv)
main: chat template example:
<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
Hello<|im_end|>
<|im_start|>assistant
Hi there<|im_end|>
<|im_start|>user
How are you?<|im_end|>
<|im_start|>assistant


system_info: n_threads = 8 (n_threads_batch = 8) / 16 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VNNI = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |

main: interactive mode on.
sampler seed: 3054892969
sampler params:
        repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
        dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 32000
        top_k = 20, top_p = 0.800, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.700
        mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000
sampler chain: logits -> logit-bias -> penalties -> dry -> top-n-sigma -> top-k -> typical -> top-p -> min-p -> xtc -> temp-ext -> dist
generate: n_ctx = 32000, n_batch = 256, n_predict = -1, n_keep = 0

== Running in interactive mode. ==
 - Press Ctrl+C to interject at any time.
 - Press Return to return control to the AI.
 - To return control without starting a new line, end your input with '/'.
 - If you want to submit another line, end your input with '\'.
 - Not using system message. To change it, set a different value via -sys PROMPT


> Hey
Hello
Thread 1 "llama-cli" received signal SIGSEGV, Segmentation fault.
0x00007fffecaad59f in ?? () from /opt/rocm-7.1.0/lib/libamdhip64.so.7
(gdb) bt
#0  0x00007fffecaad59f in ?? () from /opt/rocm-7.1.0/lib/libamdhip64.so.7
#1  0x00007fffecaaddf4 in ?? () from /opt/rocm-7.1.0/lib/libamdhip64.so.7
#2  0x00007fffecb03a85 in ?? () from /opt/rocm-7.1.0/lib/libamdhip64.so.7
#3  0x00007fffecaae95e in ?? () from /opt/rocm-7.1.0/lib/libamdhip64.so.7
#4  0x00007fffecac677e in ?? () from /opt/rocm-7.1.0/lib/libamdhip64.so.7
#5  0x00007ffff04a3950 in void __device_stub__rocblas_gemvtsm_kernel<false, 256, float, float, float>(int, int, float, long, float const*, long, int, long, float const*, long, int, long, float, long, float*, long, int, long) () from /opt/rocm-7.1.0/lib/librocblas.so.5
#6  0x00007ffff04d0c7f in rocblas_status_ rocblas_internal_gemv_launcher<float, float, float>(_rocblas_handle*, rocblas_operation_, int, int, float const*, long, float const*, long, long, long, float const*, long, long, long, float const*, long, float*, long, long, long, int, float*) ()
   from /opt/rocm-7.1.0/lib/librocblas.so.5
#7  0x00007ffff078d433 in rocblas_status_ rocblas_internal_gemm_ex<false, float const*, float const*, float const*, float*>(_rocblas_handle*, rocblas_operation_, rocblas_operation_, int, int, int, float const*, float const*, long, int, long, float const*, long, int, long, float const*, float const*, long, int, long, float*, long, int, long, int, rocblas_gemm_algo_, int, rocblas_gemm_flags_) () from /opt/rocm-7.1.0/lib/librocblas.so.5
#8  0x00007ffff0786e6b in rocblas_status_ gemm_ex_typecasting<false, float, float, float>(_rocblas_handle*, rocblas_operation_, rocblas_operation_, int, int, int, void const*, void const*, long, int, long, void const*, long, int, long, void const*, void const*, long, int, long, void*, long, int, long, int, rocblas_gemm_algo_, int, rocblas_gemm_flags_) () from /opt/rocm-7.1.0/lib/librocblas.so.5
#9  0x00007ffff0785a8d in rocblas_status_ rocblas_gemm_ex_template<false>(_rocblas_handle*, rocblas_operation_, rocblas_operation_, int, int, int, void const*, void const*, rocblas_datatype_, long, int, long, void const*, rocblas_datatype_, long, int, long, void const*, void const*, rocblas_datatype_, long, int, long, void*, rocblas_datatype_, long, int, long, int, rocblas_datatype_, rocblas_gemm_algo_, int, unsigned int) () from /opt/rocm-7.1.0/lib/librocblas.so.5
#10 0x00007ffff077d114 in rocblas_gemm_strided_batched_ex () from /opt/rocm-7.1.0/lib/librocblas.so.5
#11 0x00007ffff69c6801 in hipblasGemmStridedBatchedEx () from /opt/rocm-7.1.0/lib/libhipblas.so.3
#12 0x00007ffff636d6b1 in ggml_cuda_mul_mat_batched_cublas_impl<(ggml_type)0> (ctx=..., src0=<optimized out>, src1=<optimized out>, dst=<optimized out>) at /home/[---]/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:1944
#13 ggml_cuda_mul_mat_batched_cublas (ctx=..., src0=src0@entry=0x7ff2fcaf9580, src1=src1@entry=0x7ff2fcaf92a0, dst=dst@entry=0x7ff2fcaf96f0) at /home/[---]/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:2006
#14 0x00007ffff636bdb6 in ggml_cuda_mul_mat (ctx=..., src0=src0@entry=0x7ff2fcaf9580, src1=0x7ff2fcaf92a0, dst=dst@entry=0x7ff2fcaf96f0) at /home/[---]/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:2250
#15 0x00007ffff6368eed in ggml_cuda_compute_forward (ctx=..., dst=0x7ff2fcaf96f0) at /home/[---]/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:2605
#16 evaluate_and_capture_cuda_graph (cuda_ctx=cuda_ctx@entry=0x55555a069400, cgraph=cgraph@entry=0x5555593d38a8, graph_evaluated_or_captured=@0x7fffffff7865: false, use_cuda_graph=@0x7fffffff7867: false, cuda_graph_update_required=@0x7fffffff7866: false)
    at /home/[---]/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:3518
#17 0x00007ffff6367d3f in ggml_backend_cuda_graph_compute (backend=<optimized out>, cgraph=0x5555593d38a8) at /home/[---]/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:3632
#18 0x00007ffff7edb06d in ggml_backend_graph_compute_async (backend=0x55555a01b010, cgraph=0x5555593d38a8) at /home/[---]/llama.cpp/ggml/src/ggml-backend.cpp:359
#19 0x00007ffff7ee00bc in ggml_backend_sched_compute_splits (sched=0x555559449e10) at /home/[---]/llama.cpp/ggml/src/ggml-backend.cpp:1559
#20 0x00007ffff7ee0ee4 in ggml_backend_sched_graph_compute_async (sched=0x555559449e10, graph=0x7ff2fc9c8030) at /home/[---]/llama.cpp/ggml/src/ggml-backend.cpp:1757
#21 0x00007ffff7a3da42 in llama_context::graph_compute (this=0x555556b7aeb0, gf=0x7ff2fc9c8030, batched=false) at /home/[---]/llama.cpp/src/llama-context.cpp:1488
#22 0x00007ffff7a3a6c4 in llama_context::process_ubatch (this=0x555556b7aeb0, ubatch=..., gtype=LLM_GRAPH_TYPE_DECODER, mctx=0x555563a99b30, ret=@0x7fffffffba14: 21845) at /home/[---]/llama.cpp/src/llama-context.cpp:809
#23 0x00007ffff7a3bc48 in llama_context::decode (this=0x555556b7aeb0, batch_inp=...) at /home/[---]/llama.cpp/src/llama-context.cpp:1113
#24 0x00007ffff7a42b31 in llama_decode (ctx=0x555556b7aeb0, batch=...) at /home/[---]/llama.cpp/src/llama-context.cpp:2780
#25 0x00005555555e8831 in main (argc=21, argv=0x7fffffffe1c8) at /home/[---]/llama.cpp/tools/main/main.cpp:679
(gdb) thread apply all bt full

Thread 16 (Thread 0x7ff23c7f96c0 (LWP 15651) "llama-cli"):
#0  futex_wait (addr=0x55555a4fec54, val=1192) at ../../../src/libgomp/config/linux/x86/futex.h:97
        err = -512
        err = <optimized out>
#1  do_wait (addr=<optimized out>, val=1192) at ../../../src/libgomp/config/linux/wait.h:67
No locals.
#2  gomp_barrier_wait_end (bar=0x55555a4fec50, state=1192) at ../../../src/libgomp/config/linux/bar.c:48
No locals.
#3  0x00007ffff7e05779 in gomp_simple_barrier_wait (bar=<optimized out>) at ../../../src/libgomp/config/posix/simple-bar.h:60
No locals.
#4  gomp_thread_start (xdata=<optimized out>) at ../../../src/libgomp/team.c:133
        team = 0x55555a001840
        task = 0x55555a0023a0
        data = <optimized out>
        thr = <optimized out>
        pool = <optimized out>
        local_fn = 0x7ffff7226633 <ggml_graph_compute._omp_fn.0>
        local_data = 0x7fffffff75d0
#5  0x00007ffff6a9caa4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447
        ret = <optimized out>
        pd = <optimized out>
        out = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140678373807808, -3498611537169754479, 140678373807808, -160, 0, 140737488316560, -3498611537190725999, -3500991281709721967}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#6  0x00007ffff6b29c6c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
No locals.

Thread 15 (Thread 0x7ff23cffa6c0 (LWP 15650) "llama-cli"):
#0  futex_wait (addr=0x55555a4fec54, val=1192) at ../../../src/libgomp/config/linux/x86/futex.h:97
        err = -512
        err = <optimized out>
#1  do_wait (addr=<optimized out>, val=1192) at ../../../src/libgomp/config/linux/wait.h:67
No locals.
#2  gomp_barrier_wait_end (bar=0x55555a4fec50, state=1192) at ../../../src/libgomp/config/linux/bar.c:48
No locals.
#3  0x00007ffff7e05779 in gomp_simple_barrier_wait (bar=<optimized out>) at ../../../src/libgomp/config/posix/simple-bar.h:60
No locals.
#4  gomp_thread_start (xdata=<optimized out>) at ../../../src/libgomp/team.c:133
        team = 0x55555a001840
        task = 0x55555a0022c0
        data = <optimized out>
        thr = <optimized out>
        pool = <optimized out>
        local_fn = 0x7ffff7226633 <ggml_graph_compute._omp_fn.0>
        local_data = 0x7fffffff75d0
#5  0x00007ffff6a9caa4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447
        ret = <optimized out>
        pd = <optimized out>
        out = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140678382200512, -3498612637218253167, 140678382200512, -160, 0, 140737488316560, -3498612637239224687, -3500991281709721967}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#6  0x00007ffff6b29c6c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
No locals.

Thread 14 (Thread 0x7ff23d7fb6c0 (LWP 15649) "llama-cli"):
#0  futex_wait (addr=0x55555a4fec54, val=1192) at ../../../src/libgomp/config/linux/x86/futex.h:97
        err = -512
        err = <optimized out>
#1  do_wait (addr=<optimized out>, val=1192) at ../../../src/libgomp/config/linux/wait.h:67
No locals.
#2  gomp_barrier_wait_end (bar=0x55555a4fec50, state=1192) at ../../../src/libgomp/config/linux/bar.c:48
No locals.
#3  0x00007ffff7e05779 in gomp_simple_barrier_wait (bar=<optimized out>) at ../../../src/libgomp/config/posix/simple-bar.h:60
No locals.
#4  gomp_thread_start (xdata=<optimized out>) at ../../../src/libgomp/team.c:133
        team = 0x55555a001840
        task = 0x55555a0021e0
        data = <optimized out>
        thr = <optimized out>
        pool = <optimized out>
        local_fn = 0x7ffff7226633 <ggml_graph_compute._omp_fn.0>
        local_data = 0x7fffffff75d0
#5  0x00007ffff6a9caa4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447
        ret = <optimized out>
        pd = <optimized out>
        out = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140678390593216, -3498613737266751855, 140678390593216, -160, 0, 140737488316560, -3498613737287723375, -3500991281709721967}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#6  0x00007ffff6b29c6c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
No locals.

Thread 13 (Thread 0x7ff23dffc6c0 (LWP 15648) "llama-cli"):
#0  futex_wait (addr=0x55555a4fec54, val=1192) at ../../../src/libgomp/config/linux/x86/futex.h:97
--Type <RET> for more, q to quit, c to continue without paging--c
        err = -512
        err = <optimized out>
#1  do_wait (addr=<optimized out>, val=1192) at ../../../src/libgomp/config/linux/wait.h:67
No locals.
#2  gomp_barrier_wait_end (bar=0x55555a4fec50, state=1192) at ../../../src/libgomp/config/linux/bar.c:48
No locals.
#3  0x00007ffff7e05779 in gomp_simple_barrier_wait (bar=<optimized out>) at ../../../src/libgomp/config/posix/simple-bar.h:60
No locals.
#4  gomp_thread_start (xdata=<optimized out>) at ../../../src/libgomp/team.c:133
        team = 0x55555a001840
        task = 0x55555a002100
        data = <optimized out>
        thr = <optimized out>
        pool = <optimized out>
        local_fn = 0x7ffff7226633 <ggml_graph_compute._omp_fn.0>
        local_data = 0x7fffffff75d0
#5  0x00007ffff6a9caa4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447
        ret = <optimized out>
        pd = <optimized out>
        out = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140678398985920, -3498614837315250543, 140678398985920, -160, 0, 140737488316560, -3498614837336222063, -3500991281709721967}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#6  0x00007ffff6b29c6c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
No locals.

Thread 12 (Thread 0x7ff23e7fd6c0 (LWP 15647) "llama-cli"):
#0  futex_wait (addr=0x55555a4fec54, val=1192) at ../../../src/libgomp/config/linux/x86/futex.h:97
        err = -512
        err = <optimized out>
#1  do_wait (addr=<optimized out>, val=1192) at ../../../src/libgomp/config/linux/wait.h:67
No locals.
#2  gomp_barrier_wait_end (bar=0x55555a4fec50, state=1192) at ../../../src/libgomp/config/linux/bar.c:48
No locals.
#3  0x00007ffff7e05779 in gomp_simple_barrier_wait (bar=<optimized out>) at ../../../src/libgomp/config/posix/simple-bar.h:60
No locals.
#4  gomp_thread_start (xdata=<optimized out>) at ../../../src/libgomp/team.c:133
        team = 0x55555a001840
        task = 0x55555a002020
        data = <optimized out>
        thr = <optimized out>
        pool = <optimized out>
        local_fn = 0x7ffff7226633 <ggml_graph_compute._omp_fn.0>
        local_data = 0x7fffffff75d0
#5  0x00007ffff6a9caa4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447
        ret = <optimized out>
        pd = <optimized out>
        out = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140678407378624, -3498615937363749231, 140678407378624, -160, 0, 140737488316560, -3498615937384720751, -3500991281709721967}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#6  0x00007ffff6b29c6c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
No locals.

Thread 11 (Thread 0x7ff23effe6c0 (LWP 15646) "llama-cli"):
#0  futex_wait (addr=0x55555a4fec54, val=1192) at ../../../src/libgomp/config/linux/x86/futex.h:97
        err = -512
        err = <optimized out>
#1  do_wait (addr=<optimized out>, val=1192) at ../../../src/libgomp/config/linux/wait.h:67
No locals.
#2  gomp_barrier_wait_end (bar=0x55555a4fec50, state=1192) at ../../../src/libgomp/config/linux/bar.c:48
No locals.
#3  0x00007ffff7e05779 in gomp_simple_barrier_wait (bar=<optimized out>) at ../../../src/libgomp/config/posix/simple-bar.h:60
No locals.
#4  gomp_thread_start (xdata=<optimized out>) at ../../../src/libgomp/team.c:133
        team = 0x55555a001840
        task = 0x55555a001f40
        data = <optimized out>
        thr = <optimized out>
        pool = <optimized out>
        local_fn = 0x7ffff7226633 <ggml_graph_compute._omp_fn.0>
        local_data = 0x7fffffff75d0
#5  0x00007ffff6a9caa4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447
        ret = <optimized out>
        pd = <optimized out>
        out = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140678415771328, -3498617037412247919, 140678415771328, -160, 0, 140737488316560, -3498617037433219439, -3500991281709721967}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#6  0x00007ffff6b29c6c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
No locals.

Thread 10 (Thread 0x7ff23f7ff6c0 (LWP 15645) "llama-cli"):
#0  futex_wait (addr=0x55555a4fec54, val=1192) at ../../../src/libgomp/config/linux/x86/futex.h:97
        err = -512
        err = <optimized out>
#1  do_wait (addr=<optimized out>, val=1192) at ../../../src/libgomp/config/linux/wait.h:67
No locals.
#2  gomp_barrier_wait_end (bar=0x55555a4fec50, state=1192) at ../../../src/libgomp/config/linux/bar.c:48
No locals.
#3  0x00007ffff7e05779 in gomp_simple_barrier_wait (bar=<optimized out>) at ../../../src/libgomp/config/posix/simple-bar.h:60
No locals.
#4  gomp_thread_start (xdata=<optimized out>) at ../../../src/libgomp/team.c:133
        team = 0x55555a001840
        task = 0x55555a001e60
        data = <optimized out>
        thr = <optimized out>
        pool = <optimized out>
        local_fn = 0x7ffff7226633 <ggml_graph_compute._omp_fn.0>
        local_data = 0x7fffffff75d0
#5  0x00007ffff6a9caa4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447
        ret = <optimized out>
        pd = <optimized out>
        out = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140678424164032, -3498618137460746607, 140678424164032, -160, 0, 140737488316560, -3498618137481718127, -3500991281709721967}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#6  0x00007ffff6b29c6c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
No locals.

Thread 9 (Thread 0x7ffda97ff6c0 (LWP 15644) "llama-cli"):
#0  __GI___ioctl (fd=3, request=3222817548) at ../sysdeps/unix/sysv/linux/ioctl.c:36
        args = {{gp_offset = 16, fp_offset = 21845, overflow_arg_area = 0x7ffda97f6b30, reg_save_area = 0x7ffda97f6af0}}
        arg = <optimized out>
        r = -512
#1  0x00007fffbe142d28 in ?? () from /opt/rocm-7.1.0/lib/libhsa-runtime64.so.1
No symbol table info available.
#2  0x00007fffbe13939f in ?? () from /opt/rocm-7.1.0/lib/libhsa-runtime64.so.1
No symbol table info available.
#3  0x00007fffbe0a0083 in ?? () from /opt/rocm-7.1.0/lib/libhsa-runtime64.so.1
No symbol table info available.
#4  0x00007fffbe039e21 in ?? () from /opt/rocm-7.1.0/lib/libhsa-runtime64.so.1
No symbol table info available.
#5  0x00007ffff6a9caa4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447
        ret = <optimized out>
        pd = <optimized out>
        out = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140727447189184, -3500065094762899823, 4294967295, -160, 0, 140737488327072, -3500065094783871343, -3500991281709721967}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#6  0x00007ffff6b29c6c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
No locals.

Thread 8 (Thread 0x7ffdabbff6c0 (LWP 15643) "llama-cli"):
#0  0x00007ffff6a98d71 in __futex_abstimed_wait_common64 (private=32753, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x555555986df8 <common_log_main()::log+88>) at ./nptl/futex-internal.c:57
        sc_cancel_oldtype = 0
        sc_ret = <optimized out>
        resultvar = <optimized out>
        __arg6 = <optimized out>
        __arg5 = <optimized out>
        __arg4 = <optimized out>
        __arg3 = <optimized out>
        __arg2 = <optimized out>
        __arg1 = <optimized out>
        _a6 = <optimized out>
        _a5 = <optimized out>
        _a4 = <optimized out>
        _a3 = <optimized out>
        _a2 = <optimized out>
        _a1 = <optimized out>
#1  __futex_abstimed_wait_common (cancel=true, private=32753, abstime=0x0, clockid=0, expected=0, futex_word=0x555555986df8 <common_log_main()::log+88>) at ./nptl/futex-internal.c:87
        err = <optimized out>
        clockbit = 256
        op = 393
        err = <optimized out>
        clockbit = <optimized out>
        op = <optimized out>
#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x555555986df8 <common_log_main()::log+88>, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
No locals.
#3  0x00007ffff6a9b7ed in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x555555986da0 <common_log_main()::log>, cond=0x555555986dd0 <common_log_main()::log+48>) at ./nptl/pthread_cond_wait.c:503
        spin = 0
        buffer = {__routine = 0x7ffff6a9b4b0 <__condvar_cleanup_waiting>, __arg = 0x7ffdabbf6ba0, __canceltype = -155181008, __prev = 0x0}
        cbuffer = {wseq = 1972, cond = 0x555555986dd0 <common_log_main()::log+48>, mutex = 0x555555986da0 <common_log_main()::log>, private = 0}
        err = <optimized out>
        g = 0
        flags = <optimized out>
        g1_start = <optimized out>
        maxspin = 0
        signals = <optimized out>
        result = 0
        wseq = 1972
        seq = 986
        private = 0
        maxspin = <optimized out>
        err = <optimized out>
        result = <optimized out>
        wseq = <optimized out>
        g = <optimized out>
        seq = <optimized out>
        flags = <optimized out>
        private = <optimized out>
        signals = <optimized out>
        done = <optimized out>
        g1_start = <optimized out>
        spin = <optimized out>
        buffer = <optimized out>
        cbuffer = <optimized out>
        s = <optimized out>
#4  ___pthread_cond_wait (cond=0x555555986dd0 <common_log_main()::log+48>, mutex=0x555555986da0 <common_log_main()::log>) at ./nptl/pthread_cond_wait.c:627
No locals.
#5  0x00005555557f14eb in std::condition_variable::wait<common_log::resume()::{lambda()#1}::operator()() const::{lambda()#1}>(std::unique_lock<std::mutex>&, common_log::resume()::{lambda()#1}::operator()() const::{lambda()#1}) (this=0x555555986dd0 <common_log_main()::log+48>, __lock=..., __p=...) at /usr/include/c++/13/condition_variable:105
No locals.
#6  0x00005555557f08a7 in common_log::resume()::{lambda()#1}::operator()() const (__closure=0x555558556498) at /home/[---]/llama.cpp/common/log.cpp:286
        lock = {_M_device = 0x555555986da0 <common_log_main()::log>, _M_owns = true}
        this = 0x555555986da0 <common_log_main()::log>
#7  0x00005555557f3090 in std::__invoke_impl<void, common_log::resume()::{lambda()#1}>(std::__invoke_other, common_log::resume()::{lambda()#1}&&) (__f=...) at /usr/include/c++/13/bits/invoke.h:61
No locals.
#8  0x00005555557f304b in std::__invoke<common_log::resume()::{lambda()#1}>(common_log::resume()::{lambda()#1}&&) (__fn=...) at /usr/include/c++/13/bits/invoke.h:96
No locals.
#9  0x00005555557f2fec in std::thread::_Invoker<std::tuple<common_log::resume()::{lambda()#1}> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) (this=0x555558556498) at /usr/include/c++/13/bits/std_thread.h:292
No locals.
#10 0x00005555557f2fbc in std::thread::_Invoker<std::tuple<common_log::resume()::{lambda()#1}> >::operator()() (this=0x555558556498) at /usr/include/c++/13/bits/std_thread.h:299
No locals.
#11 0x00005555557f2f9c in std::thread::_State_impl<std::thread::_Invoker<std::tuple<common_log::resume()::{lambda()#1}> > >::_M_run() (this=0x555558556490) at /usr/include/c++/13/bits/std_thread.h:244
No locals.
#12 0x00007ffff6eecdb4 in std::execute_native_thread_routine (__p=0x555558556490) at ../../../../../src/libstdc++-v3/src/c++11/thread.cc:104
        __t = <optimized out>
#13 0x00007ffff6a9caa4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447
        ret = <optimized out>
        pd = <optimized out>
        out = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140727484937920, -3500070042565224815, 140727484937920, -160, 0, 140737488317712, -3500070042586196335, -3500991281709721967}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#14 0x00007ffff6b29c6c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
No locals.

Thread 2 (Thread 0x7fffb1bff6c0 (LWP 15637) "llama-cli"):
#0  __GI___ioctl (fd=3, request=3222817548) at ../sysdeps/unix/sysv/linux/ioctl.c:36
        args = {{gp_offset = 16, fp_offset = 0, overflow_arg_area = 0x7fffb1bf6a80, reg_save_area = 0x7fffb1bf6a40}}
        arg = <optimized out>
        r = -512
#1  0x00007fffbe142d28 in ?? () from /opt/rocm-7.1.0/lib/libhsa-runtime64.so.1
No symbol table info available.
#2  0x00007fffbe13939f in ?? () from /opt/rocm-7.1.0/lib/libhsa-runtime64.so.1
No symbol table info available.
#3  0x00007fffbe0be386 in ?? () from /opt/rocm-7.1.0/lib/libhsa-runtime64.so.1
No symbol table info available.
#4  0x00007fffbe0a0448 in ?? () from /opt/rocm-7.1.0/lib/libhsa-runtime64.so.1
No symbol table info available.
#5  0x00007fffbe039e21 in ?? () from /opt/rocm-7.1.0/lib/libhsa-runtime64.so.1
No symbol table info available.
#6  0x00007ffff6a9caa4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447
        ret = <optimized out>
        pd = <optimized out>
        out = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140736175535808, -3501138767867423087, 4294967295, -160, 0, 140737488320928, -3501138767888394607, -3500991281709721967}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#7  0x00007ffff6b29c6c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
No locals.

Thread 1 (Thread 0x7fffee1f98c0 (LWP 15631) "llama-cli"):
#0  0x00007fffecaad59f in ?? () from /opt/rocm-7.1.0/lib/libamdhip64.so.7
No symbol table info available.
#1  0x00007fffecaaddf4 in ?? () from /opt/rocm-7.1.0/lib/libamdhip64.so.7
No symbol table info available.
#2  0x00007fffecb03a85 in ?? () from /opt/rocm-7.1.0/lib/libamdhip64.so.7
No symbol table info available.
#3  0x00007fffecaae95e in ?? () from /opt/rocm-7.1.0/lib/libamdhip64.so.7
No symbol table info available.
#4  0x00007fffecac677e in ?? () from /opt/rocm-7.1.0/lib/libamdhip64.so.7
No symbol table info available.
#5  0x00007ffff04a3950 in void __device_stub__rocblas_gemvtsm_kernel<false, 256, float, float, float>(int, int, float, long, float const*, long, int, long, float const*, long, int, long, float, long, float*, long, int, long) () from /opt/rocm-7.1.0/lib/librocblas.so.5
No symbol table info available.
#6  0x00007ffff04d0c7f in rocblas_status_ rocblas_internal_gemv_launcher<float, float, float>(_rocblas_handle*, rocblas_operation_, int, int, float const*, long, float const*, long, long, long, float const*, long, long, long, float const*, long, float*, long, long, long, int, float*) () from /opt/rocm-7.1.0/lib/librocblas.so.5
No symbol table info available.
#7  0x00007ffff078d433 in rocblas_status_ rocblas_internal_gemm_ex<false, float const*, float const*, float const*, float*>(_rocblas_handle*, rocblas_operation_, rocblas_operation_, int, int, int, float const*, float const*, long, int, long, float const*, long, int, long, float const*, float const*, long, int, long, float*, long, int, long, int, rocblas_gemm_algo_, int, rocblas_gemm_flags_) () from /opt/rocm-7.1.0/lib/librocblas.so.5
No symbol table info available.
#8  0x00007ffff0786e6b in rocblas_status_ gemm_ex_typecasting<false, float, float, float>(_rocblas_handle*, rocblas_operation_, rocblas_operation_, int, int, int, void const*, void const*, long, int, long, void const*, long, int, long, void const*, void const*, long, int, long, void*, long, int, long, int, rocblas_gemm_algo_, int, rocblas_gemm_flags_) () from /opt/rocm-7.1.0/lib/librocblas.so.5
No symbol table info available.
#9  0x00007ffff0785a8d in rocblas_status_ rocblas_gemm_ex_template<false>(_rocblas_handle*, rocblas_operation_, rocblas_operation_, int, int, int, void const*, void const*, rocblas_datatype_, long, int, long, void const*, rocblas_datatype_, long, int, long, void const*, void const*, rocblas_datatype_, long, int, long, void*, rocblas_datatype_, long, int, long, int, rocblas_datatype_, rocblas_gemm_algo_, int, unsigned int) () from /opt/rocm-7.1.0/lib/librocblas.so.5
No symbol table info available.
#10 0x00007ffff077d114 in rocblas_gemm_strided_batched_ex () from /opt/rocm-7.1.0/lib/librocblas.so.5
No symbol table info available.
#11 0x00007ffff69c6801 in hipblasGemmStridedBatchedEx () from /opt/rocm-7.1.0/lib/libhipblas.so.3
No symbol table info available.
#12 0x00007ffff636d6b1 in ggml_cuda_mul_mat_batched_cublas_impl<(ggml_type)0> (ctx=..., src0=<optimized out>, src1=<optimized out>, dst=<optimized out>) at /home/[---]/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:1944
        err_ = <optimized out>
        sma = 0
        smb = 1
        src0_alloc = {pool = 0x555559666b20, ptr = 0x0, actual_size = 0}
        src1_alloc = {pool = 0x555559666b20, ptr = 0x0, actual_size = 0}
        dst_temp = {pool = 0x555559666b20, ptr = 0x0, actual_size = 0}
        alpha_f32 = 1
        beta_f32 = 0
        ne01 = <optimized out>
        ne02 = <optimized out>
        ne03 = <optimized out>
        nb00 = <optimized out>
        nb01 = <optimized out>
        nb02 = <optimized out>
        nb03 = <optimized out>
        ne10 = <optimized out>
        ne11 = <optimized out>
        ne12 = <optimized out>
        ne13 = <optimized out>
        nb10 = <optimized out>
        nb11 = <optimized out>
        nb12 = <optimized out>
        nb13 = <optimized out>
        ne0 = <optimized out>
        ne1 = <optimized out>
        ne_dst = <optimized out>
        main_stream = <optimized out>
        dst_ddf = <optimized out>
        ts_src1 = <optimized out>
        s11 = <optimized out>
        s12 = <optimized out>
        s13 = <optimized out>
        src0_ptr = <optimized out>
        src1_ptr = <optimized out>
        is_src0_cont_2 = <optimized out>
        is_src1_cont_2 = <optimized out>
        nbd2 = <optimized out>
        nbd3 = <optimized out>
        cu_compute_type = HIPBLAS_COMPUTE_32F
        cu_data_type = HIP_R_32F
        cu_data_type_a = HIP_R_32F
        cu_data_type_b = HIP_R_32F
        dst_t = <optimized out>
        beta = <optimized out>
        alpha = 0x7ffff189ce3c <batched_mul_mat_traits<(ggml_type)0>::get_alpha()::val>
        cc = <optimized out>
        r2 = <optimized out>
        ne00 = <optimized out>
        ne2 = <optimized out>
        ne3 = <optimized out>
        nb0 = <optimized out>
        nb1 = <optimized out>
        nb2 = <optimized out>
        nb3 = <optimized out>
        id = <optimized out>
        r3 = <optimized out>
        err_ = <optimized out>
        ne_src1 = <optimized out>
        convert_func = <optimized out>
        ptrs_src = <optimized out>
        ptrs_dst = <optimized out>
        ne23 = <optimized out>
        src1_stride_size = <optimized out>
        threads_x = <optimized out>
        threads_y = <optimized out>
        block_dims = <optimized out>
        grid_dims = <optimized out>
        err_ = <optimized out>
        err_ = <optimized out>
        sma = <optimized out>
        smb = <optimized out>
        err_ = <optimized out>
        to_fp32_cuda = <optimized out>
#13 ggml_cuda_mul_mat_batched_cublas (ctx=..., src0=src0@entry=0x7ff2fcaf9580, src1=src1@entry=0x7ff2fcaf92a0, dst=dst@entry=0x7ff2fcaf96f0) at /home/[---]/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:2006
No locals.
#14 0x00007ffff636bdb6 in ggml_cuda_mul_mat (ctx=..., src0=src0@entry=0x7ff2fcaf9580, src1=0x7ff2fcaf92a0, dst=dst@entry=0x7ff2fcaf96f0) at /home/[---]/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:2250
        bad_padding_clear = <optimized out>
        use_mul_mat_vec_f = false
        use_mul_mat_f = false
        use_mul_mat_vec_q = false
        use_mul_mat_q = false
        any_gpus_with_slow_fp16 = <optimized out>
        cc = <optimized out>
        use_batched_cublas_f16 = <optimized out>
        use_batched_cublas_bf16 = <optimized out>
        split = <optimized out>
        use_batched_cublas_f32 = <optimized out>
#15 0x00007ffff6368eed in ggml_cuda_compute_forward (ctx=..., dst=0x7ff2fcaf96f0) at /home/[---]/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:2605
        err = <optimized out>
        err_ = <optimized out>
#16 evaluate_and_capture_cuda_graph (cuda_ctx=cuda_ctx@entry=0x55555a069400, cgraph=cgraph@entry=0x5555593d38a8, graph_evaluated_or_captured=@0x7fffffff7865: false, use_cuda_graph=@0x7fffffff7867: false, cuda_graph_update_required=@0x7fffffff7866: false) at /home/[---]/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:3518
        node = 0x7ff2fcaf96f0
        ok = <optimized out>
        i = 43
        prev_i = 0
        disable_fusion = false
        integrated = false
#17 0x00007ffff6367d3f in ggml_backend_cuda_graph_compute (backend=<optimized out>, cgraph=0x5555593d38a8) at /home/[---]/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:3632
        use_cuda_graph = false
        cuda_graph_update_required = false
        graph_evaluated_or_captured = false
        cuda_ctx = 0x55555a069400
#18 0x00007ffff7edb06d in ggml_backend_graph_compute_async (backend=0x55555a01b010, cgraph=0x5555593d38a8) at /home/[---]/llama.cpp/ggml/src/ggml-backend.cpp:359
No locals.
#19 0x00007ffff7ee00bc in ggml_backend_sched_compute_splits (sched=0x555559449e10) at /home/[---]/llama.cpp/ggml/src/ggml-backend.cpp:1559
        ec = GGML_STATUS_SUCCESS
        split = 0x5555593d37a0
        split_backend_id = 0
        split_backend = 0x55555a01b010
        split_id = 7
        splits = 0x5555593d2e00
        prev_ids_tensor = 0x0
        ids = std::vector of length 0, capacity 0
        used_ids = std::vector of length 0, capacity 0
#20 0x00007ffff7ee0ee4 in ggml_backend_sched_graph_compute_async (sched=0x555559449e10, graph=0x7ff2fc9c8030) at /home/[---]/llama.cpp/ggml/src/ggml-backend.cpp:1757
No locals.
#21 0x00007ffff7a3da42 in llama_context::graph_compute (this=0x555556b7aeb0, gf=0x7ff2fc9c8030, batched=false) at /home/[---]/llama.cpp/src/llama-context.cpp:1488
        n_threads = 8
        tp = 0x55556406a740
        status = 32767
        __func__ = "graph_compute"
#22 0x00007ffff7a3a6c4 in llama_context::process_ubatch (this=0x555556b7aeb0, ubatch=..., gtype=LLM_GRAPH_TYPE_DECODER, mctx=0x555563a99b30, ret=@0x7fffffffba14: 21845) at /home/[---]/llama.cpp/src/llama-context.cpp:809
        __func__ = "process_ubatch"
        res = 0x55555983d420
        gf = 0x7ff2fc9c8030
        gparams = {arch = LLM_ARCH_QWEN3NEXT, hparams = {vocab_only = false, rope_finetuned = false, use_par_res = false, swin_norm = false, n_ctx_train = 262144, n_embd = 2048, n_embd_features = 0, n_layer = 48, n_layer_kv_from_start = -1, n_rot = 64, n_embd_head_k = 256, n_embd_head_v = 256, n_expert = 512, n_expert_used = 10, n_rel_attn_bkts = 0, n_embd_head_k_mla = 0, n_embd_head_v_mla = 0, posnet = {n_embd = 0, n_layer = 0}, convnext = {n_embd = 0, n_layer = 0}, n_shortconv_l_cache = 0, n_head_arr = {_M_elems = {16 <repeats 48 times>, 0 <repeats 464 times>}}, n_head_kv_arr = {_M_elems = {2 <repeats 48 times>, 0 <repeats 464 times>}}, n_ff_arr = {_M_elems = {5120 <repeats 48 times>, 0 <repeats 464 times>}}, n_layer_dense_lead = 0, n_lora_q = 0, n_lora_kv = 0, n_ff_exp = 512, n_ff_shexp = 512, n_ff_chexp = 0, n_expert_shared = 0, n_norm_groups = 0, n_expert_groups = 0, n_group_used = 0, n_group_experts = 0, expert_group_scale = 0.0500000007, expert_weights_scale = 0, expert_weights_norm = false, expert_gating_func = 0, moe_every_n_layers = 0, nextn_predict_layers = 0, f_norm_eps = 0, f_norm_rms_eps = 9.99999997e-07, f_norm_group_eps = 0, f_attn_logit_softcapping = 50, f_router_logit_softcapping = 30, f_final_logit_softcapping = 30, rescale_every_n_layers = 0, time_mix_extra_dim = 0, time_decay_extra_dim = 0, wkv_head_size = 0, token_shift_count = 2, n_lora_decay = 0, n_lora_iclr = 0, n_lora_value_res_mix = 0, n_lora_gate = 0, rope_attn_factor = 1, rope_freq_base_train = 10000000, rope_freq_base_train_swa = 10000000, rope_freq_scale_train = 1, rope_freq_scale_train_swa = 1, n_ctx_orig_yarn = 262144, rope_yarn_log_mul = 0, yarn_ext_factor = -1, yarn_attn_factor = 1, yarn_beta_fast = 32, yarn_beta_slow = 1, rope_sections = {_M_elems = {0, 0, 0, 0}}, swa_type = LLAMA_SWA_TYPE_NONE, n_swa = 0, swa_layers = {_M_elems = {false <repeats 512 times>}}, ssm_d_conv = 4, ssm_d_inner = 4096, ssm_d_state = 128, ssm_dt_rank = 32, ssm_n_group = 16, recurrent_layer_arr = {_M_elems = {true, true, true, false, true, true, true, false, true, true, true, false, true, true, true, false, true, true, true, false, true, true, true, false, true, true, true, false, true, true, true, false, true, true, true, false, true, true, true, false, true, true, true, false, true, true, true, false <repeats 465 times>}}, ssm_dt_b_c_rms = false, f_clamp_kqv = 0, f_max_alibi_bias = 0, f_logit_scale = 0, f_residual_scale = 0, f_embedding_scale = 0, f_attention_scale = 0, f_attn_out_scale = 0, attn_temp_length = 0, causal_attn = true, use_alibi = false, attn_soft_cap = false, use_kq_norm = false, n_cls_out = 1, n_moe_layer_step = 0, n_no_rope_layer_step = 4, n_attn_temp_floor_scale = 8192, f_attn_temp_scale = 0.100000001, n_altup = 4, i_altup_act = 0, laurel_rank = 64, n_embd_altup = 256, dense_2_feat_in = 0, dense_2_feat_out = 0, dense_3_feat_in = 0, dense_3_feat_out = 0, xielu_alpha_n = {_M_elems = {0 <repeats 512 times>}}, xielu_alpha_p = {_M_elems = {0 <repeats 512 times>}}, xielu_beta = {_M_elems = {0 <repeats 512 times>}}, xielu_eps = {_M_elems = {0 <repeats 512 times>}}, n_deepstack_layers = 0, dec_start_token_id = -1, dec_n_layer = 0, pooling_type = LLAMA_POOLING_TYPE_NONE, rope_type = LLAMA_ROPE_TYPE_NEOX, rope_scaling_type_train = LLAMA_ROPE_SCALING_TYPE_LINEAR}, cparams = {n_ctx = 32000, n_ctx_seq = 32000, n_batch = 256, n_ubatch = 128, n_seq_max = 1, n_threads = 8, n_threads_batch = 8, rope_freq_base = 10000000, rope_freq_scale = 1, n_ctx_orig_yarn = 262144, yarn_ext_factor = 0, yarn_attn_factor = 1, yarn_beta_fast = 32, yarn_beta_slow = 1, embeddings = false, causal_attn = true, offload_kqv = true, flash_attn = true, no_perf = false, warmup = false, op_offload = true, kv_unified = false, pooling_type = LLAMA_POOLING_TYPE_NONE, cb_eval = 0x0, cb_eval_user_data = 0x0}, ubatch = {b_equal_seqs = 1, n_tokens = 1, n_seq_tokens = 1, n_seqs = 1, n_seqs_unq = 1, n_pos = 1, token = 0x55555ee31f20, embd = 0x0, pos = 0x55555e4447f0, n_seq_id = 0x55556407c120, seq_id = 0x55555ee2e460, seq_id_unq = 0x55556437d510, seq_idx = 0x555565ea5b90, output = 0x555559990e90 "\001Lb1PU", data = std::shared_ptr<llama_ubatch::data_t> (use count 5, weak count 0) = {get() = 0x5555641f1a20}}, gtype = LLM_GRAPH_TYPE_DECODER, sched = 0x555559449e10, backend_cpu = 0x555559ff83c0, cvec = 0x555556b7af10, loras = 0x555556b7af60, mctx = 0x555563a99b30, cross = 0x555556b7af98, n_outputs = 1, cb = {<std::_Maybe_unary_or_binary_function<void, llama_ubatch const&, ggml_tensor*, char const*, int>> = {<No data fields>}, <std::_Function_base> = {static _M_max_size = 16, static _M_max_align = 8, _M_functor = {_M_unused = {_M_object = 0x555556b7aeb0, _M_const_object = 0x555556b7aeb0, _M_function_pointer = 0x555556b7aeb0, _M_member_pointer = (void (std::_Undefined_class::*)(std::_Undefined_class * const)) 0x555556b7aeb0}, _M_pod_data = "\260\256\267VUU\000\000\000\000\000\000\000\000\000"}, _M_manager = 0x7ffff7a448c8 <std::_Function_handler<void(const llama_ubatch&, ggml_tensor*, char const*, int), llama_context::graph_get_cb() const::<lambda(const llama_ubatch&, ggml_tensor*, char const*, int)> >::_M_manager(std::_Any_data &, const std::_Any_data &, std::_Manager_operation)>}, _M_invoker = 0x7ffff7a44837 <std::_Function_handler<void(const llama_ubatch&, ggml_tensor*, char const*, int), llama_context::graph_get_cb() const::<lambda(const llama_ubatch&, ggml_tensor*, char const*, int)> >::_M_invoke(const std::_Any_data &, const llama_ubatch &, ggml_tensor *&&, const char *&&, int &&)>}, res = 0x55555983d420}
        status = GGML_STATUS_SUCCESS
#23 0x00007ffff7a3bc48 in llama_context::decode (this=0x555556b7aeb0, batch_inp=...) at /home/[---]/llama.cpp/src/llama-context.cpp:1113
        status = 21845
        ubatch = @0x5555636c19d0: {b_equal_seqs = 1, n_tokens = 1, n_seq_tokens = 1, n_seqs = 1, n_seqs_unq = 1, n_pos = 1, token = 0x55555ee31f20, embd = 0x0, pos = 0x55555e4447f0, n_seq_id = 0x55556407c120, seq_id = 0x55555ee2e460, seq_id_unq = 0x55556437d510, seq_idx = 0x555565ea5b90, output = 0x555559990e90 "\001Lb1PU", data = std::shared_ptr<llama_ubatch::data_t> (use count 5, weak count 0) = {get() = 0x5555641f1a20}}
        res = 0x0
        t_logits = 0x0
        t_embd = 0x0
        __func__ = "decode"
        vocab = @0x555556dd5c60: {pimpl = std::unique_ptr<llama_vocab::impl> = {get() = 0x55555627f310}}
        hparams = @0x555556dd1ec8: {vocab_only = false, rope_finetuned = false, use_par_res = false, swin_norm = false, n_ctx_train = 262144, n_embd = 2048, n_embd_features = 0, n_layer = 48, n_layer_kv_from_start = -1, n_rot = 64, n_embd_head_k = 256, n_embd_head_v = 256, n_expert = 512, n_expert_used = 10, n_rel_attn_bkts = 0, n_embd_head_k_mla = 0, n_embd_head_v_mla = 0, posnet = {n_embd = 0, n_layer = 0}, convnext = {n_embd = 0, n_layer = 0}, n_shortconv_l_cache = 0, n_head_arr = {_M_elems = {16 <repeats 48 times>, 0 <repeats 464 times>}}, n_head_kv_arr = {_M_elems = {2 <repeats 48 times>, 0 <repeats 464 times>}}, n_ff_arr = {_M_elems = {5120 <repeats 48 times>, 0 <repeats 464 times>}}, n_layer_dense_lead = 0, n_lora_q = 0, n_lora_kv = 0, n_ff_exp = 512, n_ff_shexp = 512, n_ff_chexp = 0, n_expert_shared = 0, n_norm_groups = 0, n_expert_groups = 0, n_group_used = 0, n_group_experts = 0, expert_group_scale = 0.0500000007, expert_weights_scale = 0, expert_weights_norm = false, expert_gating_func = 0, moe_every_n_layers = 0, nextn_predict_layers = 0, f_norm_eps = 0, f_norm_rms_eps = 9.99999997e-07, f_norm_group_eps = 0, f_attn_logit_softcapping = 50, f_router_logit_softcapping = 30, f_final_logit_softcapping = 30, rescale_every_n_layers = 0, time_mix_extra_dim = 0, time_decay_extra_dim = 0, wkv_head_size = 0, token_shift_count = 2, n_lora_decay = 0, n_lora_iclr = 0, n_lora_value_res_mix = 0, n_lora_gate = 0, rope_attn_factor = 1, rope_freq_base_train = 10000000, rope_freq_base_train_swa = 10000000, rope_freq_scale_train = 1, rope_freq_scale_train_swa = 1, n_ctx_orig_yarn = 262144, rope_yarn_log_mul = 0, yarn_ext_factor = -1, yarn_attn_factor = 1, yarn_beta_fast = 32, yarn_beta_slow = 1, rope_sections = {_M_elems = {0, 0, 0, 0}}, swa_type = LLAMA_SWA_TYPE_NONE, n_swa = 0, swa_layers = {_M_elems = {false <repeats 512 times>}}, ssm_d_conv = 4, ssm_d_inner = 4096, ssm_d_state = 128, ssm_dt_rank = 32, ssm_n_group = 16, recurrent_layer_arr = {_M_elems = {true, true, true, false, true, true, true, false, true, true, true, false, true, true, true, false, true, true, true, false, true, true, true, false, true, true, true, false, true, true, true, false, true, true, true, false, true, true, true, false, true, true, true, false, true, true, true, false <repeats 465 times>}}, ssm_dt_b_c_rms = false, f_clamp_kqv = 0, f_max_alibi_bias = 0, f_logit_scale = 0, f_residual_scale = 0, f_embedding_scale = 0, f_attention_scale = 0, f_attn_out_scale = 0, attn_temp_length = 0, causal_attn = true, use_alibi = false, attn_soft_cap = false, use_kq_norm = false, n_cls_out = 1, n_moe_layer_step = 0, n_no_rope_layer_step = 4, n_attn_temp_floor_scale = 8192, f_attn_temp_scale = 0.100000001, n_altup = 4, i_altup_act = 0, laurel_rank = 64, n_embd_altup = 256, dense_2_feat_in = 0, dense_2_feat_out = 0, dense_3_feat_in = 0, dense_3_feat_out = 0, xielu_alpha_n = {_M_elems = {0 <repeats 512 times>}}, xielu_alpha_p = {_M_elems = {0 <repeats 512 times>}}, xielu_beta = {_M_elems = {0 <repeats 512 times>}}, xielu_eps = {_M_elems = {0 <repeats 512 times>}}, n_deepstack_layers = 0, dec_start_token_id = -1, dec_n_layer = 0, pooling_type = LLAMA_POOLING_TYPE_NONE, rope_type = LLAMA_ROPE_TYPE_NEOX, rope_scaling_type_train = LLAMA_ROPE_SCALING_TYPE_LINEAR}
        n_vocab = 151936
        n_embd = 2048
        output_all = false
        n_tokens_all = 1
        n_outputs_all = 1
        did_optimize = false
        mctx = std::unique_ptr<llama_memory_context_i> = {get() = 0x555563a99b30}
        n_outputs_prev = 0
#24 0x00007ffff7a42b31 in llama_decode (ctx=0x555556b7aeb0, batch=...) at /home/[---]/llama.cpp/src/llama-context.cpp:2780
        ret = 32767
        __func__ = "llama_decode"
#25 0x00005555555e8831 in main (argc=21, argv=0x7fffffffe1c8) at /home/[---]/llama.cpp/tools/main/main.cpp:679
        n_eval = 1
        i = 0
        max_embd_size = 31996
        params = {n_predict = -1, n_ctx = 32000, n_batch = 256, n_ubatch = 128, n_keep = 0, n_chunks = -1, n_parallel = 1, n_sequences = 1, grp_attn_n = 1, grp_attn_w = 512, n_print = -1, rope_freq_base = 0, rope_freq_scale = 0, yarn_ext_factor = -1, yarn_attn_factor = -1, yarn_beta_fast = -1, yarn_beta_slow = -1, yarn_orig_ctx = 0, devices = std::vector of length 0, capacity 0, n_gpu_layers = 999, main_gpu = 0, tensor_split = {0 <repeats 128 times>}, split_mode = LLAMA_SPLIT_MODE_LAYER, cpuparams = {n_threads = 8, cpumask = {false <repeats 512 times>}, mask_valid = false, priority = GGML_SCHED_PRIO_NORMAL, strict_cpu = false, poll = 50}, cpuparams_batch = {n_threads = 8, cpumask = {false <repeats 512 times>}, mask_valid = false, priority = GGML_SCHED_PRIO_NORMAL, strict_cpu = false, poll = 50}, cb_eval = 0x0, cb_eval_user_data = 0x0, numa = GGML_NUMA_STRATEGY_DISABLED, rope_scaling_type = LLAMA_ROPE_SCALING_TYPE_UNSPECIFIED, pooling_type = LLAMA_POOLING_TYPE_UNSPECIFIED, attention_type = LLAMA_ATTENTION_TYPE_UNSPECIFIED, flash_attn_type = LLAMA_FLASH_ATTN_TYPE_ENABLED, sampling = {seed = 4294967295, n_prev = 64, n_probs = 0, min_keep = 0, top_k = 20, top_p = 0.800000012, min_p = 0.0500000007, xtc_probability = 0, xtc_threshold = 0.100000001, typ_p = 1, temp = 0.699999988, dynatemp_range = 0, dynatemp_exponent = 1, penalty_last_n = 64, penalty_repeat = 1, penalty_freq = 0, penalty_present = 0, dry_multiplier = 0, dry_base = 1.75, dry_allowed_length = 2, dry_penalty_last_n = 32000, mirostat = 0, top_n_sigma = -1, mirostat_tau = 5, mirostat_eta = 0.100000001, ignore_eos = false, no_perf = false, timing_per_token = false, user_sampling_config = 0, dry_sequence_breakers = std::vector of length 4, capacity 4 = {"\n", ":", "\"", "*"}, samplers = std::vector of length 9, capacity 9 = {COMMON_SAMPLER_TYPE_PENALTIES, COMMON_SAMPLER_TYPE_DRY, COMMON_SAMPLER_TYPE_TOP_N_SIGMA, COMMON_SAMPLER_TYPE_TOP_K, COMMON_SAMPLER_TYPE_TYPICAL_P, COMMON_SAMPLER_TYPE_TOP_P, COMMON_SAMPLER_TYPE_MIN_P, COMMON_SAMPLER_TYPE_XTC, COMMON_SAMPLER_TYPE_TEMPERATURE}, grammar = "", grammar_lazy = false, grammar_triggers = std::vector of length 0, capacity 0, preserved_tokens = std::set with 0 elements, logit_bias = std::vector of length 0, capacity 0, logit_bias_eog = std::vector of length 5, capacity 8 = {{token = 151643, bias = -inf}, {token = 151645, bias = -inf}, {token = 151662, bias = -inf}, {token = 151663, bias = -inf}, {token = 151664, bias = -inf}}}, speculative = {devices = std::vector of length 0, capacity 0, n_ctx = 0, n_max = 16, n_min = 0, n_gpu_layers = -1, p_split = 0.100000001, p_min = 0.75, replacements = std::vector of length 0, capacity 0, tensor_buft_overrides = std::vector of length 0, capacity 0, cache_type_k = GGML_TYPE_F16, cache_type_v = GGML_TYPE_F16, cpuparams = {n_threads = 8, cpumask = {false <repeats 512 times>}, mask_valid = false, priority = GGML_SCHED_PRIO_NORMAL, strict_cpu = false, poll = 50}, cpuparams_batch = {n_threads = 8, cpumask = {false <repeats 512 times>}, mask_valid = false, priority = GGML_SCHED_PRIO_NORMAL, strict_cpu = false, poll = 50}, model = {path = "", url = "", hf_repo = "", hf_file = "", docker_repo = ""}}, vocoder = {model = {path = "", url = "", hf_repo = "", hf_file = "", docker_repo = ""}, speaker_file = "", use_guide_tokens = false}, diffusion = {steps = 128, visual_mode = false, eps = 0, block_length = 0, algorithm = 4, alg_temp = 0, cfg_scale = 0, add_gumbel_noise = false}, model = {path = "/home/[---]/.cache/llama.cpp/unsloth_Qwen3-Next-80B-A3B-Instruct-GGUF_Qwen3-Next-80B-A3B-Instruct-Q4_0.gguf", url = "https://huggingface.co/unsloth/Qwen3-Next-80B-A3B-Instruct-GGUF/resolve/main/Qwen3-Next-80B-A3B-Instruct-Q4_0.gguf", hf_repo = "unsloth/Qwen3-Next-80B-A3B-Instruct-GGUF", hf_file = "Qwen3-Next-80B-A3B-Instruct-Q4_0.gguf", docker_repo = ""}, model_alias = "", hf_token = "", prompt = "", system_prompt = "", prompt_file = "", path_prompt_cache = "", input_prefix = "", input_suffix = "", lookup_cache_static = "", lookup_cache_dynamic = "", logits_file = "", in_files = std::vector of length 0, capacity 0, antiprompt = std::vector of length 0, capacity 0, kv_overrides = std::vector of length 0, capacity 0, tensor_buft_overrides = std::vector of length 0, capacity 0, lora_init_without_apply = false, lora_adapters = std::vector of length 0, capacity 0, control_vectors = std::vector of length 0, capacity 0, verbosity = 0, control_vector_layer_start = -1, control_vector_layer_end = -1, offline = false, ppl_stride = 0, ppl_output_type = 0, hellaswag = false, hellaswag_tasks = 400, winogrande = false, winogrande_tasks = 0, multiple_choice = false, multiple_choice_tasks = 0, kl_divergence = false, usage = false, completion = false, use_color = false, special = false, interactive = true, interactive_first = true, prompt_cache_all = false, prompt_cache_ro = false, escape = true, multiline_input = false, simple_io = false, cont_batching = true, no_perf = false, ctx_shift = false, swa_full = false, kv_unified = false, input_prefix_bos = false, use_mmap = false, use_mlock = false, verbose_prompt = false, display_prompt = true, no_kv_offload = false, warmup = true, check_tensors = false, no_op_offload = false, no_extra_bufts = false, no_host = false, single_turn = false, cache_type_k = GGML_TYPE_Q8_0, cache_type_v = GGML_TYPE_Q8_0, conversation_mode = COMMON_CONVERSATION_MODE_ENABLED, mmproj = {path = "", url = "", hf_repo = "", hf_file = "", docker_repo = ""}, mmproj_use_gpu = true, no_mmproj = false, image = std::vector of length 0, capacity 0, image_min_tokens = -1, image_max_tokens = -1, lr = {lr0 = 9.99999975e-06, lr_min = -1, decay_epochs = -1, scale_epoch = 0, wd = 0, epochs = 2, epoch = 4294958456}, optimizer = GGML_OPT_OPTIMIZER_TYPE_ADAMW, val_split = 0.0500000007, embedding = false, embd_normalize = 2, embd_out = "", embd_sep = "\n", cls_sep = "\t", port = 8080, timeout_read = 600, timeout_write = 600, n_threads_http = -1, n_cache_reuse = 0, n_ctx_checkpoints = 8, cache_ram_mib = 8192, hostname = "127.0.0.1", public_path = "", api_prefix = "", chat_template = "", use_jinja = true, enable_chat_template = true, reasoning_format = COMMON_REASONING_FORMAT_DEEPSEEK, reasoning_budget = -1, prefill_assistant = true, api_keys = std::vector of length 0, capacity 0, ssl_file_key = "", ssl_file_cert = "", default_template_kwargs = std::map with 0 elements, webui = true, endpoint_slots = true, endpoint_props = false, endpoint_metrics = false, log_json = false, slot_save_path = "", slot_prompt_similarity = 0.100000001, is_pp_shared = false, is_tg_separate = false, n_pp = std::vector of length 0, capacity 0, n_tg = std::vector of length 0, capacity 0, n_pl = std::vector of length 0, capacity 0, context_files = std::vector of length 0, capacity 0, chunk_size = 64, chunk_separator = "\n", n_junk = 250, i_pos = -1, n_out_freq = 10, n_save_freq = 0, i_chunk = 0, imat_dat = 0 '\000', process_output = false, compute_ppl = true, show_statistics = false, parse_special = false, n_pca_batch = 100, n_pca_iterations = 1000, cvector_dimre_method = DIMRE_METHOD_PCA, cvector_positive_file = "tools/cvector-generator/positive.txt", cvector_negative_file = "tools/cvector-generator/negative.txt", spm_infill = false, batched_bench_output_jsonl = false, out_file = "", load_progress_callback = 0x0, load_progress_callback_user_data = 0x0}
        sparams = @0x7fffffffd220: {seed = 4294967295, n_prev = 64, n_probs = 0, min_keep = 0, top_k = 20, top_p = 0.800000012, min_p = 0.0500000007, xtc_probability = 0, xtc_threshold = 0.100000001, typ_p = 1, temp = 0.699999988, dynatemp_range = 0, dynatemp_exponent = 1, penalty_last_n = 64, penalty_repeat = 1, penalty_freq = 0, penalty_present = 0, dry_multiplier = 0, dry_base = 1.75, dry_allowed_length = 2, dry_penalty_last_n = 32000, mirostat = 0, top_n_sigma = -1, mirostat_tau = 5, mirostat_eta = 0.100000001, ignore_eos = false, no_perf = false, timing_per_token = false, user_sampling_config = 0, dry_sequence_breakers = std::vector of length 4, capacity 4 = {"\n", ":", "\"", "*"}, samplers = std::vector of length 9, capacity 9 = {COMMON_SAMPLER_TYPE_PENALTIES, COMMON_SAMPLER_TYPE_DRY, COMMON_SAMPLER_TYPE_TOP_N_SIGMA, COMMON_SAMPLER_TYPE_TOP_K, COMMON_SAMPLER_TYPE_TYPICAL_P, COMMON_SAMPLER_TYPE_TOP_P, COMMON_SAMPLER_TYPE_MIN_P, COMMON_SAMPLER_TYPE_XTC, COMMON_SAMPLER_TYPE_TEMPERATURE}, grammar = "", grammar_lazy = false, grammar_triggers = std::vector of length 0, capacity 0, preserved_tokens = std::set with 0 elements, logit_bias = std::vector of length 0, capacity 0, logit_bias_eog = std::vector of length 5, capacity 8 = {{token = 151643, bias = -inf}, {token = 151645, bias = -inf}, {token = 151662, bias = -inf}, {token = 151663, bias = -inf}, {token = 151664, bias = -inf}}}
        __func__ = "main"
        model = 0x555556dd1ea0
        ctx = 0x555556b7aeb0
        smpl = 0x555563726c40
        chat_msgs = std::vector of length 1, capacity 1 = {{role = "user", content = "Hey", content_parts = std::vector of length 0, capacity 0, tool_calls = std::vector of length 0, capacity 0, reasoning_content = "", tool_name = "", tool_call_id = ""}}
        llama_init = {model = std::unique_ptr<llama_model> = {get() = 0x555556dd1ea0}, context = std::unique_ptr<llama_context> = {get() = 0x555556b7aeb0}, lora = std::vector of length 0, capacity 0}
        mem = 0x555556279f70
        vocab = 0x555556dd5c60
        chat_templates = std::unique_ptr<common_chat_templates> = {get() = 0x555564066b00}
        cpu_dev = 0x7ffff73d0480 <ggml_backend_cpu_reg_get_device(ggml_backend_reg*, unsigned long)::ggml_backend_cpu_device>
        reg = 0x7ffff73d0520 <ggml_backend_cpu_reg::ggml_backend_cpu_reg>
        ggml_threadpool_new_fn = 0x7ffff72256eb <ggml_threadpool_new>
        ggml_threadpool_free_fn = 0x7ffff722460e <ggml_threadpool_free>
        tpp_batch = {cpumask = {false <repeats 512 times>}, n_threads = 8, prio = GGML_SCHED_PRIO_NORMAL, poll = 50, strict_cpu = false, paused = false}
        tpp = {cpumask = {false <repeats 512 times>}, n_threads = 8, prio = GGML_SCHED_PRIO_NORMAL, poll = 50, strict_cpu = false, paused = false}
        threadpool_batch = 0x0
        threadpool = 0x55556406a740
        n_ctx_train = 262144
        n_ctx = 32000
        has_chat_template = true
        path_session = ""
        session_tokens = std::vector of length 0, capacity 0
        add_bos = false
        embd_inp = std::vector of length 9, capacity 9 = {151644, 872, 198, 18665, 151645, 198, 151644, 77091, 198}
        waiting_for_first_input = false
        chat_add_and_format = {__chat_msgs = std::vector of length 1, capacity 1 = {{role = "user", content = "Hey", content_parts = std::vector of length 0, capacity 0, tool_calls = std::vector of length 0, capacity 0, reasoning_content = "", tool_name = "", tool_call_id = ""}}, __chat_templates = std::unique_ptr<common_chat_templates> = {get() = 0x555564066b00}}
        prompt = ""
        n_matching_session_tokens = 0
        ga_i = 0
        ga_n = 1
        ga_w = 512
        is_antiprompt = false
        input_echo = true
        display = true
        need_to_save_session = false
        n_past = 9
        n_remain = -11
        n_consumed = 9
        n_session_consumed = 0
        input_tokens = std::vector of length 0, capacity 0
        output_tokens = std::vector of length 10, capacity 16 = {151644, 872, 198, 18665, 151645, 198, 151644, 77091, 198, 9707}
        output_ss = "<|im_start|>user\nHey<|im_end|>\n<|im_start|>assistant\nHello"
        assistant_ss = "Hello"
        embd = std::vector of length 1, capacity 16 = {9707}
        antiprompt_token = std::vector of length 0, capacity 0
(gdb) info registers
rax            0xc24448d48402484   874899601209959556
rbx            0x7fffffff5960      140737488312672
rcx            0x0                 0
rdx            0x7fffffff61c0      140737488314816
rsi            0x3                 3
rdi            0x7ffff04a5400      140737224791040
rbp            0x7fffffff5810      0x7fffffff5810
rsp            0x7fffffff5780      0x7fffffff5780
r8             0x0                 0
r9             0x0                 0
r10            0x0                 0
r11            0x0                 0
r12            0x0                 0
r13            0x7ffff04a5400      140737224791040
r14            0x5555561686a0      93825004897952
r15            0x0                 0
rip            0x7fffecaad59f      0x7fffecaad59f
eflags         0x10216             [ PF AF IF RF ]
cs             0x33                51
ss             0x2b                43
ds             0x0                 0
es             0x0                 0
fs             0x0                 0
gs             0x0                 0
k0             0x1                 1
k1             0x0                 0
k2             0x1ffffff           33554431
k3             0xfffeefdf          4294897631
k4             0xffffffff          4294967295
k5             0x0                 0
k6             0x0                 0
k7             0x0                 0
fs_base        0x7fffee1f98c0      140737188436160
gs_base        0x0                 0
(gdb) info threads
  Id   Target Id                                     Frame
* 1    Thread 0x7fffee1f98c0 (LWP 15631) "llama-cli" 0x00007fffecaad59f in ?? () from /opt/rocm-7.1.0/lib/libamdhip64.so.7
  2    Thread 0x7fffb1bff6c0 (LWP 15637) "llama-cli" __GI___ioctl (fd=3, request=3222817548) at ../sysdeps/unix/sysv/linux/ioctl.c:36
  8    Thread 0x7ffdabbff6c0 (LWP 15643) "llama-cli" 0x00007ffff6a98d71 in __futex_abstimed_wait_common64 (private=32753, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x555555986df8 <common_log_main()::log+88>) at ./nptl/futex-internal.c:57
  9    Thread 0x7ffda97ff6c0 (LWP 15644) "llama-cli" __GI___ioctl (fd=3, request=3222817548) at ../sysdeps/unix/sysv/linux/ioctl.c:36
  10   Thread 0x7ff23f7ff6c0 (LWP 15645) "llama-cli" futex_wait (addr=0x55555a4fec54, val=1192) at ../../../src/libgomp/config/linux/x86/futex.h:97
  11   Thread 0x7ff23effe6c0 (LWP 15646) "llama-cli" futex_wait (addr=0x55555a4fec54, val=1192) at ../../../src/libgomp/config/linux/x86/futex.h:97
  12   Thread 0x7ff23e7fd6c0 (LWP 15647) "llama-cli" futex_wait (addr=0x55555a4fec54, val=1192) at ../../../src/libgomp/config/linux/x86/futex.h:97
  13   Thread 0x7ff23dffc6c0 (LWP 15648) "llama-cli" futex_wait (addr=0x55555a4fec54, val=1192) at ../../../src/libgomp/config/linux/x86/futex.h:97
  14   Thread 0x7ff23d7fb6c0 (LWP 15649) "llama-cli" futex_wait (addr=0x55555a4fec54, val=1192) at ../../../src/libgomp/config/linux/x86/futex.h:97
  15   Thread 0x7ff23cffa6c0 (LWP 15650) "llama-cli" futex_wait (addr=0x55555a4fec54, val=1192) at ../../../src/libgomp/config/linux/x86/futex.h:97
  16   Thread 0x7ff23c7f96c0 (LWP 15651) "llama-cli" futex_wait (addr=0x55555a4fec54, val=1192) at ../../../src/libgomp/config/linux/x86/futex.h:97
(gdb)

[fuutott]

cat buildllama.sh
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
git submodule update --init --recursive
HIPCXX="$(hipconfig -l)/clang" HIP_PATH="$(hipconfig -R)"     cmake -S . -B build -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx906 -DGGML_HIP_ROCWMMA_FATTN=ON -DCMAKE_BUILD_TYPE=Debug     && cmake --build build --config Debug -- -j 16
#HIPCXX="$(hipconfig -l)/clang" HIP_PATH="$(hipconfig -R)"     cmake -S . -B build -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx906 -DGGML_HIP_ROCWMMA_FATTN=ON -DCMAKE_BUILD_TYPE=Release     && cmake --build build --config Release -- -j 16
sudo cmake --install build

[JohannesGaessler]

I am unable to reproduce the issue using ROCm 6.4 and a single MI50.

From what I can tell you are using ROCm 7 which dropped MI50 support. And the segfault is occurring specifically in hipBLAS. So it looks to me that this is an issue outside of llama.cpp/ggml that we fundamentally cannot fix unless we write our own GEMM kernel for floating-point data.

[phoyd]

My gfx906-related crash on a 3×MI50 setup produces a different diagnostic. One striking point is that it begins with an error report stating that an invalid device function was called. I’ve seen this message before when I tried to run ComfyUI with ROCm 7.0.2 on Ubuntu and missing Tensile libraries copied from Arch Linux. I had to go back to ROCm 6.3 to resolve this. Unfortunately, llama.cpp does not compile with ROCm 6.3 right now, so I cannot investigate this further.

The hipErrorInvalidDeviceFunction part:

> hi there
HellorocBLAS error from hip error code: 'hipErrorInvalidDeviceFunction':98
ROCm error: CUBLAS_STATUS_INTERNAL_ERROR
  current device: 0, in function ggml_cuda_mul_mat_batched_cublas_impl at /home/llama/Projects/llama.cpp/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:1952
  hipblasGemmStridedBatchedEx(ctx.cublas_handle(), HIPBLAS_OP_T, HIPBLAS_OP_N, ne01, ne11, ne10, alpha, src0_ptr, cu_data_type_a, nb01/nb00, sma, src1_ptr, cu_data_type_b, s11, smb, beta, dst_t, cu_data_type, ne0, ne1*ne0, ne12*ne13, cu_compute_type, HIPBLAS_GEMM_DEFAULT)
/home/llama/Projects/llama.cpp/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:90: ROCm error

The full output from llama-cli:

$ ./bin/llama-cli -m ~/.cache/llama.cpp/unsloth_Qwen3-Next-80B-A3B-Instruct-GGUF_Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL.gguf
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 3 ROCm devices:
  Device 0: AMD Radeon Graphics, gfx906:sramecc+:xnack- (0x906), VMM: no, Wave Size: 64
  Device 1: AMD Radeon Graphics, gfx906:sramecc+:xnack- (0x906), VMM: no, Wave Size: 64
  Device 2: AMD Radeon Graphics, gfx906:sramecc+:xnack- (0x906), VMM: no, Wave Size: 64
build: 7200 (ab49f094d) with cc (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0 for x86_64-linux-gnu
main: llama backend init
main: load the model and apply lora adapter, if any
llama_model_load_from_file_impl: using device ROCm0 (AMD Radeon Graphics) (0000:83:00.0) - 32704 MiB free
llama_model_load_from_file_impl: using device ROCm1 (AMD Radeon Graphics) (0000:43:00.0) - 32704 MiB free
llama_model_load_from_file_impl: using device ROCm2 (AMD Radeon Graphics) (0000:46:00.0) - 32704 MiB free
llama_model_loader: loaded meta data with 49 key-value pairs and 807 tensors from /home/llama/.cache/llama.cpp/unsloth_Qwen3-Next-80B-A3B-Instruct-GGUF_Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = qwen3next
llama_model_loader: - kv   1:                               general.type str              = model
llama_model_loader: - kv   2:                     general.sampling.top_k i32              = 20
llama_model_loader: - kv   3:                     general.sampling.top_p f32              = 0.800000
llama_model_loader: - kv   4:                      general.sampling.temp f32              = 0.700000
llama_model_loader: - kv   5:                               general.name str              = Qwen3-Next-80B-A3B-Instruct
llama_model_loader: - kv   6:                           general.finetune str              = Instruct
llama_model_loader: - kv   7:                           general.basename str              = Qwen3-Next-80B-A3B-Instruct
llama_model_loader: - kv   8:                       general.quantized_by str              = Unsloth
llama_model_loader: - kv   9:                         general.size_label str              = 80B-A3B
llama_model_loader: - kv  10:                            general.license str              = apache-2.0
llama_model_loader: - kv  11:                       general.license.link str              = https://huggingface.co/Qwen/Qwen3-Nex...
llama_model_loader: - kv  12:                           general.repo_url str              = https://huggingface.co/unsloth
llama_model_loader: - kv  13:                   general.base_model.count u32              = 1
llama_model_loader: - kv  14:                  general.base_model.0.name str              = Qwen3 Next 80B A3B Instruct
llama_model_loader: - kv  15:          general.base_model.0.organization str              = Qwen
llama_model_loader: - kv  16:              general.base_model.0.repo_url str              = https://huggingface.co/Qwen/Qwen3-Nex...
llama_model_loader: - kv  17:                               general.tags arr[str,2]       = ["unsloth", "text-generation"]
llama_model_loader: - kv  18:                      qwen3next.block_count u32              = 48
llama_model_loader: - kv  19:                   qwen3next.context_length u32              = 262144
llama_model_loader: - kv  20:                 qwen3next.embedding_length u32              = 2048
llama_model_loader: - kv  21:              qwen3next.feed_forward_length u32              = 5120
llama_model_loader: - kv  22:             qwen3next.attention.head_count u32              = 16
llama_model_loader: - kv  23:          qwen3next.attention.head_count_kv u32              = 2
llama_model_loader: - kv  24:                   qwen3next.rope.freq_base f32              = 10000000.000000
llama_model_loader: - kv  25: qwen3next.attention.layer_norm_rms_epsilon f32              = 0.000001
llama_model_loader: - kv  26:                qwen3next.expert_used_count u32              = 10
llama_model_loader: - kv  27:             qwen3next.attention.key_length u32              = 256
llama_model_loader: - kv  28:           qwen3next.attention.value_length u32              = 256
llama_model_loader: - kv  29:                     qwen3next.expert_count u32              = 512
llama_model_loader: - kv  30:       qwen3next.expert_feed_forward_length u32              = 512
llama_model_loader: - kv  31: qwen3next.expert_shared_feed_forward_length u32              = 512
llama_model_loader: - kv  32:                  qwen3next.ssm.conv_kernel u32              = 4
llama_model_loader: - kv  33:                   qwen3next.ssm.state_size u32              = 128
llama_model_loader: - kv  34:                  qwen3next.ssm.group_count u32              = 16
llama_model_loader: - kv  35:               qwen3next.ssm.time_step_rank u32              = 32
llama_model_loader: - kv  36:                   qwen3next.ssm.inner_size u32              = 4096
llama_model_loader: - kv  37:             qwen3next.rope.dimension_count u32              = 64
llama_model_loader: - kv  38:                       tokenizer.ggml.model str              = gpt2
llama_model_loader: - kv  39:                         tokenizer.ggml.pre str              = qwen2
llama_model_loader: - kv  40:                      tokenizer.ggml.tokens arr[str,151936]  = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv  41:                  tokenizer.ggml.token_type arr[i32,151936]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv  42:                      tokenizer.ggml.merges arr[str,151387]  = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
llama_model_loader: - kv  43:                tokenizer.ggml.eos_token_id u32              = 151645
llama_model_loader: - kv  44:            tokenizer.ggml.padding_token_id u32              = 151654
llama_model_loader: - kv  45:               tokenizer.ggml.add_bos_token bool             = false
llama_model_loader: - kv  46:                    tokenizer.chat_template str              = {%- if tools %}\n    {{- '<|im_start|>...
llama_model_loader: - kv  47:               general.quantization_version u32              = 2
llama_model_loader: - kv  48:                          general.file_type u32              = 15
llama_model_loader: - type  f32:  313 tensors
llama_model_loader: - type q8_0:    3 tensors
llama_model_loader: - type q4_K:  390 tensors
llama_model_loader: - type q6_K:   53 tensors
llama_model_loader: - type bf16:   48 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type   = Q4_K - Medium
print_info: file size   = 42.01 GiB (4.53 BPW)
load: printing all EOG tokens:
load:   - 151643 ('<|endoftext|>')
load:   - 151645 ('<|im_end|>')
load:   - 151662 ('<|fim_pad|>')
load:   - 151663 ('<|repo_name|>')
load:   - 151664 ('<|file_sep|>')
load: special tokens cache size = 26
load: token to piece cache size = 0.9311 MB
print_info: arch             = qwen3next
print_info: vocab_only       = 0
print_info: n_ctx_train      = 262144
print_info: n_embd           = 2048
print_info: n_embd_inp       = 2048
print_info: n_layer          = 48
print_info: n_head           = 16
print_info: n_head_kv        = 2
print_info: n_rot            = 64
print_info: n_swa            = 0
print_info: is_swa_any       = 0
print_info: n_embd_head_k    = 256
print_info: n_embd_head_v    = 256
print_info: n_gqa            = 8
print_info: n_embd_k_gqa     = 512
print_info: n_embd_v_gqa     = 512
print_info: f_norm_eps       = 0.0e+00
print_info: f_norm_rms_eps   = 1.0e-06
print_info: f_clamp_kqv      = 0.0e+00
print_info: f_max_alibi_bias = 0.0e+00
print_info: f_logit_scale    = 0.0e+00
print_info: f_attn_scale     = 0.0e+00
print_info: n_ff             = 5120
print_info: n_expert         = 512
print_info: n_expert_used    = 10
print_info: n_expert_groups  = 0
print_info: n_group_used     = 0
print_info: causal attn      = 1
print_info: pooling type     = 0
print_info: rope type        = 2
print_info: rope scaling     = linear
print_info: freq_base_train  = 10000000.0
print_info: freq_scale_train = 1
print_info: n_ctx_orig_yarn  = 262144
print_info: rope_finetuned   = unknown
print_info: ssm_d_conv       = 4
print_info: ssm_d_inner      = 4096
print_info: ssm_d_state      = 128
print_info: ssm_dt_rank      = 32
print_info: ssm_n_group      = 16
print_info: ssm_dt_b_c_rms   = 0
print_info: model type       = ?B
print_info: model params     = 79.67 B
print_info: general.name     = Qwen3-Next-80B-A3B-Instruct
print_info: vocab type       = BPE
print_info: n_vocab          = 151936
print_info: n_merges         = 151387
print_info: BOS token        = 11 ','
print_info: EOS token        = 151645 '<|im_end|>'
print_info: EOT token        = 151645 '<|im_end|>'
print_info: PAD token        = 151654 '<|vision_pad|>'
print_info: LF token         = 198 'Ċ'
print_info: FIM PRE token    = 151659 '<|fim_prefix|>'
print_info: FIM SUF token    = 151661 '<|fim_suffix|>'
print_info: FIM MID token    = 151660 '<|fim_middle|>'
print_info: FIM PAD token    = 151662 '<|fim_pad|>'
print_info: FIM REP token    = 151663 '<|repo_name|>'
print_info: FIM SEP token    = 151664 '<|file_sep|>'
print_info: EOG token        = 151643 '<|endoftext|>'
print_info: EOG token        = 151645 '<|im_end|>'
print_info: EOG token        = 151662 '<|fim_pad|>'
print_info: EOG token        = 151663 '<|repo_name|>'
print_info: EOG token        = 151664 '<|file_sep|>'
print_info: max token length = 256
load_tensors: loading model tensors, this can take a while... (mmap = true)
load_tensors: offloading 48 repeating layers to GPU
load_tensors: offloading output layer to GPU
load_tensors: offloaded 49/49 layers to GPU
load_tensors:   CPU_Mapped model buffer size = 41868.19 MiB
load_tensors:        ROCm0 model buffer size = 15088.58 MiB
load_tensors:        ROCm1 model buffer size = 14205.03 MiB
load_tensors:        ROCm2 model buffer size = 13553.88 MiB
....................................................................................................
llama_context: constructing llama_context
llama_context: n_seq_max     = 1
llama_context: n_ctx         = 4096
llama_context: n_ctx_seq     = 4096
llama_context: n_batch       = 2048
llama_context: n_ubatch      = 512
llama_context: causal_attn   = 1
llama_context: flash_attn    = auto
llama_context: kv_unified    = false
llama_context: freq_base     = 10000000.0
llama_context: freq_scale    = 1
llama_context: n_ctx_seq (4096) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
llama_context:  ROCm_Host  output buffer size =     0.58 MiB
llama_kv_cache:      ROCm0 KV buffer size =    32.00 MiB
llama_kv_cache:      ROCm1 KV buffer size =    32.00 MiB
llama_kv_cache:      ROCm2 KV buffer size =    32.00 MiB
llama_kv_cache: size =   96.00 MiB (  4096 cells,  12 layers,  1/1 seqs), K (f16):   48.00 MiB, V (f16):   48.00 MiB
llama_memory_recurrent:      ROCm0 RS buffer size =    27.22 MiB
llama_memory_recurrent:      ROCm1 RS buffer size =    25.12 MiB
llama_memory_recurrent:      ROCm2 RS buffer size =    23.03 MiB
llama_memory_recurrent: size =   75.38 MiB (     1 cells,  48 layers,  1 seqs), R (f32):    3.38 MiB, S (f32):   72.00 MiB
llama_context: pipeline parallelism enabled (n_copies=4)
llama_context: Flash Attention was auto, set to enabled
llama_context:      ROCm0 compute buffer size =  9294.38 MiB
llama_context:      ROCm1 compute buffer size =  2205.23 MiB
llama_context:      ROCm2 compute buffer size =  2380.98 MiB
llama_context:  ROCm_Host compute buffer size =  1199.12 MiB
llama_context: graph nodes  = 21985 (with bs=512), 8449 (with bs=1)
llama_context: graph splits = 213 (with bs=512), 154 (with bs=1)
common_init_from_params: added <|endoftext|> logit bias = -inf
common_init_from_params: added <|im_end|> logit bias = -inf
common_init_from_params: added <|fim_pad|> logit bias = -inf
common_init_from_params: added <|repo_name|> logit bias = -inf
common_init_from_params: added <|file_sep|> logit bias = -inf
common_init_from_params: setting dry_penalty_last_n to ctx_size = 4096
common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
main: llama threadpool init, n_threads = 12
main: chat template is available, enabling conversation mode (disable it with -no-cnv)
main: chat template example:
<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
Hello<|im_end|>
<|im_start|>assistant
Hi there<|im_end|>
<|im_start|>user
How are you?<|im_end|>
<|im_start|>assistant


system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |

main: interactive mode on.
sampler seed: 4097783396
sampler params:
	repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
	dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 4096
	top_k = 20, top_p = 0.800, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.700
	mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000
sampler chain: logits -> logit-bias -> penalties -> dry -> top-n-sigma -> top-k -> typical -> top-p -> min-p -> xtc -> temp-ext -> dist
generate: n_ctx = 4096, n_batch = 2048, n_predict = -1, n_keep = 0

== Running in interactive mode. ==
 - Press Ctrl+C to interject at any time.
 - Press Return to return control to the AI.
 - To return control without starting a new line, end your input with '/'.
 - If you want to submit another line, end your input with '\'.
 - Not using system message. To change it, set a different value via -sys PROMPT


> hi there
HellorocBLAS error from hip error code: 'hipErrorInvalidDeviceFunction':98
ROCm error: CUBLAS_STATUS_INTERNAL_ERROR
  current device: 0, in function ggml_cuda_mul_mat_batched_cublas_impl at /home/llama/Projects/llama.cpp/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:1952
  hipblasGemmStridedBatchedEx(ctx.cublas_handle(), HIPBLAS_OP_T, HIPBLAS_OP_N, ne01, ne11, ne10, alpha, src0_ptr, cu_data_type_a, nb01/nb00, sma, src1_ptr, cu_data_type_b, s11, smb, beta, dst_t, cu_data_type, ne0, ne1*ne0, ne12*ne13, cu_compute_type, HIPBLAS_GEMM_DEFAULT)
/home/llama/Projects/llama.cpp/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:90: ROCm error
[New LWP 1192245]
[New LWP 1192223]
[New LWP 1192222]
[New LWP 1192221]
[New LWP 1192220]
[New LWP 1192219]
[New LWP 1192218]
[New LWP 1192217]
[New LWP 1192216]
[New LWP 1192215]
[New LWP 1192214]
[New LWP 1192213]
[New LWP 1192179]
[New LWP 1192177]
[New LWP 1192174]

This GDB supports auto-downloading debuginfo from the following URLs:
  <https://debuginfod.ubuntu.com>
Enable debuginfod for this session? (y or [n]) [answered N; input not from terminal]
Debuginfod has been disabled.
To make this setting permanent, add 'set debuginfod enabled off' to .gdbinit.
warning: could not find '.gnu_debugaltlink' file for /lib/x86_64-linux-gnu/liblber.so.2
warning: could not find '.gnu_debugaltlink' file for /lib/x86_64-linux-gnu/libbrotlidec.so.1
warning: could not find '.gnu_debugaltlink' file for /lib/x86_64-linux-gnu/libbrotlicommon.so.1
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007e4a0c710813 in __GI___wait4 (pid=1192246, stat_loc=0x0, options=0, usage=0x0) at ../sysdeps/unix/sysv/linux/wait4.c:30
warning: 30	../sysdeps/unix/sysv/linux/wait4.c: No such file or directory
#0  0x00007e4a0c710813 in __GI___wait4 (pid=1192246, stat_loc=0x0, options=0, usage=0x0) at ../sysdeps/unix/sysv/linux/wait4.c:30
30	in ../sysdeps/unix/sysv/linux/wait4.c
#1  0x00007e4a0cd6f673 in ggml_print_backtrace () from /home/llama/Projects/llama.cpp/build-rocm7/bin/libggml-base.so.0
#2  0x00007e4a0cd6f81b in ggml_abort () from /home/llama/Projects/llama.cpp/build-rocm7/bin/libggml-base.so.0
#3  0x00007e4a0bea6e42 in ggml_cuda_error(char const*, char const*, char const*, int, char const*) () from /home/llama/Projects/llama.cpp/build-rocm7/bin/libggml-hip.so.0
#4  0x00007e4a0beb38fc in ggml_cuda_mul_mat_batched_cublas(ggml_backend_cuda_context&, ggml_tensor const*, ggml_tensor const*, ggml_tensor*) () from /home/llama/Projects/llama.cpp/build-rocm7/bin/libggml-hip.so.0
#5  0x00007e4a0beb08ed in ggml_cuda_mul_mat(ggml_backend_cuda_context&, ggml_tensor const*, ggml_tensor const*, ggml_tensor*) () from /home/llama/Projects/llama.cpp/build-rocm7/bin/libggml-hip.so.0
#6  0x00007e4a0bead20e in evaluate_and_capture_cuda_graph(ggml_backend_cuda_context*, ggml_cgraph*, bool&, bool&, bool&) () from /home/llama/Projects/llama.cpp/build-rocm7/bin/libggml-hip.so.0
#7  0x00007e4a0beac28f in ggml_backend_cuda_graph_compute(ggml_backend*, ggml_cgraph*) () from /home/llama/Projects/llama.cpp/build-rocm7/bin/libggml-hip.so.0
#8  0x00007e4a0cd8b297 in ggml_backend_sched_graph_compute_async () from /home/llama/Projects/llama.cpp/build-rocm7/bin/libggml-base.so.0
#9  0x00007e4a0cea79b1 in llama_context::graph_compute(ggml_cgraph*, bool) () from /home/llama/Projects/llama.cpp/build-rocm7/bin/libllama.so.0
#10 0x00007e4a0cea940c in llama_context::process_ubatch(llama_ubatch const&, llm_graph_type, llama_memory_context_i*, ggml_status&) () from /home/llama/Projects/llama.cpp/build-rocm7/bin/libllama.so.0
#11 0x00007e4a0ceaf7a7 in llama_context::decode(llama_batch const&) () from /home/llama/Projects/llama.cpp/build-rocm7/bin/libllama.so.0
#12 0x00007e4a0ceb06bf in llama_decode () from /home/llama/Projects/llama.cpp/build-rocm7/bin/libllama.so.0
#13 0x00005c259efd5328 in main ()
[Inferior 1 (process 1192173) detached]
Aborted (core dumped)

[Kotali-2019]

I’m experiencing the same issue on my gfx906 (MI50 cards). This error occurs only with the QWEN3-NEXT model.

The error logs are :

slot get_availabl: id  3 | task -1 | selected slot by LRU, t_last = -1
slot launch_slot_: id  3 | task -1 | sampler chain: logits -> logit-bias -> penalties -> dry -> top-n-sigma -> top-k -> typical -> top-p -> min-p -> xtc -> temp-ext -> dist
slot launch_slot_: id  3 | task 1 | processing task
slot update_slots: id  3 | task 1 | new prompt, n_ctx_slot = 32768, n_keep = 0, task.n_tokens = 9
slot update_slots: id  3 | task 1 | n_tokens = 0, memory_seq_rm [0, end)
slot update_slots: id  3 | task 1 | prompt processing progress, n_tokens = 9, batch.n_tokens = 9, progress = 1.000000
slot update_slots: id  3 | task 1 | prompt done, n_tokens = 9, batch.n_tokens = 9
rocBLAS error from hip error code: 'hipErrorInvalidDeviceFunction':98
/root/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:90: ROCm error
ROCm error: CUBLAS_STATUS_INTERNAL_ERROR
  current device: 0, in function ggml_cuda_mul_mat_batched_cublas_impl at /root/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:1953
  hipblasGemmStridedBatchedEx(ctx.cublas_handle(), HIPBLAS_OP_T, HIPBLAS_OP_N, ne01, ne11, ne10, alpha, src0_ptr, cu_data_type_a, nb01/nb00, sma, src1_ptr, cu_data_type_b, s11, smb, beta, dst_t, cu_data_type, ne0, ne1*ne0, ne12*ne13, cu_compute_type, HIPBLAS_GEMM_DEFAULT)
srv  log_server_r: request: GET /props 127.0.0.1 200
/root/llama.cpp/build-7205-rocm906/bin/libggml-base.so.0(+0x162b8)[0x7f1ee9a922b8]
/root/llama.cpp/build-7205-rocm906/bin/libggml-base.so.0(ggml_print_backtrace+0x1e4)[0x7f1ee9a92684]
/root/llama.cpp/build-7205-rocm906/bin/libggml-base.so.0(ggml_abort+0x11e)[0x7f1ee9a9280e]
/root/llama.cpp/build-7205-rocm906/bin/libggml-hip.so.0(+0x28a8ab2)[0x7f1ee8aa8ab2]
/root/llama.cpp/build-7205-rocm906/bin/libggml-hip.so.0(+0x28b7ea7)[0x7f1ee8ab7ea7]
/root/llama.cpp/build-7205-rocm906/bin/libggml-hip.so.0(+0x28b4d5d)[0x7f1ee8ab4d5d]
/root/llama.cpp/build-7205-rocm906/bin/libggml-hip.so.0(+0x28b0f36)[0x7f1ee8ab0f36]
/root/llama.cpp/build-7205-rocm906/bin/libggml-hip.so.0(+0x28ade5f)[0x7f1ee8aade5f]
/root/llama.cpp/build-7205-rocm906/bin/libggml-base.so.0(ggml_backend_sched_graph_compute_async+0x807)[0x7f1ee9aacf87]
/root/llama.cpp/build-7205-rocm906/bin/libllama.so.0(_ZN13llama_context13graph_computeEP11ggml_cgraphb+0xa1)[0x7f1ee989e541]
/root/llama.cpp/build-7205-rocm906/bin/libllama.so.0(_ZN13llama_context14process_ubatchERK12llama_ubatch14llm_graph_typeP22llama_memory_context_iR11ggml_status+0xe1)[0x7f1ee989fbd1]
/root/llama.cpp/build-7205-rocm906/bin/libllama.so.0(_ZN13llama_context6decodeERK11llama_batch+0x2a1)[0x7f1ee98a5451]
/root/llama.cpp/build-7205-rocm906/bin/libllama.so.0(llama_decode+0xb)[0x7f1ee98a634b]
./llama-server(+0x125bdc)[0x560f890cebdc]
./llama-server(+0xd4bcb)[0x560f8907dbcb]
./llama-server(+0x875ba)[0x560f890305ba]
/lib/x86_64-linux-gnu/libc.so.6(+0x2724a)[0x7f1ee924524a]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x85)[0x7f1ee9245305]
./llama-server(+0x89731)[0x560f89032731]
Aborted

The llama.cpp version 7205 (fa04659)

ROCm version 7.0.3.70003

rocBLAS version 5.0.2.70003