mithulaartigala/reasoning-summarizer-qwen3.5-0.8b-preview-gguf

Reasoning Summarizer — Qwen3.5 0.8B Preview (GGUF)

I converted my Reasoning Summarizer Qwen3.5 0.8B preview to GGUF. I fine-tuned the source model to turn plain-text reasoning chains into a JSON object with four fields:

{"title": "...", "sub_title": "...", "summary": "...", "cur_task": "..."}

title and sub_title label the reasoning section, summary describes it, and cur_task gives a short description of the current task. I trained the source checkpoint on raw reasoning text followed by a newline, without a system prompt.

Included exports

I include only the files listed below. Existing GGUF file names are retained for compatibility.

File	Size (decimal MB)
reasoning-summarizer-800m-pre-F16.gguf	1516.74 MB
reasoning-summarizer-800m-pre-Q4_K_M.gguf	529.29 MB
reasoning-summarizer-800m-pre-Q4_K_S.gguf	505.25 MB
reasoning-summarizer-800m-pre-Q5_K_M.gguf	577.99 MB
reasoning-summarizer-800m-pre-Q6_K.gguf	629.74 MB
reasoning-summarizer-800m-pre-Q8_0.gguf	811.84 MB

I also include smoke_test_results.json, which records local tests of multiple export candidates. Some candidates in that report are not present in this repository. The report is a smoke test, not a broad quality or latency benchmark; its historical sizes can differ from the uploaded files above.

Inference

Use a llama.cpp build with Qwen3.5 support and the llama-completion executable for raw text completion. Older builds may expose raw completion through llama-cli -no-cnv instead. Pass the raw reasoning text as a completion prompt and keep the trailing newline. This model is a text-only reasoning metadata model.

hf download mithulaartigala/reasoning-summarizer-qwen3.5-0.8b-preview-gguf reasoning-summarizer-800m-pre-Q6_K.gguf --local-dir ./reasoning-summarizer
llama-completion -m ./reasoning-summarizer/reasoning-summarizer-800m-pre-Q6_K.gguf -no-cnv -p "The API returns 401. I need to check authorization headers and token expiry.
" -n 256 --temp 0

Validate the generated JSON and check that it contains exactly title, sub_title, summary, and cur_task. Smaller quantizations may reduce structured-output reliability. I have not established a quality ranking across these exports.

Training and continued fine-tuning

My source checkpoint uses Qwen/Qwen3.5-0.8B-Base, LoRA SFT with rank 32 and alpha 64, a 4,096-token training context, and assistant-output loss masking. Its saved final evaluation loss is 0.7920386046171188.

I used Reasoning Summaries 61K, a public dataset hosted under my personal account. For continued training, use my source repository, which contains the merged Transformers checkpoint and adapters. This GGUF repository contains inference exports, not Transformers weights, tokenizer files, or LoRA adapters.

Limitations and license

I designed this small model for reasoning metadata extraction rather than general chat. It can produce malformed JSON, omit details, or generate an inaccurate summary. The smoke tests do not establish performance across domains.

I retain the source checkpoint's Apache-2.0 license and Qwen base-model attribution. My work fine-tunes the base weights and converts or quantizes the result for GGUF inference. See LICENSE.

Author and citation

I am Mithula Artigala, and I maintain this model and its card. Please cite my model repository:

@misc{artigala_reasoning_summarizer_qwen3_5_0_8b_preview_gguf_2026,
  author = {Artigala, Mithula},
  title = {Reasoning Summarizer — Qwen3.5 0.8B Preview (GGUF)},
  year = {2026},
  howpublished = {Hugging Face model repository},
  url = {https://huggingface.co/mithulaartigala/reasoning-summarizer-qwen3.5-0.8b-preview-gguf}
}

My model citation credits this release. Upstream models, datasets, software, and their attribution requirements remain separate.