ggml

@ggml_org

AI inference at the edge

Joined April 2025

0Following

2KFollowers

Pinned

ggml@ggml_org · Apr 23

38.0K

ggml@ggml_org · Jul 18

github.com/ggml-org/llama…

ggml_org's tweet card. target #14629 Fuse GGML_OP_ADD and GGML_OP_MUL LLAMA_SET_ROWS=1 ./scripts/compare-commits.sh master gg/metal-fuse-add -m ./models/qwen3-30b-a3b/ggml-model-q8_0.gguf -m models/gemma-3-4b/ggml-model-...

491

ggml@ggml_org · Jul 17

github.com/ggml-org/llama…

ggml_org's tweet card. target #14285 Reuse computation graphs from the previous ubatch when possible. Works with any batch size and any model. NoteTo enable this functionality, there is a temporary requirement LLAMA_SET_...

908

ggml@ggml_org · Jul 16

github.com/ggml-org/llama…

ggml_org's tweet card. target #14285 Overview Improve multi-sequence decoding performance by avoiding the cross-sequence attention compute. NoteTo enable this functionality, there is a temporary requirement LLAMA_SET_ROW...

2.0K

ggml@ggml_org · Jul 9

github.com/ggml-org/llama…

ggml_org's tweet card. This adds support for Jamba (fixes #6372). (https://arxiv.org/abs/2403.19887) (this has been open for a while, and this description was very different originally (much broader scope), feel free to ...

469

ggml@ggml_org · Jul 9

github.com/ggml-org/llama…

ggml_org's tweet card. Fixes: #13681 Summary • Add initial support for the Falcon-H1 model family. • Implements model loading, basic inference, and tokenizer integration for Falcon-H1. • Updates the build scripts and rel...

719

ggml@ggml_org · Jul 8

github.com/ggml-org/llama…

ggml_org's tweet card. Supersede #14240 Thanks @Vaibhavs10 for the initial implementation Note: you need to use --jinja to enable thinking mode: $ llama-cli -m ../models/SmolLM3-3B/model.gguf --jinja > hi <...

12.0K

ggml@ggml_org · Jul 2

github.com/ggml-org/llama…

ggml_org's tweet card. Follow-up from #8519 (comment). This should fix #7727 and fix #8519. I've implemented the fully recurrent mode of Mamba-2, because it's very similar to Mamba-1, and also because it ...

4.0K

ggml@ggml_org · Jun 25

whisper.cpp v1.7.6 github.com/ggml-org/whisp…

ggml_org's tweet card. Overview Add initial VAD support - feedback welcome and appreciated Metal FA improvements M2 Ultra Flash Attention ON: CPU Config Model Th FA Enc. Dec. Bch5 PP Commit M2 ULTRA METAL tiny 1...

2.0K

ggml Retweeted

Vaibhav (VB) Srivastav@reach_vb · May 26

You really can just do things! Use *any* Hugging Face space as a MCP server along with your Local Models! 🔥 Here in we use Qwen 3 30B A3B with @ggml_org llama.cpp and @huggingface tiny agents to create images via FLUX powered by ZeroGPU ⚡ It's quite a bit crazy to see local…

259

314

38.0K

ggml Retweeted

Xuan-Son Nguyen@ngxson · May 26

Cooking: mixed input modalities w/ llama.cpp 🧑‍🍳

5.0K

ggml@ggml_org · May 22

winget install llama.cpp

115

8.0K

ggml@ggml_org · May 20

github.com/ggml-org/llama…

ggml_org's tweet card. This tutorial demonstrates how to use the slots management feature in llama-server to optimize repeated prompt processing through KV cache reuse. Server Setup Start the server with the desired numb...

2.0K

ggml@ggml_org · May 15

github.com/ggml-org/llama…

ggml_org's tweet card. Supersede #11647 After second thought, I think PDF parsing should be a built-in functionality because of its popularity. This PR adds the ability to: Upload PDF to web UI, it will be parse as eith...

2.0K

ggml Retweeted

Xuan-Son Nguyen@ngxson · May 12

Real-time webcam demo with @huggingface SmolVLM and @ggml_org llama.cpp server. All running locally on a Macbook M3

208

1.0K

12.0K

8.0K

962.0K