llamafile

For the last couple months I've been building a fully open source LLM performance benchmark and repository - LocalScore! It measures how well YOUR computer performs on various LLM tasks, and creates a resource for the entire Local LLM community. Try it: localscore.ai/download

Introducing LocalScore! It turns Llamafiles into "GeekBench for local AI." Benchmark open models on your local GPU or CPU. Created by @cj_pais, dev work funded by @mozillabuilders. Learn more: localscore.ai/blog

Llamafile 0.9.2 is out, adding support for DeepSeek-R1, Gemma 3, and IBM Granite! github.com/Mozilla-Ocho/l…

Just got back from Mozilla Builders Demo Day. Here's some photos of the @llamafile booth as well as slides from an @openwebui talk via @MattBCool.

Writing code looks different in the age of genAI. The team at @EclipseSource is working on a fully customizable copilot system for the @Theia_IDE. They’re demoing their work integrating StarCoder2 @llamafile s directly into their editor and it’s 🤯

I wrote a blog post about all the weird programming language syntax I encountered while building a syntax highlighter for @llamafile. Read justine.lol/lex/

There's been a lot of progress in #llamafile over the past few months, including improvements to CPU performance and support for new models. We've even created "Whisperfile" which is exactly what it sounds like. Catch up here: hacks.mozilla.org/2024/10/llamaf…

Llamafile 0.8.14 is out, featuring a brand-new command line chat interface github.com/Mozilla-Ocho/l…

Can't believe this is a 1B model - it's so fast! Made a video for how to install and get it running for those new to Llamafile. youtu.be/Lqh7egmfy4o

Forget ChatGPT: why researchers now run small AIs (local LLMs) on their laptops. My latest for @Nature: nature.com/articles/d4158… Thanks, @SebastienBubeck, @Microsoft, @stlhood, @Mozilla, @sharon000, @drchristhorpe, @embl, @JohnsonThomasMD, @FollowMercy, @cyrilzakka, @Stanford,…

We will be hosting a launch party for sqlite-vec on the Mozilla Discord server next week on Tuesday August 6th at 10am PST — come through! discord.gg/4ZMAgmrwqf?eve…

🆕 pod: The Winds of AI Winter! latent.space/p/q2-2024-recap The vibes have shifted... @fanahova and @swyx celebrate 1m downloads and recap the last 3 months in AI! Discussing the Frontier Labs vibe shift between Claude 3.5, Llama 3.1, Apple Intelligence, and the expansion of…

sqlite-vec v0.1.0 is out! Vector search for SQLite that runs everywhere! - Supported on MacOS, Linux, Windows, browser WASM, Raspberry Pis - Install with Python, Node.js, Ruby, Go, Rust, Datasette... - Float and binary vectors supported Try it out! alexgarcia.xyz/blog/2024/sqli…

A new blog post: Introducing sqlite-rembed, a SQLite extension for generating text embeddings from remote APIs , all with SQL! Supported providers include: - OpenAI - Nomic - Cohere - Jina - Mixedbread - Ollama - llamafile a sqlite-vec sister project alexgarcia.xyz/blog/2024/sqli…

New #llamafile release - 0.8.10 - github.com/Mozilla-Ocho/l… Includes the first version of our new "llamafiler" server, with an endpoint that serves up embeddings at 8x the speed of llama.cpp.

The holy grail of Local AI: 🔓Open Source 🏃‍♀️Runs Fast 💻Runs on your machine 🌐Runs on all machines @Mozilla is leading the charge on the new local AI ecosystem: @JustineTunney’s work on @llamafile and @stlhood announced sqlite-vec & $100k in non-dilutive funding for all local…

🚀 Resources for Optimizing DevOps with Docker 🚀 1️⃣ Docker Desktop vs. Docker Engine: How to tell which Docker you're running: bit.ly/4bDPDdd 2️⃣ Simplify AI Development: Containerize @llamafile with Docker: bit.ly/3S3pAFk #DockerBlog #DevOps #AI

Are you building an open source AI project like llamafile? Mozilla has $100k of non-dilutive funding burning a hole in their pocket, waiting for you. I can attest that they're a good friend to have on your side. Learn more and apply: future.mozilla.org/builders/