llama.cpp

Get up and running with Kimi-K2.5, GLM-5, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

A high-throughput and memory-efficient inference and serving engine for LLMs

The all-in-one AI productivity accelerator. On device and privacy first with no annoying setup or configuration.

⏩ Source-controlled AI checks, enforceable in CI. Powered by the open-source Continue CLI

Easily deploy LLMs with Ansible. Uses Docker with llama.cpp or ollama. Secured with whitelisted IPs.

The agent engineering platform.

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

The llama-cpp-agent framework is a tool designed for easy interaction with Large Language Models (LLMs). Allowing users to chat with LLM models, execute structured function calls a

Kubernetes operator for local LLM inference with llama.cpp, vLLM, TGI, and mlx-server — multi-GPU NVIDIA + Apple Silicon Metal, autoscaling, air-gapped, production-ready

A framework for few-shot evaluation of language models.

The Swiss Army Knife of Offline AI. Chat, Speak, and Generate Images - Privacy First, Zero Internet. Download an LLM and use it on your mobile device. No data ever leaves your phon

Get up and running with Kimi-K2.5, GLM-5, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

OpenDAN is an open source Personal AI OS , which consolidates various AI modules in one place for your personal use.

Structured Outputs

Build, run, and manage agent platforms.

A distributed implementation of llama.cpp that lets you run 70B-level LLMs on your everyday devices.

Interact with your documents using the power of GPT, 100% privately, no data leaks

GITHUB HUGGING FACE MODELSCOPE DISCORD 🚨 This model mainly supports English. We will release bilingual (English and Chinese) math models soon. Introduction Over the past year, w

A web interface for chatting with Alpaca through llama.cpp. Fully dockerized, with an easy to use API.

WebAssembly binding for llama.cpp - Enabling on-browser LLM inference

日本語LLMまとめ - Overview of Japanese LLMs

Awesome LLM compression research papers and tools.

📖A curated list of Awesome LLM/VLM Inference Papers with codes: WINT8/4, FlashAttention, PagedAttention, MLA, Parallelism, etc. 🎉🎉

A collection of open source, actively maintained web apps for LLM applications

AGI Large Language Models

Search infrastructure for AI

GITHUB HUGGING FACE MODELSCOPE DEMO DISCORD Introduction The advent of advanced programming tools, which harnesses the power of large language models (LLMs), has significantly en

The most powerful AI platform for enterprises. Customize, fine-tune, and deploy AI assistants, autonomous agents, and multimodal AI with open models.

First-generation reasoning models from DeepSeek.

Pure Rust implementation of a minimal Generative Pretrained Transformer

A multi agent desktop application built with Rust and Tauri.

Checking your browser - reCAPTCHA

Gemma 2, our next generation of open models, is now available globally for researchers and developers.

Grok-1-314B-MoE — indexed from awesome-llm

Create agentic, context engineered AI systems using Haystack’s modular and customizable building blocks, built for real-world, production-ready applications.

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

A lightweight framework for building LLM-based agents

Langchain-Chatchat（原Langchain-ChatGLM）基于 Langchain 与 ChatGLM, Qwen 与 Llama 等语言模型的 RAG 与 Agent 应用 | Langchain-Chatchat (formerly langchain-ChatGLM), local knowledge based LLM (like

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, Vertex

[OPT-1.3 6.7 13 30 66B](https://arxiv.org/abs/2205.01068)

2023-07

[Llama 3.1-8 70 405B](https://llama.meta.com/)

[Llama 2-7 13 70B](https://llama.meta.com/llama2/)

Large Language Models for All, 🦙 Cult and More, Stay in touch !

2023-02

The MiniCPM family of LLMs and VLLMs.

Mistral 7B

Moonshot's Compute-efficient MoE LLM, first Scaling Up of Muon Optimizer

Evaluation suite for LLMs

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Qwen - a Qwen Collection

GITHUB HUGGING FACE MODELSCOPE DEMO DISCORD Introduction After months of efforts, we are pleased to announce the evolution from Qwen1.5 to Qwen2. This time, we bring to you: Pret

Transformers have revolutionized almost all natural language processing (NLP) tasks but suffer from memory and computational complexity that scales quadratically with sequence le

Microsoft's enterprise-flavoured framework for AI agents. .NET-first, with Python and Java siblings.

A simple, lightweight shell script to interact with OpenAI or Ollama or Mistral AI or LocalAI or ZhipuAI from the terminal, and enhancing intelligent system management without any

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd of language models

Unsloth Studio is a web UI for training and running open models like Gemma 4, Qwen3.6, DeepSeek, gpt-oss locally.

The de facto TypeScript SDK for AI apps. Streaming, tools, multi-model, and now an agent loop.

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.

Pure Rust implementation of a minimal Generative Pretrained Transformer

Fast, flexible LLM inference

MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering high-performance on-device LLMs and Edge AI.

The fastest local AI engine for Apple Silicon. 4.2x faster than Ollama, 0.08s cached TTFT, 100% tool calling. 17 tool parsers, prompt cache, reasoning separation, cloud routing. Dr

⚡ Python-free Rust inference server — OpenAI-API compatible. GGUF + SafeTensors, hot model swap, auto-discovery, single binary. FREE now, FREE forever.

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NV