ray-llm

Overview

RayLLM is a community archive repository offering tools for running large language models on the Ray distributed compute framework. The README provides specific setup and usage details, but the project is no longer actively maintained.

Best for

Best for
Developers already using Ray who need legacy code or patterns for running LLMs at scale.

Use cases

• Deploying open-source LLMs on a Ray cluster
• Scaling LLM inference across multiple nodes
• Testing Ray-based orchestration for model serving

Notes

RayLLM is a community archive repository offering tools for running large language models on the Ray distributed compute framework. The README provides specific setup and usage details, but the project is no longer actively maintained.

1,267 stars on GitHub. Last updated 2025-03-13.

Use cases

• Deploying open-source LLMs on a Ray cluster
• Scaling LLM inference across multiple nodes
• Testing Ray-based orchestration for model serving

Pros

• Leverages Ray’s distributed computing for large models
• Open source with a public archive for reference
• Straightforward integration with Ray ecosystem

Cons

• Archived and not actively maintained or updated
• Limited community support beyond existing documentation
• May lack compatibility with newer Ray versions or LLM frameworks

Indexed from awesome-llmops and enriched against its public facts.