Cerebras

Cerebras is supported in ChatFrame as an OpenAI Compatible Large Language Model (LLM) provider, known for its high-performance LLM inference service [1].

Configuration

Cerebras is supported because its API is compatible with the OpenAI specification [2].

  1. Obtain API Key: Get your API key from the Cerebras platform.
  2. Open ChatFrame Settings: Navigate to the Providers tab and select Custom Providers or the dedicated Cerebras entry if available.
  3. Configure Endpoint:
    • API Endpoint: Use the Cerebras API base URL.
    • API Key: Paste your Cerebras API key.
  4. Model Selection: Select the desired high-speed models, such as Llama 3 8B or 70B, which are often served by Cerebras.

Performance Advantage

Cerebras is recognized in the industry for its speed, often outperforming other popular LLM providers in tokens per second for models like Llama 70B [3]. This makes it an excellent choice for users prioritizing low-latency, high-throughput interactions within ChatFrame.

LiveKit. Cerebras Integration. https://docs.livekit.io/agents/integrations/cerebras/