Free LLM API in Python: 6 Providers, 1 Call

To use a free LLM API in Python, install freelm, set one or more provider keys as environment variables, and call FreeLLM.from_env().text("..."). freelm pools six free-tier providers behind one OpenAI-compatible interface and handles rate limits, key rotation, and failover for you.

pip install freelm

import freelm

llm = freelm.FreeLLM.from_env()
print(llm.text("What is a vector database?"))

Step 1: Get free API keys

Sign up for whichever free tiers you want — all are optional, and more keys means more capacity. freelm reads these environment variables automatically:

| Provider | Env var | Free signup | |---|---|---| | Google AI Studio | GEMINI_API_KEY | aistudio.google.com | | Groq | GROQ_API_KEY | console.groq.com | | Cerebras | CEREBRAS_API_KEY | cloud.cerebras.ai | | OpenRouter | OPENROUTER_API_KEY | openrouter.ai | | NVIDIA NIM | NVIDIA_API_KEY | build.nvidia.com | | Mistral | MISTRAL_API_KEY | console.mistral.ai |

Put them in a .env file (never commit it) and load it, or export them in your shell.

Step 2: Make a chat call

from_env() builds a client from whatever keys it finds. Use text() for a string, or chat() for full message control and the response object:

from freelm import FreeLLM

llm = FreeLLM.from_env(strategy="quota_aware")

resp = llm.chat(
    [{"role": "user", "content": "Write a haiku about databases."}],
    model="chat:fast",       # virtual model -> resolved per provider
)
print(resp.text, "via", resp.provider, resp.model)

model="auto" picks the best available free model; chat:fast and chat:large bias toward speed or capability.

Step 3: Stream tokens

For chat UIs, stream the response as it generates. Streaming runs through the same failover, switching providers only before the first token:

for chunk in llm.stream("Explain async I/O simply."):
    print(chunk, end="", flush=True)

Step 4: Go async

For concurrent workloads, use the async client — same API, await/async for:

import asyncio
from freelm import AsyncFreeLLM

async def main():
    async with AsyncFreeLLM.from_env() as llm:
        print(await llm.text("one line on idempotency"))
        async for chunk in llm.astream("count to three"):
            print(chunk, end="", flush=True)

asyncio.run(main())

Step 5: Drop into existing OpenAI code

Already using the OpenAI SDK? Swap the import and keep your code. The shim exposes the same chat.completions.create surface, backed by free providers:

from freelm.compat import OpenAI

client = OpenAI()
r = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "hi"}],
)
print(r.choices[0].message.content)

Step 6: See which models are live

Free model IDs change weekly, so freelm discovers them at runtime. List the current free models any time:

from freelm import list_free_models

for m in list_free_models()[:5]:
    print(m.id, m.tags)

Frequently Asked Questions

Do I need all six provider keys? No. Supply one or many — freelm uses whatever it finds. More keys means more free capacity and better failover.

Is freelm free to use? Yes, MIT-licensed. It runs on providers' free tiers, so your limits are theirs. There is no freelm subscription.

How is this different from calling a provider directly? A direct call to one provider dies when that provider rate-limits. freelm pools six providers, rotates keys, and fails over automatically — your code stays a single call.

Can I use it with the OpenAI SDK? Yes. from freelm.compat import OpenAI is a drop-in replacement, so existing OpenAI code works unchanged against free providers.

Does it work with async and streaming? Both. AsyncFreeLLM mirrors the sync API, and stream() / astream() yield tokens across providers with pre-first-token failover.

---

Written by Shihab Shahriar Antor — AI Engineer & Founder of Shahriar Labs. Get freelm with pip install freelm — source and docs on GitHub.