> ## Documentation Index
> Fetch the complete documentation index at: https://spacesail.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# vLLM Embedder

The vLLM Embedder provides high-performance embedding inference with support for both local and remote deployment modes. All models are downloaded from HuggingFace.

## Usage

### Local Mode

You can load local models directly using the vLLM library, without any need to host a model on a server.

```python vllm_embedder.py theme={null}
from agno.knowledge.embedder.vllm import VLLMEmbedder
from agno.knowledge.knowledge import Knowledge
from agno.vectordb.pgvector import PgVector

# Get embeddings directly
embeddings = VLLMEmbedder(
    id="intfloat/e5-mistral-7b-instruct",
    dimensions=4096,
    enforce_eager=True,
    vllm_kwargs={
        "disable_sliding_window": True,
        "max_model_len": 4096,
    },
).get_embedding("The quick brown fox jumps over the lazy dog.")

print(f"Embeddings: {embeddings[:5]}")
print(f"Dimensions: {len(embeddings)}")

# Use with Knowledge
knowledge = Knowledge(
    vector_db=PgVector(
        db_url="postgresql+psycopg://ai:ai@localhost:5532/ai",
        table_name="vllm_embeddings",
        embedder=VLLMEmbedder(
            id="intfloat/e5-mistral-7b-instruct",
            dimensions=4096,
            enforce_eager=True,
            vllm_kwargs={
                "disable_sliding_window": True,
                "max_model_len": 4096,
            },
        ),
    ),
    max_results=2,
)
```

### Remote Mode

You can connect to a running vLLM server via an OpenAI-compatible API.

```python vllm_embedder_remote.py theme={null}
# Remote mode (for production deployments)
knowledge_remote = Knowledge(
    vector_db=PgVector(
        db_url="postgresql+psycopg://ai:ai@localhost:5532/ai",
        table_name="vllm_embeddings_remote",
        embedder=VLLMEmbedder(
            id="intfloat/e5-mistral-7b-instruct",
            dimensions=4096,
            base_url="http://localhost:8000/v1",  # Example endpoint for local development
            api_key="your-api-key",  # Optional
        ),
    ),
    max_results=2,
)
```

## Params

| Parameter        | Type                       | Default                             | Description                                    |
| ---------------- | -------------------------- | ----------------------------------- | ---------------------------------------------- |
| `id`             | `str`                      | `"intfloat/e5-mistral-7b-instruct"` | Model identifier (HuggingFace model name)      |
| `dimensions`     | `int`                      | `4096`                              | Embedding vector dimensions                    |
| `base_url`       | `Optional[str]`            | `None`                              | Remote vLLM server URL (enables remote mode)   |
| `api_key`        | `Optional[str]`            | `getenv("VLLM_API_KEY")`            | API key for remote server authentication       |
| `enable_batch`   | `bool`                     | `False`                             | Enable batch processing for multiple texts     |
| `batch_size`     | `int`                      | `10`                                | Number of texts to process per batch           |
| `enforce_eager`  | `bool`                     | `True`                              | Use eager execution mode (local mode)          |
| `vllm_kwargs`    | `Optional[Dict[str, Any]]` | `None`                              | Additional vLLM engine parameters (local mode) |
| `request_params` | `Optional[Dict[str, Any]]` | `None`                              | Additional request parameters (remote mode)    |
| `client_params`  | `Optional[Dict[str, Any]]` | `None`                              | OpenAI client configuration (remote mode)      |

## Developer Resources

* View [Cookbook](https://github.com/agno-agi/agno/tree/main/cookbook/knowledge/embedders/vllm_embedder.py)