NV-Embed Model

In this guide, we will walk you through how to download, install, and use the NV-Embed-v1 model. This powerful AI model enables you to perform advanced text embedding tasks efficiently. By the end of this tutorial, you’ll have everything set up and ready to go. Let’s dive in!

Key Features of NV-Embed Model

Latent Attention Layer: Enhances token embedding pooling for more expressive and accurate embeddings, crucial for retrieval tasks.
Two-Stage Instruction Tuning: Combines contrastive training with non-retrieval tasks, boosting performance across applications like classification and clustering.
Removal of Causal Attention Mask: Improves representation learning by attending to both past and future tokens during contrastive learning.
High Performance on Benchmarks: Achieves top scores in the Massive Text Embedding Benchmark (MTEB), excelling in retrieval, reranking, and classification tasks.

1. Setting Up Your Hugging Face Account

Sign Up for an Account

  • Go to huggingface.co.
  • Click on “Sign Up” and create an account using your email address.
Generate an Access Token

  • Once logged in, click on your profile picture in the upper-right corner and select “Settings.”
  • In the settings menu, find the “Access Tokens” section.
  • Click on “New Token,” name it (e.g., “NV-Embed Access”), and make sure to grant it both read and write permissions.
  • Save the token somewhere secure; you’ll need it later.
Image 1
 Image 2
 Image 3

2. Preparing Your Computer for NV-Embed Model

Install Git and Git LFS

  • Windows: Download and install Git from here. Make sure to check the option to install Git LFS during the installation.
  • macOS: Open Terminal and run:
    macOS Command
    brew install git git-lfs
  • Linux: Open your terminal and run:
    Linux Command
    sudo apt-get install git git-lfs
Set Up Git LFS

  • After installation, run the following command in your terminal to initialize Git LFS:
    Git LFS Command
    git lfs install
Image 4
 Image 5

3. Downloading the NV-Embed-v1 Model

Clone the Model Repository

  • Open your terminal or command prompt.
  • Run the following command to clone the NV-Embed-v1 model repository:
    Clone Command
    git clone https://huggingface.co/nvidia/NV-Embed-v1
  • When prompted, enter the Hugging Face access token that you generated earlier.
Image 6
 Image 7

4. Installing Required Python Libraries

Install Python

Install Necessary Python Packages

  • Once Python is installed, open your terminal and run the following commands to install the required libraries:
    Install Packages Command
    pip install transformers sentence-transformers torch safetensors
Image 8

5. Using the NV-Embed-v1 Model

Create a Python Script

  • Open a text editor (like Notepad, VS Code, or any Python IDE) and create a new file called use_nv_embed.py.
  • Copy and paste the following code into the file:
    Python Code

    from sentence_transformers import SentenceTransformer

    # Load the NV-Embed-v1 model from the local directory
    model_path = "./NV-Embed-v1"
    model = SentenceTransformer(model_path, trust_remote_code=True)

    # Example queries to generate embeddings
    queries = ["What is the capital of France?", "How do you make pasta?"]

    # Generate embeddings
    embeddings = model.encode(queries)

    # Print the embeddings
    print(embeddings)

Image 9
 Image 10

6. Troubleshooting

Run the Python Script

  • In your terminal, navigate to the directory where you saved the use_nv_embed.py file using the cd command. “cd C:\Users\YourUsername\Documents\Projects”
  • Run the script by typing:
    Run Script Command
    python use_nv_embed.py
Image 11
Loading Errors

  • If you see errors related to trust_remote_code, make sure that the trust_remote_code=True argument is included when loading the model, as shown in the script above.
Dependencies Issues

  • If you encounter problems related to package versions, ensure that all necessary Python packages are up to date by running:

    Update Packages Command
    pip install --upgrade transformers sentence-transformers torch safetensors
This guide provides everything you need to successfully download, set up, and use the NVIDIA NV-Embed-v1 model from Hugging Face. With these steps, you’ll be able to leverage the power of this AI model for your text embedding tasks. Whether you’re working on a personal project or deploying in a production environment, you’re now equipped to get started. Happy coding!