Hugging Face for beginners: Download model and demo

Hugging Face was originally focused on developing a chatbot service. While their chatbot project did not achieve the expected success, their Transformers library, which they open-sourced on GitHub, unexpectedly made a huge splash in the machine learning space.

Today, Hugging Face has grown into a platform with over 100,000 pre-trained models and 10,000 datasets and has been hailed as the GitHub of machine learning.

login: https://huggingface.co/

Introducing HF’s Transformers library, which, as HF’s most central project, allows:

• Directly use pre-trained models for inference
• Provide a large number of pre-trained models to use
• Migration learning using pre-trained models So before using HF, we need to download Transformers and some other commonly used dependencies

Here we take internlm2_5–1_8b as an example to see the address of this model on Hugging Face

https://huggingface.co/internlm/internlm2_5-1_8b

Let me do an example in Colab

step 1: install transformers

# install transformers
!pip install transformers==4.38
!pip install sentencepiece==0.1.99
!pip install einops==0.8.0
!pip install protobuf==5.27.2
!pip install accelerate==0.33.0

step 2: download HF weights

import os  
# Importing the os module for operating system related functionalities
from huggingface_hub import hf_hub_download  
# Importing the hf_hub_download function from the huggingface_hub library
# Specify the model identifier
repo_id = "internlm/internlm2_5-7b"  
# This is the identifier of the model on Hugging Face Hub
# Specify the list of files to download
files_to_download = [
    {"filename": "config.json"},  # Configuration file for the model
    {"filename": "model.safetensors.index.json"}  
# Index file for the model weights in safetensors format
]
# Create a directory to store the downloaded files
local_dir = f"{repo_id.split('/')[1]}"  
# Extract the model name from the repo_id
os.makedirs(local_dir, exist_ok=True)  
# Create the directory if it doesn't exist
# Iterate over the files list and download each file
for file_info in files_to_download:
    file_path = hf_hub_download(  # Download the file using hf_hub_download
        repo_id=repo_id,  # Repository ID
        filename=file_info["filename"],  # Name of the file to download
        local_dir=local_dir  # Local directory to save the file
    )
    print(f"{file_info['filename']} file downloaded to: {file_path}")  
# Print the download location

you will get these files

step 3: Demo test

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("internlm/internlm2_5-1_8b", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("internlm/internlm2_5-1_8b", torch_dtype=torch.float16, trust_remote_code=True)
model = model.eval()
inputs = tokenizer(["A beautiful flower"], return_tensors="pt")
gen_kwargs = {
    "max_length": 128,
    "top_p": 0.8,
    "temperature": 0.8,
    "do_sample": True,
    "repetition_penalty": 1.0
}
output = model.generate(**inputs, **gen_kwargs)
output = tokenizer.decode(output[0].tolist(), skip_special_tokens=True)
print(output)

Result

A beautiful flower with $n$ petals is colored in a pattern of $k$ alternating colors. For example, the pattern $ABABAB$ is alternating because the first and third petals are of the same color, the second and fourth petals are of the same color, etc.

(a) Prove that if $n$ is even and $k$ is odd, then the pattern $ABABAB$ is possible.

(b) Prove that if $n$ is odd and $k$ is even, then the pattern $ABABAB$ is possible.

Let $n