Skip to content

Commit 1fd5c2a

Browse files
author
Guang Yang
committed
Improve setup guide
1 parent 6a7e83f commit 1fd5c2a

File tree

2 files changed

+85
-26
lines changed

2 files changed

+85
-26
lines changed

README.md

Lines changed: 78 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,14 @@ Optimum ExecuTorch enables efficient deployment of transformer models using Meta
2020

2121
## ⚡ Quick Installation
2222

23-
Install from source:
23+
### 1. Create a virtual environment:
24+
Install [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html) on your machine. Then, create a virtual environment to manage our dependencies.
25+
```
26+
conda create -n optimum-executorch python=3.11
27+
conda activate optimum-executorch
28+
```
29+
30+
### 2. Install optimum-executorch from source:
2431
```
2532
git clone https://github.com/huggingface/optimum-executorch.git
2633
cd optimum-executorch
@@ -29,11 +36,61 @@ pip install .
2936

3037
- 🔜 Install from pypi coming soon...
3138

39+
### [Optional] 3. Install dependencies in dev mode
40+
You can install `executorch` and `transformers` from source, where you can access new ExecuTorch
41+
compatilbe models from `transformers` and new features from `executorch` as both repos are under
42+
rapid deployment.
43+
44+
Follow these steps manually:
45+
46+
#### 3.1. Clone and Install ExecuTorch from Source:
47+
From the root directory where `optimum-executorch` is cloned,
48+
```
49+
# Clone the ExecuTorch repository
50+
git clone https://github.com/pytorch/executorch.git
51+
cd executorch
52+
# Checkout the stable branch to ensure stability
53+
git checkout viable/strict
54+
# Install ExecuTorch in editable mode
55+
pip install -e .
56+
cd ..
57+
```
58+
59+
#### 3.2. Clone and Install Transformers from Source
60+
From the root directory where `optimum-executorch` is cloned:
61+
```
62+
# Clone the Transformers repository
63+
git clone https://github.com/huggingface/transformers.git
64+
cd transformers
65+
# Install Transformers in editable mode
66+
pip install -e .
67+
cd ..
68+
```
69+
3270
## 🎯 Quick Start
3371

3472
There are two ways to use Optimum ExecuTorch:
3573

36-
### Option 1: Export and Load Separately
74+
### Option 1: Export and Load in One Python API
75+
```python
76+
from optimum.executorch import ExecuTorchModelForCausalLM
77+
from transformers import AutoTokenizer
78+
79+
# Load and export the model on-the-fly
80+
model_id = "meta-llama/Llama-3.2-1B"
81+
model = ExecuTorchModelForCausalLM.from_pretrained(model_id, recipe="xnnpack")
82+
83+
# Generate text right away
84+
tokenizer = AutoTokenizer.from_pretrained(model_id)
85+
generated_text = model.text_generation(
86+
tokenizer=tokenizer,
87+
prompt="Simply put, the theory of relativity states that",
88+
max_seq_len=128
89+
)
90+
print(generated_text)
91+
```
92+
93+
### Option 2: Export and Load Separately
3794

3895
#### Step 1: Export your model
3996
Use the CLI tool to convert your model to ExecuTorch format:
@@ -61,33 +118,34 @@ generated_text = model.text_generation(
61118
prompt="Simply put, the theory of relativity states that",
62119
max_seq_len=128
63120
)
121+
print(generated_text)
64122
```
65123

66-
### Option 2: Python API
67-
```python
68-
from optimum.executorch import ExecuTorchModelForCausalLM
69-
from transformers import AutoTokenizer
124+
## Supported Models
70125

71-
# Load and export the model on-the-fly
72-
model_id = "meta-llama/Llama-3.2-1B"
73-
model = ExecuTorchModelForCausalLM.from_pretrained(model_id, recipe="xnnpack")
126+
Optimum ExecuTorch currently supports the following transformer models:
127+
128+
- **meta-llama/Llama-3.2-1B (and its variants):** A robust large language model designed for a wide range of natural language tasks.
129+
- **HuggingFaceTB/SmolLM2-135M (and its variants):** A lightweight model optimized for rapid inference with a smaller computational footprint.
130+
- **Qwen/Qwen2.5-0.5B (and its variants):** An efficient model delivering balanced performance, especially suited for resource-constrained environments.
131+
- **deepseek-ai/DeepSeek-R1-Distill-Llama-8B:** A distilled version of the Llama model, offering faster inference while retaining strong performance.
132+
- **google/gemma-2-2b (and its variants):** A cutting-edge model from Google, optimized for diverse deployment scenarios.
133+
- **allenai/OLMo-1B-hf:** A specialized model from Allen AI, tailored for advanced language understanding tasks.
134+
135+
*Note: This list is continuously expanding. As we continue to expand support, more models and variants will be added.*
136+
137+
138+
## Supported Recipes
139+
140+
Optimum ExecuTorch currently only supports [`XNNPACK` Backend](https://pytorch.org/executorch/main/backends-xnnpack.html).
74141

75-
# Generate text right away
76-
tokenizer = AutoTokenizer.from_pretrained(model_id)
77-
generated_text = model.text_generation(
78-
tokenizer=tokenizer,
79-
prompt="Simply put, the theory of relativity states that",
80-
max_seq_len=128
81-
)
82-
```
83142

84143
## 🛠️ Advanced Usage
85144

86145
Check our [ExecuTorch GitHub repo](https://github.com/pytorch/executorch) directly for:
87-
- Custom model export configurations
88-
- Performance optimization guides
146+
- More backends and performance optimization options
89147
- Deployment guides for Android, iOS, and embedded devices
90-
- Additional examples
148+
- Additional examples and benchmarks
91149

92150
## 🤝 Contributing
93151

setup.py

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -12,18 +12,19 @@
1212
assert False, "Error: Could not open '%s' due %s\n" % (filepath, error)
1313

1414
INSTALL_REQUIRE = [
15-
"optimum~=1.24",
15+
"accelerate>=0.26.0",
16+
"datasets",
1617
"executorch>=0.4.0",
18+
"optimum~=1.24",
19+
"safetensors",
20+
"sentencepiece",
21+
"tiktoken",
1722
"transformers>=4.46",
1823
]
1924

2025
TESTS_REQUIRE = [
21-
"accelerate>=0.26.0",
22-
"pytest",
2326
"parameterized",
24-
"sentencepiece",
25-
"datasets",
26-
"safetensors",
27+
"pytest",
2728
]
2829

2930

0 commit comments

Comments
 (0)