Instructions to use smcleish/Llama-3.2-non-recurrent-posttrained with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use smcleish/Llama-3.2-non-recurrent-posttrained with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="smcleish/Llama-3.2-non-recurrent-posttrained")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("smcleish/Llama-3.2-non-recurrent-posttrained")
model = AutoModelForCausalLM.from_pretrained("smcleish/Llama-3.2-non-recurrent-posttrained", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use smcleish/Llama-3.2-non-recurrent-posttrained with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "smcleish/Llama-3.2-non-recurrent-posttrained"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "smcleish/Llama-3.2-non-recurrent-posttrained",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/smcleish/Llama-3.2-non-recurrent-posttrained

SGLang

How to use smcleish/Llama-3.2-non-recurrent-posttrained with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "smcleish/Llama-3.2-non-recurrent-posttrained" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "smcleish/Llama-3.2-non-recurrent-posttrained",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "smcleish/Llama-3.2-non-recurrent-posttrained" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "smcleish/Llama-3.2-non-recurrent-posttrained",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use smcleish/Llama-3.2-non-recurrent-posttrained with Docker Model Runner:
```
docker model run hf.co/smcleish/Llama-3.2-non-recurrent-posttrained
```

Llama-3.2-non-recurrent-posttrained / README.md

smcleish

Update README.md

004e96a verified 4 months ago

preview code

Raw History Blame Contribute Delete

1.54 kB

metadata

language:
  - en
tags:
  - causal-lm
library_name: transformers
license: apache-2.0
datasets:
  - nvidia/Nemotron-CC-Math-v1
base_model:
  - meta-llama/Llama-3.2-1B

Llama-3.2-non-recurrent-posttrained

Llama-3.2-non-recurrent-posttrained is a non-recurrent meta-llama/Llama-3.2-1B based baseline for the Retrofitting Recurrence set of models. A set of depth recurrent models trained by taking layers from pretrained feedforward language models (link to paper).

Training

We train using https://github.com/mcleish7/retrofitting-recurrence using AMD MI300A GPUs on Tuolumne at Lawrence Livermore National Laboratory.

Data

Train and validation data is taken from non-overlapping subsets of raw text data. As such it is not an instruction model.

Licence

This model is released under the apache-2.0 licence.

Contact

Please, feel free to contact us with any questions, or open a discussion thread.

Citation

@article{mcleish2025teaching,
    title={Teaching Pretrained Language Models to Think Deeper with Retrofitted Recurrence}, 
    author={Sean McLeish and Ang Li and John Kirchenbauer and Dayal Singh Kalra and Brian R. Bartoldson and Bhavya Kailkhura and Avi Schwarzschild and Jonas Geiping and Tom Goldstein and Micah Goldblum},
    journal={arXiv preprint arXiv:2511.07384},
    year={2025}
}