You need to agree to share your Hugging Face information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

LUSA and companion selector weights

LUSA: Lightweight Utility-Guided Sparse Attention for Long-Context Prefill

Paper1 final selected weights

Seven deployment files: two backbone models and five ablations, selected on development data.

paper1/minicpm_kl21/best-step140.pt
paper1/minicpm_fixed_mixing_s42/selected-best-step20.pt
paper1/minicpm_listwise_only_s42/selected-best-step260.pt
paper1/minicpm_no_boundary_s42/selected-best-step60.pt
paper1/minicpm_no_worst_s42/selected-best-step180.pt
paper1/minicpm_single_subspace_s42/selected-best-step180.pt
paper1/nosa_kl21/selected-best.pt

Load selector PT files with torch.load(path, map_location="cpu", weights_only=True) and use the matching sparse-model integration. These selector files require the original frozen backbone.

Inventory and companion collections

The root manifest contains 96 weight paths across all collections. Provenance maps original and published checksums.

attention/sparda/baseline-step2000/indexer-deploy.pt contains projection tensors and inference-only sparse_config; load model_state_dict into the matching MiniCPM sparse implementation.

attention/depth-attention/baseline-step50000/model.safetensors requires the matching custom Depth-Attention implementation and a compatible tokenizer. Its accompanying configuration is provided.

Apply each companion artifact's upstream license. Paper1 results, analyses and manuscripts are stored in the private research repository. Optimizer/RNG state and intermediate Paper1 checkpoints are excluded from the current release.

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support