NVIDIA L40S GPU's for MXFP4 quantization

#100

by lordim - opened Aug 9, 2025

Are NVIDIA L40S GPU's also compatible for MXFP4 quantization? I'm trying to load gpt-oss-20b on this machine, but it seems to default to bf16.

Did it work with bf16?

@lordim MXFP4 is supported by Blackwell architecture, so I don't think it is compatible. At least not natively.

hi, does bf16 work with nvidia l40s?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment