Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+gemma-3-27b-it-abliterated.q2_k.gguf filter=lfs diff=lfs merge=lfs -text
+gemma-3-27b-it-abliterated.q3_k_m.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,39 @@
+---
+license: gemma
+library_name: transformers
+pipeline_tag: image-text-to-text
+base_model: google/gemma-3-27b-it
+tags:
+- autoquant
+- gguf
+---
+
+# 💎 Gemma 3 27B IT Abliterated
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/WjFfc8hhj20r5XK07Yny9.png)
+<center><a href="https://huggingface.co/mlabonne/gemma-3-4b-it-abliterated">Gemma 3 4B Abliterated</a> • <a href="https://huggingface.co/mlabonne/gemma-3-12b-it-abliterated">Gemma 3 12B Abliterated</a></center>
+
+This is an uncensored version of [google/gemma-3-27b-it](https://huggingface.co/google/gemma-3-27b-it) created with a new abliteration technique.
+See [this article](https://huggingface.co/blog/mlabonne/abliteration) to know more about abliteration.
+
+I was playing with model weights and noticed that Gemma 3 was much more resilient to abliteration than other models like Qwen 2.5. 
+I experimented with a few recipes to remove refusals while preserving most of the model capabilities.
+
+Note that this is fairly experimental, so it might not turn out as well as expected.
+
+I recommend using these generation parameters: `temperature=1.0`, `top_k=64`, `top_p=0.95`.
+
+## ✂️ Layerwise abliteration
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/HnBRigUfoQaCnpz96jnun.png)
+
+In the original technique, a refusal direction is computed by comparing the residual streams between target (harmful) and baseline (harmless) samples.
+
+Here, the model was abliterated by computing a refusal direction based on hidden states (inspired by [Sumandora's repo](https://github.com/Sumandora/remove-refusals-with-transformers/)) for each layer, independently.
+This is combined with a refusal weight of 1.5 to upscale the importance of this refusal direction in each layer.
+
+This created a very high acceptance rate (>90%) and still produced coherent outputs.
+
+## ⚡️ Quantization
+
+TBD.

gemma-3-27b-it-abliterated.q2_k.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:89a8a00f2e58bdcfb3da7d41196830988c56c1e4669f2413cb50a22aa394315a
+size 10503720672

gemma-3-27b-it-abliterated.q3_k_m.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8a5d2259cd1cba809f9f7681574f70e015dba40ffdcf58701e60e7c1ddc8f653
+size 13437640416