EldritchLabs/Kraken-Karcher-12B-v1

⚠️ Note: This model requires Mistral Tekken chat template.

🦑‍ Kraken Karcher 12B v1

Overview

This is a merge of pre-trained language models created using mergekit.

This is the smartest of the 3 tested variants: Della, Karcher, and Model_Stock. Kraken v1 combines 52 finetunes with the Nemo base model, no merges included. It was designed primarily for creativity and roleplay. The early terminations bug with v0 has been solved by removing all ChatML based models, leaving only default Mistral Tekken tokenizers. This version took 16 hours to merge via the karcher method while running for 500 iterations.

Merge Details

Merge Details
This model was merged using the karcher merge method.

The graph_v18.py patch was helpful to use 8GB VRAM for acceleration.

Models Merged
The following models were included in the merge:

• aixonlab/Aether-12b
• aixonlab/Zinakha-12b
• allura-org/Bigger-Body-12b
• allura-org/MN-12b-RP-Ink
• allura-org/remnant-mn-12b
• anthracite-org/magnum-v4-12b
• ArliAI/Mistral-Nemo-12B-ArliAI-RPMax-v1.2
• Babsie/Opulus-12B-v3
• BeaverAI/mistral-doryV2-12b
• crestf411/nemo-sunfall-v0.6.1
• EpistemeAI2/Fireball-Mistral-Nemo-12B-Philos
• EpistemeAI/Mistral-Nemo-Instruct-12B-Philosophy-Math
• Fizzarolli/MN-12b-Rosier-v1
• HumanLLMs/Human-Like-Mistral-Nemo-Instruct-2407
• IIEleven11/Kalypso
• intervitens/mini-magnum-12b-v1.1
• jtatman/mistral_nemo_12b_reasoning_psychology_lora
• KOOWEEYUS/BlackSheep-RP-12B
• Lambent/Arsenic-Shahrazad-12B-v2
• Lambent/Arsenic-Shahrazad-12B-v3
• Lambent/arsenic-nemo-unleashed-12B
• Lambent/Gilded-Arsenic-12B
• mistralai/Mistral-Nemo-Instruct-2407
• nbeerbower/Lyra-Gutenberg-mistral-nemo-12B
• nbeerbower/Lyra4-Gutenberg-12B
• nbeerbower/mistral-nemo-bophades-12B
• nbeerbower/mistral-nemo-gutenberg-12B-v3
• nbeerbower/mistral-nemo-gutenberg-12B-v4
• nbeerbower/Mistral-Nemo-Gutenberg-Doppel-12B
• nbeerbower/Mistral-Nemo-Gutenberg-Encore-12B
• nbeerbower/Mistral-Nemo-Gutenberg-Vitus-12B
• nbeerbower/mistral-nemo-wissenschaft-12B
• NeverSleepHistorical/lumi-nemo-e2.0
• NeverSleep/Lumimaid-v0.2-12B
• nothingiisreal/Celeste-12B-V1.6
• nothingiisreal/MN-12B-Celeste-V1.9
• PocketDoc/Dans-DangerousWinds-V1.1.0-12b
• ReadyArt/Dark-Nexus-12B-v2.0
• ReadyArt/Forgotten-Safeword-12B-v4.0
• ReadyArt/Omega-Darker_The-Final-Directive-12B
• romaingrx/red-teamer-mistral-nemo
• Sao10K/MN-12B-Lyra-v1
• Sao10K/MN-12B-Lyra-v4
• shisa-ai/shisa-v2-mistral-nemo-12b
• sleepdeprived3/Christian-Bible-Expert-v2.0-12B
• SuperbEmphasis/MN-12b-RP-Ink-RP-Longform
• SuperbEmphasis/Omega-Darker_The-Final-Directive-Longform-Stage2-ERP-12B-v0.2
• TheDrummer/Rivermind-12B-v1
• TheDrummer/Rocinante-12B-v1
• TheDrummer/Rocinante-X-12B-v1
• Trappu/Nemo-Picaro-12B
• Undi95/LocalC-12B-e2.0
• VAGOsolutions/SauerkrautLM-Nemo-12b-Instruct

Configuration

The following YAML configuration was used to produce this model:

base_model: B:/12B/models--mistralai--Mistral-Nemo-Instruct-2407
models:
  - model: B:/12B/models--aixonlab--Aether-12b
  - model: B:/12B/models--aixonlab--Zinakha-12b
  - model: B:/12B/models--allura-org--Bigger-Body-12b
  - model: B:/12B/models--allura-org--MN-12b-RP-Ink
  - model: B:/12B/models--allura-org--remnant-mn-12b
  - model: B:/12B/models--anthracite-org--magnum-v4-12b
  - model: B:/12B/models--ArliAI--Mistral-Nemo-12B-ArliAI-RPMax-v1.2
  - model: B:/12B/models--Babsie--Opulus-12B-v3
  - model: B:/12B/models--BeaverAI--mistral-doryV2-12b
  - model: B:/12B/models--crestf411--nemo-sunfall-v0.6.1
  - model: B:/12B/models--EpistemeAI2--Fireball-Mistral-Nemo-12B-Philos
  - model: B:/12B/models--EpistemeAI--Mistral-Nemo-Instruct-12B-Philosophy-Math
  - model: B:/12B/models--Fizzarolli--MN-12b-Rosier-v1
  - model: B:/12B/models--HumanLLMs--Human-Like-Mistral-Nemo-Instruct-2407
  - model: B:/12B/models--IIEleven11--Kalypso
  - model: B:/12B/models--intervitens--mini-magnum-12b-v1.1
  - model: B:/12B/models--jtatman--mistral_nemo_12b_reasoning_psychology_lora
  - model: B:/12B/models--KOOWEEYUS--BlackSheep-RP-12B
  - model: B:/12B/models--Lambent--Arsenic-Shahrazad-12B-v2
  - model: B:/12B/models--Lambent--Arsenic-Shahrazad-12B-v3
  - model: B:/12B/models--Lambent--arsenic-nemo-unleashed-12B
  - model: B:/12B/models--Lambent--Gilded-Arsenic-12B
  - model: B:/12B/models--mistralai--Mistral-Nemo-Instruct-2407
  - model: B:/12B/models--nbeerbower--Lyra-Gutenberg-mistral-nemo-12B
  - model: B:/12B/models--nbeerbower--Lyra4-Gutenberg-12B
  - model: B:/12B/models--nbeerbower--mistral-nemo-bophades-12B
  - model: B:/12B/models--nbeerbower--mistral-nemo-gutenberg-12B-v3
  - model: B:/12B/models--nbeerbower--mistral-nemo-gutenberg-12B-v4
  - model: B:/12B/models--nbeerbower--Mistral-Nemo-Gutenberg-Doppel-12B
  - model: B:/12B/models--nbeerbower--Mistral-Nemo-Gutenberg-Encore-12B
  - model: B:/12B/models--nbeerbower--Mistral-Nemo-Gutenberg-Vitus-12B
  - model: B:/12B/models--nbeerbower--mistral-nemo-wissenschaft-12B
  - model: B:/12B/models--NeverSleepHistorical--lumi-nemo-e2.0
  - model: B:/12B/models--NeverSleep--Lumimaid-v0.2-12B
  - model: B:/12B/models--nothingiisreal--Celeste-12B-V1.6
  - model: B:/12B/models--nothingiisreal--MN-12B-Celeste-V1.9
  - model: B:/12B/models--PocketDoc--Dans-DangerousWinds-V1.1.0-12b
  - model: B:/12B/models--ReadyArt--Dark-Nexus-12B-v2.0
  - model: B:/12B/models--ReadyArt--Forgotten-Safeword-12B-v4.0
  - model: B:/12B/models--ReadyArt--Omega-Darker_The-Final-Directive-12B
  - model: B:/12B/models--romaingrx--red-teamer-mistral-nemo
  - model: B:/12B/models--Sao10K--MN-12B-Lyra-v1
  - model: B:/12B/models--Sao10K--MN-12B-Lyra-v4
  - model: B:/12B/models--shisa-ai--shisa-v2-mistral-nemo-12b
  - model: B:/12B/models--sleepdeprived3--Christian-Bible-Expert-v2.0-12B
  - model: B:/12B/models--SuperbEmphasis--MN-12b-RP-Ink-RP-Longform
  - model: B:/12B/models--SuperbEmphasis--Omega-Darker_The-Final-Directive-Longform-Stage2-ERP-12B-v0.2
  - model: B:/12B/models--TheDrummer--Rivermind-12B-v1
  - model: B:/12B/models--TheDrummer--Rocinante-12B-v1
  - model: B:/12B/models--TheDrummer--Rocinante-X-12B-v1
  - model: B:/12B/models--Trappu--Nemo-Picaro-12B
  - model: B:/12B/models--Undi95--LocalC-12B-e2.0
  - model: B:/12B/models--VAGOsolutions--SauerkrautLM-Nemo-12b-Instruct
merge_method: karcher
parameters:
  max_iter: 500
  tol: 1.0e-9
dtype: float32
out_dtype: bfloat16
tokenizer:  
  source: union
chat_template: auto
name: 🦑‍ Kraken-Karcher-12B-v1

Audits

Magnitude Analysis & Data Points

ID    | Status     | Delta Norm   | Orig Size    | Model Name
----------------------------------------------------------------------------------------------------
#1    | OK         | 1.6926       | 74448896     | models--aixonlab--Aether-12b
#2    | OK         | 2.4361       | 74448896     | models--aixonlab--Zinakha-12b
#3    | OK         | 0.0407       | 74448896     | models--allura-org--Bigger-Body-12b
#4    | OK         | 1.6611       | 74448896     | models--allura-org--MN-12b-RP-Ink
#5    | OK         | 0.0866       | 74449920     | models--allura-org--remnant-mn-12b
#6    | OK         | 1.5070       | 74448896     | models--anthracite-org--magnum-v4-12b
#7    | OK         | 0.7476       | 74448896     | models--ArliAI--Mistral-Nemo-12B-ArliAI-RPMax-v1.2
#8    | OK         | 4.6310       | 74448896     | models--Babsie--Opulus-12B-v3
#9    | HIGH MAG   | 5.2080       | 74448896     | models--BeaverAI--mistral-doryV2-12b
#10   | OK         | 0.3196       | 74448896     | models--crestf411--nemo-sunfall-v0.6.1
#11   | HIGH MAG   | 5.7044       | 74448896     | models--EpistemeAI2--Fireball-Mistral-Nemo-12B-Philos
#12   | OK         | 2.3099       | 74448896     | models--EpistemeAI--Mistral-Nemo-Instruct-12B-Philosophy-Math
#13   | HIGH MAG   | 5.2074       | 74448896     | models--Fizzarolli--MN-12b-Rosier-v1
#14   | OK         | 0.0452       | 74448896     | models--HumanLLMs--Human-Like-Mistral-Nemo-Instruct-2407
#15   | OK         | 1.6716       | 74448896     | models--IIEleven11--Kalypso
#16   | HIGH MAG   | 5.1134       | 74449408     | models--intervitens--mini-magnum-12b-v1.1
#17   | OK         | 2.0833       | 74448896     | models--jtatman--mistral_nemo_12b_reasoning_psychology_lora
#18   | OK         | 2.9633       | 74448896     | models--KOOWEEYUS--BlackSheep-RP-12B
#19   | OK         | 2.8456       | 74448896     | models--Lambent--Arsenic-Shahrazad-12B-v2
#20   | OK         | 2.8456       | 74448896     | models--Lambent--Arsenic-Shahrazad-12B-v3
#21   | OK         | 2.8456       | 74448896     | models--Lambent--arsenic-nemo-unleashed-12B
#22   | OK         | 2.8461       | 74448896     | models--Lambent--Gilded-Arsenic-12B
#23   | OK         | 0.0000       | 74448896     | models--mistralai--Mistral-Nemo-Instruct-2407
#24   | OK         | 3.8395       | 74448896     | models--nbeerbower--Lyra-Gutenberg-mistral-nemo-12B
#25   | OK         | 0.4548       | 74448896     | models--nbeerbower--Lyra4-Gutenberg-12B
#26   | OK         | 0.0451       | 74448896     | models--nbeerbower--mistral-nemo-bophades-12B
#27   | HIGH MAG   | 5.1134       | 74449408     | models--nbeerbower--mistral-nemo-gutenberg-12B-v3
#28   | OK         | 1.9967       | 74448896     | models--nbeerbower--mistral-nemo-gutenberg-12B-v4
#29   | OK         | 0.0272       | 74448896     | models--nbeerbower--Mistral-Nemo-Gutenberg-Doppel-12B
#30   | OK         | 0.0350       | 74448896     | models--nbeerbower--Mistral-Nemo-Gutenberg-Encore-12B
#31   | OK         | 0.0501       | 74448896     | models--nbeerbower--Mistral-Nemo-Gutenberg-Vitus-12B
#32   | OK         | 0.0353       | 74448896     | models--nbeerbower--mistral-nemo-wissenschaft-12B
#33   | HIGH MAG   | 5.1626       | 74449408     | models--NeverSleepHistorical--lumi-nemo-e2.0
#34   | HIGH MAG   | 5.1647       | 74449408     | models--NeverSleep--Lumimaid-v0.2-12B
#35   | OK         | 0.0208       | 74448896     | models--nothingiisreal--Celeste-12B-V1.6
#36   | OK         | 4.1274       | 74448896     | models--nothingiisreal--MN-12B-Celeste-V1.9
#37   | HIGH MAG   | 5.4769       | 74448896     | models--PocketDoc--Dans-DangerousWinds-V1.1.0-12b
#38   | OK         | 2.4160       | 74448896     | models--ReadyArt--Dark-Nexus-12B-v2.0
#39   | OK         | 2.4154       | 74448896     | models--ReadyArt--Forgotten-Safeword-12B-v4.0
#40   | OK         | 0.0281       | 74448896     | models--ReadyArt--Omega-Darker_The-Final-Directive-12B
#41   | OK         | 0.0000       | 74448896     | models--romaingrx--red-teamer-mistral-nemo
#42   | OK         | 3.8395       | 74448896     | models--Sao10K--MN-12B-Lyra-v1
#43   | OK         | 0.4543       | 74448896     | models--Sao10K--MN-12B-Lyra-v4
#44   | OK         | 2.9147       | 74448896     | models--shisa-ai--shisa-v2-mistral-nemo-12b
#45   | OK         | 0.0123       | 74448896     | models--sleepdeprived3--Christian-Bible-Expert-v2.0-12B
#46   | OK         | 1.6650       | 74448896     | models--SuperbEmphasis--MN-12b-RP-Ink-RP-Longform
#47   | OK         | 0.1050       | 74448896     | models--SuperbEmphasis--Omega-Darker_The-Final-Directive-Longform-Stage2-ERP-12B-v0.2
#48   | OK         | 1.6959       | 74448896     | models--TheDrummer--Rivermind-12B-v1
#49   | OK         | 1.9967       | 74448896     | models--TheDrummer--Rocinante-12B-v1
#50   | OK         | 2.2716       | 74448896     | models--TheDrummer--Rocinante-X-12B-v1
#51   | OK         | 4.4899       | 74448896     | models--Trappu--Nemo-Picaro-12B
#52   | HIGH MAG   | 5.1950       | 74448896     | models--Undi95--LocalC-12B-e2.0
#53   | OK         | 1.4549       | 74448896     | models--VAGOsolutions--SauerkrautLM-Nemo-12b-Instruct