Instructions to use num1notsvn/wicara-56m-base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use num1notsvn/wicara-56m-base with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="num1notsvn/wicara-56m-base")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("num1notsvn/wicara-56m-base")
model = AutoModelForCausalLM.from_pretrained("num1notsvn/wicara-56m-base", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use num1notsvn/wicara-56m-base with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "num1notsvn/wicara-56m-base"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "num1notsvn/wicara-56m-base",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/num1notsvn/wicara-56m-base

SGLang

How to use num1notsvn/wicara-56m-base with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "num1notsvn/wicara-56m-base" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "num1notsvn/wicara-56m-base",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "num1notsvn/wicara-56m-base" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "num1notsvn/wicara-56m-base",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use num1notsvn/wicara-56m-base with Docker Model Runner:
```
docker model run hf.co/num1notsvn/wicara-56m-base
```

Wicara 56M Base

Weight-efficient Indonesian Conversational Architecture, Research Artifact

A 56-million-parameter Indonesian language model that we trained from scratch on a single consumer laptop (NVIDIA RTX 4050 6 GB) over 1.12 billion tokens of curated text.

Wicara (from Sanskrit vicāra): speech, discourse.

Data, tokenizer, architecture, and training pipeline were built directly in PyTorch — no off-the-shelf model, no external API. This is the foundational base model before SFT. For the conversational version, see num1notsvn/wicara-56m-chat.

Author: Bagus Ardin Prayoga (@num1notsvn)
Language: Indonesian (id)
Parameters: 56.0M (45.5M non-embedding)
Architecture: LLaMA-style (Pre-norm RMSNorm, RoPE, SwiGLU, Grouped-Query Attention, Tied Embeddings)
License: Apache-2.0
Source Code: GitHub - bagusardin25/WicaraLLM

Architecture Specifications

Component	Specification
Total Parameters	56,027,520
Non-Embedding Parameters	45,541,120
Layers (`n_layer`)	10
Hidden Dimension (`d_model`)	640
Attention Mechanism	Grouped-Query Attention (10 query heads / 5 KV heads, GQA 2:1)
Feed-Forward Dimension (`d_ffn`)	1,728 (SwiGLU)
Head Dimension (`head_dim`)	64
Context Length	512 tokens
Vocabulary Size	16,384 byte-level BPE (4.26 chars/token on Indonesian text)
Positional Embedding	RoPE ($\theta = 10000.0$)
Normalization	Pre-norm RMSNorm ($\epsilon = 10^{-5}$)
Embedding Tying	Yes (`lm_head.weight == tok_emb.weight`)

Pretraining Details

Dataset Composition

Trained on 1.12 billion tokens (3,985,535 documents) curated exclusively from natural Indonesian texts:

Source	Role	Tokens	Share	Filter Pass Rate
OpenSubtitles v2024	Conversational dialogue	484M	43.2%	93.9%
FineWeb-2 `ind_Latn`	Informal web writing	210M	18.7%	99.4%
Indonesian Wikipedia	Structured encyclopedia	174M	15.5%	83.1%
Cendol v2	Instruction data	129M	11.5%	73.3%
Aya Collection	Multilingual Q&A	121M	10.8%	79.7%
TED2020	Spoken lectures	3M	0.3%	98.7%

Training Progression & Hardware

Hardware: Single laptop with NVIDIA GeForce RTX 4050 Laptop GPU (6 GB VRAM).
Duration: 11.1 hours (17,010 steps, 1 epoch, 100% corpus coverage).
Throughput: 28,017 tokens/second.
Precision: bf16 mixed precision with AdamW ($\beta_1=0.9, \beta_2=0.95$, cosine decay with warmup).
Validation Loss: 3.0505 (Perplexity: 21.2).
Training Loss: 2.937 (generalization gap: 0.12).

Quickstart & Usage

As a base model, wicara-56m-base is trained for next-token prediction (text completion), not multi-turn instruction following. If you are looking for an interactive conversational assistant, please use num1notsvn/wicara-56m-chat.

Text Completion Example

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "num1notsvn/wicara-56m-base"
device = "cuda" if torch.cuda.is_available() else "cpu"

# Load tokenizer and base model
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16 if torch.cuda.is_available() else torch.float32,
).to(device)

# Provide a prefix prompt for completion
prompt = "Indonesia adalah sebuah negara kepulauan yang"
inputs = tokenizer(prompt, return_tensors="pt").to(device)

# Generate continuation
outputs = model.generate(
    **inputs,
    max_new_tokens=64,
    temperature=0.7,
    top_p=0.9,
    repetition_penalty=1.15,
    do_sample=True,
    eos_token_id=tokenizer.eos_token_id,
)

continuation = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(continuation)

Intended Use & Fine-Tuning

This base model is intended as a foundation for:

Downstream Fine-Tuning: Supervised fine-tuning (SFT) for Indonesian NLP tasks (sentiment analysis, intent classification, named entity recognition, or custom domain adaptation).
Edge SLM Research: Exploring the boundaries of lightweight language models (<100M parameters) under strict compute and VRAM budgets.
Academic & Educational Use: Studying transformer pretraining dynamics from scratch.

Limitations

No Instruction Tuning: This model does not understand conversational turns or system prompts out of the box.
Hallucination & Factual Consistency: Small-capacity models (56M parameters) prioritize surface-level linguistic fluency and cannot reliably serve as factual knowledge repositories without external retrieval (RAG).
Context Length: The native maximum context length is 512 tokens.

Citation & License

Released under the Apache-2.0 License.

@misc{ardin2026wicara,
  title={Wicara: A Weight-efficient Indonesian Conversational Architecture Built from Scratch},
  author={Bagus Ardin Prayoga},
  year={2026},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/num1notsvn/wicara-56m-base}}
}

Downloads last month: 236

Safetensors

Model size

66.5M params

Tensor type

BF16

Model tree for num1notsvn/wicara-56m-base

Finetunes

1 model