---
language:
- en
- ru
- uz
license: apache-2.0
tags:
- ner
- intent-classification
- language-detection
- multilingual
- uzbek
- russian
- multi-task-learning
- bert
- bilstm
datasets:
- custom
metrics:
- f1
- accuracy
pipeline_tag: token-classification
---

# AIBA BERT-BiLSTM Multi-Task Model

Multi-task BERT-BiLSTM model for banking transaction processing in multiple languages.

## 🎯 Tasks

1. **Named Entity Recognition (NER)** - Extract transaction entities (receiver name, INN, amount, etc.)
2. **Intent Classification** - Classify transaction intent (create, list, partial)
3. **Language Detection** - Detect input language (Uzbek Latin/Cyrillic, Russian, English, Mixed)

## 📊 Performance

| Task | Metric | Score |
|------|--------|-------|
| **NER** | F1 | **0.9941** |
| **Intent** | F1 | **0.9611** |
| **Language** | F1 | **0.9146** |
| **Average** | F1 | **0.9566** |

## 🚀 Quick Start

### Installation
```bash
pip install transformers torch safetensors huggingface-hub
```

### Basic Usage
```python
from nn_model import load_model_and_tokenizer
import torch

# Load model
model, tokenizer, config = load_model_and_tokenizer("YOUR_USERNAME/aiba-bert-bilstm")

# Example text
text = "Qabul qiluvchi Omad Biznes MCHJ STIR 123456789 summa 500000 UZS"

# Tokenize
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=192)

# Predict
with torch.no_grad():
    outputs = model(**inputs)

# Get results
ner_logits = outputs['ner_logits']
intent_logits = outputs['intent_logits']
lang_logits = outputs['lang_logits']
```

### Using the Inference Script
```python
from inference import predict

result = predict("Получатель ООО Прогресс ИНН 987654321 сумма 1000000 руб")

print(f"Intent: {result['intent']}")
print(f"Language: {result['language']}")
print(f"Entities: {result['entities']}")
```

## 📦 Model Architecture

- **Base**: `google-bert/bert-base-multilingual-uncased`
- **NER Head**: BiLSTM (256 hidden units) + Linear classifier
- **Pooling**: Attention-based pooling for sequence tasks
- **Intent Head**: 3-layer MLP with GELU activation
- **Language Head**: 2-layer MLP with GELU activation

## 🏷️ Labels

### NER Entities (16)
agreement_number_indicator, amount, date, description, indicator_amount, indicator_date, indicator_description, indicator_receiver_hr, indicator_receiver_inn, indicator_receiver_name, indicator_status, mfo, receiver_hr, receiver_inn, receiver_name, status

### Intents (3)
create_transaction, list_transaction, partial_entities

### Languages (5)
en, mixed, ru, uz_cyrl, uz_latn

## 📂 Repository Structure
```
.
├── config.json              # Model configuration
├── pytorch_model.bin        # Model weights
├── model.safetensors       # Model weights (safetensors format)
├── tokenizer_config.json   # Tokenizer configuration
├── nn_model.py             # Model architecture code
├── inference.py            # Example inference script
├── requirements.txt        # Python dependencies
└── README.md              # This file
```

## 🔧 Training Details

- **Epochs**: 3
- **Learning Rate**: 8e-06
- **Batch Size**: 48
- **Max Sequence Length**: 192
- **Optimizer**: AdamW
- **Loss Weights**: NER=0.5, Intent=0.35, Lang=0.15

## 💡 Use Cases

- **Banking Chatbots**: Extract transaction details from user messages
- **Payment Processing**: Automatically parse payment instructions
- **Transaction Routing**: Classify transaction type and route appropriately
- **Multilingual Support**: Handle inputs in Uzbek, Russian, and English

## ⚠️ Limitations

- Maximum sequence length: 192 tokens
- Trained on banking/financial domain data
- Best performance on transaction-related text

## 📄 Citation
```bibtex
@misc{aiba-bert-bilstm,
  title = {AIBA BERT-BiLSTM Multi-Task Model},
  author = {YOUR_USERNAME},
  year = {2024},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/YOUR_USERNAME/aiba-bert-bilstm}}
}
```

## 📧 Contact

For questions or issues, please open an issue on the [repository](https://huggingface.co/YOUR_USERNAME/aiba-bert-bilstm/discussions).

## 📜 License

Apache 2.0