Mickeymarsh02/layoutlmv3-multimodel-finetuned-invoices03

layoutlmv3-finetuned-invoices

This model is a fine-tuned version of microsoft/layoutlmv3-base on a custom invoice dataset for document understanding tasks.
It was trained using the Hugging Face Trainer API with early stopping and mixed precision.

Results

The model achieves the following results on the held-out validation set after 10 epochs:

Epoch	Train Loss	Val Loss	Precision	Recall	F1
1	0.8329	0.7094	0.7184	0.6524	0.6838
5	0.3815	0.3104	0.8625	0.8559	0.8592
8	0.2988	0.2350	0.8999	0.8803	0.8900
10	0.2499	0.2254	0.9037	0.8872	0.8954

---

Model description

• Architecture: LayoutLMv3
• Base model: microsoft/layoutlmv3-base
• Task: Token classification for invoice understanding (e.g., extracting key fields).
• Input: Scanned invoices (images + text tokens + bounding boxes).
• Output: Predicted entity labels (e.g., Invoice Number, Date, Vendor, Total).

---

Intended uses & limitations

• Use cases:

  • Automatic information extraction from invoices, receipts, and financial documents.
  • Document AI pipelines for expense management and automation.

• Limitations:

  • Fine-tuned only on a limited invoice dataset.
  • May not generalize well to other document types (contracts, ID cards, etc.).
  • Sensitive to OCR quality — better input text = better results.

---

Training details

Hyperparameters

• Learning rate: 3e-5
• Train batch size: 4
• Eval batch size: 4
• Epochs: 10 (with early stopping)
• Optimizer: AdamW
• Weight decay: 0.01
• Mixed precision (fp16): ✅
• Workers: 2

Framework versions

• Transformers 4.56.0
• PyTorch 2.8.0+cu126
• Datasets 4.0.0
• Tokenizers 0.22.0

---

How to use

from transformers import AutoModelForTokenClassification, AutoProcessor

repo_id = "your-username/layoutlmv3-finetuned-invoices"

model = AutoModelForTokenClassification.from_pretrained(repo_id)
processor = AutoProcessor.from_pretrained(repo_id)

# Example inference
from PIL import Image
image = Image.open("sample_invoice.png")

encoding = processor(image, return_tensors="pt")
outputs = model(**encoding)