finsql-mlx-qwen3-4b-instruct-4bit

This is a LoRA adapter for financial SQL generation, fine-tuned on mlx-community/Qwen3-4B-Instruct-2507-4bit.

Latest Finetuning

HF Model Path: gccmorgoth/finsql-mlx-qwen3-4b-instruct-4bit

Finetuning Details

• Method: Direct Preference Optimization (DPO)
• Checkpoint: Iteration 300
• Validation Loss: 0.048
• Training Loss: 0.122
• Learning Rate: Cosine decay with warmup
• LoRA Rank: 16

Performance

• Validation loss: 0.048 (optimal convergence point)
• Selected at iteration 300 to prevent overfitting
• DPO training for improved preference alignment on financial SQL tasks

Model Selection

• Checkpoint: Iteration 300 selected based on validation loss curve
• Rationale: Optimal balance between training convergence and generalization
• Training Dynamics: Early stopping before overfitting (val loss increased at iter 700+)

Dataset

This model was fine-tuned on financial text-to-sql data pairs, specifically the FinSQLBull dataset, to improve SQL query generation for financial databases and tables.

Usage

Recommended prompt format to specify:

Database: [database_name]

[Schema information]

Task

[Natural language question about the data]

Constraint: [Any specific constraints]

SQL: [Model Generated SQL Query]

Sample Prompt Format

Database: company_financials

Table: revenue (id, company, year, revenue, profit)

Task

What was the total revenue for all companies in 2023?

SQL: [Model Generated SQL Query]

Python

from mlx_lm import load, generate

model, tokenizer = load("gccmorgoth/finsql-mlx-qwen3-4b-instruct-4bit")
response = generate(model, tokenizer, prompt="Your prompt here")