whisper-small-basque

This model is a fine-tuned version of openai/whisper-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1890
• Wer: 19.7202

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 128
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 10000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.2729	0.17	500	0.3845	34.4761
0.161	0.33	1000	0.2777	25.3278
0.1342	0.5	1500	0.2439	21.7685
0.1113	0.66	2000	0.2236	18.3402
0.1043	0.83	2500	0.2119	26.6704
0.0962	0.99	3000	0.2023	15.9923
0.0692	1.16	3500	0.2023	18.3152
0.0669	1.32	4000	0.1959	15.5302
0.0677	1.49	4500	0.1915	17.1475
0.0652	1.65	5000	0.1887	15.0556
0.0605	1.82	5500	0.1880	17.2537
0.0582	1.98	6000	0.1897	14.8807
0.0458	2.15	6500	0.1883	15.1305
0.0396	2.31	7000	0.1907	15.6301
0.0412	2.48	7500	0.1882	21.8371
0.0409	2.64	8000	0.1882	17.1787
0.0402	2.81	8500	0.1866	15.9985
0.0422	2.97	9000	0.1863	16.1109
0.0313	3.14	9500	0.1887	18.4901
0.0323	3.3	10000	0.1890	19.7202

Framework versions

• Transformers 4.38.0
• Pytorch 2.1.1+cu121
• Datasets 2.8.0
• Tokenizers 0.15.2