Qwen-SEA-LION-v4-32B-IT-OV-4BIT

Last updated: 2026-05-20

Model creator: AI Singapore
Original model: Qwen-SEA-LION-v4-32B-IT

SEA-LION (Southeast Asian Languages In One Network) is a collection of Large Language Models (LLMs) which have been pretrained and instruct-tuned for the Southeast Asia (SEA) region.

Qwen-SEA-LION-v4-32B-IT excels at Southeast Asian (SEA) tasks when compared to other open models with fewer than 200 billion parameters, and demonstrates performance comparable to that of larger and top closed models.

This repository contains the Qwen-SEA-LION-v4-32B-IT model exported to OpenVINO™ IR format with weights compressed to INT4 using Optimum-Intel and NNCF. This variant is designed to run on Intel AI PCs without requiring a data centre GPU.

For tokenization, the model employs the default tokenizer used in Qwen3-32B.

Developed by: AI Products Pillar, AI Singapore
Funded by: Singapore NRF
Shared by: AI Products Pillar, AI Singapore
Model type: Decoder
Context Length: 32k tokens
Language(s) (NLP): Burmese, English, Indonesian, Khmer, Lao, Malay, Mandarin, Tagalog, Tamil, Thai, and Vietnamese
License: MIT
Quantized from model: Qwen-SEA-LION-v4-32B-IT

Quantization Parameters

Weight compression was performed using optimum-cli export openvino with the following parameters:

weight-format: int4
ratio: 1.0
group-size: 128

For more information on quantization, see the OpenVINO model optimization guide.

Compatibility

The provided OpenVINO™ IR model is compatible with:

OpenVINO version 2025.4.0 and higher
OpenVINO GenAI version 2025.4.0 and higher
Neural Network Compression Framework (NNCF) 2.19.0 or higher
Optimum Intel 1.26.1 and higher

Running Model Inference with OpenVINO GenAI

1. Install required packages

pip install openvino openvino-tokenizers openvino-genai
pip install huggingface_hub

2. Download the model from Hugging Face Hub

import huggingface_hub as hf_hub

model_id = "aisingapore/Qwen-SEA-LION-v4-32B-IT-OV-4BIT"
model_path = "Qwen-SEA-LION-v4-32B-IT-OV-4BIT"

hf_hub.snapshot_download(model_id, local_dir=model_path)

3. Run model inference

import openvino_genai as ov_genai

pipe = ov_genai.LLMPipeline(model_path, device="GPU")

prompt = "What are some traditional dishes across Southeast Asia?"
result = pipe.generate(prompt, max_new_tokens=200)
print(result)

More GenAI usage examples can be found in OpenVINO GenAI library docs and samples

Available Quantized Versions

Model	Format
Qwen-SEA-LION-v4-32B-IT	BF16 (full precision)
Qwen-SEA-LION-v4-32B-IT-8BIT	GGUF INT8
Qwen-SEA-LION-v4-32B-IT-4BIT	GGUF INT4
Qwen-SEA-LION-v4-32B-IT-OV-4BIT	OpenVINO INT4
Qwen-SEA-LION-v4-32B-IT-OV-8BIT	OpenVINO INT8

Training Details

This OpenVINO model is derived from the original base model via weight-only quantization; no additional training was performed.

Bias, Risks, and Limitations

For full details on limitations, please refer to the original model card.

Acknowledgement

This work was made possible through a collaboration with Intel. We deeply appreciate the support and tools provided.

The SEA-LION project is supported by the National Research Foundation Singapore and the Infocomm Media Development Authority (IMDA), Singapore under its National Large Language Model Funding Initiative.

Team

Intel

Ramesh Perumal (Intel), Malcolm Chan Hao Xian (Intel)

AI Singapore

Ahmed Dabeer, Ahn Jeongmi, Antonyrex Sajeban, Chan Hok Teng Adwin, Cheng Zi Yi Nicholas, Choa Hsueh Mei Esther, Heng Jonathan, Huang Yuli, Jann Railey Estrada Montalan, Lee Chwan Ren, Leong Wai Yi, Leong Wei Qi, Liew Rachel, Limkonchotiwat Peerat, Muhammad Ridzuan Bin Mokhtar, Nagarajan Karthik, Ng Boon Cheong Raymond, Ngee Chia Tai, Ngui Jian Gang, Nguyen Thanh Ngan, Ong Tat-Wee David, Ong Zhi Hao, Pereira Mark, Poon Joseph, Rengarajan Hamsawardhini, Siow Wei Kang Bryan, Susanto Yosephine, Sutaveephamochanon Anocha, Tan Choon Meng, Tan Chor Phin Evelyn, Tan Siao Wei Jessica, Tan Yixian, Tee Jun Yun, Teng Kok Wai Walter, Teo Eng Sipp Leslie, Tjhi William, Wu Donghang, Yeo Yeow Tong, Yong Xianbin, Zhang Haoyang, Zhang Zhou