FujitsuResearch · FKKimura · Aug 7, 2026 · Aug 12, 2026 · Aug 12, 2026 · Aug 12, 2026
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,5 +1,34 @@
 # Change log
 
+## [v1.4.0(WIP)+feature/mdbf-llamacpp-export] 2026-09-10
+
+### New Features
+
+- Added GGUF fallback export for plain and rotated MDBF checkpoints by reconstructing dense weights before llama.cpp conversion.
+
+### Bug Fixes
+
+- Reject forced `direct` and `mixed` GGUF modes for non-GPTQ layouts and rotated checkpoints instead of entering an incompatible exporter.
+
+## [v1.4.0(WIP)+feature/openvino-2026.03] 2026-09-10
+
+### Environment
+
+- Add an isolated OpenVINO 2026.3.1 environment with locked conversion dependencies and a
+  OneComp GPTQ 4-bit export example for text-generation models. Flat checkpoint
+  metadata is normalized in a temporary copy without modifying the source checkpoint.
+
+## [v1.4.0] 2026-08-dd
+
+### Breaking Changes
+
+- Remove Runner's layer-wise multi-GPU quantization feature. The `multi_gpu` and
+  `gpu_ids` options and the associated implementation have been removed.
+
+### Enhancement
+
+- Support fine-tuning router after quantizing expert's of MoE, which can be used as the postprocess. see [GEMQ](https://arxiv.org/abs/2605.23078).
+
 ## [v1.3.3] 2026-09-03
 
 ### Enhancement

diff --git a/README.md b/README.md
@@ -126,7 +126,7 @@ Then install OneComp from PyPI (see step 2 below). GPTQ quantization and Hugging
 > - GPTQ (`run_gptq`): Hessian and weights are moved to CPU for the full column-wise loop (including inverse-Hessian Cholesky). If that loop stayed on MPS, `quantize()` would call `maxq.item()` once per column; each call triggers **per-column host sync** (wait for pending MPS ops, then read one scalar—not a full Hessian/weight copy every column)—often several times slower than CPU on Apple Silicon (e.g. ~4× in internal benchmarks with PyTorch 2.12). Keeping GPTQ on CPU avoids that overhead. With `mse=True`, `find_params` also calls `quantize()` in a grid loop and benefits from the same CPU placement.
 > - QEP weight correction (`adjust_weight`, when QEP correction runs—typically `qep=True` with error propagation enabled): Per-layer work stays on MPS (e.g. `weight @ delta_hatX`, diagonal damping). Only the Cholesky solve uses CPU via `_safe_cholesky_and_solve` (one solve per layer, not per column); moving all of QEP to CPU does not materially improve speed. The subsequent GPTQ step still uses the CPU path above.
 >
-> DBF-based AutoBit fallback and multi-GPU quantization are not supported on MPS.
+> DBF-based AutoBit fallback is not supported on MPS.
 
 #### 2. Install `onecomp`
 
@@ -136,7 +136,7 @@ Once PyTorch is installed, you can install `onecomp`:
 pip install onecomp
 ```
 
-To enable multi-GPU training features (DeepSpeed), install with the `distributed` extra:
+To enable multi-GPU training for Global PTQ (DeepSpeed), install with the `distributed` extra:
 
 ```bash
 pip install "onecomp[distributed]"
@@ -182,7 +182,7 @@ See the **MPS device placement (GPTQ vs QEP)** note under [macOS (MPS)](#macos-m
 
 Adding `--extra dev` installs development tools (black, pre-commit, pytest, pylint).
 Adding `--extra visualize` installs matplotlib for visualization features.
-Adding `--extra distributed` installs DeepSpeed for multi-GPU training.
+Adding `--extra distributed` installs DeepSpeed for Global PTQ multi-GPU training.
 Adding `--extra hydra` installs `hydra-core` for the example scripts and `model_validation/` runners that use Hydra-based configuration.
 
 To use vLLM for serving quantized models on Linux, add `--extra vllm` together with `--extra cu130`:
@@ -308,6 +308,7 @@ See [`notebook/README.md`](./notebook/README.md) for local setup, or the
 | | [example_global_ptq.py](./example/post_process/example_global_ptq.py) | Global PTQ with packed buffers by default and HF-compatible safetensors output |
 | | [example_global_ptq_dbf.py](./example/post_process/example_global_ptq_dbf.py) | Global PTQ with the DBF backend and HF-compatible safetensors output |
 | | [example_global_ptq_distributed.py](./example/post_process/example_global_ptq_distributed.py) | Multi-GPU Global PTQ with DeepSpeed / torchrun and safetensors output |
+| | [example_router_fine_tuning.py](./example/post_process/example_router_fine_tuning.py) | Router-only next-token fine-tuning for a quantized MoE model |
 | | [example_lora_sft.py](./example/post_process/example_lora_sft.py) | LoRA SFT post-quantization fine-tuning |
 | | [example_lora_sft_knowledge.py](./example/post_process/example_lora_sft_knowledge.py) | LoRA SFT knowledge injection |
 | | [example_lora_sft_knowledge_jointq.py](./example/post_process/example_lora_sft_knowledge_jointq.py) | LoRA SFT knowledge injection on a JointQ-quantized model |

diff --git a/docs/api/post_process.md b/docs/api/post_process.md
@@ -24,6 +24,12 @@ Post-quantization process classes for improving quantized model accuracy.
     options:
       show_source: false
 
+## Router Fine-Tuning
+
+::: onecomp.post_process.RouterFineTuning
+    options:
+      show_source: false
+
 ## LoRA SFT
 
 ::: onecomp.post_process.PostProcessLoraSFT

diff --git a/docs/getting-started/installation.md b/docs/getting-started/installation.md
@@ -107,7 +107,7 @@ To enable visualization features (matplotlib), install with the `visualize` extr
 pip install onecomp[visualize]
 ```
 
-To enable multi-GPU training features (DeepSpeed), install with the `distributed` extra:
+To enable multi-GPU training for Global PTQ (DeepSpeed), install with the `distributed` extra:
 
 ```bash
 pip install "onecomp[distributed]"
@@ -152,7 +152,7 @@ See the [macOS / MPS guide](../user-guide/mps.md) for device placement and usage
 
 Adding `--extra dev` installs development tools (black, pytest, pylint).
 Adding `--extra visualize` installs matplotlib for visualization features.
-Adding `--extra distributed` installs DeepSpeed for multi-GPU training.
+Adding `--extra distributed` installs DeepSpeed for Global PTQ multi-GPU training.
 
 To use vLLM for serving quantized models on Linux, add `--extra vllm` together with `--extra cu130`:
 

diff --git a/docs/getting-started/quickstart.md b/docs/getting-started/quickstart.md
@@ -190,7 +190,7 @@ model, tokenizer = load_quantized_model("./output/quantized_model")
 
 - [CLI Reference](../user-guide/cli.md) -- full CLI options and usage
 - [Configuration](../user-guide/configuration.md) -- detailed explanation of `ModelConfig`, `QEPConfig`, `LPCDConfig`, and `Runner` parameters
-- [Examples](../user-guide/examples.md) -- more usage patterns including multi-GPU and chunked calibration
+- [Examples](../user-guide/examples.md) -- more usage patterns including chunked calibration
 - [Evaluation](../user-guide/evaluation.md) -- `onecomp-eval` for MT-Bench and throughput on vLLM-served models
 - [Algorithms](../algorithms/overview.md) -- learn about the quantization algorithms available in OneComp
 - [macOS / MPS](../user-guide/mps.md) -- Apple Silicon setup, limitations, and inference
diff --git a/docs/user-guide/configuration.md b/docs/user-guide/configuration.md
@@ -28,7 +28,7 @@ model_config = ModelConfig(
 
 !!! note "macOS (MPS)"
     On Apple Silicon, set `device="mps"` for GPTQ / AutoBit (GPTQ-only) quantization.
-    Only GPTQ quantizers are supported on MPS; DBF fallback and multi-GPU are not.
+    Only GPTQ quantizers are supported on MPS; DBF fallback is not.
     See the [macOS / MPS guide](mps.md) for details.
 
 ## Runner
@@ -65,13 +65,6 @@ runner = Runner(
 | `lpcd`                      | `bool`              | Enable LPCD                                      | `False`          |
 | `lpcd_config`               | `LPCDConfig`        | LPCD configuration                               | `None`           |
 
-### Advanced Parameters
-
-| Parameter     | Type        | Description                                      | Default  |
-|---------------|-------------|--------------------------------------------------|----------|
-| `multi_gpu`   | `bool`      | Enable multi-GPU layer-wise parallel quantization| `False`  |
-| `gpu_ids`     | `list[int]` | Specific GPU IDs to use                          | `None`   |
-
 !!! note
     When `calibration_config` is `None`, a `CalibrationConfig()` with default values is created automatically.
 
@@ -124,13 +117,12 @@ calib_config = CalibrationConfig(
 
 ### Valid Parameter Combinations
 
-| `quantizers` | `qep`  | `multi_gpu` | `calibration_config.batch_size` |
-|:------------:|:------:|:-----------:|:-------------------------------:|
-| Specified    | False  | False       | Specified                       |
-| None         | True   | False       | None                            |
-| None         | False  | True        | None                            |
-| None         | False  | False       | Specified                       |
-| None         | False  | False       | None                            |
+| `quantizers` | `qep`  | `calibration_config.batch_size` |
+|:------------:|:------:|:-------------------------------:|
+| Specified    | False  | Specified                       |
+| None         | True   | None                            |
+| None         | False  | Specified                       |
+| None         | False  | None                            |
 
 ## QEPConfig
 

diff --git a/docs/user-guide/cpu-inference.md b/docs/user-guide/cpu-inference.md
@@ -90,9 +90,9 @@ export_to_gguf("./model", "./model.gguf")  # mode="auto" by default
 | `jointq`, `rtn`           | same AutoGPTQ layout           | direct   | yes |
 | `mixed_gptq`              | per-layer bit-widths           | mixed    | 4/8-bit yes, 2/3-bit no |
 | `dbf`, `autobit`          | binary factorization / mixed   | fallback | no (re-quantized) |
-| `gptq`/`mixed_gptq` + `rotated=true` | online Hadamard on down_proj | fallback | no (re-quantized) |
-| `onebit`                  | —                              | unsupported (by request) |
-| `mdbf`                    | —                              | unsupported (not implemented yet) |
+| `mdbf`                    | multi-path binary factorization | fallback | no (re-quantized) |
+| supported method + `rotated=true` | online Hadamard on down_proj | fallback | no (re-quantized) |
+| `onebit`                  | —                              | unsupported (by request) | — |
 
 **QEP** only changes the GPTQ *integer codes* (via pre-quantization weight
 adjustment), so QEP-corrected checkpoints export through the very same lossless
@@ -192,6 +192,16 @@ This **re-quantizes** the weights, so the GPTQ/QEP error correction is lost and
 quality is comparable to a stock `Q4_K_M` GGUF. It requires the
 `llama-quantize` binary (set `$LLAMA_QUANTIZE_BIN` or put it on `PATH`).
 
+MDBF uses only this fallback path, including rotated checkpoints. The exporter
+reconstructs dense weights and folds any online `down_proj` Hadamard into them,
+so stock llama.cpp needs no MDBF-specific kernel. The resulting GGUF does not
+retain MDBF's 1–2-bit compression.
+
+Re-quantizing MDBF weights to the default `Q4_K_M` adds another quantization
+step while producing a larger file than the original MDBF checkpoint. To carry
+the reconstructed weights with less added error, use `qtype=None` for f16
+(Python API only), or `Q8_0` with either Python or the CLI.
+
 ## Running inference
 
 ```python

diff --git a/docs/user-guide/examples.md b/docs/user-guide/examples.md
@@ -220,28 +220,6 @@ runner.run()
 !!! info
     Chunked calibration is mathematically exact -- it accumulates \(X^T X\) across batches without approximation.
 
-## Multi-GPU Quantization
-
-Distribute layer-wise quantization across multiple GPUs:
-
-```python
-runner = Runner(
-    model_config=model_config,
-    quantizer=gptq,
-    multi_gpu=True,
-)
-runner.run()
-
-# Or specify particular GPUs
-runner = Runner(
-    model_config=model_config,
-    quantizer=gptq,
-    multi_gpu=True,
-    gpu_ids=[0, 2, 3],
-)
-runner.run()
-```
-
 ## Comparing Multiple Quantizers
 
 Run multiple quantizers in a single session with shared calibration data:

diff --git a/docs/user-guide/mps.md b/docs/user-guide/mps.md
@@ -22,7 +22,6 @@ quantization steps intentionally run on CPU for performance. See
 | vLLM / GemLite serving | No | Linux + CUDA only |
 | DBF, RTN, JointQ, and other quantizers | No | — |
 | AutoBit DBF fallback | No | — |
-| Multi-GPU quantization | No | — |
 
 ## Installation
 
@@ -126,7 +125,6 @@ with an NVIDIA GPU. See the [vLLM Inference guide](vllm-inference.md).
 
 - Only **GPTQ** quantizers are allowed (or **AutoBitQuantizer** whose candidates are all GPTQ).
 - **AutoBit DBF fallback** is rejected when the target bitwidth would require DBF-only assignment.
-- **`multi_gpu=True`** is not supported.
 
 To avoid DBF fallback on MPS, either set an explicit `wbits` within the GPTQ candidate range
 or ensure VRAM estimation yields a bitwidth that does not trigger DBF-only paths.

diff --git a/docs/user-guide/post-process.md b/docs/user-guide/post-process.md
@@ -1,9 +1,10 @@
-# Post-Process (Global PTQ / Block-wise PTQ / LoRA SFT)
+# Post-Process (Global PTQ / Block-wise PTQ / Router Fine-Tuning / LoRA SFT)
 
-OneComp supports **post-quantization processing** — additional steps applied to a quantized model to improve accuracy or inject domain-specific knowledge. Three implementations are available:
+OneComp supports **post-quantization processing** — additional steps applied to a quantized model to improve accuracy or inject domain-specific knowledge. Four implementations are available:
 
 - **Global PTQ** — Globally optimises quantization parameters (scales, zeros, scaling factors) via KL distillation from a full-precision teacher model
 - **Block-wise PTQ** — Minimises intermediate-representation MSE against an FP16 teacher model at Transformer-block granularity. No training data labelling required.
+- **Router Fine-Tuning** — Recovers quantized MoE quality by training only router parameters with next-token prediction loss while experts and all other weights remain frozen.
 - **LoRA SFT** — Fine-tunes quantized models using Low-Rank Adaptation (LoRA) adapters with SFT loss, optional teacher distillation, and intermediate block alignment.
 
 ## Overview
@@ -338,6 +339,55 @@ See the [API Reference](../api/post_process.md) for the full parameter list.
 
 ---
 
+## Router Fine-Tuning for Quantized MoE Models
+
+Quantization changes expert outputs even when the router itself remains in full
+precision. `RouterFineTuning` adapts routing decisions to those quantized expert
+outputs using standard shifted next-token prediction loss. Before training, all
+parameters are frozen and only parameters below exact module-name components
+`router`, `gate`, and `shared_expert_gate` are enabled. Exact matching means
+expert layers such as `gate_proj` remain frozen.
+
+```python
+from onecomp import GPTQ, CalibrationConfig, ModelConfig, RouterFineTuning, Runner
+
+model_config = ModelConfig(model_id="Qwen/Qwen3-30B-A3B", device="cuda:0")
+runner = Runner(
+    model_config=model_config,
+    quantizer=GPTQ(wbits=4, groupsize=128),
+    calibration_config=CalibrationConfig(max_length=512, num_calibration_samples=128),
+    post_processes=[
+        RouterFineTuning(
+            dataset_name="Salesforce/wikitext",
+            dataset_config_name="wikitext-2-raw-v1",
+            max_train_samples=512,
+            max_length=512,
+            epochs=1,
+            batch_size=1,
+            gradient_accumulation_steps=8,
+            lr=1e-5,
+        )
+    ],
+)
+runner.run()
+```
+
+For architectures with another router name, pass exact path components via
+`router_modules=("custom_router",)`. Local `.json`, `.jsonl`, `.csv`, `.txt`,
+and `.parquet` files are accepted through `data_files`; set `text_column` when
+the text field is not named `text`.
+
+During training, packed GPTQ layers are temporarily unpacked so gradients can
+flow through quantized experts to routing scores. Their incoming packed state is
+restored afterward. The process introduces no custom module type, so the result
+uses the normal `save_quantized_model()` and `load_quantized_model()` workflow.
+
+!!! tip
+    A complete baseline-versus-fine-tuned perplexity example is available at
+    [`example/post_process/example_router_fine_tuning.py`](https://github.com/FujitsuResearch/OneCompression/blob/main/example/post_process/example_router_fine_tuning.py).
+
+---
+
 ## LoRA SFT: Accuracy Recovery
 
 The most common use case is recovering accuracy lost during quantization. Provide a general-purpose dataset (e.g., WikiText-2) to fine-tune the quantized model:

diff --git a/envs/openvino/README.md b/envs/openvino/README.md
@@ -0,0 +1,92 @@
+# Export OneComp GPTQ Models to OpenVINO 2026.3
+
+This directory provides an isolated Python 3.12 environment and a conversion example for
+exporting local OneComp GPTQ 4-bit checkpoints to OpenVINO IR.
+
+- `example_export_openvino.py`: GPTQ 4-bit export for text-generation models
+
+Run the following commands from the repository root.
+
+## 1. Create the isolated environment
+
+uv creates the project environment at `envs/openvino/.venv`. The lock file pins OpenVINO
+2026.3.1 and the matching conversion dependencies. An explicit sync is optional because the
+first `uv run --project envs/openvino ...` command also creates and synchronizes `.venv`.
+
+```bash
+uv sync --project envs/openvino --locked
+
+uv lock --check --project envs/openvino
+```
+
+## 2. Prepare a OneComp GPTQ 4-bit checkpoint
+
+Start from an existing local OneComp GPTQ 4-bit checkpoint containing model weights and a
+`quantization_config` in `config.json`. Its architecture must be supported by Transformers,
+Optimum Intel, and OpenVINO.
+
+When `modules_in_block_to_quantize` uses the flat `List[str]` shape, the exporter
+copies the checkpoint to a temporary directory and normalizes only the copied config to the
+`List[List[str]]` shape that current Transformers and Optimum require. The source checkpoint
+is not modified. Set `TMPDIR` to a large local filesystem when exporting a large checkpoint
+and the default temporary directory does not have enough capacity. Create the target
+directory before running the exporter; otherwise Python silently falls back to a different
+temporary directory such as `/tmp`.
+
+## 3. Update the model path in the export example
+
+Open `envs/openvino/example_export_openvino.py` and edit the two constants at the top of the
+file: set `MODEL_PATH` to the OneComp GPTQ 4-bit checkpoint from step 2, and `OUT_DIR` to the
+directory that should receive the OpenVINO IR.
+
+```python
+# Replace this placeholder with the path to your local OneComp GPTQ 4-bit model.
+MODEL_PATH = "CHANGE_TO_ONECOMP_GPTQ_MODEL_PATH"
+
+# Directory that receives the OpenVINO IR and tokenizer files.
+OUT_DIR = Path("ov_gptq_int4_model_from_onecomp")
+```
+
+## 4. Run the export
+
+```bash
+uv run --project envs/openvino --locked \
+  python envs/openvino/example_export_openvino.py
+```
+
+The example keeps the checkpoint's GPTQ 4-bit weights, so it passes neither
+`OVWeightQuantizationConfig` nor another OpenVINO weight-compression option. It writes the
+model IR, the Hugging Face tokenizer files, and the OpenVINO tokenizer and detokenizer IR
+required by OpenVINO GenAI, then parses every generated model IR as a check.
+
+## 5. VLM checkpoints
+
+The example uses `OVModelForCausalLM`, which does not handle multimodal models. For a VLM,
+use the model-specific Optimum class such as `OVModelForVisualCausalLM` and keep the rest of
+the flow, including the `modules_in_block_to_quantize` normalization from step 2. Text input
+and output need only the tokenizer files that the checkpoint already contains. Processor
+metadata such as `processor_config.json` is required for image or audio input, and can be
+taken from a separately pinned upstream revision when the quantized checkpoint omits it.
+
+A VLM export produces several component IR files, such as the language model, text
+embeddings, per-layer embeddings, and vision embeddings. The GPTQ weights cover only the
+language model, so the embedding components stay uncompressed. Those submodels alone can be
+reduced with NNCF weight compression (`nncf.compress_weights` with
+`CompressWeightsMode.INT8_ASYM`), leaving the GPTQ language model untouched. NNCF is already
+part of this locked environment.
+
+## 6. Run inference on an NPU machine
+
+Copy the exported directory to an NPU machine with the same environment, then run inference
+with OpenVINO GenAI. For a text-generation export:
+
+```python
+import openvino_genai as ov_genai
+
+pipe = ov_genai.LLMPipeline("COPIED_MODEL_DIR", "NPU")
+result = pipe.generate(["YOUR_PROMPT"], max_new_tokens=100)
+print(result.texts[0])
+```
+
+For VLM inference, use `ov_genai.VLMPipeline` instead. Pass a plain prompt string because the
+pipeline applies the model's chat template; do not apply it yourself.