Qualcomm AI Engine Direct - [GenAI Pipeline] PR5: Model preparation & quantization strategy implementations by qti-horodnic · Pull Request #21899 · pytorch/executorch

qti-horodnic · 2026-08-17T22:18:06Z

Summary

This PR implements the model preparation and quantization strategy implementations, replacing the NotImplementedError stubs with real logic. Each strategy delegates to injectable adapter interfaces (from PR4) for testability.

What's included

Strategy implementations (2 files + 1 `init` fix):

ExecuTorchModelPreparationStrategy: 5-step flow
- load_model → load_tokenizer (via ModelLoaderAdapter)
- generate_calibration_data (via separately-injectable CalibrationDataAdapter)
- Optional tokenizer export for on-device runtime
- Chat template extraction from tokenizer (with extra_options fallback)
- Validates input config (model_name, soc_model required)
ExecuTorchQuantizationStrategy: Full PT2E single-graph pipeline via QuantizerAdapter
- export → make_quantizer → prepare_pt2e → calibrate → convert_pt2e
- Supports quant_dtype, quant_recipe, and per-channel options via extra_options
- Handles any Iterable as calibration data (lists, DataLoaders, generators)
- Validates calibration data is non-empty before export
- Warns (does not fail) when training_data is provided (QAT deferred)
strategies/model_preparation/__init__.py: adds missing ExecuTorchModelPreparationStrategy import to __all__

Unit tests:

test_executorch_model_preparation_strategy.py
test_executorch_quantization_strategy.py
test_default_model_preparation_adapter.py
test_default_model_preparation_adapter.py

PR Review Checklist

All new classes follow single responsibility (one class per file) - Yes.
All dependencies are injected via constructor with sensible defaults - Yes.
All external calls are behind injectable interfaces - Yes (adapter pattern).
Unit tests cover every public method - Yes (100% coverage on strategy impls).
Docstrings on all public classes and methods - Yes.
Type annotations on all function signatures - Yes.
Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step).

Related PRs

PR 1: Core data model, engine routing & exceptions: Qualcomm AI Engine Direct - [GenAI Pipeline] PR1: Core Data Model, Engine Routing & Exceptions #20409
PR 2: Strategy interfaces & stage wrappers: Qualcomm AI Engine Direct - [GenAI Pipeline] PR2: Strategy Interfaces & Stage Wrappers #20795
PR 3: Pipeline orchestrator: Qualcomm AI Engine Direct - [GenAI Pipeline] PR3: Pipeline Orchestrator #21149
PR 4: Adapter interfaces, default implementations & dataset providers: Qualcomm AI Engine Direct - [GenAI Pipeline] PR4: Adapter interfaces, default implementations & dataset providers #21751
PR 5: Model preparation & quantization strategies: this pr.
PR 6: Compilation & inference strategy implementations: pending.
PR 7: Integration & E2E tests: pending.

Test plan

Run only tests added in this PR:

python -m pytest \
  backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \
  backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \
  -v

Run only this PR's tests with coverage:

python -m pytest \
  backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \
  backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \
  --cov=backends/qualcomm/genai_pipeline/strategies/model_preparation \
  --cov=backends/qualcomm/genai_pipeline/strategies/quantization \
  --cov-config=backends/qualcomm/.coveragerc \
  --cov-report=term-missing

Result:

Name                                                                                                     Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py      68      0     14      0   100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py                        9      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py                  7      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py                61      0     18      0   100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py                            7      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py                                9      0      0      0   100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                                                                      161      0     32      0   100%

Run all `genai_pipeline` tests:

python -m pytest backends/qualcomm/genai_pipeline/tests/ -v

Run all `genai_pipeline` tests with coverage:

python -m pytest backends/qualcomm/genai_pipeline/tests/ \
  --cov=backends/qualcomm/genai_pipeline \
  --cov-config=backends/qualcomm/.coveragerc \
  --cov-report=term-missing

Result:

Name                                                                                                     Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py                                        11      0      0      0   100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py                                        8      0      0      0   100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py                                          12      0      0      0   100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py                                          9      0      0      0   100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py                                   7      0      0      0   100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py                                 12      0      0      0   100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py                                       13      0      0      0   100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py                                       5      0      0      0   100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py                                        5      0      0      0   100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py                               26      0      6      0   100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py                                  14      0      2      0   100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py                                           5      0      0      0   100%
backends/qualcomm/genai_pipeline/engine_proxy.py                                                            20      0      4      0   100%
backends/qualcomm/genai_pipeline/exceptions.py                                                              20      0      6      0   100%
backends/qualcomm/genai_pipeline/genai_pipeline.py                                                          99      7     12      1    93%   190-201
backends/qualcomm/genai_pipeline/pipeline_context.py                                                        52      0     14      0   100%
backends/qualcomm/genai_pipeline/pipeline_stage.py                                                           5      0      0      0   100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py                                                14      0      0      0   100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py                                                  14      0      0      0   100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py                                          14      2      0      0    86%   30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py                                               14      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py                              7      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py                                 11      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py                   7      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py                              14      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py                       7      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py                                  7      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py      68      0     14      0   100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py                        9      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py                  7      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py                61      0     18      0   100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py                            7      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py                                9      0      0      0   100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                                                                      593      9     76      1    99%

… quantization strategy implementations

pytorch-bot · 2026-08-17T22:18:10Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21899

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⚠️ 12 Awaiting Approval

As of commit 1fee845 with merge base 9ffbe36 ():

AWAITING APPROVAL - The following workflows need approval before CI can run:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

qti-horodnic · 2026-08-17T22:18:43Z

@pytorchbot label "release notes: qualcomm"

Qualcomm AI Engine Direct - [GenAI Pipeline] PR5: Model preparation &…

1fee845

… quantization strategy implementations

qti-horodnic requested review from abhinaykukkadapu and psiddh as code owners August 17, 2026 22:18

meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 17, 2026

pytorch-bot Bot added the release notes: qualcomm Changes to the Qualcomm backend delegate label Aug 17, 2026

Navigation Menu

Uh oh!

Qualcomm AI Engine Direct - [GenAI Pipeline] PR5: Model preparation & quantization strategy implementations - #21899

Qualcomm AI Engine Direct - [GenAI Pipeline] PR5: Model preparation & quantization strategy implementations#21899
qti-horodnic wants to merge 1 commit into
pytorch:mainfrom
CodeLinaro:pr4b

qti-horodnic commented Aug 17, 2026

Uh oh!

pytorch-bot Bot commented Aug 17, 2026 •

edited

Loading

Uh oh!

qti-horodnic commented Aug 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

qti-horodnic commented Aug 17, 2026

Summary

What's included

Strategy implementations (2 files + 1 __init__ fix):

Unit tests:

PR Review Checklist

Related PRs

Test plan

Run only tests added in this PR:

Run only this PR's tests with coverage:

Run all genai_pipeline tests:

Run all genai_pipeline tests with coverage:

Uh oh!

pytorch-bot Bot commented Aug 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21899

⚠️ 12 Awaiting Approval

Uh oh!

qti-horodnic commented Aug 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Strategy implementations (2 files + 1 `init` fix):

Run all `genai_pipeline` tests:

Run all `genai_pipeline` tests with coverage:

pytorch-bot Bot commented Aug 17, 2026 •

edited

Loading