Transformers Pipeline 推断指南基于 Transformers 的统一多模态推理 API 与参数配置详解【免费下载链接】transformers Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.项目地址: https://gitcode.com/GitHub_Trending/tra/transformers本指南以 docs/source/en/pipeline_tutorial.md 为核心主线围绕 Transformers 的Pipeline推断 API 展开。Pipeline是一套简单却强大的推理接口可基于 Hugging Face Hub 上的任意模型快速完成文本、图像、音频与多模态任务的推断。读完本文你将掌握如何按任务挑选 pipeline、在多类硬件GPU/CPU/Apple Silicon上配置运行设备、启用批处理与 Chunk 批处理、用生成器处理大规模数据集并通过半精度权重、device_mapauto与量化等手段在有限显存下运行大模型。Transformers 库将加载模型 预处理 前向推断 后处理封装为一条开箱即用的推理流水线开发者只需把输入数据丢给Pipeline其余繁琐环节全部交给框架处理。本文从使用姿势讲起逐步深入到源码实现与参数原理。1. 什么是PipelinePipeline是一个面向文本、视觉、音频及多模态等多种机器学习任务的统一推断 API可直接搭配 Hub 上的任意模型使用。你可以通过任务专属参数例如为会议记录的自动语音识别 pipeline 加上时间戳对流水线进行裁剪。[Pipeline] 支持 GPU、Apple Silicon 与半精度权重用于加速推理并节约显存。在 Transformers 中实际存在两类 pipeline 类通用基类Pipeline抽象推断流程base.py 中定义了preprocess/_forward/postprocess与__call__/get_iterator/run_single等执行逻辑大量任务专属 pipeline例如TextGenerationPipelinetext_generation.py、AutomaticSpeechRecognitionPipelineautomatic_speech_recognition.py、ImageClassificationPipelineimage_classification.py等。两类 pipeline 的加载方式是统一的在pipeline()入口中通过task参数传入任务标识符即可实例化对应的专属类。任务标识符可在 pipelines/init.py 的TASK_ALIASES/SUPPORTED_TASKS注册表中查到。每个任务都预置了一个默认的预训练模型与预处理器若想换用别的模型用model参数覆盖即可。例如用TextGenerationPipeline搭配 Gemma 2模型说明见 docs/source/en/model_doc/gemma2.md只需设置tasktext-generation、modelgoogle/gemma-2-2bfrom transformers import pipeline pipeline pipeline(tasktext-generation, modelgoogle/gemma-2-2b) pipeline(the secret to baking a really good cake is ) [{generated_text: the secret to baking a really good cake is 1. the right ingredients 2. the}]注示例中模型按 task 默认配好了配套的 tokenizer/processor若你的模型在 Hub 上的配置齐全pipeline()会依据模型的pipeline_tag推断任务甚至可以不传task前提是get_task能从模型卡片读取pipeline_tag见 get_task。多输入推断当输入多于一个时把它们放进一个列表即可pipeline 会自动按顺序逐个推理并返回对应的结果列表from transformers import pipeline from accelerate import Accelerator device Accelerator().device pipeline pipeline(tasktext-generation, modelgoogle/gemma-2-2b, devicedevice) pipeline([the secret to baking a really good cake is , a baguette is ]) [[{generated_text: the secret to baking a really good cake is 1. the right ingredients 2. the}], [{generated_text: a baguette is 100% bread.\n\na baguette is 100%}]]2. 面向多模态的 Task 用法示例Pipeline覆盖了文本、视觉、音频等多个模态的大量任务。以下分别展示语音识别、图像分类与视觉问答三个典型场景。自动语音识别automatic speech recognition传入音频文件的 URL 即可完成转写。底层由AutomaticSpeechRecognitionPipelineautomatic_speech_recognition.py处理from transformers import pipeline pipeline pipeline(taskautomatic-speech-recognition, modelopenai/whisper-large-v3) pipeline(https://huggingface.co/datasets/Narsil/asr_dummy/resolve/main/mlk.flac) {text: I have a dream that one day this nation will rise up and live out the true meaning of its creed.}图像分类image classificationImageClassificationPipelineimage_classification.py接受图像路径或 URL返回带置信度分数score的 Top-K 标签列表from transformers import pipeline pipeline pipeline(taskimage-classification, modelgoogle/vit-base-patch16-224) pipeline(imageshttps://huggingface.co/datasets/huggingface/documentation-images/resolve/main/pipeline-cat-chonk.jpeg) [{label: lynx, catamount, score: 0.43350091576576233}, {label: cougar, puma, catamount, mountain lion, painter, panther, Felis concolor, score: 0.034796204417943954}, {label: snow leopard, ounce, Panthera uncia, score: 0.03240183740854263}, {label: Egyptian cat, score: 0.02394474856555462}, {label: tiger cat, score: 0.02288915030658245}]视觉问答visual question answering同时传入图像与问题文本VQA 模型如Salesforce/blip-vqa-base即可得到自然语言答案from transformers import pipeline pipeline pipeline(taskvisual-question-answering, modelSalesforce/blip-vqa-base) pipeline( imagehttps://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/idefics-few-shot.jpg, questionWhat is in the image?, ) [{answer: statue of liberty}]从 pipelines/init.py 的SUPPORTED_TASKS注册表可看到任务到具体 pipeline 类的映射关系同一张表还记录了每个任务允许的模型类型如text-generation对应AutoModelForCausalLM系列与默认模型。需要列出所有受支持任务时可直接调用pipeline模块的get_supported_tasks()见 get_supported_tasks。3. 核心参数详解Pipeline最少只需要任务标识符 模型 合适输入三者。除此之外还提供大量参数涵盖任务专属行为与性能优化两大类。这里介绍几个最重要的通用参数。3.1 Device硬件设备选择Pipeline兼容 GPU、CPU、Apple Silicon 等多种硬件。通过device参数指定运行设备默认行为不传device时pipeline 会自动把模型放到第一个可用的加速器上CUDA GPU、Apple Silicon 的 MPS、XPU 等仅当没有任何加速器可用时才回退到 CPU。可传devicecpu强制使用 CPU。GPU——把device设为对应的 CUDA 设备号即可例如device0表示在第一张 GPU 上运行from transformers import pipeline pipeline pipeline(tasktext-generation, modelgoogle/gemma-2-2b, device0) pipeline(the secret to baking a really good cake is )也可以交给 Accelerate 自动决定如何把模型权重加载/放置到合适设备这对拥有多张设备的情形尤其有用。Accelerate 会优先把权重放到最快的设备再按需把多余权重卸载到 CPU、硬盘等更慢的设备。设置device_mapauto即可启用前置条件需要安装 Accelerate。!pip install -U acceleratefrom transformers import pipeline pipeline pipeline(tasktext-generation, modelgoogle/gemma-2-2b, device_mapauto) pipeline(the secret to baking a really good cake is )从 pipeline 工厂函数签名 可以看到device支持int | str | torch.device而device_map支持str | dict——这意味着你既可以写device_mapauto也可以传入{cuda:0: 10GiB, cpu: 20GiB}这类自定义分配字典。Apple Silicon——在苹果芯片上运行时把device设为mpsfrom transformers import pipeline pipeline pipeline(tasktext-generation, modelgoogle/gemma-2-2b, devicemps) pipeline(the secret to baking a really good cake is )3.2 dtype精度选择工厂函数中还内置了dtype参数默认auto。它支持torch.float16与torch.bfloat16等类型半精度能显著提速并节约显存对大模型而言精度损失通常可忽略。若硬件支持可改用torch.bfloat16以获得更大的数值表示范围详见下文大模型一节。4. Batch inference批量推断用batch_size参数可以对一批输入做批量推断在 GPU 上通常能提升吞吐。但提速并非必然——硬件、数据与模型本身都会影响最终收益因此框架默认关闭批量推断。以 4 个输入、batch_size2为例pipeline 每次把 2 个输入组成一个 batch 送给模型from transformers import pipeline from accelerate import Accelerator device Accelerator().device pipeline pipeline(tasktext-generation, modelgoogle/gemma-2-2b, devicedevice, batch_size2) pipeline([the secret to baking a really good cake is, a baguette is, paris is the, hotdogs are]) [[{generated_text: the secret to baking a really good cake is to use a good cake mix.\n\ni’}], [{generated_text: a baguette is}], [{generated_text: paris is the most beautiful city in the world.\n\ni’ve been to paris 3}], [{generated_text: hotdogs are a staple of the american diet. they are a great source of protein and can}]]批量推断的另一个典型场景是流式处理数据集。借助KeyDataset定义于 pt_utils.py用于把数据集元素字典中的某个键抽取出来作为 pipeline 输入可以一边迭代一边按批推断from transformers import pipeline from accelerate import Accelerator from transformers.pipelines.pt_utils import KeyDataset import datasets device Accelerator().device # KeyDataset 是返回数据集中某个 key 的值的工具类 dataset load_dataset(stanfordnlp/imdb, nameplain_text, splitunsupervised) pipeline pipeline(tasktext-classification, modeldistilbert/distilbert-base-uncased-finetuned-sst-2-english, devicedevice) for out in pipeline(KeyDataset(dataset, text), batch_size8, truncationonly_first): print(out)在 base.py 的get_iterator实现中可以看到底层机制输入会先被包装为PipelineDataset/PipelineIterator再经DataLoader按batch_size切分collate_fnno_collate_fn或pad_collate_fn负责把各样本 padding 到同一长度最终依次执行self.forward与self.postprocess产出结果。注意传入关键字truncationonly_first等 tokenizer 参数会被_sanitize_parameters归类进预处理参数中生效例如 text_classification.py 的_sanitize_parameters接收**tokenizer_kwargs。何时应该用批量推断六条经验法则唯一可靠的办法是在你的模型、数据、硬件上实测性能。若受延迟约束例如在线实时推理产品不要批量推断。在 CPU 上运行时不要批量推断。若不知道数据的sequence_length不要批量推断。应先实测逐步加大序列长度并加入 OOM显存溢出检查以便失败恢复。若你的数据sequence_length比较规整可以批量推断并不断加大batch_size直到触发 OOM。GPU 越大批量推断收益越明显。如果决定做批量推断务必确保能妥善处理 OOM 错误。一个值得注意的细节在 Pipeline.call中当同一个 pipeline 对象在 GPU 上被顺序调用超过 10 次时框架会提示请使用 dataset 方式输入以最大化效率——这正是把批量/流式接口设计为返回迭代器而非一次性物化所有结果的原因。5. Task 专属参数Pipeline会透传各任务 pipeline 支持的任意参数。使用前应查阅具体任务 pipeline 的 API 文档确认可用参数。以下给出两个典型示例。ASR返回逐词时间戳给自动语音识别 pipeline 传return_timestampsword即可拿到每个单词出现的时间区间。返回结果中的chunks列表每条包含text与(start, end)时间戳from transformers import pipeline pipeline pipeline(taskautomatic-speech-recognition, modelopenai/whisper-large-v3) pipeline(audiohttps://huggingface.co/datasets/Narsil/asr_dummy/resolve/main/mlk.flac, return_timestampsword) {text: I have a dream that one day this nation will rise up and live out the true meaning of its creed., chunks: [{text: I, timestamp: (0.0, 1.1)}, {text: have, timestamp: (1.1, 1.44)}, {text: a, timestamp: (1.44, 1.62)}, {text: dream, timestamp: (1.62, 1.92)}, {text: that, timestamp: (1.92, 3.7)}, {text: one, timestamp: (3.7, 3.88)}, {text: day, timestamp: (3.88, 4.24)}, {text: this, timestamp: (4.24, 5.82)}, {text: nation, timestamp: (5.82, 6.78)}, {text: will, timestamp: (6.78, 7.36)}, {text: rise, timestamp: (7.36, 7.88)}, {text: up, timestamp: (7.88, 8.46)}, {text: and, timestamp: (8.46, 9.2)}, {text: live, timestamp: (9.2, 10.34)}, {text: out, timestamp: (10.34, 10.58)}, {text: the, timestamp: (10.58, 10.8)}, {text: true, timestamp: (10.8, 11.04)}, {text: meaning, timestamp: (11.04, 11.4)}, {text: of, timestamp: (11.4, 11.64)}, {text: its, timestamp: (11.64, 11.8)}, {text: creed., timestamp: (11.8, 12.3)}]}从 automatic_speech_recognition.py 的_sanitize_parameters可以看到ASR pipeline 还支持return_language、chunk_length_s、stride_length_s等更多选项return_timestamps会被标记为_forward阶段的参数与 word-level 对齐_align_to逻辑协同完成逐词时间戳。文本生成返回多个序列与仅返回生成文本TextGenerationPipeline.__call__还额外支持来自GenerationMixin.generate的全部关键字参数。传num_return_sequences4可以让每个输入生成 4 条候选序列传return_full_textFalse则只返回新生成的文本而不含原始 promptfrom transformers import pipeline pipeline pipeline(tasktext-generation, modelopenai-community/gpt2) pipeline(the secret to baking a good cake is, num_return_sequences4, return_full_textFalse) [{generated_text: how easy it is for me to do it with my hands. You must not go nuts, or the cake is going to fall out.}, {generated_text: to prepare the cake before baking. The key is to find the right type of icing to use and that icing makes an amazing frosting cake.\n\nFor a good icing cake, we give you the basics}, {generated_text: to remember to soak it in enough water and dont worry about it sticking to the wall. In the meantime, you could remove the top of the cake and let it dry out with a paper towel.\n}, {generated_text: the best time to turn off the oven and let it stand 30 minutes. After 30 minutes, stir and bake a cake in a pan until fully moist.\n\nRemove the cake from the heat for about 12}]查看 text_generation.py 的_sanitize_parameters可知诸如max_length、handle_long_generation、stop_sequence、prefix、truncation、clean_up_tokenization_spaces等参数在预处理阶段被消化而num_return_sequences等**generate_kwargs会被直接转交给_forward中的模型生成调用与GenerationMixin.generate的采样参数temperature、top_p、do_sample等无缝衔接。6. Chunk batching切块批处理有些场景需要把输入切块再喂给模型某些数据类型下单个输入本身需要被拆成多段才能处理例如一条特别长的音频文件某些任务如 zero-shot 分类、问答下单个输入需要多次前向计算此时与batch_size参数会产生冲突。ChunkPipeline类就是为上述场景设计的实现于 base.py。它与普通Pipeline的用法完全一致区别在于它能自动管理内部的批处理你无需操心单个输入会触发多少次前向只需独立地调优batch_size。两者执行流程的对比如下preprocess产出可迭代的输入块时普通Pipeline.run_single假定每次预处理只产出一次前向所需输入而ChunkPipeline.run_single会循环消费预处理产生的所有块# ChunkPipeline all_model_outputs [] for preprocessed in pipeline.preprocess(inputs): model_outputs pipeline.model_forward(preprocessed) all_model_outputs.append(model_outputs) outputs pipeline.postprocess(all_model_outputs) # Pipeline preprocessed pipeline.preprocess(inputs) model_outputs pipeline.forward(preprocessed) outputs pipeline.postprocess(model_outputs)注文档中的model_forward在真实源码中即基类Pipeline.forward见 base.py它负责把输入张量确保移动到模型所在设备、以torch.no_grad()上下文调用_forward再把输出统一拉回 CPU。ChunkPipeline通过重写run_single与get_iterator后者使用PipelineChunkIterator/PipelinePackIterator包装实现对每块输入分别前向、最后统一后处理的效果。ChunkPipeline的真实实现代码如下简化自 base.pyclass ChunkPipeline(Pipeline): def run_single(self, inputs, preprocess_params, forward_params, postprocess_params): all_outputs [] for model_inputs in self.preprocess(inputs, **preprocess_params): model_outputs self.forward(model_inputs, **forward_params) all_outputs.append(model_outputs) outputs self.postprocess(all_outputs, **postprocess_params) return outputs7. 大数据集推断对超大数据集做推断时可以直接迭代数据集本身而无需一次性把整个数据集物化到内存也不必手动构造 batch。推荐配合batch_size试验批量推断是否提升性能from transformers.pipelines.pt_utils import KeyDataset from transformers import pipeline from accelerate import Accelerator from datasets import load_dataset device Accelerator().device dataset load_dataset(stanfordnlp/imdb, nameplain_text, splitunsupervised) pipeline pipeline(tasktext-classification, modeldistilbert/distilbert-base-uncased-finetuned-sst-2-english, devicedevice) for out in pipeline(KeyDataset(dataset, text), batch_size8, truncationonly_first): print(out)在 Pipeline.call中可以看到框架对不同类型的输入走不同分支list 输入被急切地消费一次性返回完整的输出 listDataset 与生成器generator输入则惰性流式返回迭代器消费多少计算多少内存占用与输入规模解耦单条输入 ChunkPipeline走内部迭代器分支。用生成器/迭代器驱动另一种常见做法是传入生成器或迭代器适合逐条产出输入如流式爬取或在线数据源def data(): for i in range(1000): yield fMy example {i} pipeline pipeline(modelopenai-community/gpt2, device0) generated_characters 0 for out in pipeline(data()): generated_characters len(out[0][generated_text])注意此例未显式传taskpipeline 工厂会尝试通过模型卡片的pipeline_tag自动推断任务见 get_task离线模式下该推断不可用。8. 大模型的资源优化Accelerate 为Pipeline运行大模型提供了若干优化手段先安装依赖!pip install -U accelerate8.1 自动设备放置device_mapauto会把模型自动分布到最快设备GPU优先放不下时再调度到更慢的设备CPU、硬盘。这与第 3.1 节的介绍一致对单机显存不足以装下完整模型权重如 Gemma 7B 的全精度版本的场景尤其关键。8.2 半精度权重Pipeline支持以torch.float16半精度加载权重通常能明显提速并节约显存对大多数模型尤其大模型精度损失可忽略。若硬件支持torch.bfloat16可改用该类型以换取更大的数值范围。注意输入会在内部自动转换为torch.float16并且该优化只对 PyTorch 后端的模型生效。在真实调用中通过工厂函数参数直接指定精度即可import torch from transformers import pipeline pipeline pipeline(modelgoogle/gemma-7b, dtypetorch.bfloat16, device_mapauto) pipeline(the secret to baking a good cake is )从 pipeline 工厂函数签名 可知dtype接受字符串或torch.dtype例如dtypefloat16、dtypetorch.bfloat16均可。8.3 量化加载Pipeline同样接受量化模型以进一步降低显存占用。先安装 bitsandbytes再把quantization_config放进model_kwargsimport torch from transformers import pipeline, BitsAndBytesConfig pipeline pipeline(modelgoogle/gemma-7b, dtypetorch.bfloat16, device_mapauto, model_kwargs{quantization_config: BitsAndBytesConfig(load_in_8bitTrue)}) pipeline(the secret to baking a good cake is ) [{generated_text: the secret to baking a good cake is 1. the right ingredients 2. the right}]model_kwargs会被工厂函数原样转交给模型加载流程load_model见 base.py 附近因此除了quantization_config你还可以通过它传递attn_implementation、revision、torch_dtype等任何from_pretrained支持的加载选项。BitsAndBytesConfig定义于库的量化配置模块通过from transformers import BitsAndBytesConfig导入支持load_in_8bitTrue、load_in_4bitTrue等模式。9. 完整 API 文档见 docs/source/en/main_classes/pipelines.md 对应的源码与类注释各任务专属 pipeline 类的行为可分别查阅 text_generation.py、automatic_speech_recognition.py、image_classification.py、pt_utils.pyKeyDataset等工具类。想了解 pipeline 注册机制与任务标识符 → 实现类 → 默认模型的完整映射可阅读 pipelines/init.py 与 base.py 中的PipelineRegistry实现。测试用例可参考 tests/pipelines 目录下各任务的测试文件它们展示了大量输入形态与参数组合的合法用法。【免费下载链接】transformers Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.项目地址: https://gitcode.com/GitHub_Trending/tra/transformers创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考