SAM 图像分割实战：自动 Mask、框提示、点提示与 COCO 对比

SAM 图像分割实战自动 Mask、框提示、点提示与 COCO 对比这篇教程根据我复现 SAM 图像分割流程时整理重点演示自动 mask 生成、框提示分割、点提示分割以及把 SAM 结果和真实 COCO 标注做对比。SAM 是很多分割工作流的基础模块。本文更适合当成分割交互工具的起点先理解自动分割和 prompt 分割再扩展到自己的数据。本文会重点跑通以下流程安装 SAM 与可视化依赖下载 checkpoint 并准备示例图片使用自动 mask 生成器完成整图分割使用框提示和点提示做交互式分割用 COCO 标注数据和 SAM 结果做对比如果你正在系统学习目标检测、实例分割、OCR、多目标跟踪或视觉大模型建议收藏本文配套 notebook、示例图片和运行环境说明后续会继续整理。如果环境配置卡住可以在评论区说明具体报错。 文章目录SAM 图像分割实战自动 Mask、框提示、点提示与 COCO 对比⚙️ 环境准备⬇️ 下载权重与示例图片 加载 SAM 模型 自动 Mask 生成 框提示分割 点提示分割 COCO 标注对比 小结 同系列教程汇总⚙️ 环境准备先检查运行环境并安装依赖。建议优先使用带 NVIDIA GPU 的环境避免推理和训练阶段显存不足。!nvidia-smiimportos HOMEos.getcwd()print(HOME:,HOME)!pip install-qgithttps://github.com/facebookresearch/segment-anything.git!pip install-qgithttps://github.com/facebookresearch/segment-anything.git!pip install-q jupyter_bbox_widget supervision0.23.0dataclasses-json⬇️ 下载权重与示例图片先把 SAM checkpoint 和示例图片准备好。!mkdir-p{HOME}/weights !wget-q https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth-P{HOME}/weightsimportos CHECKPOINT_PATHos.path.join(HOME,weights,sam_vit_h_4b8939.pth)print(CHECKPOINT_PATH,; exist:,os.path.isfile(CHECKPOINT_PATH))!mkdir-p{HOME}/data# 请从数据集后台下载示例图片并放到 {HOME}/data 目录。# 文件名保持为 dog.jpeg、dog-2.jpeg、dog-3.jpeg、dog-4.jpeg。 加载 SAM 模型加载 SAM 后就可以切换自动分割和提示分割两种方式。importtorch DEVICEtorch.device(cuda:0iftorch.cuda.is_available()elsecpu)MODEL_TYPEvit_hfromsegment_anythingimportsam_model_registry,SamAutomaticMaskGenerator,SamPredictor samsam_model_registry[MODEL_TYPE](checkpointCHECKPOINT_PATH).to(deviceDEVICE) 自动 Mask 生成自动生成整图 mask适合快速浏览分割候选。mask_generatorSamAutomaticMaskGenerator(sam)importos IMAGE_NAMEdog.jpegIMAGE_PATHos.path.join(HOME,data,IMAGE_NAME)importcv2importsupervisionassv image_bgrcv2.imread(IMAGE_PATH)image_rgbcv2.cvtColor(image_bgr,cv2.COLOR_BGR2RGB)sam_resultmask_generator.generate(image_rgb)print(sam_result[0].keys()) 框提示分割框提示适合把模型的注意力锁定到某个对象上。mask_annotatorsv.MaskAnnotator(color_lookupsv.ColorLookup.INDEX)detectionssv.Detections.from_sam(sam_resultsam_result)annotated_imagemask_annotator.annotate(sceneimage_bgr.copy(),detectionsdetections)sv.plot_images_grid(images[image_bgr,annotated_image],grid_size(1,2),titles[source image,segmented image])masks[mask[segmentation]formaskinsorted(sam_result,keylambdax:x[area],reverseTrue)]sv.plot_images_grid(imagesmasks,grid_size(8,int(len(masks)/8)),size(16,16))mask_predictorSamPredictor(sam)importos IMAGE_NAMEdog.jpegIMAGE_PATHos.path.join(HOME,data,IMAGE_NAME)# helper function that loads an image before adding it to the widgetimportbase64defencode_image(filepath):withopen(filepath,rb)asf:image_bytesf.read()encodedstr(base64.b64encode(image_bytes),utf-8)returndata:image/jpg;base64,encodedIS_COLABTrueifIS_COLAB:fromgoogle.colabimportoutput output.enable_custom_widget_manager()fromjupyter_bbox_widgetimportBBoxWidget widgetBBoxWidget()widget.imageencode_image(IMAGE_PATH)widgetwidget.bboxes 点提示分割点提示能进一步缩小分割范围。importnumpyasnp# default_box is going to be used if you will not draw any box on image abovedefault_box{x:68,y:247,width:555,height:678,label:}boxwidget.bboxes[0]ifwidget.bboxeselsedefault_box boxnp.array([box[x],box[y],box[x]box[width],box[y]box[height]])importcv2importnumpyasnpimportsupervisionassv image_bgrcv2.imread(IMAGE_PATH)image_rgbcv2.cvtColor(image_bgr,cv2.COLOR_BGR2RGB)mask_predictor.set_image(image_rgb)masks,scores,logitsmask_predictor.predict(boxbox,multimask_outputTrue)box_annotatorsv.BoxAnnotator(colorsv.Color.RED,color_lookupsv.ColorLookup.INDEX)mask_annotatorsv.MaskAnnotator(colorsv.Color.RED,color_lookupsv.ColorLookup.INDEX)detectionssv.Detections(xyxysv.mask_to_xyxy(masksmasks),maskmasks)detectionsdetections[detections.areanp.max(detections.area)]source_imagebox_annotator.annotate(sceneimage_bgr.copy(),detectionsdetections)segmented_imagemask_annotator.annotate(sceneimage_bgr.copy(),detectionsdetections)sv.plot_images_grid(images[source_image,segmented_image],grid_size(1,2),titles[source image,segmented image])importsupervisionasv sv.plot_images_grid(imagesmasks,grid_size(1,4),size(16,4)) COCO 标注对比最后把 SAM 的结果和 COCO 标注做对比检查分割质量。importnumpyasnpfromdataclassesimportdataclassfromtypingimportList,Tuple,Union,Optionalfromdataclasses_jsonimportdataclass_jsonfromsupervisionimportDetectionsdataclass_jsondataclassclassCOCOCategory:id:intname:strsupercategory:strdataclass_jsondataclassclassCOCOImage:id:intwidth:intheight:intfile_name:strlicense:intdate_captured:strcoco_url:Optional[str]Noneflickr_url:Optional[str]Nonedataclass_jsondataclassclassCOCOAnnotation:id:intimage_id:intcategory_id:intsegmentation:List[List[float]]area:floatbbox:Tuple[float,float,float,float]iscrowd:intdataclass_jsondataclassclassCOCOLicense:id:intname:strurl:strdataclass_jsondataclassclassCOCOJson:images:List[COCOImage]annotations:List[COCOAnnotation]categories:List[COCOCategory]licenses:List[COCOLicense]defload_coco_json(json_file:str)-COCOJson:importjsonwithopen(json_file,r)asf:json_datajson.load(f)returnCOCOJson.from_dict(json_data)classCOCOJsonUtility:staticmethoddefget_annotations_by_image_id(coco_data:COCOJson,image_id:int)-List[COCOAnnotation]:return[annotationforannotationincoco_data.annotationsifannotation.image_idimage_id]staticmethoddefget_annotations_by_image_path(coco_data:COCOJson,image_path:str)-Optional[List[COCOAnnotation]]:imageCOCOJsonUtility.get_image_by_path(coco_data,image_path)ifimage:returnCOCOJsonUtility.get_annotations_by_image_id(coco_data,image.id)else:returnNonestaticmethoddefget_image_by_path(coco_data:COCOJson,image_path:str)-Optional[COCOImage]:forimageincoco_data.images:ifimage.file_nameimage_path:returnimagereturnNonestaticmethoddefannotations2detections(annotations:List[COCOAnnotation])-Detections:class_id,xyxy[],[]forannotationinannotations:x_min,y_min,width,heightannotation.bbox class_id.append(annotation.category_id)xyxy.append([x_min,y_min,x_minwidth,y_minheight])returnDetections(xyxynp.array(xyxy,dtypeint),class_idnp.array(class_id,dtypeint))%cd{HOME}# 如需使用真实 COCO 数据请先从数据集后台下载并解压到本地。fromtypesimportSimpleNamespace DATASET_DIR/content/dataset# 修改为数据集后台导出的数据集目录datasetSimpleNamespace(locationDATASET_DIR)importos DATA_SET_SUBDIRECTORYtestANNOTATIONS_FILE_NAME_annotations.coco.jsonIMAGES_DIRECTORY_PATHos.path.join(dataset.location,DATA_SET_SUBDIRECTORY)ANNOTATIONS_FILE_PATHos.path.join(dataset.location,DATA_SET_SUBDIRECTORY,ANNOTATIONS_FILE_NAME)coco_dataload_coco_json(json_fileANNOTATIONS_FILE_PATH)CLASSES[category.nameforcategoryincoco_data.categoriesifcategory.supercategory!none]IMAGES[image.file_nameforimageincoco_data.images]CLASSES# set random seed to allow easy reproduction of the experimentimportrandom random.seed(10)EXAMPLE_IMAGE_NAMErandom.choice(IMAGES)EXAMPLE_IMAGE_PATHos.path.join(dataset.location,DATA_SET_SUBDIRECTORY,EXAMPLE_IMAGE_NAME)# load dataset annotationsannotationsCOCOJsonUtility.get_annotations_by_image_path(coco_datacoco_data,image_pathEXAMPLE_IMAGE_NAME)ground_truthCOCOJsonUtility.annotations2detections(annotationsannotations)# small hack - coco numerate classes from 1, model from 0 we drop first redundant class from coco jsonground_truth.class_idground_truth.class_id-1# load imageimage_bgrcv2.imread(EXAMPLE_IMAGE_PATH)image_rgbcv2.cvtColor(image_bgr,cv2.COLOR_BGR2RGB)# initiate annotatorbox_annotatorsv.BoxAnnotator(colorsv.Color.RED,color_lookupsv.ColorLookup.INDEX)mask_annotatorsv.MaskAnnotator(colorsv.Color.RED,color_lookupsv.ColorLookup.INDEX)# annotate ground truthannotated_frame_ground_truthbox_annotator.annotate(sceneimage_bgr.copy(),detectionsground_truth)# run SAM inferencemask_predictor.set_image(image_rgb)masks,scores,logitsmask_predictor.predict(boxground_truth.xyxy[0],multimask_outputTrue)detectionssv.Detections(xyxysv.mask_to_xyxy(masksmasks),maskmasks)detectionsdetections[detections.areanp.max(detections.area)]annotated_imagemask_annotator.annotate(sceneimage_bgr.copy(),detectionsdetections)sv.plot_images_grid(images[annotated_frame_ground_truth,annotated_image],grid_size(1,2),titles[source image,segmented image]) 小结SAM 的优势是通用性强、交互直观。实际复现时自动 mask 适合探索框和点提示更适合精修而 COCO 对比可以帮助你快速检查效果。这一类 notebook 建议按“先环境、再数据、再单样例、最后批量推理”的顺序复现。遇到报错时优先检查 GPU、依赖版本、数据集目录和模型权重路径。后续我会继续按源项目顺序整理同系列中的目标检测、实例分割、OCR、多目标跟踪和视觉大模型教程。 同系列教程汇总Google Gemini 3.5 Flash 零样本目标检测教程从提示词到可视化结果GLM-OCR 文档识别实战教程从验证码、公式到车牌 OCRRF-DETR ByteTrack 多目标跟踪实战教程从命令行到 Python 视频轨迹可视化SAM 3 图像分割实战教程文本、框和点提示的多种分割方式SAM 图像分割实战自动 Mask、框提示、点提示与 COCO 对比