BiRefNet variants explained: which one to use for portraits, products, and complex scenes

BiRefNet models explained: what General, Portrait, Matting, HR and COD each do best, and which checkpoint to pick for products, portraits or hair edges.

Z.Tools9 分钟阅读
Z.Tools blog OG image: birefnet-background-removal-series

BiRefNet is easy to misunderstand because the name gets used two ways. Sometimes people mean the research architecture from the CAAI AIR paper "Bilateral Reference for High-Resolution Dichotomous Image Segmentation." Other times they mean one of the public checkpoints built from that architecture: BiRefNet General, BiRefNet Portrait, BiRefNet Matting, BiRefNet HR, BiRefNet COD, and a few related variants.

Background removal is not one problem. A catalog shoe on a white sweep, a person with flyaway hair, a glass perfume bottle, and a bird hidden in branches all ask for different behavior from the mask. A general checkpoint can cut a product cleanly and still clip hair; a matting checkpoint can keep soft edges and still blur the edge of a hard plastic object (see the Matting section; not tested here).

Pick the checkpoint that matches the image.

What the BiRefNet paper contributes

The original BiRefNet paper frames the task as dichotomous image segmentation, which is a formal way of saying the model separates the important foreground from everything else. The research contribution is the bilateral reference mechanism. In plain English, BiRefNet tries to keep the whole image in mind while still paying attention to local edge detail.

A model that sees only the whole frame can identify the subject and still shave off hair, fingers, strings, cables or lace. A model that looks only at small patches can keep edge texture and lose track of what belongs to the subject.

BiRefNet combines global localization with detail reconstruction. The paper reports broad benchmark gains across high resolution dichotomous segmentation, camouflaged object detection, high resolution salient object detection, and ordinary salient object detection. The public Hugging Face model card also notes that the main weights are MIT licensed, around 0.2 billion parameters, and widely used through Spaces and downstream tools.

The paper's benchmark gains say nothing about which variant suits a given image. The variants differ in training data, and that changes their behavior.

BiRefNet General

Start with BiRefNet General when you do not have a strong reason to choose something else. It is the most sensible first pass for mixed content: products, ordinary objects, people in simple scenes, food, furniture, and the usual "remove the background from this image" jobs.

General is not specialized for one kind of subject: it does not assume the subject is a person, and a matting-focused model is more likely to soften an edge that should stay sharp (not tested here).

Use it first for e-commerce product photos, marketplace thumbnails, social posts, object cutouts and images where the subject is visually clear. If the result is already clean, there is no reason to switch; keep the specialist variants for images where General fails.

BiRefNet Portrait

BiRefNet Portrait is for people. More specifically, it is for people in normal photographic compositions: headshots, profile photos, fashion images, creator thumbnails, staff pages, and full body shots where the person is the obvious subject.

The Portrait checkpoint was trained with portrait-oriented data including P3M-10k and TR-humans. On the P3M portrait test set, the public card reports an S-measure of 0.983 and mean absolute error of 0.006. Those numbers come from the card's test set, not from your own photos.

Try Portrait when General clips shoulders, hair, hands or clothing edges. It is a weaker fit when the person is part of a crowded scene, partly hidden, heavily motion-blurred, or dressed in colors that blend into the background; in those images the scene itself is ambiguous, and switching to Portrait will not fix that.

BiRefNet Matting

BiRefNet Matting is for images where the edge itself is the job.

A normal segmentation mask is close to a yes-or-no cutout; matting estimates partial transparency. Hair, fur, smoke, translucent fabric, veils, mesh, glass, motion blur and any boundary where the foreground fades into the background need that.

The public BiRefNet Matting card lists training data from several matting sets, including P3M-10k, AM-2k, AIM-500, Human-2k, Distinctions-646, HIM2K, and PPM-100. Its reported P3M non-portrait test result includes an S-measure of 0.979 and mean squared error of 0.003. That is a result on the card's test set; whether individual strands survive on your image is untested here.

The trade-off is softness. Because Matting estimates partial transparency, it can soften the boundary of a hard-edged product (not tested here). For a metal chair, a phone case or a cardboard box, General is the safer first choice because the edge should stay firm.

BiRefNet HR

BiRefNet HR is trained for higher resolution input, with public documentation describing 2048 by 2048 training. Use it when your source image has enough real detail to justify the extra work: print assets, large product photos, detailed illustrations, high resolution packshots, and close crops where a weak edge will be obvious.

The HR model card reports a gap on DIS validation data when compared at high resolution. BiRefNet HR shows an S-measure of 0.927 and mean absolute error of 0.026 at 2048 by 2048, while the standard general checkpoint is listed at 0.898 and 0.037 in the same high resolution comparison.

On a small or compressed upload, or one destined for a tiny thumbnail, HR can only spend more compute on detail that is not there. Use it when the output will be inspected, composited, printed or zoomed.

BiRefNet HRSOD

BiRefNet HRSOD sits close to HR. HRSOD means high resolution salient object detection. Salient object detection is about finding the visually dominant subject in an image.

That makes HRSOD useful when the subject is obvious to a human viewer but the image contains a lot of high resolution texture: a bag on a patterned rug, a ceramic piece on a busy table, a plant against detailed wallpaper. It looks for the main subject in a detailed frame and does not assume the subject is a person.

Between HR and HRSOD, try HR for high resolution cutout fidelity and HRSOD when the subject is prominent but the scene is visually busy.

BiRefNet COD

BiRefNet COD is trained for camouflaged object detection (COD). The official model zoo lists COD training on COD10K and CAMO data, which are built around objects that blend into their surroundings.

That training data suggests wildlife photography, but the same problem appears in commercial images: a beige product on a beige surface, a patterned dress against patterned wallpaper, white packaging on a white set, or a pet on a textured blanket.

For clean product photos, start with General instead. Use COD when the normal variants lose the subject because the background has the same color, pattern, or contrast.

BiRefNet DIS and the large training variants

You may also see BiRefNet DIS or large DIS5K-flavored variants in provider lists. These are closer to the research side of the family. DIS is the benchmark task behind the original paper, and DIS5K is one of the important datasets in that world.

For normal background removal, DIS is a poor first choice unless you are testing segmentation behavior or comparing research checkpoints. In a production editing workflow, General, Portrait, Matting, HR, HRSOD and COD are easier to choose between because their names describe the image problem.

How BiRefNet compares with Bria RMBG

Bria RMBG v2.0 is also built on the BiRefNet architecture. Bria's public repository says RMBG v2.0 uses BiRefNet with Bria's own dataset and training scheme. It also says the model was trained on more than 15,000 manually labeled, fully licensed images covering objects, people, animals, text, photorealistic content, and non-photorealistic content.

So the comparison is between open BiRefNet variants and a commercially packaged, BiRefNet-based model trained on licensed data.

If your main concern is cutout quality across everyday images, BiRefNet General and Bria RMBG are both worth testing. If training data provenance or commercial licensing matters, Bria states that its images are fully licensed. If you need a specialist for a hard image, the BiRefNet family has more checkpoints to choose from: Portrait for people, Matting for soft edges, HR for large images, COD for low-contrast subjects.

Where you can run it

The official weights are on Hugging Face, and the GitHub repository points users to the model zoo, demos, and deployment options. The project page also mentions hosted inference through fal, while Replicate has a public BiRefNet endpoint with millions of runs. Runware exposes a background removal tool with several BiRefNet variants, including General, Portrait, Matting, HRSOD, COD, and speed-oriented options.

You can run it locally if you are comfortable with Python and GPU setup, or use a hosted provider if you only need the output. Unless you already run a GPU, I would use hosted access: it skips the environment setup, and the fee on Z.Tools is $0.0006 per image.

AI 智能抠图

AI 智能抠图

AI 驱动的一键抠图,精准分离主体与背景

继续阅读