Amazon Tests Marketplace Content Quality System

Amazon Marketplace uses a multimodel system on Amazon SageMaker AI to evaluate A+ Content, the combination of text, images, and video on product pages, and recommend improvements. Amazon says selling partners with high-quality A+ Content boost sales by 8–20 percent compared with partners without it. Since the content has no strict image-ratio or word-count rules, quality varies among brands with large design teams and bootstrapped sellers. The evaluation renders each page as one snapshot so a Vision-Language Model (a model that interprets images and text together) can judge how its modules work as a whole.

The system keeps the VLM and its rubric fixed, then feeds its scores into a CatBoost classifier trained on expert labels. The rubric scores four dimensions—Visual Storytelling, Feature Presentation, A+ Layout, and Conversion Optimization—from 0 to 10, with confidence values and written rationales. One ecommerce marketing expert and four annotators rated 5,478 product pages across 10 categories as high or low quality; majority votes formed the labels, with 49.6 percent marked high quality. Annotator agreement against the consensus ranged from 0.845 to 0.915 on Cohen’s kappa, a measure of agreement adjusted for chance.

On a 20 percent holdout set of about 1,100 pages, a baseline that averaged the four VLM scores reached 0.80 accuracy and 0.84 F1 for the high-quality class. CatBoost using VLM scores reached 0.89 accuracy and 0.90 F1. The team added optical character recognition (OCR, software that detects text in images) signals for text density, character density, spacing, and overall readability. These measure layout, so pages with identical wording can receive different scores. OCR-only CatBoost reached 0.82 accuracy and 0.83 F1; combining the four VLM and four OCR features produced 0.94 accuracy and 0.95 F1, gains of 0.14 accuracy and 0.11 F1 over the baseline.

The team used SHAP, a method for estimating how much each input feature influences a model’s decision, to identify what drove page ratings. Across the holdout set, the most influential features were OCR Character Density, with a mean absolute SHAP value of 1.1; OCR Overall Readability and VLM Visual Storytelling, both at 1.0; A+ Layout at 0.8; Feature Presentation at 0.6; and Conversion Optimization at 0.4. For low-quality pages, the system sends the most negative feature contributions, product category, ecommerce best practices, and VLM rationales to an LLM to produce prioritized actions. In one example, an A+ Layout contribution of -1.2 ranked ahead of Visual Storytelling at -0.3. After sellers revised content, average quality scores rose 1.4 points on a 10-point scale in a statistically significant sample.

An additional experiment changed the VLM backbone on a single-category dataset of 499 content projects. With a lighter VLM variant, the baseline scored 0.74 accuracy and 0.72 F1; the article’s final sentence ends before reporting further results.

Amazon Marketplace uses a multimodel system on Amazon SageMaker AI to evaluate A+ Content, the combination of text, images, and video on product pages, and recommend improvements. Amazon says selling partners with high-quality A+ Content boost sales by 8–20 percent compared with partners without it. Since the content has no strict image-ratio or word-count rules, quality varies among brands with large design teams and bootstrapped sellers. The evaluation renders each page as one snapshot so a Vision-Language Model (a model that interprets images and text together) can judge how its modules work as a whole.

The system keeps the VLM and its rubric fixed, then feeds its scores into a CatBoost classifier trained on expert labels. The rubric scores four dimensions—Visual Storytelling, Feature Presentation, A+ Layout, and Conversion Optimization—from 0 to 10, with confidence values and written rationales. One ecommerce marketing expert and four annotators rated 5,478 product pages across 10 categories as high or low quality; majority votes formed the labels, with 49.6 percent marked high quality. Annotator agreement against the consensus ranged from 0.845 to 0.915 on Cohen’s kappa, a measure of agreement adjusted for chance.

On a 20 percent holdout set of about 1,100 pages, a baseline that averaged the four VLM scores reached 0.80 accuracy and 0.84 F1 for the high-quality class. CatBoost using VLM scores reached 0.89 accuracy and 0.90 F1. The team added optical character recognition (OCR, software that detects text in images) signals for text density, character density, spacing, and overall readability. These measure layout, so pages with identical wording can receive different scores. OCR-only CatBoost reached 0.82 accuracy and 0.83 F1; combining the four VLM and four OCR features produced 0.94 accuracy and 0.95 F1, gains of 0.14 accuracy and 0.11 F1 over the baseline.

The team used SHAP, a method for estimating how much each input feature influences a model’s decision, to identify what drove page ratings. Across the holdout set, the most influential features were OCR Character Density, with a mean absolute SHAP value of 1.1; OCR Overall Readability and VLM Visual Storytelling, both at 1.0; A+ Layout at 0.8; Feature Presentation at 0.6; and Conversion Optimization at 0.4. For low-quality pages, the system sends the most negative feature contributions, product category, ecommerce best practices, and VLM rationales to an LLM to produce prioritized actions. In one example, an A+ Layout contribution of -1.2 ranked ahead of Visual Storytelling at -0.3. After sellers revised content, average quality scores rose 1.4 points on a 10-point scale in a statistically significant sample.

An additional experiment changed the VLM backbone on a single-category dataset of 499 content projects. With a lighter VLM variant, the baseline scored 0.74 accuracy and 0.72 F1; the article’s final sentence ends before reporting further results.