AWS published an August 4, 2026 guide showing how teams can deploy automated web insight extraction with Amazon Bedrock AgentCore Browser, Amazon Bedrock, Amazon OpenSearch Serverless and AWS Lambda. The system monitors RSS feeds, retrieves website content through a managed browser, extracts summaries and insights with AI, and makes the results searchable through a React web interface. AWS frames the workflow for design teams, marketing teams and product managers that manually track competitor products, content trends and market intelligence across dozens of websites.

The architecture separates content collection from AI processing. An Amazon EventBridge schedule triggers a Lambda function every 15 minutes to check configured RSS feeds, find new articles and deduplicate them against Amazon S3. For each new article, Lambda opens an Amazon Bedrock AgentCore Browser session and controls it with Playwright over the Chrome DevTools Protocol, allowing JavaScript-heavy pages to render before screenshots, images, HTML and metadata are saved to Amazon S3.

Amazon S3 upload events then publish messages to Amazon SQS, where a second Lambda function extracts clean text from raw HTML. Large HTML files over 1 MB are simplified with html-to-text to avoid token limits, while smaller files use Mozilla’s Readability library to extract the main content and primary image. Amazon Bedrock then generates concise summaries, themes, entities, categories, actionable insights and vector embeddings (numeric meaning representations for search).

The enriched content is indexed in Amazon OpenSearch Serverless for both keyword search and vector search, so a query such as “what are emerging design trends” can return relevant results even when those exact words do not appear in the source text. End users authenticate with Amazon Cognito and use a React-based frontend running on Amazon ECS with AWS Fargate. Programmatic access is available through a Model Context Protocol server on Amazon ECS with Fargate and Amazon CloudFront; MCP is described as an open standard that lets AI assistants connect to external data sources through a unified interface.

AWS says production deployments should add Amazon Bedrock Guardrails because the pipeline sends third-party web content to a foundation model and publishes generated output to teams. The controls include content filtering for harmful or inappropriate scraped material, denied topics and word filters for scope control, and contextual grounding checks to reduce hallucinated claims before they enter the searchable index. Processing components run inside Amazon VPC, with Amazon CloudWatch for observability and IAM for least privilege access.

The guide lists several implementation lessons. AgentCore Browser sessions took 10–30 seconds per Playwright render in AWS’s test and cost significantly more than plain HTTP requests, making URL hashing and S3 deduplication important before starting a new browser session. AWS also says raw HTML is a poor input for LLMs, SQS decoupling enables automatic retries through a dead-letter queue, and Amazon OpenSearch Serverless has a minimum-capacity cost floor. For lower-volume deployments, AWS says Amazon RDS with pgvector can provide vector search at a lower baseline cost.

AWS published an August 4, 2026 guide showing how teams can deploy automated web insight extraction with Amazon Bedrock AgentCore Browser, Amazon Bedrock, Amazon OpenSearch Serverless and AWS Lambda. The system monitors RSS feeds, retrieves website content through a managed browser, extracts summaries and insights with AI, and makes the results searchable through a React web interface. AWS frames the workflow for design teams, marketing teams and product managers that manually track competitor products, content trends and market intelligence across dozens of websites.

The architecture separates content collection from AI processing. An Amazon EventBridge schedule triggers a Lambda function every 15 minutes to check configured RSS feeds, find new articles and deduplicate them against Amazon S3. For each new article, Lambda opens an Amazon Bedrock AgentCore Browser session and controls it with Playwright over the Chrome DevTools Protocol, allowing JavaScript-heavy pages to render before screenshots, images, HTML and metadata are saved to Amazon S3.

Amazon S3 upload events then publish messages to Amazon SQS, where a second Lambda function extracts clean text from raw HTML. Large HTML files over 1 MB are simplified with html-to-text to avoid token limits, while smaller files use Mozilla’s Readability library to extract the main content and primary image. Amazon Bedrock then generates concise summaries, themes, entities, categories, actionable insights and vector embeddings (numeric meaning representations for search).

The enriched content is indexed in Amazon OpenSearch Serverless for both keyword search and vector search, so a query such as “what are emerging design trends” can return relevant results even when those exact words do not appear in the source text. End users authenticate with Amazon Cognito and use a React-based frontend running on Amazon ECS with AWS Fargate. Programmatic access is available through a Model Context Protocol server on Amazon ECS with Fargate and Amazon CloudFront; MCP is described as an open standard that lets AI assistants connect to external data sources through a unified interface.

AWS says production deployments should add Amazon Bedrock Guardrails because the pipeline sends third-party web content to a foundation model and publishes generated output to teams. The controls include content filtering for harmful or inappropriate scraped material, denied topics and word filters for scope control, and contextual grounding checks to reduce hallucinated claims before they enter the searchable index. Processing components run inside Amazon VPC, with Amazon CloudWatch for observability and IAM for least privilege access.

The guide lists several implementation lessons. AgentCore Browser sessions took 10–30 seconds per Playwright render in AWS’s test and cost significantly more than plain HTTP requests, making URL hashing and S3 deduplication important before starting a new browser session. AWS also says raw HTML is a poor input for LLMs, SQS decoupling enables automatic retries through a dead-letter queue, and Amazon OpenSearch Serverless has a minimum-capacity cost floor. For lower-volume deployments, AWS says Amazon RDS with pgvector can provide vector search at a lower baseline cost.