Log Streaming for Enhanced Monitoring

Log Center supports log streaming to third-party monitoring and analytics tools. You can also stream logs using a generic HTTP connection. With these tools, Log Center users get enterprise-scale monitoring and automation capabilities. All B2C Commerce Log Center customers get log streaming at no additional cost.

Supported tools include:

  • Splunk
  • Datadoc
  • New Relic
  • Grafana Loki,
  • Sumo Logic
  • Dynatrace
  • AWS Elastic Cloud

Review this information about log streaming, and then configure it.

Log Streaming Operation

  • Stream sandbox logs.
  • To verify that logs have originated from Log Center, specify parameters when you configure log streaming. When you stream logs, these parameters can pass extra attributes as metadata to the third-party vendors. Most of these vendors can parse Firehose HTTP header X-Amz-Firehose-Common-Attributes, and can receive this metadata.
  • To turn log streaming on and off, use the stop and play button.
    Important
    Important If you disable and re-enable streaming, streaming resumes for only new logs. When log streaming is off, data for the time period is lost.

Log Streaming Configuration Guidance

  • If you configure log streaming multiple times for the same realm, the result can be duplicate logs. To avoid this situation, create separate log streaming configurations for production logs and non-production logs. Separate configurations make sure that logs get sent only one time, and helps you keep your data organized.
  • After you configure a log stream, only new logs are streamed. Existing files aren’t in the log stream.

Log Streaming Frequency and Performance

  • Logs stream in batches. The time between each message can be from 20 seconds to two minutes, depending on how fast logs are cached and streamed.
  • Log streaming is a real-time service. You can't configure logs to stream periodically, for example, every 5 minutes. You also can't stream logs from a specific number of days in the past.
  • Log volume limits don't impact log streaming. Streaming continues.

Log Streaming Support

  • If you have integration problems with logs in a third-party tool, create a support ticket?
  • If the API token used for log streaming expires, it is your responsibility to update it. We don’t automatically renew tokens.

Third-Party Documentation for Log Streaming

Log streaming uses AWS Firehose as an underlying streaming service. For more information about AWS Firehose integration, see these resources.

  • Set Up Log Streaming to Third-Party Monitoring Tools
    For enterprise-scale monitoring and automation, stream B2C Commerce logs to third-party analytics platforms, including Splunk, Datadog, Grafana Loki, New Relic, Dynatrace, SumoLogic, and AWS Elastic Cloud. You can also use a generic HTTP connection to stream logs to other tools.
  • Set Log Streaming Notifications
    Create and update notifications for proactive monitoring of B2C Commerce log streaming.
  • Security Logs Streaming
    B2C Commerce security logs contain sensitive information such as login info, and client download info.