Collectord has default configuration embedded. Changing this configuration allows you to control how often data is forwarded to CloudWatch, which host logs should be forwarded, default sampling for the logs and more.
You can always review all the configuration that is applied to collectord by executing the command on one of the running
Get a list of the pods from the
kubectl get pods -n collectord-cloudwatch
The output will look like
NAME READY STATUS RESTARTS AGE collectord-cloudwatch-4n52x 1/1 Running 0 18s collectord-cloudwatch-addon-6b6bbdfdd-g8qhm 1/1 Running 0 18s
There are two deployments running. One is the daemonset that is deployed on every node and forwards host, container and application logs. Second one is a deployment that forwards Kubernetes events.
To get the configuration from the pod, run the following command (change the pod name from one of the list). The output from pod scheduled with DaemonSet will be different from the pod scheduled with Deployment.
kubectl exec -it -n collectord-cloudwatch collectord-cloudwatch-4n52x /collectord show-config
Overriding the configuration
With the installation instruction we provide a YAML template that has a ConfigMap, allowing you to override default configuration.
apiVersion: v1 kind: ConfigMap metadata: name: collectord-cloudwatch namespace: collectord-cloudwatch labels: app: collectord-cloudwatch data: 101-general.conf: | [general] # Review SLA at https://www.outcoldsolutions.com/docs/license-agreement/ and accept the license acceptLicense = false # Request the trial license with automated form https://www.outcoldsolutions.com/trial/request/ license = # If you are planning to setup log aggregation for multiple cluster, name the cluster fields.cluster = - [aws] # Specify AWS Region region = 102-daemonset.conf: | 103-addon.conf: |
Disable forwarding of host logs
To disable forwarding of the host logs, just set
true for the
102-daemonset.conf: | ... # Input all ^(([\w\-.]+\.log(.[\d\-]+)?)|(docker))$ files [input.files::logs] disabled = true # Input all ^(syslog|messages)(.\d+)?$ files [input.files::syslog] disabled = true # host logs from journald [input.journald] disabled = true
Use opt-out by default behavior for container logs
By default collectord forwards all container logs to CloudWatch. If you want to disable that, and be able to specify
with the annotations from which Pods you want to
forward logs you can change the configuration for
102-daemonset.conf: | ... [input.files] output = devnull
Sample all container logs by default
To sample all container logs by default, you can set the percent of the logs, that should be forwarded to CloudWatch
102-daemonset.conf: | ... [input.files] samplingPercent = 5
Change default retention setting for CloudWatch LogGroup
When collectord creates a new LogGroup it also updates the retention configuration. The default is 90 days.
101-general.conf: | ... [output.cloudwatch.logs] # Possible values are: 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, and 3653. retentionInDays = 90
Disabling forwarding of Kubernetes Events
Remove the Deployment
collectord-cloudwatch-addon from the installation configuration.
Collectord forwards very basic telemetry about the performance and enabled configurations. You can disable it
101-general.conf: | ... [general] # telemetry report endpoint, set it to empty string to disable telemetry telemetryEndpoint =
- Setup centralized Logging in 5 minutes.
- Automatically forward host, container and application logs.
- Test our solution with the 30 days evaluation license.
- Forwarding application logs.
- Multi-line container logs.
- Fields extraction for application and container logs (including timestamp extractions).
- Hiding sensitive data, stripping terminal escape codes and colors.
- Advanced configurations for collectord.
- Troubleshooting steps.
- Verify configuration.