Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 76 additions & 3 deletions reference/observability/observability-aws-metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@
mapped_pages:
- https://www.elastic.co/guide/en/serverless/current/observability-aws-metrics.html
- https://www.elastic.co/guide/en/observability/current/aws-metrics.html
applies_to:
stack: ga
serverless: ga
products:
- id: cloud-serverless
- id: observability
Expand All @@ -14,12 +17,28 @@

::::



## Monitor EC2 instances [monitor-ec2-instances]

To analyze EC2 instance metrics, you can select view filters based on the following predefined metrics, or you can add [custom metrics](/solutions/observability/infra-and-hosts/view-infrastructure-metrics-by-resource-type.md#custom-metrics).

:::{note}
:applies_to: stack: ga 9.3
For EC2 instances, The [Infrastructure UI](/solutions/observability/infra-and-hosts/analyze-infrastructure-host-metrics.md) and [inventory rules](/solutions/observability/incident-management/create-an-inventory-rule.md) only support metric collected by the [EC2 integration](integration-docs://reference/aws/ec2.md).
:::

### Entity definition [monitor-ec2-entity]
```{applies_to}
stack: ga 9.3
```

| | | |
| --- | --- | --- |
| **Filter** | `event.module : aws` | Used to filter relevant data. |
| **Identifier** | `cloud.instance.id` | Used to identify each entity. |
| **Display value** | `cloud.instance.name` | Used as a display friendly value. |

### Metrics [monitor-ec2-metrics]

| | |
| --- | --- |
| **CPU Usage** | Average of `aws.ec2.cpu.total.pct`. |
Expand All @@ -33,6 +52,24 @@

To analyze S3 bucket metrics, you can select view filters based on the following predefined metrics, or you can add [custom metrics](/solutions/observability/infra-and-hosts/view-infrastructure-metrics-by-resource-type.md#custom-metrics).

:::{note}
:applies_to: stack: ga 9.3
For S3 buckets, the [Infrastructure UI](/solutions/observability/infra-and-hosts/analyze-infrastructure-host-metrics.md) and [inventory rules](/solutions/observability/incident-management/create-an-inventory-rule.md) only support metric data collected by the [S3 integration](integration-docs://reference/aws/s3.md).
:::

### Entity definition [monitor-s3-entity]
```{applies_to}
stack: ga 9.3
```

| | |
| --- | --- |
| **Filter** | `event.module : aws` | Used to filter relevant data. |
| **Identifier** | `aws.s3.bucket.name` | Used to identify each entity. |
| **Display value** | `aws.s3.bucket.name` | Used as a display friendly value. |

### Metrics [monitor-s3-metrics]

| | |
| --- | --- |
| **Bucket Size** | Average of `aws.s3_daily_storage.bucket.size.bytes`. |
Expand All @@ -46,6 +83,24 @@

To analyze SQS queue metrics, you can select view filters based on the following predefined metrics, or you can add [custom metrics](/solutions/observability/infra-and-hosts/view-infrastructure-metrics-by-resource-type.md#custom-metrics).

:::{note}
:applies_to: stack: ga 9.3
For SQS queues, the [Infrastructure UI](/solutions/observability/infra-and-hosts/analyze-infrastructure-host-metrics.md) and [inventory rules](/solutions/observability/incident-management/create-an-inventory-rule.md) only support metric data collected by the [SQS integration](integration-docs://reference/aws/sqs.md).

Check notice on line 88 in reference/observability/observability-aws-metrics.md

View workflow job for this annotation

GitHub Actions / preview / vale

Elastic.Acronyms: 'SQS' has no definition.

Check notice on line 88 in reference/observability/observability-aws-metrics.md

View workflow job for this annotation

GitHub Actions / preview / vale

Elastic.Acronyms: 'SQS' has no definition.
:::

### Entity definition [monitor-sqs-entity]
```{applies_to}
stack: ga 9.3
```

| | |
| --- | --- |
| **Filter** | `event.module : aws` | Used to filter relevant data. |
| **Identifier** | `aws.sqs.queue.name` | Used to identify each entity. |
| **Display value** | `aws.sqs.queue.name` | Used as a display friendly value. |

### Metrics [monitor-sqs-metrics]

| | |
| --- | --- |
| **Messages Available** | Max of `aws.sqs.messages.visible`. |
Expand All @@ -59,6 +114,24 @@

To analyze RDS database metrics, you can select view filters based on the following predefined metrics, or you can add [custom metrics](/solutions/observability/infra-and-hosts/view-infrastructure-metrics-by-resource-type.md#custom-metrics).

:::{note}
:applies_to: stack: ga 9.3
For RDS databases, the [Infrastructure UI](/solutions/observability/infra-and-hosts/analyze-infrastructure-host-metrics.md) and [inventory rules](/solutions/observability/incident-management/create-an-inventory-rule.md) only support metric data collected by the [RDS](integration-docs://reference/aws/rds.md) integration.

Check notice on line 119 in reference/observability/observability-aws-metrics.md

View workflow job for this annotation

GitHub Actions / preview / vale

Elastic.Acronyms: 'RDS' has no definition.
:::

### Entity definition [monitor-rds-entity]
```{applies_to}
stack: ga 9.3
```

| | |
| --- | --- |
| **Filter** | `event.module : aws` | Used to filter relevant data. |
| **Identifier** | `aws.rds.db_instance.arn` | Used to identify each entity. |
| **Display value** | `aws.rds.db_instance.identifier` | Used as a display friendly value. |

### Metrics [monitor-rds-metrics]

| | |
| --- | --- |
| **CPU Usage** | Average of `aws.rds.cpu.total.pct`. |
Expand All @@ -67,4 +140,4 @@
| **Active Transactions** | Average of `aws.rds.transactions.active`. |
| **Latency** | Average of `aws.rds.latency.dml`. |

For information about the fields used by the Infrastructure UI to display AWS services metrics, see the [Infrastructure app fields](/reference/observability/fields-and-object-schemas.md).
For information about the fields used by the Infrastructure UI to display AWS services metrics, refer to the [Infrastructure app fields](/reference/observability/fields-and-object-schemas.md).
40 changes: 36 additions & 4 deletions reference/observability/observability-container-metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@
mapped_pages:
- https://www.elastic.co/guide/en/serverless/current/observability-container-metrics.html
- https://www.elastic.co/guide/en/observability/current/container-metrics.html
applies_to:
stack: ga
serverless: ga
products:
- id: cloud-serverless
- id: observability
Expand All @@ -14,11 +17,25 @@
* [Docker](#key-metrics-docker)
* [Kubernetes](#key-metrics-kubernetes)


## Docker container metrics [key-metrics-docker]

These are the key metrics displayed for Docker containers.

:::{note}
:applies_to: stack: ga 9.3
For Docker container metrics, the [Infrastructure UI](/solutions/observability/infra-and-hosts/analyze-infrastructure-host-metrics.md) and [inventory rules](/solutions/observability/incident-management/create-an-inventory-rule.md) only support metric data collected by the [Docker integration](integration-docs://reference/docker.md).
:::

### Entity definition [monitor-docker-container-entity]
```{applies_to}
stack: ga 9.3
```

| | | |
| --- | --- | --- |
| **Filter** | `event.module : "docker"` | Used to filter relevant data. |
| **Identifier** | `container.id` | Used to identify each entity. |
| **Display value** | `container.name` | Used as a display friendly value. |

### CPU usage metrics [key-metrics-docker-cpu]

Expand Down Expand Up @@ -50,10 +67,25 @@
| **Disk Write IOPS** | Average count of write operations from the device per second.<br><br>**Field Calculation:** `counter_rate(max(docker.diskio.write.ops), kql='docker.diskio.write.ops: *')`<br> |


## Kubernetes container metrics [key-metrics-kubernetes]
## {{k8s}} container metrics [key-metrics-kubernetes]

Check notice on line 70 in reference/observability/observability-container-metrics.md

View workflow job for this annotation

GitHub Actions / preview / vale

Elastic.Capitalization: 'container metrics' should use sentence-style capitalization.

These are the key metrics displayed for {{k8s}} (containerd) containers.

:::{note}
:applies_to: stack: ga 9.3
For {{k8s}} container metrics, the [Infrastructure UI](/solutions/observability/infra-and-hosts/analyze-infrastructure-host-metrics.md) and [inventory rules](/solutions/observability/incident-management/create-an-inventory-rule.md) only support metric data collected by the [{{k8s}} integration](integration-docs://reference/kubernetes.md).
:::

These are the key metrics displayed for Kubernetes (containerd) containers.
### Entity definition [monitor-k8s-container-entity]
```{applies_to}
stack: ga 9.3
```

| | | |
| --- | --- | --- |
| **Filter** | `event.module : "kubernetes"` | Used to filter relevant data. |
| **Identifier** | `container.id` | Used to identify each entity. |
| **Display value** | `container.name` | Used as a display friendly value. |

### CPU usage metrics [key-metrics-kubernetes-cpu]

Expand All @@ -66,4 +98,4 @@

| Metric | Description |
| --- | --- |
| **Memory Usage (%)** | Average memory usage for the container.<br><br>**Field Calculation:** `average(kubernetes.container.memory.usage.limit.pct)`<br> |
| **Memory Usage (%)** | Average memory usage for the container.<br><br>**Field Calculation:** `average(kubernetes.container.memory.usage.limit.pct)`<br> |
43 changes: 33 additions & 10 deletions reference/observability/observability-host-metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@
mapped_pages:
- https://www.elastic.co/guide/en/serverless/current/observability-host-metrics.html
- https://www.elastic.co/guide/en/observability/current/host-metrics.html
applies_to:
stack: ga
serverless: ga
products:
- id: cloud-serverless
- id: observability
Expand All @@ -27,7 +30,18 @@ Refer to the following sections for host metrics and field calculation formulas
* [Disk](#key-metrics-network)
* [Legacy](#legacy-metrics)

### Hosts metrics [key-metrics-hosts]
### Entity definition [monitor-rds-entity]
```{applies_to}
stack: ga 9.3
```

| | |
| --- | --- |
| **Filter** | `event.module: 'system'` or `metricset.module: 'system'` | Used to filter relevant data. |
| **Identifier** | `host.name` | Used to identify each entity. |
| **Display value** | `host.name` | Used as a display friendly value. |

### Hosts count [key-metrics-hosts]

| Metric | Description |
| --- | --- |
Expand Down Expand Up @@ -114,13 +128,24 @@ Refer to the following sections for host metrics and field calculation formulas
* [Network](#otel-metrics-network)
* [Disk](#otel-metrics-network)

### OpenTelemetry hosts metrics [otel-metrics-hosts]
### Entity definition [opentelemetry-host-entity]
```{applies_to}
stack: ga 9.3
```

| | |
| --- | --- |
| **Filter** | `data_stream.dataset: 'hostmetricsreceiver.otel'` | Used to filter relevant data. |
| **Identifier** | `host.name` | Used to identify each entity. |
| **Display value** | `host.name` | Used as a display friendly value. |

### Hosts count [otel-metrics-hosts]

| Metric | Description |
| --- | --- |
| **Hosts** | Number of hosts returned by your search criteria.<br><br>**Field Calculation**: `unique_count(host.name)`<br> |

### OpenTelemetry CPU usage metrics [otel-metrics-cpu]
### CPU usage metrics [otel-metrics-cpu]

| Metric | Description |
| --- | --- |
Expand All @@ -137,7 +162,7 @@ Refer to the following sections for host metrics and field calculation formulas
| **Load (15m)** | 15 minute load average.<br><br>Load average gives an indication of the number of threads that are runnable (either busy running on CPU, waiting to run, or waiting for a blocking IO operation to complete).<br><br>**Field Calculation**: `average(metrics.system.cpu.load_average.15m)`<br> |
| **Normalized Load** | 1 minute load average normalized by the number of CPU cores.<br><br>Load average gives an indication of the number of threads that are runnable (either busy running on CPU, waiting to run, or waiting for a blocking IO operation to complete).<br><br>100% means the 1 minute load average is equal to the number of CPU cores of the host.<br><br>Taking the example of a 32 CPU cores host, if the 1 minute load average is 32, the value reported here is 100%. If the 1 minute load average is 48, the value reported here is 150%.<br><br>**Field Calculation**: `average(metrics.system.cpu.load_average.1m) / max(metrics.system.cpu.logical.count)`<br> |

### OpenTelemetry memory metrics [otel-metrics-memory]
### Memory metrics [otel-metrics-memory]

| Metric | Description |
| --- | --- |
Expand All @@ -148,20 +173,20 @@ Refer to the following sections for host metrics and field calculation formulas
| **Memory Usage (%)** | Percentage of main memory usage excluding page cache.<br><br>This includes resident memory for all processes plus memory used by the kernel structures and code apart from the page cache.<br><br>A high level indicates a situation of memory saturation for the host. For example, 100% means the main memory is entirely filled with memory that can’t be reclaimed, except by swapping out.<br><br>**Field Calculation**: `average(system.memory.utilization, kql='state: used') + average(system.memory.utilization, kql='state: buffered') + average(system.memory.utilization, kql='state: slab_reclaimable') + average(system.memory.utilization, kql='state: slab_unreclaimable')`<br> |
| **Memory Used** | Main memory usage excluding page cache.<br><br>**Field Calculation**: `average(metrics.system.memory.usage, kql='state: used') + average(metrics.system.memory.usage, kql='state: buffered') + average(metrics.system.memory.usage, kql='state: slab_reclaimable') + average(metrics.system.memory.usage, kql='state: slab_unreclaimable')`<br> |

### OpenTelemetry log metrics [otel-metrics-log]
### Log metrics [otel-metrics-log]

| Metric | Description |
| --- | --- |
| **Log Rate** | Derivative of the cumulative sum of the document count scaled to a 1 second rate. This metric relies on the same indices as the logs.<br><br>**Field Calculation**: `cumulative_sum(doc_count)`<br> |

### OpenTelemetry network metrics [otel-metrics-network]
### Network metrics [otel-metrics-network]

| Metric | Description |
| --- | --- |
| **Network Inbound (RX)** | Number of bytes that have been received per second on the public interfaces of the hosts.<br><br>**Field Calculation**: `8 * counter_rate(max(metrics.system.network.io, kql='direction: receive')))`<br> |
| **Network Outbound (TX)** | Number of bytes that have been sent per second on the public interfaces of the hosts.<br><br>**Field Calculation**: `8 * counter_rate(max(metrics.system.network.io, kql='direction: transmit'))`<br> |

### OpenTelemetry disk metrics [otel-metrics-disk]
### Disk metrics [otel-metrics-disk]

| Metric | Description |
| --- | --- |
Expand All @@ -171,6 +196,4 @@ Refer to the following sections for host metrics and field calculation formulas
| **Disk Usage - Available (%)** | Percentage of disk space available.<br><br>**Field Calculation**: `average(system.filesystem.usage, kql='state: free')`<br> |
| **Disk Usage - Used (%)** | Percentage of disk space used. <br><br>**Field Calculation**: `1 - sum(metrics.system.filesystem.usage, kql='state: free') / sum(metrics.system.filesystem.usage)`<br> |
| **Disk Write IOPS** | Average count of write operations from the device per second.<br><br>**Field Calculation**: `counter_rate(max(system.disk.operations, kql='attributes.direction: write'))`<br> |
| **Disk Write Throughput** | Average number of bytes written from the device per second.<br><br>**Field Calculation**: `counter_rate(max(system.disk.io, kql='attributes.direction: write'))')`<br> |


| **Disk Write Throughput** | Average number of bytes written from the device per second.<br><br>**Field Calculation**: `counter_rate(max(system.disk.io, kql='attributes.direction: write'))')`<br> |
27 changes: 24 additions & 3 deletions reference/observability/observability-kubernetes-pod-metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,35 @@
mapped_pages:
- https://www.elastic.co/guide/en/serverless/current/observability-kubernetes-pod-metrics.html
- https://www.elastic.co/guide/en/observability/current/kubernetes-pod-metrics.html
applies_to:
stack: ga
serverless: ga
products:
- id: cloud-serverless
- id: observability
---

# Kubernetes pod metrics [observability-kubernetes-pod-metrics]
# {{k8s}} pod metrics [observability-kubernetes-pod-metrics]

Check notice on line 13 in reference/observability/observability-kubernetes-pod-metrics.md

View workflow job for this annotation

GitHub Actions / preview / vale

Elastic.Capitalization: 'pod metrics' should use sentence-style capitalization.

To analyze Kubernetes pod metrics, you can select view filters based on the following predefined metrics, or you can add [custom metrics](/solutions/observability/infra-and-hosts/view-infrastructure-metrics-by-resource-type.md#custom-metrics).
To analyze {{k8s}} pod metrics, you can select view filters based on the following predefined metrics, or you can add [custom metrics](/solutions/observability/infra-and-hosts/view-infrastructure-metrics-by-resource-type.md#custom-metrics).

:::{note}
:applies_to: stack: ga 9.3
For {{k8s}} pod metrics, the [Infrastructure UI](/solutions/observability/infra-and-hosts/analyze-infrastructure-host-metrics.md) and [inventory rules](/solutions/observability/incident-management/create-an-inventory-rule.md) only support metric data collected by the [{{k8s}} integration](integration-docs://reference/kubernetes.md).
:::

## Entity definition [monitor-k8s-pods-entity]
```{applies_to}
stack: ga 9.3
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should there be a serverless statement here too?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so as this page is supposed to apply universally to serverless. @bmorelli25 had suggested this earlier change in a previous review.

```

| | | |
| --- | --- | --- |
| **Filter** | `event.module: "kubernetes"` | Used to filter relevant data. |
| **Identifier** | `kubernetes.pod.uid` | Used to identify each entity. |
| **Display value** | `kubernetes.pod.name` | Used as a display friendly value. |

## Metrics [monitor-k8s-pods-metrics]

| | |
| --- | --- |
Expand All @@ -18,4 +39,4 @@
| **Inbound Traffic** | Derivative of the maximum of `kubernetes.pod.network.rx.bytes` scaled to a 1 second rate. |
| **Outbound Traffic** | Derivative of the maximum of `kubernetes.pod.network.tx.bytes` scaled to a 1 second rate. |

For information about the fields used by the Infrastructure UI to display Kubernetes pod metrics, see the [Infrastructure app fields](/reference/observability/fields-and-object-schemas.md).
For information about the fields used by the Infrastructure UI to display {{k8s}} pod metrics, see the [Infrastructure app fields](/reference/observability/fields-and-object-schemas.md).

Check notice on line 42 in reference/observability/observability-kubernetes-pod-metrics.md

View workflow job for this annotation

GitHub Actions / preview / vale

Elastic.WordChoice: Consider using 'refer to (if it's a document), view (if it's a UI element)' instead of 'see', unless the term is in the UI.
Loading
Loading