diff --git a/README.md b/README.md index dc1ccc2..eb424d8 100644 --- a/README.md +++ b/README.md @@ -26,9 +26,11 @@ and writes it virtually anywhere. * [Google Analytics](https://google.github.io/garf/fetchers/google-analytics-api/) * [Google Merchant Center](https://google.github.io/garf/fetchers/merchant-center-api/) * [Bid Manager](https://google.github.io/garf/fetchers/bid-manager/) +* [Campaign Manager](https://google.github.io/garf/fetchers/campaign-manager/) * [Google Ads](https://google.github.io/garf/fetchers/google-ads/) * [Search Ads 360](https://google.github.io/garf/fetchers/google-ads/#search-ads-360) -* [REST](fetchers/rest.md) +* [REST](https://google.github.io/garf/fetchers/rest.md) +* [Prometheus HTTP API](https://google.github.io/garf/fetchers/prometheus.md) ## Installation diff --git a/docs/fetchers/overview.md b/docs/fetchers/overview.md index 4a55a00..b9314ea 100644 --- a/docs/fetchers/overview.md +++ b/docs/fetchers/overview.md @@ -14,3 +14,4 @@ | [`youtube-analytics`](youtube.md#youtube-analytics-api) | `YouTubeAnalyticsApiReportFetcher` | `garf-youtube` | | | [`youtube-data-api`](youtube.md#youtube-data-api) | `YouTubeDataApiReportFetcher` | `garf-youtube` | `id`, `forHandle`, `forUsername`, `regionCode`, `chart`, `videoId` | | [`media-tagging`](media-tagging.md) | `MediaTaggingApiReportFetcher` | `garf-media-tagging` | `db-uri`, `endpoint` | +| [`prometheus`](prometheus.md) | `PrometheusApiReportFetcher` | `garf-prometheus` | `endpoint` | diff --git a/docs/fetchers/prometheus.md b/docs/fetchers/prometheus.md new file mode 100644 index 0000000..490abc6 --- /dev/null +++ b/docs/fetchers/prometheus.md @@ -0,0 +1,82 @@ +## garf for Prometheus HTTP API + +[![PyPI](https://img.shields.io/pypi/v/garf-prometheus?logo=pypi&logoColor=white&style=flat-square)](https://pypi.org/project/garf-prometheus) +[![Downloads PyPI](https://img.shields.io/pypi/dw/garf-prometheus?logo=pypi)](https://pypi.org/project/garf-prometheus/) + +`garf-prometheus` simplifies fetching data from [Prometheus HTTP API](https://prometheus.io/docs/prometheus/latest/querying/api/) using SQL-like queries. + +## Install + +Install `garf-prometheus` library + +/// tab | pip +``` +pip install garf-executors garf-prometheus +``` +/// + +/// tab | uv +``` +uv pip install garf-executors garf-prometheus +``` +/// + +## Usage + +### Prerequisites + +* Running Prometheus instance + +/// tab | cli +```bash +echo """ +SELECT + timestamp, + job, + instance, + value +FROM query +WHERE + query = up +" > query.sql +garf query.sql --source prometheus \ + --output csv --prometheus.endpoint=http://localhost:9090 +``` +/// + +/// tab | python + +```python +import os + +from garf.io import writer +from garf.community.prometheus import PrometheusApiReportFetcher + +query = """ +SELECT + timestamp, + job, + instance, + value +FROM query +WHERE + query = up +""" + +fetched_report = ( + PrometheusApiReportFetcher( + endpoint='http://localhost:9090' + ) + .fetch(query) +) + +csv_writer = writer.create_writer('csv') +csv_writer.write(fetched_report, 'query') +``` +/// + +### Available source parameters + +| name | values| comments | +|----- | ----- | -------- | +| `endpoint` | Base URL when Prometheus is running (`http://localhost:9090` by default) | diff --git a/libs/community/prometheus/README.md b/libs/community/prometheus/README.md new file mode 100644 index 0000000..a74ece0 --- /dev/null +++ b/libs/community/prometheus/README.md @@ -0,0 +1,70 @@ +# `garf` for Prometheus HTTP API + +[![PyPI](https://img.shields.io/pypi/v/garf-prometheus?logo=pypi&logoColor=white&style=flat-square)](https://pypi.org/project/garf-prometheus) +[![Downloads PyPI](https://img.shields.io/pypi/dw/garf-prometheus?logo=pypi)](https://pypi.org/project/garf-prometheus/) + +`garf-prometheus` simplifies fetching data from Prometheus HTTP API using SQL-like queries. + +## Prerequisites + +* Running Prometheus instance + +## Installation + +`pip install garf-prometheus` + +## Usage + +### Run as a library +``` +import os + +from garf.io import writer +from garf.community.prometheus import PrometheusApiReportFetcher + +query = """ +SELECT + timestamp, + instance, + job, + value +FROM query +WHERE query = up +""" + +fetched_report = ( + PrometheusApiReportFetcher( + endpoint='http://localhost:9090' + ) + .fetch(query) +) + +console_writer = writer.create_writer('console') +console_writer.write(fetched_report, 'query') +``` + +### Run via CLI + +> Install `garf-executors` package to run queries via CLI (`pip install garf-executors`). + +``` +garf --source prometheus \ + --output \ + --prometheus.endpoint=http://localhost:9090 +``` + +where: + +* `PATH_TO_QUERIES` - local or remove files containing queries +* `output` - output supported by [`garf-io` library](https://google.github.io/garf/usage/writers/). +* `SOURCE_PARAMETER=VALUE` - key-value pairs to refine fetching, check [available source parameters](#available-source-parameters). + +### Available source parameters + +| name | values| comments | +|----- | ----- | -------- | +| `endpoint` | Base URL when Prometheus is running (`http://localhost:9090` by default) | + +## Documentation + +You can find a documentation on `garf-prometheus` [here](https://google.github.io/garf/fetchers/prometheus/). diff --git a/libs/community/prometheus/examples/scape.sql b/libs/community/prometheus/examples/scape.sql new file mode 100644 index 0000000..50867a7 --- /dev/null +++ b/libs/community/prometheus/examples/scape.sql @@ -0,0 +1,4 @@ +SELECT + timestamp, + sum by(job) (rate(scrape_duration_seconds[1h])) AS scrape_rate +FROM query diff --git a/libs/community/prometheus/examples/up.sql b/libs/community/prometheus/examples/up.sql new file mode 100644 index 0000000..1fb13e4 --- /dev/null +++ b/libs/community/prometheus/examples/up.sql @@ -0,0 +1,7 @@ +SELECT + timestamp, + instance, + job, + value +FROM query +WHERE query = up diff --git a/libs/community/prometheus/examples/up_range.sql b/libs/community/prometheus/examples/up_range.sql new file mode 100644 index 0000000..817b885 --- /dev/null +++ b/libs/community/prometheus/examples/up_range.sql @@ -0,0 +1,11 @@ +SELECT + timestamp, + instance, + job, + value +FROM query_range +WHERE + query = up + AND start=2026-06-03T00:00:30.781Z + AND end=2026-06-03T20:11:00.781Z + AND step = 15s diff --git a/libs/community/prometheus/garf/community/prometheus/__init__.py b/libs/community/prometheus/garf/community/prometheus/__init__.py new file mode 100644 index 0000000..094416c --- /dev/null +++ b/libs/community/prometheus/garf/community/prometheus/__init__.py @@ -0,0 +1,17 @@ +# Copyright 2026 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Library for interacting with Prometheus Query API via garf.""" + +__version__ = '0.0.1' diff --git a/libs/community/prometheus/garf/community/prometheus/api_clients.py b/libs/community/prometheus/garf/community/prometheus/api_clients.py new file mode 100644 index 0000000..5bd5a97 --- /dev/null +++ b/libs/community/prometheus/garf/community/prometheus/api_clients.py @@ -0,0 +1,66 @@ +# Copyright 2026 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Handles to Prometheus HTTP API querying.""" + +import requests +from garf.community.prometheus import exceptions, query_editor +from garf.core import api_clients + + +class PrometheusApiClientError(exceptions.PrometheusApiError): + """Prometheus HTTP API specific error.""" + + +class PrometheusApiClient(api_clients.RestApiClient): + """Specifies client for interacting with Prometheus HTTP API.""" + + def get_response( + self, + request: query_editor.PrometheusApiQuery, + **kwargs: str, + ) -> api_clients.GarfApiResponse: + url = f'{self.endpoint}/api/v1/{request.resource_name}' + params = {} + for field in request.fields: + if field == 'timestamp': + continue + params['query'] = field + for element in request.filters: + key, value = element.split(' = ') + params[key.strip()] = value.strip().replace(' ', '') + headers = {k: v for k, v in kwargs.items() if not isinstance(v, bool)} + response = requests.get(url, params=params, headers=headers) + if response.status_code == self.OK: + results = response.json() + if request.resource_name == 'query_range': + results = results.get('data').get('result') + final_results = [] + for r in results: + labels = r.get('metric') + for row in r.get('values'): + values = dict(zip(['timestamp', 'value'], row)) + final_results.append({**values, **labels}) + + if request.resource_name == 'query': + results = results.get('data').get('result') + final_results = [] + for row in results: + values = dict(zip(['timestamp', 'value'], row.get('value'))) + labels = row.get('metric') + final_results.append({**values, **labels}) + return api_clients.GarfApiResponse(results=final_results) + raise PrometheusApiClientError( + 'Failed to get data from Prometheus HTTP API, reason: ', response.text + ) diff --git a/libs/community/prometheus/garf/community/prometheus/exceptions.py b/libs/community/prometheus/garf/community/prometheus/exceptions.py new file mode 100644 index 0000000..7498e13 --- /dev/null +++ b/libs/community/prometheus/garf/community/prometheus/exceptions.py @@ -0,0 +1,19 @@ +# Copyright 2026 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from garf.core import exceptions + + +class PrometheusApiError(exceptions.GarfError): + """Prometheus Query API error.""" diff --git a/libs/community/prometheus/garf/community/prometheus/query_editor.py b/libs/community/prometheus/garf/community/prometheus/query_editor.py new file mode 100644 index 0000000..e3eec56 --- /dev/null +++ b/libs/community/prometheus/garf/community/prometheus/query_editor.py @@ -0,0 +1,19 @@ +# Copyright 2026 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from garf.core import query_editor + + +class PrometheusApiQuery(query_editor.QuerySpecification): + """Query to Prometheus API.""" diff --git a/libs/community/prometheus/garf/community/prometheus/report_fetcher.py b/libs/community/prometheus/garf/community/prometheus/report_fetcher.py new file mode 100644 index 0000000..6be31e2 --- /dev/null +++ b/libs/community/prometheus/garf/community/prometheus/report_fetcher.py @@ -0,0 +1,45 @@ +# Copyright 2026 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import garf.core +from garf.community.prometheus import api_clients, query_editor + + +class PrometheusApiReportFetcher(garf.core.ApiReportFetcher): + """Defines report fetcher for Prometheus API.""" + + alias = 'prometheus' + + def __init__( + self, + api_client: api_clients.PrometheusApiClient | None = None, + parser: garf.core.parsers.DictParser = ( + garf.core.parsers.NumericConverterDictParser + ), + query_spec: query_editor.PrometheusApiQuery = ( + query_editor.PrometheusApiQuery + ), + parallel_threshold: int = 10, + **kwargs: str, + ) -> None: + """Initializes PrometheusApiReportFetcher.""" + if not api_client: + api_client = api_clients.PrometheusApiClient(**kwargs) + self.parallel_threshold = parallel_threshold + super().__init__( + api_client=api_client, + parser=parser, + query_specification_builder=query_spec, + **kwargs, + ) diff --git a/libs/community/prometheus/garf/community/prometheus/telemetry.py b/libs/community/prometheus/garf/community/prometheus/telemetry.py new file mode 100644 index 0000000..5c23cc4 --- /dev/null +++ b/libs/community/prometheus/garf/community/prometheus/telemetry.py @@ -0,0 +1,17 @@ +# Copyright 2026 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from opentelemetry import trace + +tracer = trace.get_tracer(instrumenting_module_name='garf.community.prometheus') diff --git a/libs/community/prometheus/pyproject.toml b/libs/community/prometheus/pyproject.toml new file mode 100644 index 0000000..01b1974 --- /dev/null +++ b/libs/community/prometheus/pyproject.toml @@ -0,0 +1,41 @@ +[build-system] +requires = ["setuptools >= 61.0"] +build-backend = "setuptools.build_meta" + +[project] +name = "garf-prometheus" +dependencies = [ + "garf-core[pandas]>=1.1.2", + "garf-io>=1.0.0", +] +authors = [ + {name = "Andrei Markin", email = "amarkin@google.com"}, + {name = "Google Inc. (gTech gPS CSE team)", email = "no-reply@google.com"}, +] +license = {text = "Apache 2.0"} +requires-python = ">=3.9" +description = "Garf implementation for Prometheus API" +readme = "README.md" +classifiers = [ + "Development Status :: 4 - Beta", + "Programming Language :: Python" +] + +dynamic=["version"] + +[tool.setuptools.dynamic] +version = {attr = "garf.community.prometheus.__version__"} + +[project.entry-points.garf] +prometheus = "garf.community.prometheus.report_fetcher" + +[project.optional-dependencies] +test = [ + "pytest", + "pytest-cov", + "pytest-mock", + "python-dotenv", +] +[tool.setuptools.packages.find] +where = ["."] +include=["garf.*"] diff --git a/mkdocs.yml b/mkdocs.yml index aa95921..2578ef8 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -104,3 +104,4 @@ nav: - Campaign Manager 360: fetchers/campaign-manager.md - Google Ads: fetchers/google-ads.md - Media Tagging: fetchers/media-tagging.md + - Prometheus: fetchers/prometheus.md