Repository navigation
Spine should be able to process a value regex on output or input-output. #210
Description
Implementation evidence — 2026-10-11
PRs #655 and #656 expands Spine-side tests to units-bearing real SNMP values across full/credential-change/final batches and hexadecimal normalization. Cacti develop schema cad6762d44cf71cd9980d03ff1c266ef90e6108e lacks output_regex; schema/UI/cache population and cross-poller input-output compatibility remain external integration work. This issue stays open.
Implementation and validation: #656
Checked-in measurements and acceptance documentation: #658
Benchmarks and remaining acceptance boundaries are checked in with the PR. Reported measurements do not demonstrate a speedup.
Current relevance audit
The Spine half is implemented: it reads poller_item.output_regex when the column exists and applies regex processing to SNMP output. Production-linked regression coverage exercises the output path, with further coverage in the pending series.
The feature remains blocked on Cacti. A fresh audit of Cacti/cacti develop found no output_regex column in cacti.sql, and repository code search found no output_regex implementation. Column detection therefore does not enable this feature on that schema. Do not close this issue solely because Spine contains the matching code.
Remaining completion criteria
- Implement and migrate the Cacti schema/configuration path that populates the regex consumed by Spine.
- Define compatible regex semantics across the Cacti and Spine paths.
- Validate input-output queries end to end, including a single item and both full SNMP batches and remainder batches.
- Demonstrate the original units-bearing value case against the supported paired Cacti/Spine versions.
Keep this issue open for the cross-repository dependency. Historical reports and discussion follow.
From a verbose query:
Click to show Data Query output for field 'volTotalSize2'
Total: 13.690000, Delta: 0.000000, Located input field 'volTotalSize2' [walk]
Total: 18.810000, Delta: 5.120000, Executing SNMP walk for data @ '.1.3.6.1.4.1.24681.2.2.19.1.3'
Found item [volTotalSize2='50.7 (50.7 TB)'] index: 1 [from regexp value parse]
Click to show Data Query output for field 'volUsedSize2'
Total: 18.810000, Delta: 0.000000, Located input field 'volUsedSize2' [walk]
Total: 21.320000, Delta: 2.510000, Executing SNMP walk for data @ '.1.3.6.1.4.1.24681.2.2.19.1.4'
Found item [volUsedSize2='23.0 (23.0 TB)'] index: 1 [from regexp value parse]
The XML used:
<interface>
<name>QNAP Volumes via SNMP</name>
<description>Queries a QNAP NAS for a list of monitorable volume</description>
<oid_index>.1.3.6.1.4.1.24681.2.2.19.1.1</oid_index>
<oid_num_indexes>.1.3.6.1.4.1.24681.2.2.18</oid_num_indexes>
<index_order>volDescr:volStatus:volIndex</index_order>
<index_order_type>numeric</index_order_type>
<index_title_format>|chosen_order_field|</index_title_format>
<fields>
<volIndex>
<name>Index</name>
<method>walk</method>
<source>value</source>
<direction>input</direction>
<oid>.1.3.6.1.4.1.24681.2.2.19.1.1</oid>
</volIndex>
<volDescr>
<name>Description</name>
<method>walk</method>
<source>value</source>
<direction>input</direction>
<oid>.1.3.6.1.4.1.24681.2.2.19.1.2</oid>
</volDescr>
<volTotalSize2>
<name>Total Size</name>
<method>walk</method>
<source>VALUE/REGEXP:([0-9]*(\.[0-9]*)?) TB$</source>
<direction>input-output</direction>
<oid>.1.3.6.1.4.1.24681.2.2.19.1.3</oid>
</volTotalSize2>
<volUsedSize2>
<name>Used Size</name>
<method>walk</method>
<source>VALUE/REGEXP:([0-9]*(\.[0-9]*)?) TB$</source>
<direction>input-output</direction>
<oid>.1.3.6.1.4.1.24681.2.2.19.1.4</oid>
</volUsedSize2>
<volStatus>
<name>Status</name>
<method>walk</method>
<source>value</source>
<direction>input</direction>
<oid>.1.3.6.1.4.1.24681.2.2.19.1.5</oid>
</volStatus>
</fields>
</interface>
The results received from cacti.log using spine:
2021-06-23 18:12:03 - SPINE: Poller[Main Poller] PID[15243] Device[SAN6 - QNAP ((IP removed))] HT[1] DS[SAN6 - QNAP ((IP removed)) - Space - Lun0San6_0] Graphs[SAN6 - QNAP ((IP removed)) - Space - Lun0San6_0 ] TT[2326.70] SNMP: v2: (IP removed), dsname: volTotalSize2, oid: .1.3.6.1.4.1.24681.2.2.19.1.3.1, value: 50.7
2021-06-23 18:12:01 - SPINE: Poller[Main Poller] PID[15243] Device[SAN6 - QNAP ((IP removed))] HT[1] DS[SAN6 - QNAP ((IP removed)) - Space - Lun0San6_0] Graphs[SAN6 - QNAP ((IP removed)) - Space - Lun0San6_0 ] TT[1188.25] SNMP: v2: (IP removed), dsname: volUsedSize2, oid: .1.3.6.1.4.1.24681.2.2.19.1.4.1, value: U
2021-06-23 18:12:01 - SPINE: Poller[Main Poller] PID[15243] WARNING: Invalid Response, Device[SAN6 - QNAP ((IP removed))] HT[1] DS[SAN6 - QNAP ((IP removed)) - Space - Lun0San6_0] Graphs[SAN6 - QNAP ((IP removed)) - Space - Lun0San6_0 ] SNMP: v2: (IP removed), dsname: volUsedSize2, oid: .1.3.6.1.4.1.24681.2.2.19.1.4.1, value: 23.0 TB
The results received from cacti.log using cmd.php:
2021-06-23 18:17:03 - POLLER: Poller[Main Poller] PID[17114] Device[SAN6 - QNAP ((IP removed))] DS[SAN6 - QNAP ((IP removed)) - Space - Lun0San6_0] Graphs[SAN6 - QNAP ((IP removed)) - Space - Lun0San6_0 ] TT[1237.91] SNMP: v2: (IP removed), dsname: volUsedSize2, oid: .1.3.6.1.4.1.24681.2.2.19.1.4.1, output: 23.0
2021-06-23 18:17:01 - POLLER: Poller[Main Poller] PID[17114] Device[SAN6 - QNAP ((IP removed))] DS[SAN6 - QNAP ((IP removed)) - Space - Lun0San6_0] Graphs[SAN6 - QNAP ((IP removed)) - Space - Lun0San6_0 ] TT[1302.64] SNMP: v2: (IP removed), dsname: volTotalSize2, oid: .1.3.6.1.4.1.24681.2.2.19.1.3.1, output: 50.7
Cacti and spine version: 1.2.17
Activity
TheWitness commentedon Jun 25, 2021
Increase your timeouts, I'm sure the QNAP is just not keeping up. This is a common error. Also, watch your MAX OID's at the Device level. Some SNMP Agents don't handle settings over 1 well.
godsyn commentedon Jun 28, 2021
Upped timeouts to 100000. 1 OID was already set.
Same results.
Using cmd.php, no issue.
Using spine:
2021-06-28 13:27:01 - SPINE: Poller[Main Poller] PID[2645189] WARNING: Invalid Response, Device[SAN6 - QNAP ((IP removed))] HT[1] DS[SAN6 - QNAP ((IP removed)) - Space - Lun0San6_0] Graphs[SAN6 - QNAP ((IP removed)) - Space - Lun0San6_0 ] SNMP: v2: (IP removed), dsname: volUsedSize2, oid: .1.3.6.1.4.1.24681.2.2.19.1.4.1, value: 29.3 TBnetniV commentedon Jun 28, 2021
What do you see if you run this in debug mode manually?
<path_to_spine>/spine -f <host_id> -l <host_id> -C <path to spine config>/spine.conf -R -V 5godsyn commentedon Jul 6, 2021
godsyn commentedon Jul 8, 2021
Issue persists in 1.2.18 (cacti and spine)
netniV commentedon Jul 9, 2021
Yes, we are working towards a patch this weekend that will resolve the issues.
TheWitness commentedon Sep 5, 2021
The issue is that the snmpagent is returning a suffix of "TB", which means the response is likely a string. Can you run the folloiwng command:
Post the output.
TheWitness commentedon Sep 5, 2021
Spine itself does not process the regex to remove the numeric suffix. There has been a feature request for this in the very past, but no one delivered a pull request to adopt it. So, you should move your Data Query to a Script or Script server based one.
[-]Spine not returning results when polling. Failes to honor 2nd (identical) regex. [/-][+]Spine should be able to process a value regex on output or input-output.[/+]netniV commentedon Sep 6, 2021
@TheWitness if you can remind me which regex feature/code block this relates to in the core, I can make an attempt. I'll do it on it's own branch for testing, as I don't want to mess too much with the stability for 1.2.19 since we have now got things settled down again.
TheWitness commentedon Sep 6, 2021
The real issue is this in the XML file:
<source>VALUE/REGEXP:([0-9]*(\.[0-9]*)?) TB$</source>. It implies that the output from thesnmpgetshould be regexed. This is fine, and the pcre2 regex functions make it easy. However, the problem is that there is no column in the poller_item table currently used to hold a regex. There are two columns that could be repurposed for that. Those are arg2 and arg3 (we never ever used them) in all my 19 years of Cacti.What I would support, from a design perspective, would be the following:
The issue with not doing it in the poller_item table is that it will otherwise require a bunch of file open/close activities, and some caching algo for all the potentially millions of data sources that Cacti will need to process. Not pretty.
oijkn commentedon Feb 8, 2022
My case may be similar, spine returns a
value: Uusing theGet SNMP Data (Indexed)method of Cacti.On the other hand, with the
snmpwalkcommand, the value returned is the expected one.This problem is present on my two Cacti environments.
The first one uses a multi-poller architecture (FreeBSD 12/Cacti v1.2.16/Spine v1.2.15).
The second one uses the normal mode (FreeBSD 12/Cacti v1.2.19/Spine v1.2.17).
The XML used:
Spine v1.2.15 cli result:
Spine v1.2.17 cli result:
Snmpwalk cli result:
10 remaining items
feat: detect output_regex column at runtime and fix sql.c indentation
fix(build): resolve compilation failures and warnings on develop (Cac…
fix(build): resolve compilation failures and warnings on develop (Cac…
fix(build): resolve compilation failures on develop (#514)
fix(build): resolve compilation failures and warnings on develop (Cac…
fix(build): resolve compilation failures and warnings on develop (Cac…
fix(build): resolve compilation failures and warnings on develop (Cac…
fix(poller): apply output_regex to the trailing multi-get batch
somethingwithproof commentedon Aug 31, 2026
Spine has read
poller_item.output_regexsince #514, but it applied the regex in only one of the two places OIDs are drained: inside the item loop at each MAX_OIDS boundary, and not in the pass after the loop that handles the remainder. A device whose OID count missed a boundary stored the raw value, and a single data source is always the leftover case.Fixed in #542, with an integration case that asserts on the stored value rather than on it merely being non-empty.
This is the spine half only. No Cacti branch defines
poller_item.output_regex, so the column detection never fires against a current install and the feature stays dormant until Cacti adds it.fix(poller): apply output_regex to the trailing multi-get batch
fix(poller): apply output_regex to the trailing multi-get batch
fix: poller correctness bugs, with fuzz and integration coverage (#542)
somethingwithproof commentedon Aug 31, 2026
Worth flagging before anyone closes this: the spine half is implemented, the Cacti half is not.
output_regexis on develop inspine.c,poller.candspine.h, landed in #514 and extended by #542, with unit coverage intests/unit/test_build_fixes.c. But spine gates the whole feature on the column existing:output_regexdoes not appear anywhere in Cacti/cacti develop, in the schema or in PHP. Sodb_column_exists()is false on every install today and the code never runs.This should stay open until Cacti core adds the column. The comment above also claims Cacti 1.3.1, which is a release that does not exist yet, so the version reference is worth correcting when the core side lands.