ConstraintData silently drops unit/interconnector LHS terms for mixed FCAS+energy constraints

[seb9291]

When running counterfactual dispatches, I saw large price errors up to 80% in QLD of a couple of intervals in May 2024. After I took a look, the cause is that ConstraintData misclassifies generic constraints as FCAS requirements, causing their unit and interconnector LHS coefficients to be dropped.

The bug is in nempy/historical_inputs/constraints.py in the ConstraintData class.

F_Q++86_L6 and F_Q++86_L60 are mixed constraints. Their LHS contains three components:

Component	Entry	Coefficient
Regional FCAS	QLD1 lower_6s (L6SE)	+1.0
Regional FCAS	QLD1 lower_60s (L60S)	+1.0
Unit energy	NEWENSF1, NEWENSF2, SAPHWF1 (ENOF)	-1.0 each
Interconnector	N-Q-MNSP1	+1.0
Interconnector	NSW1-QLD1	+1.0

The full constraint is QLD1_lower_6s + NSW1 → QLD1_flow + N-Q-MNSP1_flow− NEWENSF1_energy − NEWENSF2_energy − SAPHWF1_energy ≥ RHS

Because these constraints have a region FCAS factor, ConstraintData.init includes them in self.fcas_requirements (via merge of region_generic_lhs with generic_rhs). They are then excluded from the output of get_rhs_and_type_excluding_regional_fcas_constraints(), which filters out anything whose set appears in fcas_requirements.

When the market is built:

1. set_fcas_requirements_constraints(fcas_requirements) — adds the constraint as a pure regional FCAS requirement only: QLD1_lower_6s ≥ RHS
2. set_generic_constraints(generic_rhs) — does not include this constraint (it was excluded)
3. link_units_to_generic_constraints(unit_lhs) — the unit coefficients for F_Q++86_L6 are present in unit_lhs, but since the constraint is not in generic_rhs, they are ignored
4. link_interconnectors_to_generic_constraints(interconnector_lhs) — same problem; the interconnector coefficients are ignored

The LP therefore solves: QLD1_lower_6s ≥ −383.77 instead of QLD1_lower_6s + NSW1→QLD1_flow + N-Q-MNSP1_flow − NEWENSF1_E − NEWENSF2_E − SAPHWF1_E ≥ −383.77
In the affected intervals, NSW1-QLD1 flowwas -292 to -379MW (NSW importing from QLD). The missing interconnector terms make QLD lower FCAS requirement appear more binding that it actually is, causing price errors.

A ghetto workaround in my own code found that I got some clean results:

Interval	Max price error before	Max price error after
2024/05/16 10:20	22.1%	0.0%
2024/05/16 15:50	44.0%	2.2%
2024/05/17 15:25	32.3%	4.1%
2024/05/17 16:00	79.7%	0.7%
2024/05/17 16:15	19.8%	0.6%

The ghetto workaround is that after loading constraints, detect any constraint that appears in both fcas_requirements and unit_generic_lhs/interconnector_generic_lhs. Remove it from fcas_requirements and add it to generic_rhs. All three LHS components (region, unit, interconnector) are already present in their respective DataFrames and are correctly wired up by the existing link_regions_to_generic_constraints, link_units_to_generic_constraints, and link_interconnectors_to_generic_constraints calls once the constraint is in generic_rhs.

unit_sets = set(data.unit_generic_lhs['set'].unique())
ic_sets   = set(data.interconnector_generic_lhs['set'].unique())
mixed     = set(data.fcas_requirements['set'].unique()) & (unit_sets | ic_sets)

if mixed:
    full_rhs = constraint_inputs.get_rhs_and_type()
    data.generic_rhs = pd.concat(
        [data.generic_rhs, full_rhs[full_rhs['set'].isin(mixed)]],
        ignore_index=True
    )
    data.fcas_requirements = data.fcas_requirements[
        ~data.fcas_requirements['set'].isin(mixed)
    ]

The proper fix in ConstraintData itself would be to exclude from fcas_requirements any constraint that also has entries in unit_generic_lhs or interconnector_generic_lhs, and instead route those through the generic constraint pathway with all LHS terms intact.

The pattern likely affects any interval where a contingency FCAS constraint carries mixed-type LHS terms, not specific to the F_Q++86 family or to Queensland.