Scenario Sweep
=======================================

Goal
--------------------------

This class varies **both** the injection (generators / static generators / loads) and a
contingency (a line / transformer disconnection) per simulation, independently,
row-aligned: row `i` of every injection input is solved together with row `i` of the
contingency mask.

It is the 4th instantiation of the same underlying C++ template as
:class:`lightsim2grid.timeSerie.TimeSerie` / :class:`lightsim2grid.injectionSweep.InjectionSweep`
(which vary only the injection) and :class:`lightsim2grid.contingencyAnalysis.ContingencyAnalysis`
(which varies only the topology, over one shared base injection) -- see :doc:`time_series`
and :doc:`security_analysis`.

It can be used as:

.. code-block:: python

    import numpy as np
    import grid2op
    from lightsim2grid import ScenarioSweep
    from lightsim2grid import LightSimBackend

    env_name = ...
    env = grid2op.make(env_name, backend=LightSimBackend())
    obs = env.reset()

    n_simul = 10
    load_p = np.tile(obs.load_p, (n_simul, 1))
    load_p *= 1 + 0.01 * np.arange(n_simul).reshape(-1, 1)  # vary the load a bit per row

    # lightsim2grid's own line / trafo counts -- NOT grid2op's flat `env.n_line`
    # (grid2op merges lines and trafos into one "powerline" list; lightsim2grid keeps
    # them as two separate containers, and the contingency masks below follow that split)
    n_line = len(env.backend._grid.get_lines())
    n_trafo = len(env.backend._grid.get_trafos())
    line_mask = np.zeros((n_simul, n_line), dtype=bool)
    line_mask[5:, 3] = True  # disconnect line 3 for the last 5 simulations

    sweep = ScenarioSweep(env)
    sweep.modify_load_p(load_p)
    sweep.set_contingency_lines(line_mask)
    # modify_gen_p / modify_sgen_p / modify_load_q / set_contingency_trafos were never
    # called: those axes default to the grid's own current state, broadcast across
    # every row (see "Unset axes" below).
    sweep.compute()

    res_v = sweep.get_voltages()
    res_a = sweep.compute_A()
    res_p = sweep.compute_P()

    # res_p[row_id] / res_a[row_id] / res_v[row_id] are the flows / voltages for
    # simulation `row_id`, combining that row's injections and that row's contingency.

The setter-based API
----------------------------

Unlike `TimeSerie` / `InjectionSweep`'s single bundled `compute_V_from_inj` call,
`ScenarioSweep` is built up incrementally: call as many of `modify_gen_p` /
`modify_sgen_p` / `modify_load_p` / `modify_load_q` / `modify_gen_v` /
`set_contingency_lines` / `set_contingency_trafos` / `set_contingency_gens` as are
relevant, then `compute()`.

- **Row-count locking.** The first setter you call fixes the number of simulations for
  the whole object; every later setter is checked against it immediately -- a shape
  mistake raises right at the call that made it, not later inside `compute()`.
- **Unset axes.** Any injection axis you never call (eg `modify_gen_p`) defaults to the
  grid's own current target value, broadcast across every row -- not zero. Any
  contingency mask you never call (`set_contingency_lines` / `set_contingency_trafos` /
  `set_contingency_gens`) defaults to all-`False` -- nothing disconnected. `compute()` raises if you never call
  *any* setter (a degenerate call -- use `ac_pf()` / `dc_pf()` directly for a single
  powerflow).
- **`modify_gen_v` is different from the other four `modify_*` setters.** It does not
  feed the injection (`Sbus`) at all -- a PV bus's voltage magnitude is never part of
  what Newton-Raphson solves for, only of what it starts from. `modify_gen_v` instead
  re-seeds `|V|` at each voltage-regulating generator's regulated bus immediately before
  that row's solve, shape `(n_simul, n_gen)`, in pu (`vm_pu`), NOT kV. Left unset, every
  row keeps using the grid's own `target_vm_pu`, exactly as if it had never been called.
- `TimeSerie` / `InjectionSweep` also gained this same setter API (`modify_*` +
  `compute()`) alongside their legacy `compute_V_from_inj` call, which remains
  available (deprecated, not removed).

Two contingency APIs, on purpose
------------------------------------

`ScenarioSweep.set_contingency_lines` / `set_contingency_trafos` and
`ContingencyAnalysis.add_n1` / `add_nk` are **deliberately two different APIs**, not a
naming inconsistency:

- `ContingencyAnalysis` applies an arbitrary *set* of distinct contingencies to one
  shared base injection -- `add_n1(3)` registers "disconnect line 3", independent of
  any row ordering or count.
- `ScenarioSweep` pairs each contingency with that same row's own injection, so a
  dense, row-aligned boolean mask (shape `(n_simul, n_line)` / `(n_simul, n_trafo)`,
  `True` = "deactivate this branch for this simulation") is the natural fit instead.

`ScenarioSweep` does not have `add_n1` / `add_nk`; `ContingencyAnalysis` does not have
`set_contingency_lines` / `set_contingency_trafos`.

.. note::

    Dense contingency masks cost real memory at scale (eg 10k simulations x 3k branches
    ~= 30 MB) -- fine for typical use, a lighter-weight representation may be added in a
    future release if that becomes a bottleneck.

Generator contingencies
------------------------------------

`set_contingency_gens(mask)` takes a `(n_simul, n_gen)` boolean mask, `True` meaning
"disconnect this generator for this simulation". Unlike the two branch masks it does
not edit the admittance matrix -- a generator carries no admittance. It does three
things instead:

- takes the generator's active power out of that row's injection (and its reactive
  setpoint too, if it is not a voltage-regulating machine);
- re-weights the distributed slack without it, renormalised, so a row that drops a
  participating machine redistributes its share over the others. The **set** of slack
  buses is not touched: the angle reference is a property of the batch, chosen once;
- and, when the **last** generator regulating its own bus is taken out, turns that bus
  from PV to PQ for that row. Its voltage magnitude is then solved for rather than held
  at the setpoint, and the reactive power that machine was supplying is no longer
  available there.

The lost MW is picked up by the slack. If you want a redispatch instead, express it
with `modify_gen_p` -- that is what it is for.

With `redistribute_slack = True`, the active power a row takes out of the balance of the
grid's own targets is first shared on the units of the distributed slack, OpenLoadFlow
style (proportionally to their weight, each one clamped to its `[min_p, max_p]`, a
clamped unit leaving that row's distributed slack), before the powerflow. It counts:

- the set-points of the generators the row disconnects;
- the net injection of the islands cut off with `handle_disconnected_grid`;
- the imbalance of the row's own injections: `sum(gen_p - target)` over the generators
  and static generators minus `sum(load_p - target)` over the loads.

A line or transformer contingency that leaves the grid connected loses no injection:
the change in the losses it causes is left to the powerflow's distributed slack.

.. code-block:: python

    import numpy as np
    from lightsim2grid import ScenarioSweep

    sweep = ScenarioSweep(env)
    gen_mask = np.zeros((env.n_gen, env.n_gen), dtype=bool)
    np.fill_diagonal(gen_mask, True)   # one row per generator: the "generator N-1" sweep
    sweep.set_contingency_gens(gen_mask)
    sweep.compute()
    v = sweep.get_voltages()

.. note::

    The PV -> PQ relabelling would normally force a fresh symbolic factorization on
    every row, which is exactly what makes `ScenarioSweep` fast (one analysis, one
    factorization, refactorizations after that). It does not here: every bus that *can*
    flip is given a voltage-magnitude unknown and a reactive equation once, up front, so
    the Jacobian's sparsity is the union over all the rows; a row where the bus is still
    PV simply masks that reactive equation to an identity row, freezing `|V|` at the
    setpoint. That is a value-level edit, and the factorization survives it.

    The cost is one extra row and column per *switchable* bus, on every row of the
    sweep. A handful of candidate generators is negligible; masking every generator on
    the grid grows the Jacobian's dimension by roughly the number of PV buses.

.. warning::

    Only generators that regulate their **own** bus are supported for now. `compute()`
    raises if the mask names a generator that regulates a remote bus, or one standing on
    a bus whose voltage a control group holds (a remote generator, an SVC or an HVDC
    converter station). Those go through a different part of the Jacobian, with columns
    and rows of their own that this feature does not yet reserve and mask.

Topological actions
---------------------

A row can also carry a **topological action**, the same object the light environment plays
(:class:`lightsim2grid.lightEnv.TopoAction`, grid2op semantics), given as a grid2op action or
as a ``TopoAction``, one per row:

.. code-block:: python

    import grid2op
    from lightsim2grid import LightSimBackend
    from lightsim2grid.scenarioSweep import ScenarioSweep

    env = grid2op.make("l2rpn_case14_sandbox", backend=LightSimBackend())
    obs = env.reset()

    actions = [
        env.action_space({}),                                   # a plain row
        env.action_space({"set_line_status": [(3, -1)]}),       # line 3 off
        env.action_space({"set_bus": {"loads_id": [(0, -1)]}}), # load 0 off
        env.action_space({"set_bus": {"generators_id": [(1, -1)]}}),  # gen 1 off: its bus turns PQ
    ]
    sweep = ScenarioSweep(env)
    sweep.set_topo_actions(actions)
    sweep.compute()
    Vs = sweep.get_voltages()

Every action is checked against the grid when it is registered (the element exists, the
busbar exists, no contradiction) and an invalid one raises a ``ValueError`` naming the row.

A row plays:

- **disconnections**: a line or a trafo (``set_line_status -1``, or ``set_bus -1`` on one of
  its ends), a generator, a load or a storage unit (``set_bus -1``). A branch or a generator is
  disconnected exactly as the ``set_contingency_*`` masks do it -- the same admittance edit,
  the same PV -> PQ handling -- so a row is free to combine an action with injections and
  masks, as long as it does not name one element in both a mask and its action (``compute``
  refuses that);
- **reconnections and moves**: an element put on a busbar (``set_bus`` to it), a branch
  reconnected (``set_line_status 1``, or ``set_bus`` on its ends). This is what creates a bus
  (a busbar nothing stood on, now used) or merges two (a busbar left empty).

The whole sweep still runs on **one symbolic analysis**. The solver labelling is built once
for the union of the buses any row uses, the admittance entries a row writes are reserved as
stored zeros before the base case is solved, and each row is then a set of value edits: the
coefficients of the branches it moves, the injections of the elements it moves, the PV pinning
of the buses whose generators it moves (a bus a generator lands on is held at the set-point,
a bus its last generator leaves is solved for), and the masking of the buses of the union
it does not use, exactly as ``handle_disconnected_grid`` masks a stranded bus. A row that
leaves an element alone on a busbar (an island of one bus) strands it, wherever that busbar
is: without ``handle_disconnected_grid`` the row is ``NOT_SIMULATED``, as for a contingency
that strands a bus; with it, the bus is masked and the element's injection left out. A busbar
the row leaves with no element at all (a merge), or does not use, strands nothing: it is
masked, and the row solved, in either mode.

.. warning::

    Refused by ``compute`` for now: moving or reactivating a slack participant, a generator
    on a slack bus, a generator that regulates a remote bus or whose bus a control group
    holds, a storage unit that regulates voltage; a branch with one end open in the base grid
    put back on (taking it out works); ``keep_jacobian`` on a batch that moves or reactivates a
    generator; and the DC algorithm. See the TODO section of the changelog.
    ``compute_physical_violations`` follows the row: a generator the row moves or reactivates
    is checked on the bus the row gives it.

Per-row results
--------------------------

Besides the voltages, the C++ object (`ScenarioSweepCPP`, or `sweep.computer` from the
Python wrapper) reports, for every row of the last `compute()`:

- `compute_branch_results()`: active and reactive power at both ends of every branch, as
  an array of shape `(n_simul, n_branch, 4)` -- `P` and `Q` at side 1, then at side 2, in
  MW and MVAr, branches numbered lines then transformers like `get_flows`. They are what
  `get_line_res1/2` and `get_trafo_res1/2` would read after solving that row's topology
  on its own. A branch the row disconnects reads 0, and so does a row that did not
  converge. In DC, `Q` is 0 and side 2 carries the opposite of side 1.
- `get_Ybus(row)` (AC) / `get_dcYbus(row)` (DC): the admittance matrix that row was
  solved with, in grid bus numbering like `LSGrid.get_Ybus` (`solver_numbering=True` for
  the solver one). An entry only the disconnected branches contributed to is removed from
  the pattern rather than stored as an explicit zero.
- `get_row_solve_times()` / `get_row_nb_iter()`: what the algorithm reported for each
  row's solve, 0 for a row that never reached it.
- `get_gen_results()`, once `compute_gen_results = True` was set before `compute()`: the
  active and reactive output of every generator, shape `(n_simul, n_gen, 2)`, MW and
  MVAr. The rules are the ones `get_gen_res` follows after a single powerflow: the
  setpoint plus the generator's share of the distributed slack (re-weighted without the
  participants the row disconnects); for Q, the setpoint of a machine that does not
  regulate, the solver's own value for a controller it solved (a remote voltage control
  group), or a share of its bus' reactive residual proportional to each machine's reactive
  range. A generator the row disconnects reads 0. In DC, Q is 0.

With `set_topo_actions`, the branch and generator results follow the row's action: an
element it moves, reconnects or reactivates is reported on the buses the row gives it.

`compute_branch_results`, `get_gen_results`, `get_row_solve_times` and `get_row_nb_iter`
exist on every batch class (`TimeSeriesCPP`, `InjectionSweepCPP`, `ContingencyAnalysisCPP` too); `get_Ybus` /
`get_dcYbus` only where the topology varies per row (`ScenarioSweepCPP`,
`ContingencyAnalysisCPP`).

.. code-block:: python

    computer = sweep.computer
    res = computer.compute_branch_results()   # (n_simul, n_branch, 4)
    p_or, q_or, p_ex, q_ex = res[..., 0], res[..., 1], res[..., 2], res[..., 3]
    ybus_row_3 = computer.get_Ybus(3)          # scipy.sparse, (total_bus, total_bus)

    computer.compute_gen_results = True        # before compute()
    computer.compute(v_init, max_iter, tol)
    gen = computer.get_gen_results()           # (n_simul, n_gen, 2): P, Q

Outside a batch, `LSGrid.get_gen_slack_shares()` gives each generator's share of the
distributed slack (`P = target_p + share * slack absorbed`). The weights it reads are set
when the grid is built and by `update_slack_weights`; `change_p_gen` does not update them.

Many sweeps in a row
--------------------------

A sweep holds its own copy of the grid, taken at construction: changing the grid
afterwards (`change_p_gen`, `change_p_load`, ...) does not reach it. A scenario that
differs from the grid by its injections is therefore expressed through the sweep, not by
editing the grid and building a new sweep:

.. code-block:: python

    computer = ScenarioSweepCPP(grid)
    computer.set_contingency_lines(line_mask)       # the outages, once
    for scenario in scenarios:
        computer.modify_gen_p(scenario.gen_p)        # one row per simulation
        computer.modify_load_p(scenario.load_p)
        computer.modify_load_q(scenario.load_q)
        computer.compute(v_init, max_iter, tol)
        assert computer.base_case_was_reused()       # nothing rebuilt but the injections

What one `compute()` builds is kept for the next, and only what an input changed is
rebuilt:

- new injections (`modify_*`) keep everything: the admittance matrix, the base-case
  powerflow, the symbolic analysis of the Jacobian;
- new contingency masks (`set_contingency_*`) rebuild the base case (which branches each
  row disconnects is part of it) but keep what was read off the grid. A fixed outage
  list across scenarios is the fastest way to use one sweep.

Building a new sweep per scenario stays correct, only slower; it is what to do when the
grid itself changes (its parameters, its topology).

Threads and processes
--------------------------

`compute()` releases Python's GIL. Two ways to use several cores follow from that:

- a pool of Python threads, each running its own sweep (one object per thread: a sweep
  is not meant to be used by two threads at once). This works because the GIL is
  released for the whole solve;
- `nb_thread` on one sweep, which splits its rows over C++ threads. It defaults to 1.

In a caller that already runs one process per core, keep `nb_thread = 1`: more threads
than cores only adds contention. `nb_thread > 1` pays one symbolic analysis per thread,
so it is worth it for many rows rather than for a few.

Handling disconnected grids and limit violations
------------------------------------------------------

`ScenarioSweep` has the same `handle_disconnected_grid` mode and inline limit-violation
checking (`compute_limit_violations` / `violation_threshold` / `violation_rel_tol` / `get_violations` /
`get_violations_n`) as `ContingencyAnalysis` -- see :doc:`security_analysis` for the full
description of what each does. Same names, same semantics, and both classes'
`get_violations` / `get_violations_n` return the same `LimitViolation` objects.

.. code-block:: python

    sweep = ScenarioSweep(env)
    sweep.compute_limit_violations = True  # see warning below: set this FIRST
    sweep.handle_disconnected_grid = True
    sweep.modify_load_p(load_p)
    sweep.set_contingency_lines(line_mask)
    sweep.compute()

    row_violations = sweep.get_violations()  # one list of LimitViolation per row
    n_violations = sweep.get_violations_n()  # violations of the shared pre-batch base case

    sa_res = sweep.run()  # PreContingencyResult / ContingencyResult / SecurityAnalysisResult,
                           # reusing lightsim2grid.contingencyAnalysis's dataclasses --
                           # contingency_name is always None here (no such concept on ScenarioSweep)

.. warning::

    Just like on `ContingencyAnalysis`, setting `compute_limit_violations` through the
    property **clears the whole object** -- registered injections, contingency masks,
    `handle_disconnected_grid`, everything, the same reset `clear()` performs. Set it
    *first*, before calling any `modify_*` / `set_contingency_*` setter or
    `handle_disconnected_grid`, or those calls will be silently wiped.

    Unlike `ContingencyAnalysis`, `ScenarioSweep` has no `(grid, compute_limit_violations)`
    constructor shortcut to dodge this ordering -- the property is the only way to set it.

.. note::

    There is deliberately **no** `converged` / `converged_n` on `ScenarioSweep` (unlike
    `ContingencyAnalysis`). A non-converged row's `get_violations()` entry already carries
    a single `LimitViolation` with `element_type == ViolationElementType.GRID` and
    `violation_type` either `NOT_SIMULATED` (skipped before the solver ran, eg it splits the
    grid and `handle_disconnected_grid` is off) or `DIVERGENCE` (the solver ran but did not
    converge) -- a separate convergence flag would be redundant. If the shared pre-batch "n"
    powerflow itself diverges, that same sentinel is stamped into `get_violations_n()` and
    into every row of `get_violations()`, rather than leaving them as empty lists
    indistinguishable from "converged, nothing found".

A future release may add a "combine mode" axis (this row-aligned / zipped behavior vs. a
cartesian "every contingency x every injection profile") -- not yet available.

Detailed usage
--------------------------

.. automodule:: lightsim2grid.scenarioSweep
    :members:
    :autosummary:

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`