# Pipelines

# Pipelines

Change channels as they arrive, before they are recorded. Use a pipeline to
drop noise, rescale a value, decode CAN frames, shrink video or add a cleaned-up
copy of a signal.

Open **Pipelines** in Phloem Computer.

## How rules work

A pipeline is a list of **rules**. Each rule has a channel pattern and a chain
of steps.

1. A channel arrives. Phloem checks the rules from the top.
2. The first rule whose pattern matches claims the channel and runs its steps.
3. Later rules never see that channel. A channel no rule matches passes
   through unchanged.

So order matters. The list is headed **Rules · first match wins**.

| On a rule's row | Means |
|---|---|
| number | its place in the list |
| count | live channels this rule claims |
| `⊘ shadowed` | its pattern matches, but earlier rules claim every channel first |
| switch | off keeps the rule in its place but passes data through |

Drag a row to reorder it, or press Alt+↑ / Alt+↓. An open rule also has
**move to #**.

Calibration runs before any rule. A rule shows whether its channels are
calibrated, with a link to set it on **Inputs**:
[Scale a raw signal into engineering units](https://phloem.ai/docs/guides/calibrate/).

## Add a rule

1. Press **+ Rule**.
2. Type a pattern, like `/motor/**`. `**` matches everything below a path.
   Give the rule a name if you want one.
3. Press **Add step**, the **+** between steps, or ⌘/ (Ctrl+/ on Windows and
   Linux). The **Transform library** opens. Pick a step.
4. Fill in the step's fields. A red `*` marks a required field.
5. Press **Save & apply**. **Discard** throws away unsaved edits.

The header shows how many errors block saving. Click it to see them.

## Steps

The library groups steps by what they work on. Steps that do not fit at that
point in the chain are listed under **Not applicable at this point**.
Video steps need raw frames, and number steps need decoded values.

| Step | What it does |
|---|---|
| **Filter** | keeps samples where an expression is true, like `value > 0.0` |
| **Transform** | replaces each value with an expression, like `value * 0.1` |
| **Derive channel** | adds a new channel beside the original, run through a filter or function |
| **Decimate** | keeps at most N samples per second on each channel |
| **Route** | renames the channel path, like `ev/motor/{path}` |
| **Drop channels** | removes channels matching a pattern, like `**/*_covariance/**` |
| **Array slice** | keeps array positions from one index to another |
| **Set frame** | names the 3D frame that point clouds or cameras sit in |
| **Parse DBC** | decodes raw CAN frames into named signals, using a `.dbc` you upload |
| **Resize video** | shrinks camera frames to fit a width and height |
| **Throttle FPS** | drops frames above N frames per second |
| **Grayscale** | removes colour from camera frames |
| **Mark** | records a mark when a value crosses a line |
| **Alarm** | a mark with a severity (`info`, `warn`, `critical`) and who to notify |

Expressions read `value`, `path` and `t`. They can use `sqrt`, `abs`, `pow`,
`exp`, `log`, `log10`, `hypot`, `floor`, `ceil`, `round` and `clamp`.

**Mark** and **Alarm** fire on a crossing: `rises`, `falls`, `above`, `below`,
`within` or `outside`. Set `until` to make a span that closes on a second
crossing. Marks show on **Live** and **Replay** — see [Marks](https://phloem.ai/docs/review/marks/).

## Derived channels

A derived channel is a new channel computed from another one. The original
stays as it was.

There are two kinds:

| Kind | Where | Use it for |
|---|---|---|
| **Derive channel** step | inside a rule | a per-sample filter: `lowpass`, `highpass`, `bandpass`, `notch`, `butterworth`, `ewma`, `delta`, `deriv`, `integral`, `moving_mean`, `moving_rms` |
| **Derived · windowed** | above the rules | anything, including functions over a window: `mean`, `rms`, `min`, `max`, `stdev`, `peak_to_peak`, `histogram`, `psd`, `band_power`, `peak_hz`, `expr` |

To add a windowed one, press **+** beside **Derived · windowed**. Set `fn`,
`in` (the source channel) and `out` (the new channel's path).

Each derived channel is recorded and synced to the cloud like any channel.
Turn its cloud switch off to keep it live-only; it then does not appear in
Replay. Press **view live** to open it on **Live**.

## Test before you save

The panel on the right checks the open rule.

| Tab | Shows |
|---|---|
| **Match** | the live channels the pattern matches, and any an earlier rule claims |
| **Test** | runs the rule on sample data and shows each step's output |
| **Errors** | every problem that blocks saving; click one to jump to it |
| **Lineage** | which channels the rules produce, and from what |
| **Library** | your own saved functions; **Use** adds one as a derived channel |

**Test** takes three sources:

- **Tap live** — **Start tap** runs the rule on live data. Nothing is saved.
- **File** — **Choose file…** runs a log or recording through it.
- **Paste** — paste sample data and press **Run test**.

Results show on each step, so you see a value before and after.

## YAML

Switch **Flow** to **YAML** to edit the saved rules as text. It shows saved
rules only. Rules that another tool adds for a short time run but are not
in this text. They are listed on **Flow** under **Tool · ephemeral**.

- [Name channels and units](https://phloem.ai/docs/inputs/channels/): Name a signal, set its unit
- [Decode a CAN bus with your DBC](https://phloem.ai/docs/guides/decode-can/): Turn CAN frames into named signals
- [Scale a raw signal into engineering units](https://phloem.ai/docs/guides/calibrate/): Linear, polynomial and table calibration