Constants

Every tuning bound, offset and weight in one place.

All frozen numbers in one place: DEBRA’s tuning constants and the classic Dust RGB recipe.

Every (MIN, MAX) scaling bound, offset, and weight from Miller et al. (2017), JGR Atmospheres, doi:10.1002/2017JD027365, using the 26 Feb 2020 erratum versions of Eqs. 7, 21, 22, 24, 25. The paper notes “minor retuning” may be needed per sensor; retune here, nowhere else.

Sign/orientation conventions follow the printed equations exactly.

The sensor facts here (Band and the channel maps) are not from the paper; they are what each instrument calls the roles the algorithms read.

class shachen.constants.Bounds

Bases: object

(MIN, MAX) pair for the Eq. 3 normalization primitive.

min: float

max: float

__init__(min: float, max: float) → None

class shachen.constants.Band

Bases: StrEnum

Spectral roles the algorithms read (nominal wavelengths in um).

Seven of these are DEBRA’s inputs (DEBRA_BANDS); TIR_112 exists only for the classic Dust RGB green gun.

VIS_064 = 'vis_064'

cloud mask + day baseline image

NIR_160 = 'nir_160'

daytime cloud test (reserved)

SWIR_39 = 'swir_39'

night thin-cirrus test CM4

WV_62 = 'wv_62'

deep-convection test CM2

TIR_86 = 'tir_86'

dust test DT2 (8.4-8.6 um)

TIR_104 = 'tir_104'

clean window reference (10.3-10.4 um)

TIR_112 = 'tir_112'

classic Dust RGB green gun only (11.2 um; not a DEBRA input)

TIR_123 = 'tir_123'

dirty window, RSW / DT1 (12.3 um)

__new__(value)

shachen.constants.ABI_BANDS: dict[Band, str] = {Band.NIR_160: 'C05', Band.SWIR_39: 'C07', Band.TIR_104: 'C13', Band.TIR_112: 'C14', Band.TIR_123: 'C15', Band.TIR_86: 'C11', Band.VIS_064: 'C02', Band.WV_62: 'C08'}

Spectral role -> GOES-R ABI channel name (satpy dataset names).

shachen.constants.AHI_BANDS: dict[Band, str] = {Band.NIR_160: 'B05', Band.SWIR_39: 'B07', Band.TIR_104: 'B13', Band.TIR_112: 'B14', Band.TIR_123: 'B15', Band.TIR_86: 'B11', Band.VIS_064: 'B03', Band.WV_62: 'B08'}

Spectral role -> Himawari AHI band name (satpy dataset names).

shachen.constants.DEBRA_BANDS: tuple[Band, ...] = (Band.VIS_064, Band.NIR_160, Band.SWIR_39, Band.WV_62, Band.TIR_86, Band.TIR_104, Band.TIR_123)

The seven DEBRA algorithm input roles (Miller et al. 2017 Table 1). Band.TIR_112 exists only for the classic Dust RGB comparison baseline (shachen.dustrgb); pipelines that feed DEBRA alone load these.

shachen.constants.BAND_CENTER_UM: dict[Band, float] = {Band.NIR_160: 1.61, Band.SWIR_39: 3.9, Band.TIR_104: 10.33, Band.TIR_112: 11.19, Band.TIR_123: 12.3, Band.TIR_86: 8.44, Band.VIS_064: 0.64, Band.WV_62: 6.19}

Nominal central wavelengths (um) used to interpolate the UWBF emissivity hinge points to sensor band centers (ABI values; AHI within tolerance).

class shachen.constants.CloudMaskConstants

Bases: object

Eqs. 1-12.

cm1_cold_offset_k: float = 50.0

CM1 (Eq. 1): 1 - N(BT10.4; T_skin - cm1_cold_offset_k, T_skin)

cm2: Bounds

CM2 (Eq. 4): N(BT10.4 - BT6.2)

cm3: Bounds

CM3 (Eq. 5): N(BT10.4 - BT12.3), day/night thin cirrus

cm4: Bounds

CM4 (Eq. 6): N(BT3.9 - BT10.4), night-only thin cirrus

r1: Bounds

R1 (Eq. 7, erratum): N(BT12.3 - BT10.4) * (1 - CM1)

r2: Bounds

R2 (Eqs. 8-9): N(BT8.6 - BT10.4) * restoral weights

cm_norm: Bounds

Eq. 12: CM_norm = N(CM)

__init__(cm1_cold_offset_k: float = 50.0, cm2: Bounds = <factory>, cm3: Bounds = <factory>, cm4: Bounds = <factory>, r1: Bounds = <factory>, r2: Bounds = <factory>, cm_norm: Bounds = <factory>) → None

class shachen.constants.DustTestConstants

Bases: object

Eqs. 13-15.

dt1_max_rsw_k: float = 3.5

DT1 (Eq. 13): (RSW_obs - RSW_bg) / (max_rsw_k - RSW_bg), RSW = BT12.3 - BT10.4

dt2_max_btd_k: float = 3.0

DT2 (Eq. 14): (BTD_obs - BTD_bg) / (max_btd_k - BTD_bg), BTD = BT8.6 - BT10.4

dt2_fixed_interval: Bounds | None = None

A second DT2 reading, on a fixed interval instead of Eq. 14’s dynamic background: N(BTD_obs; min, max), emitted as dt2_fixed next to dt2 and read by Eqs. 17-18 only (Eq. 16 keeps Eq. 14). None, the default, is the published chain exactly. The hook exists for ZHOUYE: on 1.19 M station-hours over North China (42 dust days, all 24 hours) the raw 8.6 - 10.4 um difference ranks dust at AUC 0.85 / 0.80 / 0.80 (day / terminator / night) while Eq. 14’s output ranks at 0.64 / 0.60 / 0.59 with 95% of the rows clipped to zero – the background subtraction, the clip at zero and the 3 K range discard the clear-sky side of the signal that the classic Dust RGB’s green gun keeps. By day that side buys no equal-FAR detection, so the daytime sum is left as DEBRA printed it.

dt3_shift_land_k: float = -10.0

DT3 (Eq. 15): (BT10.4 - (T_merra - S - dt3_depth_k)) / dt3_depth_k

dt3_shift_ocean_k: float = 5.0

dt3_depth_k: float = 50.0

__init__(dt1_max_rsw_k: float = 3.5, dt2_max_btd_k: float = 3.0, dt2_fixed_interval: Bounds | None = None, dt3_shift_land_k: float = -10.0, dt3_shift_ocean_k: float = 5.0, dt3_depth_k: float = 50.0) → None

shachen.constants.NIGHT_CF_SCALE: float = 0.5

Ratio of the Eq. 18 night ceiling to the Eq. 16 day ceiling. Both are sums of DT terms that each saturate at 1: day is DT1 + DT2 + DT3 = 3.0, night is max(DT1, DT2) + dt3_weight_ngt * DT3 = 1.5 – night has no independent second test, because without solar heating the 8.6 um emissivity contrast stops being independent of the split window. One shared Eq. 19 interval therefore squeezes CF_ngt onto the lower half of [0, 1]; scaling the night interval by this factor puts the two branches back on one scale. The floor scales with it for the same reason: DT3’s clear-sky bias enters Eq. 18 at half the weight it has in Eq. 16.

class shachen.constants.ConfidenceConstants

Bases: object

Eqs. 16-22 (Eqs. 21-22 per erratum).

Eq. 19 is normalization with one interval, but Eqs. 16 and 18 do not produce values on one scale (see NIGHT_CF_SCALE), so the interval is split here: cf_norm_day normalizes CF_day, cf_norm_ngt normalizes CF_ngt, and the terminator branch interpolates between them on the Eq. 20 weight, which is 1 on the day side of 90 deg and 0 beyond 105 deg – so CF_trm meets whichever neighbour Eq. 22 is handing it to. cf_norm_trm overrides that interpolation with a fixed interval; it is None by default, so nothing here departs from that scheme unless a caller asks it to.

cf_norm is accepted as a constructor-only alias that sets both intervals to the value given, which is the pre-split behaviour exactly. It is an InitVar, so it reads back as None, never as an interval: code that used to do constants.cf_norm.min raises instead of quietly normalizing with the wrong bounds. Read cf_norm_day / cf_norm_ngt. (It stays a declared field, rather than being deleted off the class, so that dataclasses.replace() keeps working on this dataclass.)

dt3_weight_trm: float = 0.5

Eq. 17: CF_trm weights DT3 by this factor; Eq. 18: CF_ngt likewise.

dt3_weight_ngt: float = 0.5

cf_norm_day: Bounds

Eq. 19 for CF_day: the paper’s printed interval.

cf_norm_ngt: Bounds

Eq. 19 for CF_ngt: the same interval scaled by NIGHT_CF_SCALE.

cf_norm_trm: Bounds | None = None

Eq. 19 for CF_trm. None, the default, is the published behaviour: CF_trm has no interval of its own and is normalized on the day and night intervals interpolated by the Eq. 20 weight. Setting an interval here switches CF_trm to it – a hook for callers who want to try a terminator scale of their own, not a change to what DEBRA does.

blend_exponent: float = 1.5

Eqs. 20-21: terminator blending on cos(theta_sun), exponent 1.5.

ngt_trm_zenith_deg: Bounds

night/terminator interface: N(cos theta; cos 105 deg, cos 90 deg)

trm_day_zenith_deg: Bounds

terminator/day interface: N(cos theta; cos 90 deg, cos 75 deg)

corroborate_dt3: bool = False

Corroboration: pixels where DT1 and DT2 are both zero contribute nothing to Eqs. 17-18 (their raw sums are set to 0 before Eq. 19). Eq. 16 is not touched, so by day the product stays DEBRA’s. False, the default, is the published behaviour. This is the ZHOUYE scheme’s third item: a pixel whose only evidence is DT3 – the thermal-contrast test that reads the same skin-temperature gap as cloud mask CM1 – is mid-level cloud or a skin-temperature error far more often than dust (the false-yellow cases of the operational imagery).

cf_norm: dataclasses.InitVar[shachen.constants.Bounds | None] = None

Deprecated alias: sets cf_norm_day and cf_norm_ngt to the same interval (the pre-0.3 single-interval behaviour). Constructor-only.

__init__(dt3_weight_trm: float = 0.5, dt3_weight_ngt: float = 0.5, cf_norm_day: Bounds = <factory>, cf_norm_ngt: Bounds = <factory>, cf_norm_trm: Bounds | None = None, blend_exponent: float = 1.5, ngt_trm_zenith_deg: Bounds = <factory>, trm_day_zenith_deg: Bounds = <factory>, corroborate_dt3: bool = False, cf_norm: dataclasses.InitVar[shachen.constants.Bounds | None] = None) → None

class shachen.constants.ImageryConstants

Bases: object

Eqs. 23-29 (Eqs. 24-25 per erratum).

bg_blend_zenith_deg: Bounds

Eq. 25 (erratum): B_bg = 1 - N(theta_sun; 79 deg, 89 deg)^1.5

bg_blend_exponent: float = 1.5

cf_cap: float = 0.5

Eqs. 27-29: RED = GRN = BI*(1 - min(CF, cf_cap)) + CF; BLU = BI*(1 - min(CF, cf_cap)) + blue_dimming * CF

blue_dimming: float = 0.1

gun_max: float = 1.2

Per-gun rescale [0, gun_max] -> [0, 255]

__init__(bg_blend_zenith_deg: Bounds = <factory>, bg_blend_exponent: float = 1.5, cf_cap: float = 0.5, blue_dimming: float = 0.1, gun_max: float = 1.2) → None

shachen.constants.COLOR_DIMMING: dict[str, tuple[float, float, float]] = {'blue': (0.25, 0.25, 1.0), 'green': (0.1, 1.0, 0.1), 'pink': (1.0, 0.25, 0.25), 'yellow': (1.0, 1.0, 0.1)}

Per-gun dimming triples (D_red, D_grn, D_blu) generalizing Eqs. 27-29: each gun is BI*(1 - min(CF, cf_cap)) + D_gun*CF, a full gun has D = 1.0. Presets from the paper’s section 4.2 text (“dust can be made pink by applying D = 0.25 to equations (28) and (29), green by applying D = 0.10 to equations (27) and (29), or blue by applying D = 0.25 to (27) and (28)”).

shachen.constants.COMPOSITE_WINDOW_DAYS: int = 14

Cloud-cleared composite background (paper section 3.2, Option B): rolling window length (“multiday (e.g., 14 day) composites”) and the fewest composite days accepted at load time before the background is refused.

class shachen.constants.DustRGBConstants

Bases: object

One classic Dust RGB stretch set. Not part of DEBRA (Eqs. 1-29).

Each gun is N(x; MIN, MAX) ** (1/gamma) over the GOES-R/Himawari band mix of the CIRA Dust RGB Quick Guide (satpy’s dust composite): 12.3-10.3, 11.2-8.4, 10.3 — note the green minuend is 11.2 um, not the 10.8 um of SEVIRI’s IR10.8 - IR8.7.

The stretch values are per sensor: DUST_RGB (SEVIRI) and DUST_RGB_ABI are the two published sets, dispatched by DUST_RGB_BY_READER. See shachen.dustrgb for the references.

red: Bounds

RED: BT12.3 - BT10.4 (split window), [-4, +2] K, gamma 1.

green: Bounds

GRN: BT11.2 - BT8.6, [0, 15] K, gamma 2.5 (note: 11.2 um, not 10.4).

green_gamma: float = 2.5

blue: Bounds

BLU: BT10.4, [261, 289] K, gamma 1.

__init__(red: Bounds = <factory>, green: Bounds = <factory>, green_gamma: float = 2.5, blue: Bounds = <factory>) → None

shachen.constants.DUST_RGB = DustRGBConstants(red=Bounds(min=-4.0, max=2.0), green=Bounds(min=0.0, max=15.0), green_gamma=2.5, blue=Bounds(min=261.0, max=289.0))

The original SEVIRI stretches (Lensky and Rosenfeld 2008, as formalised in EUMeTrain’s recipe compilation; satpy’s generic dust_default enhancement). Applied to Himawari here, which is what satpy does too: it ships no dust_ahi override. Also the fallback for an unknown sensor.

shachen.constants.DUST_RGB_ABI = DustRGBConstants(red=Bounds(min=-6.7, max=2.6), green=Bounds(min=-0.5, max=20.0), green_gamma=2.5, blue=Bounds(min=261.2, max=288.7))

The ABI-adjusted stretches from the CIRA GOES-R Dust RGB Quick Guide (satpy’s sensor-specific dust_abi enhancement), converted from the Quick Guide’s degrees Celsius. Berndt et al. (2018) is the method behind the adjustment: ABI’s channels do not sit where SEVIRI’s do, so the SEVIRI numbers put the clear-sky land background at the wrong end of the red stretch.

shachen.constants.DUST_RGB_BY_READER: dict[str, DustRGBConstants] = {'abi_l1b': DustRGBConstants(red=Bounds(min=-6.7, max=2.6), green=Bounds(min=-0.5, max=20.0), green_gamma=2.5, blue=Bounds(min=261.2, max=288.7)), 'ahi_hsd': DustRGBConstants(red=Bounds(min=-4.0, max=2.0), green=Bounds(min=0.0, max=15.0), green_gamma=2.5, blue=Bounds(min=261.0, max=289.0))}

satpy reader -> the stretch set that sensor’s operational product uses. shachen.pipeline.run_dust_rgb() reads this off scene.attrs["reader"] so a baseline rendered here matches the image forecasters see. The same rule is why there is no single canonical recipe to freeze: the scheme is retuned per imager (satpy carries ash_abi, convection_abi and the night-microphysics variants for the same reason).

class shachen.constants.DebraConstants

Bases: object

The whole tuning surface, one group per stage of the algorithm.

cloud_mask: CloudMaskConstants

dust_tests: DustTestConstants

confidence: ConfidenceConstants

imagery: ImageryConstants

__init__(cloud_mask: CloudMaskConstants = <factory>, dust_tests: DustTestConstants = <factory>, confidence: ConfidenceConstants = <factory>, imagery: ImageryConstants = <factory>) → None

shachen.constants.DEFAULTS = DebraConstants(cloud_mask=CloudMaskConstants(cm1_cold_offset_k=50.0, cm2=Bounds(min=0.0, max=25.0), cm3=Bounds(min=2.0, max=4.5), cm4=Bounds(min=5.0, max=8.0), r1=Bounds(min=0.0, max=3.5), r2=Bounds(min=-1.0, max=3.0), cm_norm=Bounds(min=0.45, max=0.8)), dust_tests=DustTestConstants(dt1_max_rsw_k=3.5, dt2_max_btd_k=3.0, dt2_fixed_interval=None, dt3_shift_land_k=-10.0, dt3_shift_ocean_k=5.0, dt3_depth_k=50.0), confidence=ConfidenceConstants(dt3_weight_trm=0.5, dt3_weight_ngt=0.5, cf_norm_day=Bounds(min=0.25, max=2.5), cf_norm_ngt=Bounds(min=0.125, max=1.25), cf_norm_trm=None, blend_exponent=1.5, ngt_trm_zenith_deg=Bounds(min=105.0, max=90.0), trm_day_zenith_deg=Bounds(min=90.0, max=75.0), corroborate_dt3=False), imagery=ImageryConstants(bg_blend_zenith_deg=Bounds(min=79.0, max=89.0), bg_blend_exponent=1.5, cf_cap=0.5, blue_dimming=0.1, gun_max=1.2))

The paper-tuned default constant set.

shachen.constants.ABI_TUNED = DebraConstants(cloud_mask=CloudMaskConstants(cm1_cold_offset_k=50.0, cm2=Bounds(min=0.0, max=25.0), cm3=Bounds(min=2.0, max=4.5), cm4=Bounds(min=5.0, max=8.0), r1=Bounds(min=0.0, max=3.5), r2=Bounds(min=-1.0, max=3.0), cm_norm=Bounds(min=0.45, max=0.8)), dust_tests=DustTestConstants(dt1_max_rsw_k=3.5, dt2_max_btd_k=3.0, dt2_fixed_interval=None, dt3_shift_land_k=-10.0, dt3_shift_ocean_k=5.0, dt3_depth_k=50.0), confidence=ConfidenceConstants(dt3_weight_trm=0.5, dt3_weight_ngt=0.5, cf_norm_day=Bounds(min=0.4, max=2.5), cf_norm_ngt=Bounds(min=0.2, max=1.25), cf_norm_trm=None, blend_exponent=1.5, ngt_trm_zenith_deg=Bounds(min=105.0, max=90.0), trm_day_zenith_deg=Bounds(min=90.0, max=75.0), corroborate_dt3=False), imagery=ImageryConstants(bg_blend_zenith_deg=Bounds(min=79.0, max=89.0), bg_blend_exponent=1.5, cf_cap=0.5, blue_dimming=0.1, gun_max=1.2))

Optional retune for the ABI + MERRA-2 + CAMEL stack. Single deviation from the paper: the Eq. 19 lower bound is raised 0.25 -> 0.40. Rationale: on this ancillary stack DT3 carries a 0.2-0.5 clear-sky floor over land (MERRA-2 skin T runs warmer than BT10.4, plus the -10 K land shift), which leaks through low-cloud-mask pixels as a faint yellow tint over vegetated and low-cloud areas. Sweeping the bound over the three reference cases: 0.40 removes 92% of the tinted area (SE-US box, fraction CF > 0.05: 0.37 -> 0.028) while the 2017-03-23 plume mean CF drops only 5% (0.476 -> 0.452) and the 2020-12-23 plume core stays distinct (p90 0.40). The paper itself anticipates “minor retuning” per sensor. The night interval follows it through NIGHT_CF_SCALE (0.20, 1.25): the same DT3 floor enters Eq. 18 at half the weight, so the floor that suppresses it halves too.

shachen.constants.ZHOUYE = DebraConstants(cloud_mask=CloudMaskConstants(cm1_cold_offset_k=50.0, cm2=Bounds(min=0.0, max=25.0), cm3=Bounds(min=2.0, max=4.5), cm4=Bounds(min=5.0, max=8.0), r1=Bounds(min=0.0, max=3.5), r2=Bounds(min=-1.0, max=3.0), cm_norm=Bounds(min=0.45, max=0.8)), dust_tests=DustTestConstants(dt1_max_rsw_k=3.5, dt2_max_btd_k=3.0, dt2_fixed_interval=Bounds(min=-0.5, max=1.5), dt3_shift_land_k=-10.0, dt3_shift_ocean_k=5.0, dt3_depth_k=50.0), confidence=ConfidenceConstants(dt3_weight_trm=0.5, dt3_weight_ngt=0.5, cf_norm_day=Bounds(min=0.4, max=2.5), cf_norm_ngt=Bounds(min=0.1, max=1.55), cf_norm_trm=Bounds(min=0.6, max=2.1), blend_exponent=1.5, ngt_trm_zenith_deg=Bounds(min=105.0, max=90.0), trm_day_zenith_deg=Bounds(min=90.0, max=75.0), corroborate_dt3=True), imagery=ImageryConstants(bg_blend_zenith_deg=Bounds(min=79.0, max=89.0), bg_blend_exponent=1.5, cf_cap=0.5, blue_dimming=0.1, gun_max=1.2))

ZHOUYE (昼夜, “day and night”): a diurnally consistent dust confidence scheme built on DEBRA’s tests and cloud mask. It is a second algorithm in this package, not a retune of DEBRA – DEFAULTS and ABI_TUNED do not change when it does. Three departures, each a hook that is off in the DEBRA presets, and all three act on the terminator and night sums (Eqs. 17-18) only: Eq. 16 and its interval are DEBRA’s, so wherever the Eq. 20 weight is 1 (solar zenith below 75 deg) CF_comb is ABI_TUNED’s bit for bit, and the scheme is a night-side extension that blends into DEBRA across 75-90 deg. The goal it is built to is one reading for one dust around the clock: no fading at dusk (DEBRA) and no brightening after dark.

1. DT2 on a fixed interval (dt2_fixed_interval) in Eqs. 17-18 instead of Eq. 14: the same 8.6 - 10.4 um channels, but the clear-sky side of the difference is kept, which is where most of the night-time ranking lives. Eq. 16 keeps Eq. 14: by day the fixed interval buys no equal-FAR detection and only re-labels the plume upward. The interval is (-0.5, +1.5) K. The equal-FAR optimum on the 42 dust days is the 1 K step (-0.5, +0.5) (night POD at DEBRA’s FAR@0.2 0.165 -> 0.323, 19/19 leave-one-event-out folds), but a step saturates on nearly every dust pixel and, through Eq. 18’s max, puts every lit night pixel on one plateau that no Eq. 19 interval can spread into the day’s 0.2-0.8 gradation; the +1.5 K ceiling lets the reading grade with BTD and costs 0.04 of that equal-FAR POD (0.281, still +0.12 over DEBRA).

2. DT3 needs corroboration (corroborate_dt3) in Eqs. 17-18: a pixel lit by the thermal-contrast test alone contributes nothing there.

3. Eq. 19 intervals for the terminator and night sums fitted so that lit dust reads as it does by day (cf_norm_ngt, cf_norm_trm): the ceiling-proportional split of ABI_TUNED was right for DEBRA’s sums, where DT2 is mostly zero, but with the fixed-interval DT2 the night sum sits far higher relative to its ceiling than the day sum does. The intervals here are the grid cells that bring the p25/p50/p75/p90 of CF_comb on lit dust rows (PM10 >= 600, CF >= 0.1) at night (> 105 deg) and across the terminator (75-105 deg) closest to the day rows’ under DEBRA on the 42 dust days, among cells that light no larger share of clean rows than the day does: night 0.22 / 0.38 / 0.59 / 0.73 and terminator 0.20 / 0.34 / 0.59 / 0.77 against the day’s 0.19 / 0.33 / 0.58 / 0.78 (DEBRA’s own night interval reads 0.18 / 0.31 / 0.51 / 0.77 on the few pixels it lights). Eq. 19 is a monotone map, so this fixes what a threshold means and leaves the ranking above the floor untouched. The night ceiling 1.5 maps to CF 0.97.

What it costs, measured on 30 no-dust days (711 seven-band scenes): night POFD at 0.2 doubles (0.0029 -> 0.0054), which at a dust-season base rate of 0.555% is FAR 0.758 -> 0.771; the terminator at the same threshold is 0.472 -> 0.432. Over quartz sand seas (clear-sky BTD background -3 K and below) the fixed interval is on the miss side, not the false-alarm side: a plume must lift the BTD about twice as far as Eq. 14 needs before it reads. The cloud-mask constants and the day interval are ABI_TUNED’s. Shares DEBRA’s tests and cloud mask; it is not an independent retrieval.