Skip to content

Open questions

What is genuinely unresolved, as of 2026-09-01. This page exists so that the honest gaps are findable in one place rather than buried in the write-up of whichever experiment first ran into them.

It is deliberately separate from Experiments, which records what was measured and settled. Something belongs here only if a concrete next step exists and has not been taken. When an item is closed it moves to the experiments register with a verdict, and its row here is deleted rather than annotated.

Ranked by expected value over cost

1. Keep turning the flywheel

More human-verified Pakistani installations, retrain, measure. Every shipped improvement in the experiments register is smaller than this one. In-domain training chips tripled large-array recall; nothing architectural has come close.

2. building_table's roof term counts a ground array that clips a shed as rooftop PV

parcel_pv_area's rule 3 skips any installation at or above YARD_MAX_INSTALLATION_M2 (400 m²), because above that floor ground-mount is segmentation's instrument and the atlas already prices it. The roof term next to it has no such guard: it intersects every mapped PV polygon with every VIDA footprint regardless of placement or size. Harmless while ground-mount above the floor was rare in the quadrats; not harmless now.

Measured on Kalat Rural (added 2026-08-17, sited deliberately to include ground-mount): 69 of the 89 buildings the roof term labels has-PV are labelled solely because a >= 400 m² ground array clips them, on roofs whose median footprint is 29 m². That is a 21.24% base rate in a 46.5 bldg/km2 rural box, higher than Mardan's. Folding it in unchanged would fit the sparse density band's coverage ratio on ground-mount, extrapolate that across millions of rural buildings, and double-count against segmentation's own est_mwp_rc_ground.

The step: extend rule 3 to the roof term, then re-measure every existing quadrat's base_rate/pv_area_true_m2 to see how much it moves them (expected small outside Kalat, since ground-mount above the floor is rare elsewhere, but that is an assumption until measured). Until then Kalat Rural must be excluded from any refit by hand -- roofclf.discover_quadrats globs the label directory, so it is picked up automatically. See Calibration boxes's Box 17.

3. The correction that prices most of the atlas is fit outside the density range it is applied to

Reach dropped an order of magnitude since this was first measured, apparently as a side effect of an unrelated refit, not a deliberate fix -- re-verify before treating this as closed. As measured 2026-08-16: coverage_ratio and area_recall (83% of Best) were fit on quadrats spanning 872 to 4,195 bldg/km2 while 84.1% of in-domain buildings sit in cells sparser than every one of them (building-weighted median 252 bldg/km2), pricing 9,842 MWp (54% of published Best) with a multiplier fit entirely on denser ground. Re-run (scripts/trust_gate_density_audit.py) as of the 2026-08-17 parcel-label refit and again after the 2026-08-20 Box 18 quadrats (Attock/Layyah/Lodhran) were folded in: the sparsest quadrat surviving the trust gate is now 124 bldg/km2 (bahawalnagar_rural, whose own rate_ratio moved 2.19 → 1.78 → 1.70 across the three fits and crossed the 2.0 ceiling sometime between 2026-08-16 and 2026-08-17), only 13.5% of in-domain buildings sit sparser than every fit quadrat, and only ~5% of published Best is priced out of calibration range -- essentially unchanged between the 2026-08-17 and 2026-08-20 fits, so the Box 18 quadrats did not move this finding further. Nobody appears to have deliberately checked Bahawalnagar Rural's rate_ratio against this page when it crossed the gate -- worth asking the owner whether that shift was noticed, since it is the whole reason this item's severity changed. Full before/after table: roofclf calibration density mismatch.

The trust gate is still not density-neutral (Spearman(density, rate_ratio) -0.36 on the current 30-quadrat fit, versus -0.58 on 2026-08-16), so the mechanism is unfixed even though its reach is now small. nasirabad_rural and tank_rural no longer lack n_buildings (both are now scored -- 189 and 225 buildings respectively -- by every roof-classifier run since 2026-08-13), so that specific blocker from the original measurement is resolved; both still fail the gate on their own rate_ratio (tank_rural 0.307, below the 0.5 floor; nasirabad_rural undefined, zero true positives), which is a legitimate exclusion, not a data gap. Remaining open work: settle the gate band explicitly now that its (much smaller) confounding is measured, and directly extend the fit's own density support into the remaining gap (50-125 bldg/km2, where the 13.5% still-uncovered population actually sits) rather than the stale 150-600 bldg/km2 target. Eight unscreened candidate quadrats targeting exactly that band, spanning Punjab/Sindh/Khyber Pakhtunkhwa/ Balochistan, are proposed and awaiting the owner's own imagery-recency check (no automated check exists) in data/labels/candidate_quadrats/*_gap_calib_2km_candidate.geojson -- see Calibration boxes's Box 19.

4. France answered half of this; the EarthPV half is still open

Partly closed 2026-09-04. The original item asked whether France could supply an external reference for the sub-400 m² half of the atlas, which Germany's register cannot reach. It can, and the register-side work is done and written up in Validation against the French register: fourteen exhaustively hand-mapped communes against dated vintages of a complete national register put DEFAULT_KWP_PER_M2_MODULE at 0.150 kWp/m² against the assumed 0.180, and settled that the below-floor capacity share is not a transferable constant.

The independent test of the 400 m² floor is also done, 2026-09-12: EarthPV's recall against the hand-mapped communes climbs with installation size while OpenPVMapper's, on the same truth at sub-metre resolution, does not, and retraining on French data does not move it. That is in the experiments register as the detection floor measured against sub-metre truth.

What that leaves genuinely open, and it is the part this project actually needs:

An external coverage_ratio reference below the floor, which is now the only part of this that France can still be asked for and probably cannot give. Both coverage_ratio and area_recall are still fit on Pakistani quadrats alone. The blocker named here originally, that the composites did not exist yet, is gone: the national compose finished 2026-09-05 and EarthPV has been run over the communes three times (v4, v5, v6). The area-recall-versus-size curve is measured, and is the floor result above.

What is left is coverage_ratio, which is a roofclf quantity: true mapped PV area over flagged roof area. France cannot supply it, because roofclf does not flag French roofs at a usable rate in the first place (0.627 AUC within size band, 16 buildings flagged out of 44,314 at precision 0.5). So this needs either a third country whose arrays are large against a pixel and which has both a register and hand-mapped truth, or more Pakistani quadrats in the sparse band. Adoption rates do not transfer, so a French coverage_ratio could never have been applied to Pakistan directly even if it existed.

A second uncensored pose source (item 11). OpenPVMapper carries tilt and azimuth per installation, derived from imagery rather than self-reported, over exactly the small installations MaStR omits. Not yet used, and worth doing: the glint-fitted pose work has no external check at all.

The trap has not gone away. OpenPVMapper is a model output with ~74-75% published precision, not ground truth. Measured against the hand-mapped communes it recalls 0.67 of real installations, and the fourteen communes are its own manual-correction layer, so its precision there is an optimistic bound. EarthPV agreeing with it would not be validation. The human sweeps are the part that can serve as truth.

5. The calibration quadrats are purposive, not a probability sample

This is the single largest caveat on every capacity number the project publishes, and no amount of additional modelling removes it. All 27 quadrats were chosen by a researcher to span landscape types, not drawn at random from a defined national frame, so the intervals on the evidence atlas are not a design-based margin of error and cannot be made into one by adding more purposive quadrats.

What would actually close it is a probability sample drawn from the national building-density frame, which does not exist yet. That is new mapping work rather than new code. A cheaper partial route, not yet tried, is model-assisted estimation: post-stratify the existing quadrats on auxiliary variables known for the whole population (building density, roof-size distribution, nightlights, region) to get a design-consistent national total with a variance estimate.

6. Ground-truth completeness is relative to the mapping imagery, not the satellite composite

Quadrats are mapped against OpenStreetMap's background imagery, whose capture date is generally older than the Sentinel-2 composite the model reads. So Rule 1 certifies completeness as of the mapping imagery, and the freshest installations cannot be mapped at all. Known bias directions: measured precision is a lower bound, base_rate is a lower bound, and rate_ratio is therefore an upper bound. Recall over mapped installations is unaffected.

The magnitude is bounded but not resolved. Pooled over 13 quadrats the effect is only 5.8% of apparent false positives, but per quadrat it dominates exactly where the sub-400 m² work lives: 68.4% in coastal Karachi, 23.7% in Quetta, 11.7% in Lahore. The imagery_layer/imagery_date fields exist in results/calibration_quadrats.csv and are still empty for all 27 quadrats, which is why the per-quadrat magnitude is unknown rather than merely unstated. Backfilling them against Esri Wayback is the cheap first step; Calibration imagery dating has the full costing. Another starting point is the Maxar Open Data Program, which provides dated imagery over several areas in Pakistan (see Open Issue #3)

7. Small ground-mounted installations have no instrument at all

Both sub-400 m² instruments are per-building classifiers: they score a footprint. A small free-standing ground array has no footprint to score, and the segmentation model was trained with everything below its floor burned as ignore, so it gets no signal there either. This is a distinct open gap rather than a smaller version of the rooftop one, and the partial mitigations that work for rooftops do not generalise to it even in principle.

It probably matters more in Pakistan than the size of the gap suggests: solar-powered irrigation tubewells are a large and fast-growing class of small ground-mounted arrays in Punjab and Sindh farmland. Note that the rural quadrats' near-zero PV findings do not bound this, because those are building-scoped measurements and a tubewell array sits on bare ground.

No longer just a plausible concern -- externally quantified, 2026-08-13. TransitionZero's satellite estimate (5.64 GW ground-mounted, mostly tube-wells) and PRIED's independent household survey (5.04 GW agricultural) converge to within 12% of each other despite disagreeing on the national total by ~6 GW -- and this project's own published ground-mount total, everything combined (utility, industrial, agricultural), is 2.2 GW. This is now the single largest, most confidently-explainable component of the gap to those external estimates.

Measured 2026-08-16, and the open part narrowed. The quadrats hold 262 strict sub-400 m² ground-mount installations, not the 15 a five-quadrat hand count found, and 98.5% of them sit within 30 m of a VIDA building (98.5% too on an independent national OSM sample, against 39.9% for random points in the same cells). So the candidate universe does not have to be invented: a 30 m ring around the building index roofclf already scores covers a tenth of the country and keeps the population. What does not work is detection at that unit, which lands under 1% precision at 30% recall in every stratum tested. What stays genuinely open is the size of the prize: a density-stratified estimate puts it at 1,492 MWp with a 90% interval of 435-3,526 MWp, and the whole spread sits in the < 150 bldg/km² stratum, which holds 22.9% of national buildings and is represented by four quadrats covering 16 km², all drawn around villages rather than in farmland. Mapping cropland quadrats is the step that resolves it. See Small ground-mounted instrument for the full measurements and what they rule in and out.

8. Manual review of the small size bins

In the 100 to 500 m² band, p_real is only pinned to [0.10, 0.89]. earthpv calibrate-sample emits a stratified sample of unmapped candidates for human verdicts, and roughly twenty verdicts per bin would collapse the widest remaining term in the calibration table.

9. The out-of-domain population cannot currently be validated by eye

The published Best estimate no longer carries this component: it was dropped 2026-08-15, precisely because it was the one part of the total that could not be validated where it was applied. The question it leaves open is not the headline figure but the coverage of the domain itself -- roughly a third of the country's cells (holding ~5% of its buildings) are outside it and are now reported as nothing rather than as an extrapolation. The JOSM validation batch built to test whether the domain could be widened with evidence (results/pakistan_roofclf_validation_outdomain/) turned out not to be reviewable, because the reference imagery there is too old to confirm or refute recently-installed small PV. Confirmed again 2026-08-17, this time on purpose-drawn quadrats rather than random review cells. Three low-density boxes were drawn specifically to widen the domain, mapped, and all three came back with zero installations against 83 roofclf-AND-SPPI flagged buildings and 475.4 kWp claimed. The imagery over all three is very old, so the zeros are uninterpretable in both directions, and none was registered: as confirmed zeros they would have moved the domain floor to 11.8 bldg/km2 (66.3% -> 95.6% of cells) and pulled the sparse band's coverage ratio toward zero on evidence that does not support it. See Calibration boxes's Box 17. The blocker is now known to apply to any box drawn in the sparse remainder, not just to the cells the random sampler happened to pick, so it gates the domain-widening programme itself.

This is item 6 blocking item 9. Contemporaneous high-resolution imagery would unblock both; free options worth checking first are Planet/NICFI monthly basemaps, which cover roughly to 30 degrees north and so include Sindh but not Punjab.

10. Two random-cell validation batches are generated but unreviewed

results/pakistan_roofclf_validation_domain/ and ..._outdomain/ were drawn to measure roofclf's precision against an unbiased population rather than the curated quadrats. Neither has been through JOSM review. The protocol is in roofclf random-cell validation, and the results belong in results/roofclf_random_validation_log.csv so precision against that population accumulates across batches.

11. MaStR carries an uncensored installed-pose distribution, which the glint model assumes

glint_opportunity.py states the problem in its own comments: the installed-pose prior has to be assumed, because this project's pose survey was fitted from observed glints, so every pose in it satisfies the glint condition by construction and it cannot supply an uncensored distribution. DEFAULT_POSE_PRIOR is therefore a guess, dict(tilt_mu=25.0, tilt_sd=8.0, az_mu=180.0, az_sd=25.0), and it propagates into expected_opportunities, the sensitivity inversion, and from there into p_unmapped.

Germany's register records pose at registration, independently of whether anything ever glinted. solar_extended carries Hauptausrichtung (azimuth), HauptausrichtungNeigungswinkel (tilt band), the same pair for a secondary orientation, and EinheitlicheAusrichtungUndNeigungswinkel (is the array single-pose at all). 97.4% of 4.44M rooftop units have azimuth and tilt; 225,138 have pose and coordinates (119,429 of them at or above 72 kWp), plus 14,100 ground-mount. For comparison, the existing German glint study covers 113 targets.

Checked against the assumed prior (2026-09-01), it is much too narrow in both axes:

Measured (Germany) DEFAULT_POSE_PRIOR
tilt > 40° 23.8% 3.04%
tilt mean / sd 32.2 / 12.6 25.0 / 8.0
azimuth >45° off south 15.7% 7.19%

It is also structurally unable to represent two real categories: 348,524 units are Ost-West (dual-pitch, two opposed azimuths, no single pose) and 3,775 are tracked. A unimodal Gaussian has nowhere to put either.

Three steps, cheapest first. (1) Swap the German distribution into expected_opportunities, which already takes the prior as an argument, and re-derive the figure the glint-date feature was rejected on -- "only 1-2% of a plausible installed population can ever satisfy the specular geometry on any single date" was computed under this prior, and a rejection resting on an assumed distribution should be re-checked against a measured one. It may well survive. (2) A direct per-target test: for the 119,429 units with pose and coordinates above the segmentation floor, run the glint detector and check whether the validation rate tracks predicted specular compatibility. That is a falsifiable test of the mechanism rather than another rate. (3) The same for the 14,100 ground-mount units, where tilt is far more uniform.

The blocking caveat is transfer. The prior is applied to Pakistan, where roofs are largely flat; Germany's steep tilts reflect pitched-roof construction, so these numbers do not correct the Pakistani prior -- the same non-transferability this project already documents for adoption rates. What they do establish is that a narrow Gaussian is the wrong shape for a national rooftop population, and they give one country's real distribution to test the machinery against. Lesser caveats: tilt is banded in ~20° steps where specular geometry is sensitive to a few degrees; 33.4% of geolocated units are flagged non-uniform; coordinates are address-level and pose is self-reported; and nothing exists below 30 kWp. See Validation against MaStR and Solar glint.

12. A per-locality pose calibration

The panel pose survey fits a national pose distribution from glint. Fitting a local pose from whatever installations a subdivision already has, rather than assuming a national standard, is untried and would improve the yield modelling that turns capacity into energy.

13. Sentinel-1 backscatter variance as a false-positive filter

Distinct from the corner-reflector hypothesis that failed. Multi-temporal backscatter variance separates permanent structures from seasonally changing fields, and greenhouse metal frames give a bright return, the opposite of PV. Cheap, and not ruled out by the negative result on corner reflection.

14. Per-pixel glint anomaly counting

The statistic the cell-aggregate glint test should have used. A 90th-percentile statistic over a whole cell only moves if roughly 10% of the cell brightens at once; scoring each pixel against its own baseline does not have that problem. Prototyped, never evaluated.

One piece of evidence now bears on it, and it cuts against the optimistic reading. A pixel-level PSF matched filter, the most favourable per-pixel statistic available since it is the optimal linear detector for a known shape, loses to the aggregate p98-minus-annulus statistic on verified positives and negatives. Per-pixel work is not automatically better than an aggregate here, because the glinting patch's shape and position are unknown and vary by date. A per-pixel counting statistic that does not assume a shape is still untested and remains the open item.

15. Glint scene coverage is silently incomplete

The tile-major glint fetch is 22x faster and numerically identical, but revalidation found that token expiry silently drops scenes for a large share of targets rather than erroring. The try/except fix moved the failure from loud to silent without removing it. Re-running the full study after a proper token-refresh fix is the only way to get a scene-count comparison that means what it appears to mean. See Glint tile-batched coverage.

16. Growth as a product

Per-epoch density estimates would make capacity a time series, so the 2022 to 2026 boom becomes measurable per district and independently checkable against NEPRA net-metering registrations and customs import series. A single growth map exists; an annual series does not (composites for 2022/23 to 2024/25 are being built).

The two-epoch diff itself failed its first sanity check and now carries a persistence gate (2026-08-20). Almost all PV detected pre-boom should still exist post-boom, yet only 26.6 percent of pre-boom segmentation candidates had any current-epoch detection within 50 m (11.4 percent of model-geometry ones; rooftop 86 percent, ground_adjacent 44 percent, no_building 5.6 percent). The vanishing mass was concentrated above latitude 34: winter snow in the 2021-10 to 2022-01 composite, scored by a precision calibration that never saw snow, and let through check-density by Gilgit-Baltistan's ratio-check exemption. It inflated the pre-boom level enough to turn the national ground-mount delta negative. growth.persistence_gate now drops pre-boom candidates with no current-epoch corroboration before the pre-boom level is recomputed (pre-boom segmentation 5,646.5 to 4,408.6 MWp, ground 1,253.0 to 346.5; national delta 3,497.5 to 4,415.5 MWp), with the ungated values kept per cell as *_preboom_ungated.

What remains open, in order of expected effect on the delta:

  • The gate is one-directional. Current-epoch-only false positives cannot be gated (they are indistinguishable from genuine new installations here) and are priced by p_real alone, so the residual false-positive bias on the delta is now upward where it was previously downward and much larger. A snow/SCL mask on the pre-boom composite, or a winter-free compose window, would remove the FP mode at the source instead of at the diff.
  • The roofclf halves are epoch-insensitive rather than unstable. They passed the same persistence check (94 percent of pre-boom flagged cells still flagged, 3 to 4 percent of capacity on dropped cells), but the pre-boom roofclf level reads about 79 percent of the current one for a boom that multiplied the true sub-400 stock. Much of roofclf's signal is adopter-propensity building appearance that predates the panels, so the sub-400 and >= 400 roofclf deltas are floors on true rooftop growth. Measuring roofclf's epoch sensitivity directly (score the same mapped quadrat buildings on both composites and compare against known install-date installations, e.g. from the step-change onset set) is the concrete next step.
  • No composed credible interval on the delta, and the pre-boom epoch has no calibration of its own, so every stated pre-boom level remains an artifact of current-epoch calibration transfer; only diffs of the fixed instrument are meaningful.

17. Nightlights as a substitute or complement to VIDA buildings

VIDA Open Buildings is a single load-bearing input across the pipeline: it is the unit roofclf classifies (every VIDA footprint gets a has-PV score), the population density.CALIBRATED_BLDG_DENSITY_KM2 stratifies by, the layer postprocess joins against for rooftop/ground/no-building placement, and the filter compose uses to decide which 0.1° cells even get composited. It carries three costs this project has already hit: it is a single present-day snapshot (a building built after the pre-boom epoch reads as bare ground in that epoch's table -- growth.persistence_gate, item 16 above), its footprints are imagery-derived and routinely undersized or a metre or two offset (the reason the parcel label's yard-overhang term exists, see The rooftop classifier), and its per-country availability is not guaranteed -- it had to be fetched separately for Germany (data/vida/DEU.parquet, 27.9M rows, 2026-08-23), which was one of three blockers to a MaStR-comparable atlas there (Validation against MaStR); the remaining blocker is mapped calibration quadrats, since the register cannot localise the sub-30 kWp population roofclf needs. VIIRS nightlights is free, global, monthly, and none of those three problems apply to it, which makes it worth asking whether it can substitute for VIDA anywhere, or at least flag where VIDA is deficient.

What already exists points away from a straightforward fix. scripts/build_pv_external_comparison.py correlates a 500 m VIIRS VNL v2.1 composite against per-cell Best estimate: Pearson r = 0.76 raw, but only r = 0.55 once log roof area is partialled out. Most of nightlights' apparent signal is therefore redundant with what VIDA-derived building density already supplies, not independent information a stratification model would gain by adding it. 2026-08-26: registered and rerun against the current 18,826.7 MWp atlas -- both numbers barely moved (0.548 to 0.545 partial) despite a 14% shift in the headline total since the script's only prior run (2026-08-12), so this looks like a stable property of the estimate's geography, not an artifact of one refit; full numbers, including the same result for the Relative Wealth Index, now live in Experiments. Read that alongside Capacity density's own three failed attempts at finding any coarse per-cell proxy (candidate density, roofclf's predicted rate, SPPI agreement rate) for which quadrat-regime a national cell resembles -- a fourth coarse proxy succeeding where three did not would be the surprise, not the default expectation.

Resolution rules it out as a footprint replacement outright. roofclf needs the footprint geometry itself, not just a settlement signal: about half of Pakistani VIDA buildings are already sub-pixel at Sentinel-2's 10 m GSD and fall back to a representative point, and VIIRS DNB's ~500 m pixel is 50x coarser again on a side (2,500x by area). A brighter or dimmer pixel does not tell postprocess which building under it is rooftop versus ground, or give sub400_capacity a footprint area to divide by. Any role nightlights can play is at the cell or region level, not the building level.

Where it could plausibly still help, none of it tried:

  • A VIDA-coverage plausibility check. A cell that is bright in VIIRS but has implausibly low VIDA building density is a candidate for "VIDA is missing structures here" -- the same failure mode Germany hits at the country level, just at cell granularity. Cheap to compute (VIDA building count per cell against zonal_ntl()'s already-implemented per-cell mean radiance), and it would slot into plausibility.py alongside the existing ground:rooftop and single-cell-concentration checks rather than needing a new stage.
  • Unblocking compose's cell-selection step for a country with no VIDA file. The "populated cells" filter currently reads VIDA building points; a country that fails scripts/new_region.py's check_vida gate has no path through that step today. A VIIRS-derived built-up mask is a strictly worse proxy for "has roofs" than building points, but it is better than no proxy at all -- it would only need to unblock imagery composition, not per-building scoring, which would still need VIDA or an equivalent footprint source before roofclf or sub400-capacity could run.
  • A staleness cross-check for the growth work. VIIRS's monthly cadence is the one input in this pipeline that is not a single snapshot. Comparing VIIRS radiance change against growth.py's persistence-gated delta (item 16 above), even coarsely, is an independent check on the direction of the 2022-2026 boom that costs nothing to compute against data already downloaded for the external-comparison script.

Concrete next step: test the VIDA-density-vs-nightlights ratio specifically as a check-density-style plausibility flag (the first bullet above), since that is the one candidate role where VIIRS's coarseness is not disqualifying and it is the one bullet above still untried. Item 4's post-stratification use should stay lower priority given the partial-correlation result, unless nightlight variability over time, rather than the mean level, turns out to behave differently from building density -- untested either way.

18. compose downloads 31 times more than it keeps (2026-09-19)

Measured on the running Nigerian compose, over four minutes of a healthy 6.8 MB/s link:

downloaded   410 MB per composited cell
written       13.4 MB per composited cell   (24,788 MB over 1,854 cells)

annual_composite pulls roughly twelve scenes by ten bands and keeps a per-pixel median, so about 31 of every 32 bytes crossing the link are discarded on arrival. For Nigeria's remaining 4,829 cells that is about 2 TB of transfer to produce 65 GB of composites: five days against under three hours at the same link speed.

This is the binding constraint on the whole programme, not a Nigeria problem. Compose is the only stage measured in days, every new country pays it in full, and the scaling guide names eight more countries. It is listed last here only because the ranking above weighs accuracy; on wall-clock per country it would sit near the top.

The fix is to composite next to the data and download only the result. Three routes, in increasing order of risk:

  • Run the existing code on a co-located VM. sentinel-cogs lives in AWS us-west-2 and Planetary Computer's blobs in Azure West Europe; same-region reads are free and fast. Nothing about the science changes: same imagery.py, same twelve-least-cloudy selection, same SCL classes, same Collection-1 offsets, byte-identical output. compose is already resumable per cell and EARTHPV_STAC_PROVIDER already pins the provider to match the region. Egress for a country is tens of GB.
  • openEO on the Copernicus Data Space Ecosystem, which is compositing as an API: send a process graph, download the result. Every user gets 10,000 free credits a month.
  • Google Earth Engine, same shape via COPERNICUS/S2_SR_HARMONIZED.

What makes the last two risky is specific to this project. annual_composite is not a generic median: twelve least-cloudy scenes, per-pixel SCL masking on classes ⅘/6/7, and the Collection-1 radiometric offset. The production checkpoint was fine-tuned on exactly that radiometry, and this repository has already discarded a set of composites built before the 2026-07-26 offset fix. A reimplementation that is subtly off produces imagery the model reads as domain shift, and nothing would surface it until the capacity numbers came out wrong.

Concrete next step, which is cheap and settles the risk: composite a few dozen cells of an existing AOI on a co-located VM, copy them back, and diff against the locally built ones. If they are byte-identical, route the next country that way and keep the recipe unchanged. Only if that is impractical is it worth porting the recipe to openEO, and then the same cell-level diff is the acceptance test.

Note that the obvious free option is gone: Microsoft retired the Planetary Computer Hub in June 2024 and archived its repository in June 2026. The STAC API and the data stay free, but there is no hosted notebook to run this in.

19. Temporal unmixing deserves one test on an annual stack (2026-09-19)

Temporal unmixing was rejected on the imagery this project composites, and the reason points at the imagery rather than the method. The estimator infers a pixel's PV fraction from how much its temporal spread is damped relative to its local background, so it needs a background that moves. Measured across 22 quadrats, the open background moves only 1.12x as much as a roof, and PV pixels vary 1.02x as much as PV-free roofs, which is no signal at all.

That is a property of the window, not of Pakistan. annual_composite takes a dry-season range of the twelve least-cloudy scenes precisely to suppress phenological and atmospheric variation, and a stable composite is the right choice for every other stage. Cropland in the monsoon is a different denominator entirely.

The test is cheap and well-defined, because the machinery exists: scene_stack and scripts/compose_scene_stacks.py already build the per-scene cube, and scripts/run_temporal_unmix_ablation.py already prices the block with a conditional test against measured background dynamism. Re-fetch the stacks for a handful of the most agricultural quadrats (Muzaffargarh, Bahawalnagar, Khairpur, Nasirabad, Malok) with --window spanning a full year, re-run the invariance diagnostic first, and only continue if open_over_roof clears something like 2x. If it does, the conditional result already measured (Spearman +0.401, p=0.071; the dynamic half gained in 9 of 11 folds) says where to look.

Two cautions carried from the rejected run. The gain in the dynamic half was +0.0015 AUC, so even a favourable window has to move this by an order of magnitude to matter. And an annual stack reintroduces everything the dry-season window exists to exclude, monsoon cloud above all, so the scene count per pixel will fall and n_obs has to be watched.

20. A gradient-boosted roofclf calibrates better; price it end to end (2026-09-19)

The model-class test found the two objectives pulling apart. Against the shipped linear model, gradient boosting ranks worse (-0.023 AUC within size band, better in 4 of 30 folds, p < 0.001) and calibrates better (median per-quadrat adoption-rate error 0.0308 to 0.0165, closer in 20 of 29 quadrats, Wilcoxon p = 0.006), while keeping the full dispersion of true rates (0.0806 against 0.0798) rather than hedging toward the mean.

That is not enough to change the classifier, because the two halves of the product want different things. Per-building ranking drives the MapRoulette leads and the precision-thresholded population the coverage ratio is fitted on; the per-cell adoption rate drives the capacity estimate that PyPSA consumes. The linear model is better at the first and worse at the second.

Update 2026-09-19: the cheap substitute does not work. A monotone recalibration cannot change a ranking, so if the gradient-boosted model were merely better calibrated, isotonic or Platt on the linear model's scores would capture the gain for nothing. Measured: the best variant (isotonic within roof-area terciles) recovers 38% of it at Wilcoxon p = 0.069, and Platt recovers none by construction. The advantage is a reordering, not a calibration curve, so it has to be deployed rather than approximated. An area-weighted loss was tested in the same run and is not the answer either: it improves the area ratio it optimises (1.084 to 1.035) while losing 0.0111 AUC and worsening the count rate error to 0.0395.

What would settle it is pricing a GBM through the capacity chain rather than the AUC table: fit it on the same 30 quadrats, run sub400-capacity and ge400-roof-capacity with the same coverage-ratio and area-recall machinery, and compare the per-cell capacity against the quadrats it was not fitted on. If the better rate calibration survives that, it is worth the deployment work; if the coverage ratio simply absorbs the difference, it is not.

Two obstacles are known. save_model/score_buildings_national assume a linear model serialised as coefficients, so national scoring needs real plumbing rather than a swap. And fit_logistic leaves its intercept unpenalised specifically so the fitted base rate is not shrunk, which is a deliberate calibration choice a tree ensemble makes differently -- any comparison has to keep that in view rather than treating the two as interchangeable.

21. A second imagery epoch is the only new information left; is it worth the download? (2026-09-20)

The pre-boom epoch difference is the one intervention in the 2026-09-19/20 sweep that adds INFORMATION rather than removing noise, and its controls hold: a split-half placebo is null, the difference beats the undifferenced levels 6x, and the change is a darkening concentrated in the bulk rather than in the extreme tail, so it is PV and not construction.

What is open is whether it is worth deploying, and the honest number is the small one. Against the shipped read path it is +0.0234 within size band; against the noise-reduced one it is +0.0067 (23 of 29 folds, p = 0.002), because four fifths of it is already captured by the trimmed mean and area-weighted zonal means. A project that ships the reducer package gets under a hundredth of AUC from a second epoch, at the cost of compositing the country twice. The depth question is closed: doubling the old epoch's frames (median 7 to 16) moved it 0.0047 to 0.0067 and the curve has not flattened, so a deeper old epoch is the cheapest remaining improvement. Two things remain open:

  1. Does it survive the capacity chain? The deployment framing on the shipped base is +3.2 points of recall at precision 0.500 (0.6298 to 0.6621). The same measurement has not been made on the noise-reduced base, where the AUC gain is 5x smaller.
  2. Does it transfer? Pakistan's boom is post-2022, which is what makes a 2019 composite a clean pre-installation look. A country whose PV predates the archive has no such epoch, and Germany's register-calibrated half has no quadrats to measure it on, so this is a Pakistan-shaped result until something tests it elsewhere.

Cost note: the stacks here are quadrat-clipped (223 MB for 31 boxes). A national second epoch is the same order as the recomposite item 18 prices, and unlike that one it does not replace what already exists, so both epochs have to be kept.

Known defects carried on purpose

These are understood, measured, and currently accepted rather than pending.

The plausibility gate reports three failures on the published state. Khyber Pakhtunkhwa, Balochistan and Islamabad Capital Territory fail the single-cell-concentration check. Checked rather than assumed: all three flagged cells are the calibration quadrats' own cities, and the failures appeared only because shrinking a real ground-mount overstatement mechanically raised the visible concentration share of whatever legitimate signal remained. Published anyway, following this project's own precedent for a checked-genuine plausibility failure. Gilgit-Baltistan is separately exempted from the ground-to-rooftop ratio check, because its real rooftop base rate is near zero and the ratio is structurally uninformative there.

buildings.geoparquet's rooftop sum is not the region total's rooftop component. This is structural, not a bug: the region total sums each rooftop-placed candidate's full polygon once, while the per-building table credits each building only with its actual geometric intersection. Measured at a 46.4% gap nationally, and confirmed by the recorded building_overlap_frac. Any per-building disaggregation built from that file is a conservative roof-anchored floor and should not be expected to sum back.

Buildings with no valid composite pixel score NaN rather than being scored. The cell-edge fix excludes nodata fill from the zonal statistics, which is correct, but it leaves 0.5 to 3.0% of buildings in tile-overlap strips unscored. A targeted re-read from the correct neighbouring tile would rescue them; it is not implemented, and the current behaviour is the safe direction (nothing unscored can clear a threshold).

Two segmentation checkpoints no longer exist on disk. v2_combined and v3_combined_india were deleted at some point after producing their outputs. The Gujarat atlas is therefore built from a checkpoint that can no longer be verified against its own weights, which is stated on that page. Nothing recovers this except retraining.