Add receiver health UX, fix restart crash, add interface scan selector

Receiver management improvements for the admin Receivers tab:

- Per-receiver health: server now tracks each connection's state
  (connecting/connected/down), message count, and last-message time, exposed
  via new GET /api/admin/receiver-status and POST /api/admin/test-receivers.
- Receivers tab redesign: live "Active Receivers" panel with colored
  indicators (green=receiving, yellow=connected/no data, red=unreachable),
  a clear select -> SAVE & APPLY -> auto-test flow, and selectable scan
  results with a "USE SELECTED RECEIVERS" action.
- Network scan interface selector: scan a chosen interface/subnet instead of
  every local subnet. Oversized subnets (>/20, e.g. a docker /16) are disabled
  in the UI and rejected server-side, eliminating slow/hanging scans.

Critical fix — server crash on receiver restart:

- Receiver connection tasks were awaited inside main()'s top-level
  asyncio.gather. Restarting receivers cancels those tasks, and the resulting
  CancelledError propagated into the gather and killed the whole server
  process (every action after a restart then failed). Receiver tasks now run
  as independent background tasks; restart cancels and awaits them with
  return_exceptions=True so cancellation can never tear down the server.

Also cache-bust admin.css/admin.js/pixel-editor.js (?v=) so updated assets
load without a manual hard-refresh, and surface a clear message instead of a
silent "Loading…" when the status endpoint is unavailable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
root
2026-06-09 12:49:45 -07:00
parent 214ee8dd90
commit 7a80057cb5
4 changed files with 480 additions and 35 deletions
+93
View File
@@ -592,6 +592,99 @@ textarea {
font-size: 11px;
}
/* ===================================================================
Receivers tab — status indicators & selectable scan results
=================================================================== */
.rx-section-label {
font-family: 'Press Start 2P', monospace;
font-size: 10px;
color: #fcd444;
margin: 22px 0 12px;
padding-bottom: 6px;
border-bottom: 1px solid rgba(92, 148, 252, 0.3);
}
.rx-summary {
font-family: 'Courier New', monospace;
font-size: 12px;
color: #b4d4ec;
}
#receiver-status-list {
display: flex;
flex-direction: column;
gap: 8px;
margin-bottom: 12px;
}
.rx-status-item {
display: flex;
align-items: center;
gap: 10px;
background: rgba(0, 0, 0, 0.3);
border: 1px solid rgba(92, 148, 252, 0.4);
border-radius: 6px;
padding: 8px 12px;
font-size: 13px;
flex-wrap: wrap;
}
.rx-dot {
width: 12px;
height: 12px;
border-radius: 50%;
flex-shrink: 0;
box-shadow: 0 0 6px currentColor;
}
.rx-dot.green { background: #54fc54; color: #54fc54; }
.rx-dot.yellow { background: #fcd444; color: #fcd444; }
.rx-dot.red { background: #fc5454; color: #fc5454; }
.rx-dot.gray { background: #888; color: #888; box-shadow: none; }
.rx-ip {
font-family: 'Courier New', monospace;
font-weight: bold;
color: #fcfcfc;
min-width: 110px;
}
.rx-label {
font-family: 'Press Start 2P', monospace;
font-size: 8px;
}
.rx-green { color: #54fc54; }
.rx-yellow { color: #fcd444; }
.rx-red { color: #fc5454; }
.rx-gray { color: #888; }
.rx-detail {
color: #b4d4ec;
font-size: 11px;
margin-left: auto;
}
.rx-scan-item {
display: flex;
align-items: center;
gap: 10px;
padding: 6px 4px;
cursor: pointer;
border-bottom: 1px solid rgba(92, 148, 252, 0.2);
}
.rx-scan-item input[type="checkbox"] {
width: 16px;
height: 16px;
accent-color: #fcd444;
cursor: pointer;
}
.rx-scan-ip {
font-family: 'Courier New', monospace;
font-weight: bold;
color: #fcfcfc;
}
/* ===================================================================
Pixel Editor
=================================================================== */
+44 -12
View File
@@ -7,7 +7,7 @@
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Press+Start+2P&display=swap" rel="stylesheet">
<link rel="stylesheet" href="/admin/admin.css">
<link rel="stylesheet" href="/admin/admin.css?v=20260609b">
</head>
<body>
<!-- Login Screen -->
@@ -76,37 +76,69 @@
<!-- Receivers Tab -->
<section id="tab-receivers" class="tab-pane hidden">
<h2>Receiver Configuration</h2>
<div id="network-info-box" class="info-box">
<h3>Host Network</h3>
<div id="network-info">Loading...</div>
<!-- Live status of the receivers the app is currently using -->
<div id="receiver-status-box" class="info-box">
<h3>Active Receivers <span id="receiver-status-summary" class="rx-summary"></span></h3>
<div id="receiver-status-list">Loading...</div>
<div class="button-row">
<button id="test-receivers-btn" class="btn">TEST CONNECTIONS</button>
<button id="restart-receivers-btn" class="btn">RESTART CONNECTIONS</button>
</div>
<p class="hint">🟢 receiving data &nbsp; 🟡 connected, no data yet &nbsp; 🔴 unreachable</p>
</div>
<!-- Step 1: choose how receivers are selected -->
<div class="rx-section-label">1. Select Receivers</div>
<div class="form-group">
<label>Discovery Mode</label>
<select id="receiver-mode">
<option value="AUTO">AUTO (scan network)</option>
<option value="MANUAL">MANUAL (specify IPs)</option>
<option value="AUTO">AUTO (use all receivers found on the network)</option>
<option value="MANUAL">MANUAL (use only the IPs I specify)</option>
</select>
</div>
<div id="manual-ips-group" class="form-group hidden">
<label>Receiver IPs (one per line)</label>
<textarea id="receiver-ips-input" rows="4" placeholder="192.168.1.100&#10;192.168.1.101"></textarea>
<span class="hint">The app connects to every IP listed here.</span>
</div>
<div class="form-group">
<label>Receiver Port</label>
<input type="number" id="receiver-port" value="30003" min="1" max="65535">
</div>
<!-- Step 2: save + apply -->
<div class="rx-section-label">2. Save &amp; Apply</div>
<div class="button-row">
<button id="save-receivers-btn" class="btn btn-primary">SAVE &amp; APPLY</button>
</div>
<p class="hint">Saves your selection, reconnects immediately, then re-tests the receivers below.</p>
<!-- Find receivers on the network -->
<div class="rx-section-label">Find Receivers on Network</div>
<div class="form-group">
<label>Custom Subnet (optional)</label>
<input type="text" id="scan-subnet" placeholder="e.g. 10.0.0.0/24 (leave empty to scan all local subnets)">
<label>Interface / Subnet to Scan</label>
<select id="scan-interface"></select>
<span class="hint">Pick a network interface to scan, or scan all local subnets.</span>
</div>
<div class="form-group">
<label>Custom Subnet (optional override)</label>
<input type="text" id="scan-subnet" placeholder="e.g. 10.0.0.0/24 — overrides the selection above">
</div>
<div class="button-row">
<button id="scan-receivers-btn" class="btn">SCAN NOW</button>
<button id="restart-receivers-btn" class="btn">RESTART CONNECTIONS</button>
<button id="save-receivers-btn" class="btn btn-primary">SAVE</button>
</div>
<div id="scan-results" class="info-box hidden">
<h3>Scan Results</h3>
<div id="scan-results-list"></div>
<div class="button-row" id="scan-use-row" style="display:none">
<button id="use-selected-btn" class="btn btn-primary">USE SELECTED RECEIVERS</button>
</div>
</div>
<div id="network-info-box" class="info-box">
<h3>Host Network</h3>
<div id="network-info">Loading...</div>
</div>
</section>
@@ -265,7 +297,7 @@
<div id="toast" class="toast hidden"></div>
</div>
<script src="/admin/pixel-editor.js"></script>
<script src="/admin/admin.js"></script>
<script src="/admin/pixel-editor.js?v=20260609b"></script>
<script src="/admin/admin.js?v=20260609b"></script>
</body>
</html>
+225 -18
View File
@@ -78,6 +78,12 @@
themesManagerLoaded = true;
loadThemeManager();
}
// Live receiver health polling only while the Receivers tab is open
if (tab.dataset.tab === 'receivers') {
startReceiverStatusPolling();
} else {
stopReceiverStatusPolling();
}
});
});
@@ -191,11 +197,44 @@
});
html += '</table>';
el.innerHTML = html;
populateScanInterfaces(ifaces);
} catch (e) {
el.textContent = 'Error loading network info';
}
}
// Build the "Interface / Subnet to Scan" dropdown from the host interfaces.
// Subnets larger than /20 (>4094 hosts) are disabled — the server rejects
// them anyway, and scanning e.g. a docker /16 is impractically slow.
const MAX_SCAN_HOSTS = 4094;
function populateScanInterfaces(ifaces) {
const sel = document.getElementById('scan-interface');
if (!sel) return;
sel.innerHTML = '';
const allOpt = document.createElement('option');
allOpt.value = '';
allOpt.textContent = 'All local subnets (slower)';
sel.appendChild(allOpt);
let firstScannable = null;
ifaces.forEach(i => {
const prefix = parseInt((i.network.split('/')[1]) || '0', 10);
const hosts = Math.max(0, Math.pow(2, 32 - prefix) - 2);
const opt = document.createElement('option');
opt.value = i.network;
const tooBig = hosts > MAX_SCAN_HOSTS;
opt.textContent = `${i.name}${i.network} (${hosts} hosts)` + (tooBig ? ' — too large' : '');
opt.disabled = tooBig;
sel.appendChild(opt);
if (!tooBig && firstScannable === null) firstScannable = i.network;
});
// Default to the first reasonably-sized interface for a fast scan.
if (firstScannable !== null) sel.value = firstScannable;
}
// ------- Receivers -------
function toggleManualIps(mode) {
document.getElementById('manual-ips-group').classList.toggle('hidden', mode === 'AUTO');
@@ -210,7 +249,10 @@
btn.textContent = 'SCANNING...';
btn.disabled = true;
try {
const subnet = (document.getElementById('scan-subnet').value || '').trim();
// Custom subnet text overrides the interface dropdown selection.
const custom = (document.getElementById('scan-subnet').value || '').trim();
const iface = document.getElementById('scan-interface').value || '';
const subnet = custom || iface;
const fetchOpts = { method: 'POST' };
if (subnet) {
fetchOpts.headers = { 'Content-Type': 'application/json' };
@@ -218,16 +260,7 @@
}
const res = await fetch('/api/admin/scan-receivers', fetchOpts);
const data = await res.json();
const box = document.getElementById('scan-results');
const list = document.getElementById('scan-results-list');
box.classList.remove('hidden');
if (data.error) {
list.textContent = data.error;
} else if (data.receivers && data.receivers.length) {
list.textContent = data.receivers.join(', ');
} else {
list.textContent = 'No receivers found on network';
}
renderScanResults(data);
} catch (e) {
toast('Scan failed', true);
}
@@ -235,17 +268,77 @@
btn.disabled = false;
});
// Render scan results as a selectable checklist.
function renderScanResults(data) {
const box = document.getElementById('scan-results');
const list = document.getElementById('scan-results-list');
const useRow = document.getElementById('scan-use-row');
box.classList.remove('hidden');
list.innerHTML = '';
if (data.error) {
list.textContent = data.error;
useRow.style.display = 'none';
return;
}
const found = data.receivers || [];
if (!found.length) {
list.textContent = 'No receivers found on network';
useRow.style.display = 'none';
return;
}
// Pre-check IPs already in the manual list.
const current = new Set(document.getElementById('receiver-ips-input').value
.split('\n').map(s => s.trim()).filter(Boolean));
found.forEach(ip => {
const row = document.createElement('label');
row.className = 'rx-scan-item';
row.innerHTML = `
<input type="checkbox" value="${ip}" ${current.has(ip) ? 'checked' : ''}>
<span class="rx-scan-ip">${ip}</span>`;
list.appendChild(row);
});
useRow.style.display = '';
}
// "USE SELECTED" -> switch to MANUAL, populate IPs, save & apply.
document.getElementById('use-selected-btn').addEventListener('click', async () => {
const checked = Array.from(
document.querySelectorAll('#scan-results-list input[type="checkbox"]:checked')
).map(cb => cb.value);
if (!checked.length) { toast('Select at least one receiver', true); return; }
document.getElementById('receiver-mode').value = 'MANUAL';
toggleManualIps('MANUAL');
document.getElementById('receiver-ips-input').value = checked.join('\n');
await saveAndApplyReceivers();
});
document.getElementById('restart-receivers-btn').addEventListener('click', async () => {
const btn = document.getElementById('restart-receivers-btn');
const orig = btn.textContent;
btn.textContent = 'RESTARTING…';
btn.disabled = true;
try {
const res = await fetch('/api/admin/restart-receivers', { method: 'POST' });
if (res.ok) toast('Receivers restarted');
if (res.ok) { toast('Receivers restarted'); await testReceivers(); }
else toast('Failed to restart', true);
} catch (e) {
toast('Error restarting receivers', true);
}
btn.textContent = orig;
btn.disabled = false;
});
document.getElementById('save-receivers-btn').addEventListener('click', async () => {
document.getElementById('save-receivers-btn').addEventListener('click', saveAndApplyReceivers);
// Save the receiver selection, reconnect, then re-test — the full
// select -> save -> verify loop in one action.
async function saveAndApplyReceivers() {
const btn = document.getElementById('save-receivers-btn');
const orig = btn.textContent;
const mode = document.getElementById('receiver-mode').value;
const port = parseInt(document.getElementById('receiver-port').value) || 30003;
const body = { receiver_port: port };
@@ -255,11 +348,122 @@
} else {
const ips = document.getElementById('receiver-ips-input').value
.split('\n').map(s => s.trim()).filter(Boolean);
body.receivers = ips.length ? ips : 'AUTO';
if (!ips.length) { toast('Enter at least one receiver IP', true); return; }
body.receivers = ips;
}
await saveConfig(body);
});
btn.textContent = 'APPLYING…';
btn.disabled = true;
try {
const saved = await saveConfig(body, true);
if (saved) {
// Apply immediately so the active list reflects the new selection.
await fetch('/api/admin/restart-receivers', { method: 'POST' });
await testReceivers();
toast('Saved & applied — testing connections');
}
} catch (e) {
toast('Failed to apply', true);
}
btn.textContent = orig;
btn.disabled = false;
}
// ------- Receiver status / health -------
let receiverStatusInterval = null;
function startReceiverStatusPolling() {
loadReceiverStatus();
if (!receiverStatusInterval) {
receiverStatusInterval = setInterval(loadReceiverStatus, 4000);
}
}
function stopReceiverStatusPolling() {
if (receiverStatusInterval) {
clearInterval(receiverStatusInterval);
receiverStatusInterval = null;
}
}
async function loadReceiverStatus() {
const list = document.getElementById('receiver-status-list');
try {
const res = await fetch('/api/admin/receiver-status');
if (!res.ok) {
// 404 here almost always means the server is running an older
// build without this endpoint — tell the user instead of
// leaving a silent "Loading…".
if (res.status === 404) {
list.innerHTML = '<span class="no-sprite">Status endpoint not found — ' +
'restart the server to load the latest build.</span>';
}
return;
}
const data = await res.json();
renderReceiverStatus(data.receivers || [], data.configured);
} catch (e) {
// Only replace the initial placeholder; don't clobber good data on
// a transient poll failure.
if (/Loading/.test(list.textContent)) {
list.innerHTML = '<span class="no-sprite">Could not reach server.</span>';
}
}
}
document.getElementById('test-receivers-btn').addEventListener('click', testReceivers);
async function testReceivers() {
const btn = document.getElementById('test-receivers-btn');
const orig = btn.textContent;
btn.textContent = 'TESTING…';
btn.disabled = true;
try {
const res = await fetch('/api/admin/test-receivers', { method: 'POST' });
if (res.ok) {
const data = await res.json();
renderReceiverStatus(data.receivers || [], undefined, true);
} else {
toast('Test failed', true);
}
} catch (e) {
toast('Test error', true);
}
btn.textContent = orig;
btn.disabled = false;
}
function renderReceiverStatus(statuses, configured, tested) {
const list = document.getElementById('receiver-status-list');
const summary = document.getElementById('receiver-status-summary');
if (configured !== undefined) {
const modeLabel = (configured === 'AUTO') ? 'AUTO' : 'MANUAL';
summary.textContent = `(${statuses.length}${modeLabel})`;
}
if (!statuses.length) {
list.innerHTML = '<span class="no-sprite">No receivers configured. ' +
'Use AUTO mode or add IPs below, then SAVE &amp; APPLY.</span>';
return;
}
list.innerHTML = '';
statuses.forEach(s => {
const row = document.createElement('div');
row.className = 'rx-status-item';
const detail = s.last_msg_age != null
? `${s.msg_count} msgs · last ${s.last_msg_age}s ago`
: (s.state === 'down' ? (tested ? 'no response on port' : 'not connected')
: 'waiting for data…');
row.innerHTML = `
<span class="rx-dot ${s.color}"></span>
<span class="rx-ip">${s.ip}</span>
<span class="rx-label rx-${s.color}">${s.label}</span>
<span class="rx-detail">${detail}</span>`;
list.appendChild(row);
});
}
// ------- Location -------
document.getElementById('browser-location-btn').addEventListener('click', () => {
@@ -580,7 +784,7 @@
}
// ------- Helpers -------
async function saveConfig(body) {
async function saveConfig(body, silent) {
try {
const res = await fetch('/api/admin/config', {
method: 'PUT',
@@ -588,13 +792,16 @@
body: JSON.stringify(body)
});
if (res.ok) {
toast('Settings saved');
if (!silent) toast('Settings saved');
return true;
} else {
const data = await res.json();
toast(data.error || 'Save failed', true);
return false;
}
} catch (e) {
toast('Connection error', true);
return false;
}
}
+118 -5
View File
@@ -170,6 +170,69 @@ receivers: List[str] = []
receiver_tasks: List[asyncio.Task] = []
emitter_categories: Dict[str, str] = {} # ICAO hex -> ADS-B emitter category (A1, A3, A7, etc.)
# Live per-receiver connection health, keyed by IP. Each entry:
# {"state": "connecting"|"connected"|"down", "last_msg": ts|None,
# "msg_count": int, "connected_since": ts|None, "error": str}
receiver_status: Dict[str, dict] = {}
# A receiver is considered "receiving" if it produced a message this recently.
RECEIVER_DATA_FRESH_SECONDS = 30
def reset_receiver_status(ips):
"""Replace the tracked receiver set, marking each as connecting."""
global receiver_status
receiver_status = {
ip: {"state": "connecting", "last_msg": None,
"msg_count": 0, "connected_since": None, "error": ""}
for ip in ips
}
def describe_receiver(ip, reachable=None):
"""Build a display-friendly status object for one receiver.
state -> (label, color):
down/missing -> DOWN (red)
connecting -> CONNECTING (gray)
connected + data -> RECEIVING (green)
connected, no data -> NO DATA (yellow) (socket up; maybe no aircraft in range)
reachable: optional active port-probe result that overrides a stale state.
"""
st = receiver_status.get(ip)
now = time.time()
if st is None:
state, last_age, msg_count = "down", None, 0
else:
state = st["state"]
last_age = (now - st["last_msg"]) if st["last_msg"] else None
msg_count = st["msg_count"]
# An active probe result is authoritative about reachability.
if reachable is False:
state = "down"
elif reachable is True and state == "down":
state = "connected"
if state == "down":
label, color = "DOWN", "red"
elif state == "connecting":
label, color = "CONNECTING", "gray"
elif last_age is not None and last_age <= RECEIVER_DATA_FRESH_SECONDS:
label, color = "RECEIVING", "green"
else:
label, color = "NO DATA", "yellow"
return {
"ip": ip,
"state": state,
"label": label,
"color": color,
"msg_count": msg_count,
"last_msg_age": round(last_age, 1) if last_age is not None else None,
"reachable": reachable,
}
@dataclass
class SBSMessage:
@@ -246,10 +309,17 @@ async def connect_to_receiver(host: str, port: int = 30003):
"""Connect to an SBS receiver and process messages"""
reconnect_delay = config['tuning']['receiver_reconnect_seconds']
print(f"Connecting to receiver at {host}:{port}...")
# Ensure a status entry exists even if this task started outside a reset.
receiver_status.setdefault(host, {"state": "connecting", "last_msg": None,
"msg_count": 0, "connected_since": None, "error": ""})
while True:
try:
receiver_status[host]["state"] = "connecting"
reader, writer = await asyncio.open_connection(host, port)
print(f"Connected to {host}:{port}")
receiver_status[host]["state"] = "connected"
receiver_status[host]["connected_since"] = time.time()
receiver_status[host]["error"] = ""
while True:
line = await reader.readline()
@@ -259,6 +329,8 @@ async def connect_to_receiver(host: str, port: int = 30003):
line = line.decode('utf-8', errors='ignore')
msg = parse_sbs_message(line)
if msg and msg.icao:
receiver_status[host]["last_msg"] = time.time()
receiver_status[host]["msg_count"] += 1
if msg.icao not in flights:
flights[msg.icao] = {
'icao': msg.icao,
@@ -295,10 +367,14 @@ async def connect_to_receiver(host: str, port: int = 30003):
flight['ground'] = msg.ground
flight['last_seen'] = time.time()
# readline returned empty -> peer closed the connection.
receiver_status[host]["state"] = "down"
writer.close()
await writer.wait_closed()
except Exception as e:
print(f"Receiver {host}:{port} error: {e}")
receiver_status[host]["state"] = "down"
receiver_status[host]["error"] = str(e)
await asyncio.sleep(reconnect_delay)
@@ -992,10 +1068,15 @@ async def restart_receiver_connections():
"""Cancel existing receiver tasks and start new ones."""
global receivers, receiver_tasks
# Cancel old tasks
for task in receiver_tasks:
task.cancel()
# Cancel old tasks and let them unwind cleanly. Awaiting the cancelled
# tasks here (with return_exceptions) absorbs their CancelledError so it
# can never propagate into the main event loop.
old_tasks = list(receiver_tasks)
receiver_tasks.clear()
for task in old_tasks:
task.cancel()
if old_tasks:
await asyncio.gather(*old_tasks, return_exceptions=True)
# Determine receivers
receiver_config = config["receivers"]
@@ -1015,6 +1096,9 @@ async def restart_receiver_connections():
print(f"Error determining receivers: {e}")
receivers = []
# Reset health tracking to the new receiver set.
reset_receiver_status(receivers)
# Start new tasks
for r in receivers:
task = asyncio.create_task(connect_to_receiver(r, receiver_port))
@@ -1030,6 +1114,28 @@ async def handle_admin_restart_receivers(request):
return web.json_response({"ok": True, "receivers": receivers})
@require_auth
async def handle_admin_receiver_status(request):
"""GET /api/admin/receiver-status - Live health of configured receivers."""
statuses = [describe_receiver(ip) for ip in receivers]
return web.json_response({
"receivers": statuses,
"configured": config["receivers"], # "AUTO" or a list of IPs
"count": len(receivers),
})
@require_auth
async def handle_admin_test_receivers(request):
"""POST /api/admin/test-receivers - Actively probe each configured receiver
and report reachability combined with live data-flow state."""
port = config["receiver_port"]
probes = await asyncio.gather(*[test_port(ip, port) for ip in receivers])
statuses = [describe_receiver(ip, reachable=reachable)
for ip, reachable in zip(receivers, probes)]
return web.json_response({"receivers": statuses, "count": len(receivers)})
# ---------------------------------------------------------------------------
# Static file serving with setup redirect
# ---------------------------------------------------------------------------
@@ -1103,6 +1209,8 @@ async def start_http_server():
app.router.add_get('/api/admin/network-info', handle_admin_network_info)
app.router.add_post('/api/admin/scan-receivers', handle_admin_scan_receivers)
app.router.add_post('/api/admin/restart-receivers', handle_admin_restart_receivers)
app.router.add_get('/api/admin/receiver-status', handle_admin_receiver_status)
app.router.add_post('/api/admin/test-receivers', handle_admin_test_receivers)
app.router.add_get('/api/admin/sprites', handle_admin_sprites)
app.router.add_post('/api/admin/sprites/{type}', handle_admin_sprite_upload)
@@ -1148,6 +1256,8 @@ async def main():
else:
print(f"Using {len(receivers)} receiver(s)")
reset_receiver_status(receivers)
# Start all tasks
tasks = [
start_http_server(),
@@ -1156,12 +1266,15 @@ async def main():
poll_aircraft_json(),
]
# Connect to all receivers
# Connect to all receivers. These run as independent background tasks and
# are deliberately NOT awaited in the gather below: restarting receivers
# cancels them, and a cancelled task inside the main gather would tear down
# the whole server. They still run as long as the event loop is alive.
for receiver in receivers:
task = asyncio.ensure_future(connect_to_receiver(receiver, receiver_port))
receiver_tasks.append(task)
tasks.append(task)
# Only the core services keep the server alive.
await asyncio.gather(*tasks)