Most efficient way to extract one field from a JSON string column in Spark (8–10B rows)
Efficient technique to extract a field from massive JSON strings in Spark SQL.
Why this scored 17
every term, weightedEngagement gained per hour since the last capture, against the fastest item on its own source
Whether that velocity is itself speeding up, as a per-hour rate
How many independent communities are talking about the same entity
Decays to zero over 14 days
Subtracted once something is big and old — we rank what's next, not what's peaked
Weights are hand-tuned, not learned — we're calibrating them against realized trends as history accumulates. On a topic's first sighting there's no previous reading to compare against, so acceleration starts from a neutral prior rather than a measurement, and velocity falls back to engagement over its whole lifetime until a second reading exists. Full methodology
Outlook
low confidence · estimate, not a guarantee7-day
~0
range 0–12
14-day
~0
range 0–15
30-day
~0
range 0–20
Signal history
7-day window (free)projected trajectory (estimate, not a guarantee)
Embed a live signal badge
[](https://www.signalcrest.app/topic/so%3A80006817)
Drop this in a README or blog post — it updates automatically as the score moves.