lobstersdevtools
Exercises in benchmarking and evals, part 7: DeepSWE, Senior SWE-Bench, napkin math, and winter tires
Benchmarking methods
CoolingWhy this scored 20
every term, weightedVelocity+10.3 / 40 max
Engagement per hour, normalized against the fastest item on its own source
Acceleration+0.0 / 25 max
Whether that velocity is itself speeding up, as a per-hour rate
Cross-source spread+0.0 / 25 max
How many independent communities are talking about the same entity
Recency+10.0 / 10 max
Decays to zero over 14 days
Saturation penalty−0.0 / 30 max
Subtracted once something is big and old — we rank what's next, not what's peaked
Composite20.2
Weights are hand-tuned, not learned — we're calibrating them against realized trends as history accumulates. On a topic's first sighting there's no previous reading to compare against, so acceleration starts from a neutral prior rather than a measurement. Full methodology
Signal history
7-day window (free)Entities
deepswe
Embed a live signal badge
[](https://www.signalcrest.app/topic/lob%3Atzr3le)
Drop this in a README or blog post — it updates automatically as the score moves.