Alert-Type Decision Ledger: Test a Service Before Paying
A reusable evaluation ledger for price, news, scanner, publisher and copy alerts, with separate evidence for delivery and execution.
Choose the unit before you count it
An alert is not one standard product. A price alert reports that a user-defined threshold fired. A news alert reports an event. A scanner lists candidates. A publisher call proposes a position. A copy instruction may place an order through an integration. Each moves a different part of the decision away from the reader. Before a trial, write one sentence describing exactly what the service promises to deliver and what you still decide yourself.
Do not score a threshold crossing as a winning trade, or a scanner hit as a completed recommendation. A comparison is meaningful only after the product type and its denominator are fixed. A provider selling more than one type needs a separate ledger for each stream.
Build five columns
For every alert in a predeclared sample, record: trigger (rule or source document), publication (original message and provider timestamp), delivery (device or webhook receipt time), actionability (first executable quote, spread, order status), and outcome (the stated scoring rule and later revisions). Keep the raw message, not just a screenshot of a later recap. Mark missing fields unknown rather than filling them from a chart after the event.
A useful compact row looks like this: price alert / threshold 100 / published 14:32:05 UTC / received 14:33:11 / first ask 100.35 / no order / threshold observed, trade result not applicable. These are invented values showing the format, not an observed provider result. The row explains why a correct notification can still be unusable as a trade entry.
Match the test to the alert type
For a price alert, reproduce the trigger from the documented market data and test duplicate or stale messages. For news, retain the original filing or announcement and compare its release time with the alert time. For a scanner, preserve the universe, rule and full candidate list so misses are not erased. For a publisher call, preserve the first version, cancellation rules and every losing or unfilled call. For copy trading, compare source and follower ledgers including rejected orders, fees and sizing differences.
Proof must fit the claim. A hash receipt can support only the fields in its documented preimage by a verified attestation time; it does not show a subscriber fill. For Vector Ridge's documented VR2 format, target and stop are not hashed. A pending OpenTimestamps submission is not a confirmed Bitcoin block. The record-verification walkthrough shows how to inspect this separately.
Decide with a dated sample
Set the trial period and minimum number of alerts before seeing results. Log every item, including cancellations, duplicates, expired signals, non-deliveries and alerts you could not act on. At the end, report coverage, median and worst delivery delay, actionability rate, and costs for the specific workflow you tested. Separate provider-reported model outcomes from paper or actual subscriber outcomes; they are different populations.
A trial can establish whether the service fits your schedule and tools. It cannot certify future returns. If a vital field is inaccessible, record that limitation and compare another product type. The delay-cost workbench can help explore how a later executable price affects an illustrative call; it is a scenario tool, not a backtest.