Properties of a real-time analytics system
When customers evaluate a real-time analytics platform, they often focus only on query latency. “Can it return an answer within 50ms?” is a reasonable question, and one most analytical engines can answer convincingly if you throw enough compute at a static dataset. The question that decides what users actually experience is harder. It is whether the system can return a 50ms answer on data that arrived a second ago, while ingestion is still running, and while other users are also querying. Data availability requires thinking about end-to-end time-to-insight, which has three components.- Time to ingest: How long does it take for newly generated data to land in the platform and become durably stored?
- Time to transform and prepare: How long does it take to clean, enrich, join, pre-aggregate, or update the serving structures (materialized views, rollups, indexes) that queries actually hit?
- Time to query: How long does it take to plan and execute the read once the data is available?