How MarketReader Processes 3M Trades/Min to Deliver US Market Trading Insights with TimescaleDB

This is an installment of our “Community Member Spotlight” series, in which we invite our customers to share their work, spotlight their success, and inspire others with new ways to use technology to solve problems.

In this edition, Web Begole, CTO and co-founder at MarketReader, a fintech startup trusted by financial leaders for US market movement insights, shares how Tiger Data simplified his architecture structure from Day 1.  

About MarketReader

We founded MarketReader in 2021 to answer the question every investor asks: “Why are markets moving and what am I missing?” 

Our outputs feed two primary surfaces. The first is our own platform, including real-time monitoring and an AI-generated morning newsletter delivered minutes before the market opens. The second is our delivery layer, via API and WebSocket, which powers most of our business. Across the API and WebSocket, clients typically receive about 700 updates every ten minutes across different stocks. On high volatility days, such as an FOMC rate announcement, that can climb to roughly 3,000 updates in the same window.

To evaluate trends in real time, we built a two-part system. First, we establish what normal behavior looks like for every U.S. listed asset. Then, we process high-volume market data at scale to deliver tick level and market movement analytics. Through our NASDAQ partnership, we ingest an average of 3M trades per minute across the U.S. equity market, covering roughly 26,000 listed and OTC names. At this scale, ingestion has to expand quickly to avoid bottlenecks, because even small delays can prevent investors from getting truly real time insights.

Early on, many retail brokerages told us they planned to build this themselves. Today, many of those same firms are MarketReader customers because they cannot replicate our preprocessing work. Customers like Public, along with institutional investors and trade compliance teams, rely on MarketReader to filter out market noise, reduce false flags, and provide the rich context needed to train large language models.

When an investor sees a stock move and asks an LLM why, they’re starting with a question, not the historical context needed to answer it well. We do the opposite. We begin with everything already happening in the market, detect unusual movement across thousands of names in real time, and attach time-aligned evidence. Our RAG is based on time, tied to the market’s reaction as information becomes available. Market context decays quickly, so it's critical we continuously keep our models current with the most recent information.  

Selecting Tiger Data and TimescaleDB

Once I defined the customer challenge MarketReader needed to solve, I looked for the simplest architecture that could handle real-time analytics with as few moving parts as possible. I trusted Postgres as a proven transactional database, so I started there. As I evaluated Postgres extensions, TimescaleDB stood out because it let me stay on Postgres while gaining purpose- built time series performance.

As I evaluated Postgres extensions, TimescaleDB stood out because it let me stay on Postgres while gaining purpose- built time series performance. - Web Begole, MarketReader CTO and co-founder

To scale analytics across the entire market, I leaned on Tiger Data’s time-series primitives. We use hypertables for automatic partitioning to ingest market data quickly and continuous aggregates and hyperfunctions to run the statistical analysis that flags unusual stock behavior. When we began pushing real-time market workloads, it became clear Tiger Data was the right fit for our core database layer.

Technology was only part of the decision. Tiger Data’s commitment to continuous improvement meant I could build on what existed today and trust that ingest and query performance would keep getting better. We now run on Tiger Cloud, which removes the operational burden from my team while preserving the performance and scalability we need.

We now run on Tiger Cloud, which removes the operational burden from my team while preserving the performance and scalability we need. - Web Begole, MarketReader CTO and co-founder

AWS and Tiger Data for the Tech Stack Backbone 

MarketReader’s architecture is intentionally modular, separating storage, compute, and serving concerns to support real-time market workloads at scale. Tiger Cloud is the system of record for all data, including high-volume market trades and unstructured analytical outputs. Both are stored in TimescaleDB, which allows MarketReader to run time-series analytics, statistical aggregation, and retrieval close to the data. AWS hosts the rest of the stack, providing elastic compute, orchestration, and application services.

Market analysis is broken into independent domains, such as social signals, newswire content, and web sourced articles. Each domain runs as its own Python-based microservice on AWS EKS, deployed as isolated Kubernetes pods. These services operate independently but coordinate through Redis for messaging and state sharing. Each service pulls only the slices of data it needs from Tiger Cloud, performs domain specific analysis, and produces structured conclusions about market activity.

Those conclusions are then passed to a large language model with strict constraints. The LLM is instructed to summarize only the provided inputs, rather than generate new information, which significantly reduces hallucinations while still producing readable explanations. To improve relevance, the system also uses vector search to attach semantically related news and social context to each market move at the stock and sector level. This is supported directly inside Tiger Cloud using Postgres-based vector tooling such as pgvector, pgvectorscale, and pgai.

To protect client experience, MarketReader adds a caching and serving layer in front of the core databases. Kubernetes services pull curated results from Tiger Cloud, cache the appropriate slices in Supabase, and serve them to customers via APIs and WebSockets. This layer acts as a buffer between backend processing and end users, so if a downstream system or database becomes unavailable, cached results continue to flow without interruption. Conceptually, the stack resolves to Tiger Cloud for storage and analytics, AWS EKS for Python microservices, Redis for coordination, Supabase for cached serving, and API and WebSocket delivery into brokerage platforms, institutional systems, and compliance workflows.

The biggest benefit of Tiger Data is that it gives us first-class time series analytics while staying on Postgres, so we didn’t have to build and maintain separate databases or specialized data pipelines. - Web Begole, MarketReader CTO and co-founder

Time-Series as a First-Class Capability with Tiger Data

The biggest benefit of Tiger Data is that it gives us first-class time series analytics while staying on Postgres, so we didn’t have to build and maintain separate databases or specialized data pipelines. Hypertables and continuous aggregates make it fast enough to model what’s normal and detect what’s unusual at market scale, which reduces the amount of custom glue needed to keep the system reliable. 

Moving to Tiger Cloud also removed the operational overhead of running the database ourselves, letting us iterate faster. And when we’ve pushed the system with an unusual workload, Tiger Data has been a highly responsive partner, from break fix support to deep technical collaboration.

Looking Ahead

Our next phase is scaling adoption as the market catches up to what it really takes to build this, collaborating with investment firms and brokerages to make their chatbots smarter and their context for LLMs more rich.

On the product side, Tiger Data lets us go deeper on time-aligned retrieval and semantic relevance without changing our foundation. We’re expanding our vector workflows, adding more automation around embeddings where it helps, and building richer explanations that stay anchored to time and evidence.

Related: Learn how Plexigrid consolidated 4 databases into Postgres and got 350x faster queries.