---
title: "CAN Bus Data Logger: Decode DBC & J1939 Signals"
description: "Decode CAN bus and J1939 signals with a DBC file, then model, store, and query the results at scale in Postgres. Includes a full code walkthrough. "
section: "Postgres for IoT"
---

> **TimescaleDB is now Tiger Data.**

## Introduction

A single heavy-duty truck's J1939 network can carry dozens of distinct parameter groups, several broadcasting every 10 to 50 milliseconds, hundreds of frames per second from one vehicle alone. Add a passenger fleet running standard OBD-II polling and a CAN bus data logger on each vehicle produces thousands of frames per second before a single one is decoded.

A raw CAN frame is not a signal. It's an arbitration ID and up to 8 bytes of payload, with no field names and no units, until it's decoded against a DBC file and a hex payload becomes EngineCoolantTemp = 87°C. Storage and query design only make sense once you understand that decode step: it's the boundary between an opaque byte stream and a structured, queryable time series.

This page covers that boundary: how raw frames get decoded into named signals, the schema and query patterns those signals need once they land in Postgres, and what to deliberately leave out of a relational database entirely. It applies whether the vehicle is passenger (OBD-II) or heavy-duty (J1939), human-driven or autonomous. This is the signal layer underneath any CAN-bus-equipped vehicle, not a survey of everything a connected fleet produces. If you're still weighing database options for the storage layer itself, see [<u>the full IoT database comparison</u>](https://www.tigerdata.com/learn/how-to-choose-an-iot-database).

Since Tiger Data is a database vendor, this page naturally treats Postgres as the destination. The engineering argument for decode-then-store stands on its own regardless of what you use downstream, and we'll say plainly where a relational database is the wrong tool.

## What is CAN bus data, and why can't you query it directly?

A raw CAN frame consists of an arbitration ID, which identifies the message, and up to 8 bytes of payload. No field names, no units, no indication of what those bytes mean. The same frame looks identical whether it's carrying engine RPM, wheel speed, or a fault code, until something on the other end knows how to read it.

That "something" is a DBC file. DBC (short for Vector's CAN database format) defines, for each message ID, the signals packed into its payload: signal name, start bit, bit length, scaling factor, offset, unit, and, for multiplexed messages, which mux value selects which signal set. Decoding is arithmetic once you have the definition: apply the DBC's scale and offset to the raw bits at engine coolant temperature's assigned position (`raw_value * scale + offset`), and you get `EngineCoolantTemp = 87.5°C` instead of two meaningless bytes.

One naming collision worth flagging: when a DBC file (or a "J1939 database" of PGN and SPN definitions) gets called a "database," that's a signal dictionary, not a storage engine. It tells you what a frame means, not where to store the decoded values, the actual database question this page addresses.

### OBD-II vs. J1939: the two dominant CAN application layers

Passenger and light-duty vehicles standardize on OBD-II, a fixed vocabulary of parameter IDs (PIDs) for things like vehicle speed and fuel level. Heavy-duty and off-road vehicles use J1939, organizing signals into Parameter Groups (PGNs) built from Suspect Parameters (SPNs), and J1939 messages can span multiple frames in ways OBD-II mostly doesn't. Both sit on the same raw CAN transport and go through the identical decode problem: raw frame plus DBC-equivalent definition equals named signal, so everything from here applies to either protocol.

## The decode pipeline: from raw frames to named signals

A few tools show up repeatedly in the Python CAN ecosystem, each handling a different stage:

1. **python-can** handles the hardware and interface layer, reading raw frames off a CAN adapter or a SocketCAN interface.
2. **cantools** and **canmatrix** parse DBC files and decode raw payloads into named, scaled signals.
3. **can_decoder** decodes directly into a pandas DataFrame, convenient for batch-processing logged data rather than streaming it live.
4. **SocketCAN's **`**candump**`, on Linux, captures and prints raw frames to the terminal for quick inspection.

To be precise about what `candump` gives you: raw hex payloads, nothing more. It can't produce a named signal. That's a separate step `cantools` or `can_decoder` perform downstream, using the DBC file as the missing piece.

If your vehicle's manufacturer doesn't publish a DBC file, you're not stuck. comma.ai maintains [<u>opendbc</u>](https://github.com/commaai/opendbc), an actively developed open-source repository of DBC-equivalent definitions for dozens of vehicle makes, built to support their openpilot ADAS project. It's a legitimate reference point, alongside hand reverse-engineering (correlating known vehicle states against captured payloads) as the fallback.

### Code walkthrough: raw CAN log to time-series rows

In production, this decode step usually runs on the vehicle itself, inside a CAN logger or Telematics Control Unit (TCU) that [<u>buffers decoded signals at the edge</u>](https://www.tigerdata.com/learn/edge-database) before forwarding them to a central database. The code below shows the decode-to-storage pipeline end to end, using `python-can` and `cantools` to read frames, decode them, and land the result in a table shaped for time-series queries.

`import can
import cantools
import pandas as pd
from datetime import datetime, timezone
import psycopg2

# Load the DBC file that defines this vehicle's signals
db = cantools.database.load_file("vehicle.dbc")

# Open the CAN interface (SocketCAN on Linux, or a USB-to-CAN adapter)
bus = can.interface.Bus(channel="can0", interface="socketcan")

rows = []
for _ in range(2000):  # capture a batch of frames
    message = bus.recv(timeout=1.0)
    if message is None:
        continue
    try:
        decoded = db.decode_message(message.arbitration_id, message.data)
    except KeyError:
        continue  # this frame isn't defined in the loaded DBC file

    timestamp = datetime.fromtimestamp(message.timestamp, tz=timezone.utc)
    for signal_name, value in decoded.items():
        rows.append({"time": timestamp, "signal_name": signal_name, "value": value})

df = pd.DataFrame(rows)

conn = psycopg2.connect("postgresql://user:password@host:5432/fleet")
cur = conn.cursor()
insert_sql = """
    INSERT INTO can_signals (time, vehicle_id, signal_name, value)
    VALUES (%s, %s, %s, %s)
"""
cur.executemany(
    insert_sql,
    [(r["time"], "truck-042", r["signal_name"], r["value"]) for _, r in df.iterrows()],
)
conn.commit()`

This is a genuine, if simplified, version of a pattern developers already run in production. CSS Electronics' CANedge tooling documents a comparable decode-to-database pipeline for their hardware loggers, and CSS Electronics itself now labels its [<u>InfluxDB-writer project</u>](https://github.com/CSS-Electronics/canedge-influxdb-writer) for that pipeline legacy, recommending a Grafana-and-Athena stack instead. That's not a claim about Tiger Data displacing a competitor. It's evidence the decode-then-store pattern is real and already validated in the market, independent of which database sits at the end of it.

## Why a CAN bus data logger produces a time-series workload

Once frames are decoded, the row-rate math gets concrete fast: 200 signals per vehicle, sampled once per second, across 500 vehicles, is 100,000 rows per second. Push engine or braking signals to a 10 Hz sampling rate and that number climbs further.

That shape holds regardless of exact volume: append-only, timestamped, high-cardinality on the combination of vehicle ID and signal name, since a mixed fleet rarely has an identical signal set across every unit. It's what makes this a time-series workload rather than a generic transactional one, the same argument Tiger Data's [<u>fleet telemetry guide</u>](https://www.tigerdata.com/learn/fleet-telemetry-database) makes for GPS and OBD-II data, applied here to decoded CAN/J1939 output. Getting decoded signals off the vehicle and into that central database is usually [<u>MQTT's job as the transport layer</u>](https://www.tigerdata.com/learn/mqtt-to-postgresql), not the decoder's.

That row-rate math also answers the too-high-frequency objection: a hundred thousand rows per second is a partitioning problem, not a reason to rule out Postgres. TimescaleDB’s [<u>hypertables</u>](https://www.tigerdata.com/docs/learn/hypertables/understand-hypertables) handle exactly that, automatic time-based partitioning that keeps inserts fast and queries scoped as volume grows.

## Schema design for decoded signals

The schema question comes down to narrow rows versus wide rows. A narrow-row schema, one row per signal reading, keyed on time, `vehicle_id, signal_name`, and `value`, handles variable, high-cardinality signal sets gracefully: a new signal from a new vehicle model is just a new value in a text column, not a schema migration. A wide-row schema, one column per signal, reads more naturally for a fixed, known signal set, but a mixed fleet with dozens of different signal sets per model turns wide rows into a maintenance problem. For most fleet CAN and J1939 workloads, narrow rows are the more durable starting point.

`CREATE TABLE can_signals (
    time        TIMESTAMPTZ NOT NULL,
    vehicle_id  TEXT NOT NULL,
    signal_name TEXT NOT NULL,
    value       DOUBLE PRECISION
);

SELECT create_hypertable('can_signals', by_range('time'));`

From there, [<u>continuous aggregates</u>](https://www.tigerdata.com/docs/build/continuous-aggregates/create-a-continuous-aggregate) handle the rollups a fleet operator queries daily, computed incrementally instead of recalculated from raw rows every time:

`-- Hourly max engine coolant temperature per vehicle
CREATE MATERIALIZED VIEW engine_temp_hourly
WITH (timescaledb.continuous) AS
SELECT
    time_bucket('1 hour', time) AS bucket,
    vehicle_id,
    max(value) AS max_engine_temp
FROM can_signals
WHERE signal_name = 'EngineCoolantTemp'
GROUP BY bucket, vehicle_id;

-- Average vehicle speed, hourly
CREATE MATERIALIZED VIEW vehicle_speed_hourly
WITH (timescaledb.continuous) AS
SELECT
    time_bucket('1 hour', time) AS bucket,
    vehicle_id,
    avg(value) AS avg_speed
FROM can_signals
WHERE signal_name = 'VehicleSpeed'
GROUP BY bucket, vehicle_id;

-- Daily fault code counts
CREATE MATERIALIZED VIEW fault_codes_daily
WITH (timescaledb.continuous) AS
SELECT
    time_bucket('1 day', time) AS bucket,
    vehicle_id,
    count(*) AS fault_count
FROM can_signals
WHERE signal_name = 'ActiveFaultCode'
GROUP BY bucket, vehicle_id;`

Add a refresh policy per view so each rollup updates on a schedule (a continuous aggregate only materializes and stays current once a policy is attached): 

`-- One policy per continuous aggregate; adjust intervals to taste.
SELECT add_continuous_aggregate_policy('engine_temp_hourly',
    start_offset      => INTERVAL '3 hours',
    end_offset        => INTERVAL '1 hour',
    schedule_interval => INTERVAL '1 hour');
-- Repeat for vehicle_speed_hourly (hourly) and
-- fault_codes_daily (e.g. end_offset => INTERVAL '1 day',
-- schedule_interval => INTERVAL '1 hour').`

Decoded signal history is a strong candidate for [<u>Hypercore's columnstore</u>](https://www.tigerdata.com/docs/learn/columnar-storage/understand-hypercore) once it ages past the window you query frequently: append-only, rarely updated, exactly the profile columnar compression is built for, and it cuts storage costs on the months of historical data most fleets keep for trend analysis and troubleshooting.

## J1939-specific considerations for heavy-duty fleets

J1939 organizes signals into Parameter Group Numbers, each grouping related Suspect Parameter Numbers, roughly analogous to how a DBC message groups its individual signals. A database-focused reader just needs enough to recognize that PGN and SPN structure maps onto the same arbitration-ID-plus-payload model covered earlier, with an added layer of grouping.

One real gotcha: some J1939 messages exceed the 8-byte single-frame limit and split across multiple CAN frames using the transport protocol (TP). Those frames need reassembly into a complete message before a decoder can extract the signal, an easy step to miss when building a pipeline from scratch instead of using tooling that already handles TP reassembly.

One forward-looking note: heavy-duty fleets often feed decoded J1939 signals into longer-term compliance and audit retention requirements, on top of the operational queries covered here. Plan schema and retention policy around that from the start if it applies to your fleet.

## What not to store in Postgres (and where it goes instead)

Not everything from a CAN bus belongs in a relational database. Raw, high-frequency, undecoded frame dumps kept for full-fidelity forensic replay, intrusion-detection research, or ML training-data mining are a file or object-storage workload: you're replaying or bulk-processing an exact original capture, not querying or aggregating by time range, and a flat file or object store handles that better and cheaper than a database built for structured queries.

The rule of thumb: if you need to replay the exact original bytes, keep the raw log in file or object storage. If you need to query, aggregate, or alert on signal values over time, decode first and store the result in Postgres, since decoded signals are squarely a time-series database problem.

## Decision framework: which fleet telemetry guide matches your workload

Tiger Data has several related guides on vehicle and fleet data. Here's how to land on the right one.

### Use this guide if:

You're logging raw CAN or J1939 frames from any CAN-bus-equipped vehicle, passenger or heavy-duty, human-driven or autonomous, and need to decode them via a DBC file into named, queryable signals.

### See the fleet telemetry guide if:

You want [<u>the four core data types a connected fleet produces</u>](https://www.tigerdata.com/learn/fleet-telemetry-database) (GPS, OBD-II/CAN bus, driver behavior, EV battery data) at a conceptual level, rather than a deep dive on the decode pipeline specifically.

### See Robot Fleet Telemetry if:

You're [<u>logging robot or AMR telemetry rather than vehicle CAN bus data</u>](https://www.tigerdata.com/learn/robot-fleet-telemetry): battery, pose, mission status, and fault codes from AMRs, industrial robots, or service robots, a different actor class entirely.

### See Physical AI Telemetry if:

You're an autonomous-vehicle fleet operator concerned with [<u>autonomous-vehicle fleet operations data specifically, not raw CAN decode</u>](https://www.tigerdata.com/learn/physical-ai-telemetry-database): disengagement events, mission logs, and vehicle state, rather than the raw signal-decode layer underneath it.

## Migrating an existing CAN/DBC pipeline to Tiger Data

If you already have a decode pipeline running, the migration path depends on where it writes today.

**From a legacy InfluxDB-based pipeline:** the decode step upstream stays the same. What changes is the destination: instead of writing decoded points to InfluxDB's line protocol, insert into a TimescaleDB hypertable using standard SQL, as in the code walkthrough above.

**From ad hoc CSV or Parquet exports:** bulk-load historical decoded signal data with `COPY` or a batch insert, partitioned the same way as live data. Older chunks then become candidates for Hypercore's columnstore, so the backlog doesn't carry the same storage cost as recent, actively queried data.

**From a single wide Postgres table:** one column per signal, one row per timestamp, migrating means unpivoting into `time`, `vehicle_id`,` signal_name`, `value rows`. Worth the effort for a fleet with more than a handful of vehicle models, since the narrow-row model absorbs new and rare signals without a schema change.

Ready to implement this? [<u>Start a Tiger Cloud free trial</u>](https://www.tigerdata.com/cloud) and see the [<u>hypertables</u>](https://www.tigerdata.com/docs/learn/hypertables/understand-hypertables) and [<u>continuous aggregates</u>](https://www.tigerdata.com/docs/learn/continuous-aggregates) documentation for setup detail.