Home Time-series basics Data Center & Infrastructure Telemetry Postgres basics Postgres guides Postgres best practices Postgres extensions

CAN Bus Data Logger: Decoding DBC and J1939 Signals into Postgres

By Tiger Data Team

Updated at Jul 27, 2026

Introduction

A single heavy-duty truck's J1939 network can carry dozens of distinct parameter groups, several broadcasting every 10 to 50 milliseconds, hundreds of frames per second from one vehicle alone. Add a passenger fleet running standard OBD-II polling and a CAN bus data logger on each vehicle produces thousands of frames per second before a single one is decoded.

A raw CAN frame is not a signal. It's an arbitration ID and up to 8 bytes of payload, with no field names and no units, until it's decoded against a DBC file and a hex payload becomes EngineCoolantTemp = 87°C. Storage and query design only make sense once you understand that decode step: it's the boundary between an opaque byte stream and a structured, queryable time series.

This page covers that boundary: how raw frames get decoded into named signals, the schema and query patterns those signals need once they land in Postgres, and what to deliberately leave out of a relational database entirely. It applies whether the vehicle is passenger (OBD-II) or heavy-duty (J1939), human-driven or autonomous. This is the signal layer underneath any CAN-bus-equipped vehicle, not a survey of everything a connected fleet produces. If you're still weighing database options for the storage layer itself, see the full IoT database comparison.

Since Tiger Data is a database vendor, this page naturally treats Postgres as the destination. The engineering argument for decode-then-store stands on its own regardless of what you use downstream, and we'll say plainly where a relational database is the wrong tool.

What is CAN bus data, and why can't you query it directly?

A raw CAN frame consists of an arbitration ID, which identifies the message, and up to 8 bytes of payload. No field names, no units, no indication of what those bytes mean. The same frame looks identical whether it's carrying engine RPM, wheel speed, or a fault code, until something on the other end knows how to read it.

That "something" is a DBC file. DBC (short for Vector's CAN database format) defines, for each message ID, the signals packed into its payload: signal name, start bit, bit length, scaling factor, offset, unit, and, for multiplexed messages, which mux value selects which signal set. Decoding is arithmetic once you have the definition: apply the DBC's scale and offset to the raw bits at engine coolant temperature's assigned position (raw_value * scale + offset), and you get EngineCoolantTemp = 87.5°C instead of two meaningless bytes.

One naming collision worth flagging: when a DBC file (or a "J1939 database" of PGN and SPN definitions) gets called a "database," that's a signal dictionary, not a storage engine. It tells you what a frame means, not where to store the decoded values, the actual database question this page addresses.

OBD-II vs. J1939: the two dominant CAN application layers

Passenger and light-duty vehicles standardize on OBD-II, a fixed vocabulary of parameter IDs (PIDs) for things like vehicle speed and fuel level. Heavy-duty and off-road vehicles use J1939, organizing signals into Parameter Groups (PGNs) built from Suspect Parameters (SPNs), and J1939 messages can span multiple frames in ways OBD-II mostly doesn't. Both sit on the same raw CAN transport and go through the identical decode problem: raw frame plus DBC-equivalent definition equals named signal, so everything from here applies to either protocol.

The decode pipeline: from raw frames to named signals

A few tools show up repeatedly in the Python CAN ecosystem, each handling a different stage:

python-can handles the hardware and interface layer, reading raw frames off a CAN adapter or a SocketCAN interface.
cantools and canmatrix parse DBC files and decode raw payloads into named, scaled signals.
can_decoder decodes directly into a pandas DataFrame, convenient for batch-processing logged data rather than streaming it live.
SocketCAN's candump, on Linux, captures and prints raw frames to the terminal for quick inspection.

To be precise about what candump gives you: raw hex payloads, nothing more. It can't produce a named signal. That's a separate step cantools or can_decoder perform downstream, using the DBC file as the missing piece.

If your vehicle's manufacturer doesn't publish a DBC file, you're not stuck. comma.ai maintains opendbc, an actively developed open-source repository of DBC-equivalent definitions for dozens of vehicle makes, built to support their openpilot ADAS project. It's a legitimate reference point, alongside hand reverse-engineering (correlating known vehicle states against captured payloads) as the fallback.

Code walkthrough: raw CAN log to time-series rows

In production, this decode step usually runs on the vehicle itself, inside a CAN logger or Telematics Control Unit (TCU) that buffers decoded signals at the edge before forwarding them to a central database. The code below shows the decode-to-storage pipeline end to end, using python-can and cantools to read frames, decode them, and land the result in a table shaped for time-series queries.

import can
import cantools
import pandas as pd
from datetime import datetime, timezone
import psycopg2

# Load the DBC file that defines this vehicle's signals
db = cantools.database.load_file("vehicle.dbc")

# Open the CAN interface (SocketCAN on Linux, or a USB-to-CAN adapter)
bus = can.interface.Bus(channel="can0", interface="socketcan")

rows = []
for _ in range(2000):  # capture a batch of frames
    message = bus.recv(timeout=1.0)
    if message is None:
        continue
    try:
        decoded = db.decode_message(message.arbitration_id, message.data)
    except KeyError:
        continue  # this frame isn't defined in the loaded DBC file

    timestamp = datetime.fromtimestamp(message.timestamp, tz=timezone.utc)
    for signal_name, value in decoded.items():
        rows.append({"time": timestamp, "signal_name": signal_name, "value": value})

df = pd.DataFrame(rows)

conn = psycopg2.connect("postgresql://user:password@host:5432/fleet")
cur = conn.cursor()
insert_sql = """
    INSERT INTO can_signals (time, vehicle_id, signal_name, value)
    VALUES (%s, %s, %s, %s)
"""
cur.executemany(
    insert_sql,
    [(r["time"], "truck-042", r["signal_name"], r["value"]) for _, r in df.iterrows()],
)
conn.commit()

This is a genuine, if simplified, version of a pattern developers already run in production. CSS Electronics' CANedge tooling documents a comparable decode-to-database pipeline for their hardware loggers, and CSS Electronics itself now labels its InfluxDB-writer project for that pipeline legacy, recommending a Grafana-and-Athena stack instead. That's not a claim about Tiger Data displacing a competitor. It's evidence the decode-then-store pattern is real and already validated in the market, independent of which database sits at the end of it.

Why a CAN bus data logger produces a time-series workload

Once frames are decoded, the row-rate math gets concrete fast: 200 signals per vehicle, sampled once per second, across 500 vehicles, is 100,000 rows per second. Push engine or braking signals to a 10 Hz sampling rate and that number climbs further.

That shape holds regardless of exact volume: append-only, timestamped, high-cardinality on the combination of vehicle ID and signal name, since a mixed fleet rarely has an identical signal set across every unit. It's what makes this a time-series workload rather than a generic transactional one, the same argument Tiger Data's fleet telemetry guide makes for GPS and OBD-II data, applied here to decoded CAN/J1939 output. Getting decoded signals off the vehicle and into that central database is usually MQTT's job as the transport layer, not the decoder's.

That row-rate math also answers the too-high-frequency objection: a hundred thousand rows per second is a partitioning problem, not a reason to rule out Postgres. TimescaleDB’s hypertables handle exactly that, automatic time-based partitioning that keeps inserts fast and queries scoped as volume grows.

Schema design for decoded signals

The schema question comes down to narrow rows versus wide rows. A narrow-row schema, one row per signal reading, keyed on time, vehicle_id, signal_name, and value, handles variable, high-cardinality signal sets gracefully: a new signal from a new vehicle model is just a new value in a text column, not a schema migration. A wide-row schema, one column per signal, reads more naturally for a fixed, known signal set, but a mixed fleet with dozens of different signal sets per model turns wide rows into a maintenance problem. For most fleet CAN and J1939 workloads, narrow rows are the more durable starting point.

CREATE TABLE can_signals (
    time        TIMESTAMPTZ NOT NULL,
    vehicle_id  TEXT NOT NULL,
    signal_name TEXT NOT NULL,
    value       DOUBLE PRECISION
);

SELECT create_hypertable('can_signals', by_range('time'));

From there, continuous aggregates handle the rollups a fleet operator queries daily, computed incrementally instead of recalculated from raw rows every time:

-- Hourly max engine coolant temperature per vehicle
CREATE MATERIALIZED VIEW engine_temp_hourly
WITH (timescaledb.continuous) AS
SELECT
    time_bucket('1 hour', time) AS bucket,
    vehicle_id,
    max(value) AS max_engine_temp
FROM can_signals
WHERE signal_name = 'EngineCoolantTemp'
GROUP BY bucket, vehicle_id;

-- Average vehicle speed, hourly
CREATE MATERIALIZED VIEW vehicle_speed_hourly
WITH (timescaledb.continuous) AS
SELECT
    time_bucket('1 hour', time) AS bucket,
    vehicle_id,
    avg(value) AS avg_speed
FROM can_signals
WHERE signal_name = 'VehicleSpeed'
GROUP BY bucket, vehicle_id;

-- Daily fault code counts
CREATE MATERIALIZED VIEW fault_codes_daily
WITH (timescaledb.continuous) AS
SELECT
    time_bucket('1 day', time) AS bucket,
    vehicle_id,
    count(*) AS fault_count
FROM can_signals
WHERE signal_name = 'ActiveFaultCode'
GROUP BY bucket, vehicle_id;

Add a refresh policy per view so each rollup updates on a schedule (a continuous aggregate only materializes and stays current once a policy is attached):

-- One policy per continuous aggregate; adjust intervals to taste.
SELECT add_continuous_aggregate_policy('engine_temp_hourly',
    start_offset      => INTERVAL '3 hours',
    end_offset        => INTERVAL '1 hour',
    schedule_interval => INTERVAL '1 hour');
-- Repeat for vehicle_speed_hourly (hourly) and
-- fault_codes_daily (e.g. end_offset => INTERVAL '1 day',
-- schedule_interval => INTERVAL '1 hour').

Decoded signal history is a strong candidate for Hypercore's columnstore once it ages past the window you query frequently: append-only, rarely updated, exactly the profile columnar compression is built for, and it cuts storage costs on the months of historical data most fleets keep for trend analysis and troubleshooting.

J1939-specific considerations for heavy-duty fleets

J1939 organizes signals into Parameter Group Numbers, each grouping related Suspect Parameter Numbers, roughly analogous to how a DBC message groups its individual signals. A database-focused reader just needs enough to recognize that PGN and SPN structure maps onto the same arbitration-ID-plus-payload model covered earlier, with an added layer of grouping.

One real gotcha: some J1939 messages exceed the 8-byte single-frame limit and split across multiple CAN frames using the transport protocol (TP). Those frames need reassembly into a complete message before a decoder can extract the signal, an easy step to miss when building a pipeline from scratch instead of using tooling that already handles TP reassembly.

One forward-looking note: heavy-duty fleets often feed decoded J1939 signals into longer-term compliance and audit retention requirements, on top of the operational queries covered here. Plan schema and retention policy around that from the start if it applies to your fleet.

What not to store in Postgres (and where it goes instead)

Not everything from a CAN bus belongs in a relational database. Raw, high-frequency, undecoded frame dumps kept for full-fidelity forensic replay, intrusion-detection research, or ML training-data mining are a file or object-storage workload: you're replaying or bulk-processing an exact original capture, not querying or aggregating by time range, and a flat file or object store handles that better and cheaper than a database built for structured queries.

The rule of thumb: if you need to replay the exact original bytes, keep the raw log in file or object storage. If you need to query, aggregate, or alert on signal values over time, decode first and store the result in Postgres, since decoded signals are squarely a time-series database problem.

Decision framework: which fleet telemetry guide matches your workload

Tiger Data has several related guides on vehicle and fleet data. Here's how to land on the right one.

Use this guide if:

You're logging raw CAN or J1939 frames from any CAN-bus-equipped vehicle, passenger or heavy-duty, human-driven or autonomous, and need to decode them via a DBC file into named, queryable signals.

See the fleet telemetry guide if:

You want the four core data types a connected fleet produces (GPS, OBD-II/CAN bus, driver behavior, EV battery data) at a conceptual level, rather than a deep dive on the decode pipeline specifically.

See Robot Fleet Telemetry if:

You're logging robot or AMR telemetry rather than vehicle CAN bus data: battery, pose, mission status, and fault codes from AMRs, industrial robots, or service robots, a different actor class entirely.

See Physical AI Telemetry if:

You're an autonomous-vehicle fleet operator concerned with autonomous-vehicle fleet operations data specifically, not raw CAN decode: disengagement events, mission logs, and vehicle state, rather than the raw signal-decode layer underneath it.

Migrating an existing CAN/DBC pipeline to Tiger Data

If you already have a decode pipeline running, the migration path depends on where it writes today.

From a legacy InfluxDB-based pipeline: the decode step upstream stays the same. What changes is the destination: instead of writing decoded points to InfluxDB's line protocol, insert into a TimescaleDB hypertable using standard SQL, as in the code walkthrough above.

From ad hoc CSV or Parquet exports: bulk-load historical decoded signal data with COPY or a batch insert, partitioned the same way as live data. Older chunks then become candidates for Hypercore's columnstore, so the backlog doesn't carry the same storage cost as recent, actively queried data.

From a single wide Postgres table: one column per signal, one row per timestamp, migrating means unpivoting into time, vehicle_id, signal_name, value rows. Worth the effort for a fleet with more than a handful of vehicle models, since the narrow-row model absorbs new and rare signals without a schema change.

Ready to implement this? Start a Tiger Cloud free trial and see the hypertables and continuous aggregates documentation for setup detail.

FAQ

How do I decode CAN bus data using a DBC file?

Parse the DBC file with a library like cantools or canmatrix, match each raw frame's arbitration ID to its signal definitions, then apply the scaling factor and offset to produce named, unit-labeled values.

What's the difference between raw CAN frames and decoded signals?

A raw frame is an arbitration ID plus up to 8 bytes of payload with no inherent meaning. A decoded signal is a named, scaled, unit-labeled value, like engine RPM or coolant temperature, produced by applying a DBC file's definitions to that payload.

Can I store CAN bus or J1939 data directly in PostgreSQL or TimescaleDB?

Yes, once it's decoded into named signals, using a narrow-row schema and hypertable pattern. Raw, undecoded frame archives are usually better suited to file or object storage.

What's the difference between OBD-II and J1939 for fleet data logging?

OBD-II is the passenger and light-duty CAN application layer with a standardized PID vocabulary. J1939 is the heavy-duty and off-road layer, built on PGNs and SPNs, with multi-packet transport-protocol messages. Both decode into the same kind of named-signal output once processed.

What is a DBC file and what does it contain?

A DBC (Vector CAN database) file defines the signals encoded in a vehicle's CAN messages: signal name, start bit and length, scaling factor, offset, unit, and multiplexing rules where applicable.

What is a J1939 PGN (Parameter Group Number)?

A PGN identifies a group of related parameters, or SPNs, within a J1939 message. It maps onto the same arbitration-ID-plus-payload model as any CAN frame, with an added grouping layer.

How do I use python-can and cantools to log decoded vehicle signals to a database?

python-can handles the hardware and interface layer to read raw frames, cantools decodes them against a DBC file, and the decoded output is inserted into a hypertable.

What's the right schema for high-frequency, high-cardinality vehicle signal data?

A narrow-row schema (time, vehicle_id, signal_name, value) handles high signal-cardinality and variable signal sets across vehicle models more gracefully than a wide-row-per-signal table.

Is CAN bus data too high-frequency to store in a relational database?

No. The row-rate math, signals multiplied by frequency multiplied by vehicle count, shows the volume is a partitioning and indexing problem, and hypertable partitioning plus columnstore compression are built for exactly that kind of workload.

Do I need a purpose-built time-series database like InfluxDB for automotive telemetry?

Not necessarily. A documented decoded-CAN-to-InfluxDB pipeline exists via CSS Electronics' CANedge tooling, but their own documentation now labels that specific integration legacy and points toward a different stack, evidence that a purpose-built time-series database isn't a universally settled requirement.

Where do DBC files come from if my vehicle manufacturer doesn't publish one?

Two paths: comma.ai's opendbc open-source repository, which covers a large number of makes, or hand reverse-engineering, correlating known vehicle states against raw payloads, when no DBC file exists.

What's the difference between SocketCAN's candump and a DBC decoder?

candump captures and displays raw CAN frames on Linux, still hex payloads with no decoding. A DBC decoder like cantools or can_decoder takes that raw output and applies a DBC file's signal definitions to produce named, scaled values. They're complementary steps, not competing tools.