Tiger Cloud: Performance, Scale, Enterprise, Free
Self-hosted products
MST
Create a hypertable partitioned on a single dimension with columnstore enabled, or create a standard Postgres relational table.
A hypertable is a specialized Postgres table that automatically partitions your data by time. All actions that work on a
Postgres table, work on hypertables. For example, ALTER TABLE and SELECT. By default,
a hypertable is partitioned on the time dimension. To add secondary dimensions to a hypertable, call
add_dimension. To convert an existing relational table into a hypertable, call
create_hypertable.
When you create a hypertable using CREATE TABLE ... WITH ..., the default partitioning
column is automatically the first column with a timestamp data type. Also, TimescaleDB creates a
columnstore policy that automatically converts your data to the columnstore, after an interval equal to the value of the chunk_interval, defined through compress_after in the policy. This columnar format enables fast scanning and
aggregation, optimizing performance for analytical workloads while also saving significant storage space. In the
columnstore conversion, hypertable chunks are compressed by up to 98%, and organized for efficient, large-scale queries.
You can customize this policy later using alter_job. However, to change after or
created_before, the compression settings, or the hypertable the policy is acting on, you must
remove the columnstore policy and add a new one.
You can also manually convert chunks in a hypertable to the columnstore.
Hypertable to hypertable foreign keys are not allowed, all other combinations are permitted.
The columnstore settings are applied on a per-chunk basis. You can change the settings by calling ALTER TABLE without first converting the entire hypertable back to the rowstore. The new settings apply only to the chunks that have not yet been converted to columnstore, the existing chunks in the columnstore do not change. Similarly, if you remove an existing columnstore policy and then add a new one, the new policy applies only to the unconverted chunks. This means that chunks with different columnstore settings can co-exist in the same hypertable.
TimescaleDB calculates default columnstore settings for each chunk when it is created. These settings apply to each chunk, and not the entire hypertable. To explicitly disable the defaults, set a setting to an empty string.
CREATE TABLE extends the standard Postgres CREATE TABLE. This page explains the features and
arguments specific to TimescaleDB.
Note
For TimescaleDB v2.23.0 and higher, the table is automatically partitioned on the first column
in the table with a timestamp data type. If multiple columns are suitable candidates as a partitioning column,
TimescaleDB throws an error and asks for an explicit definition. For earlier versions, set
partition_column to a
time column.
If you are self-hosting TimescaleDB v2.20.0 to v2.22.1
, to convert your
data to the columnstore after a specific time interval, you have to call add_columnstore_policy after you call
CREATE TABLE
If you are self-hosting TimescaleDB v2.19.3 and below, create a Postgres relational table
,
then convert it using create_hypertable. You then enable hypercore with a call
to ALTER TABLE.
Create a hypertable partitioned on the time dimension and enable columnstore:
CREATE TABLE crypto_ticks ("time" TIMESTAMPTZ,symbol TEXT,price DOUBLE PRECISION,day_volume NUMERIC) WITH (tsdb.hypertable,tsdb.segmentby='symbol',tsdb.orderby='time DESC');When you create a hypertable using
CREATE TABLE WITH, TimescaleDB automatically creates a columnstore policy that uses the chunk interval as the compression interval, with a default schedule interval of 1 day. The default partitioning column is automatically selected as the first column with a timestamp or timestampz data type.Create a hypertable partitioned on the time with fewer chunks based on time interval:
CREATE TABLE IF NOT EXISTS hypertable_control_chunk_interval(time int4 NOT NULL,device text,value float) WITH (tsdb.hypertable,tsdb.chunk_interval=3453);Create a hypertable partitioned using UUIDv7:
Enable data compression during ingestion:
When you set
timescaledb.enable_direct_compress_copyyour data gets compressed in memory during ingestion withCOPYstatements. By writing the compressed batches immediately in the columnstore, the IO footprint is significantly lower. Also, the columnstore policy you set is less important,INSERTalready produces compressed chunks.Note
Please note that this feature is a tech preview and not production-ready. Using this feature could lead to regressed query performance and/or storage ratio, if the ingested batches are not correctly ordered or are of too high cardinality.
To enable in-memory data compression during ingestion:
SET timescaledb.enable_direct_compress_copy=on;Important facts
- High cardinality use cases do not produce good batches and lead to degreaded query performance.
- The columnstore is optimized to store 1000 records per batch, which is the optimal format for ingestion per segment by.
- WAL records are written for the compressed batches rather than the individual tuples.
- Currently only
COPYis support,INSERTwill eventually follow. - Best results are achieved for batch ingestion with 1000 records or more, upper boundary is 10.000 records.
- Continous Aggregates are not supported at the moment.
- Create a hypertable:CREATE TABLE t(time timestamptz, device text, value float) WITH (tsdb.hypertable);
- Copy data into the hypertable:
You achieve the highest insert rate using binary format. CSV and text format are also supported.COPY t FROM '/tmp/t.binary' WITH (format binary);
Create a Postgres relational table:
CREATE TABLE IF NOT EXISTS relational_table(device text,value float);
The syntax is:
CREATE TABLE <table_name> (-- Standard Postgres syntax for CREATE TABLE)WITH (tsdb.hypertable = true | falsetsdb.partition_column = '<column_name> ',tsdb.chunk_interval = '<interval>'tsdb.create_default_indexes = true | falsetsdb.associated_schema = '<schema_name>',tsdb.associated_table_prefix = '<prefix>'tsdb.orderby = '<column_name> [ASC | DESC] [ NULLS { FIRST | LAST } ] [, ...]',tsdb.segmentby = '<column_name> [, ...]',tsdb.sparse_index = '<index>(<column_name>), index(<column_name>)')
| Name | Type | Default | Required | Description |
|---|---|---|---|---|
tsdb.hypertable | BOOLEAN | true | ✖ | Create a new hypertable for time-series data rather than a standard Postgres relational table. |
tsdb.partition_column | TEXT | The first column in the table with a timestamp data type | ✖ | Set the time column to automatically partition your time-series data by. |
tsdb.chunk_interval | TEXT | 7 days | ✖ | Change this to better suit your needs. For example, if you set chunk_interval to 1 day, each chunk stores data from the same day. Data from different days is stored in different chunks. |
tsdb.create_default_indexes | BOOLEAN | true | ✖ | Set to false to not automatically create indexes. The default indexes are:
|
tsdb.associated_schema | REGCLASS | _timescaledb_internal | ✖ | Set the schema name for internal hypertable tables. |
tsdb.associated_table_prefix | TEXT | _hyper | ✖ | Set the prefix for the names of internal hypertable chunks. |
tsdb.orderby | TEXT | Descending order on the time column in table_name. | ✖ | The order in which items are used in the columnstore. Specified in the same way as an ORDER BY clause in a SELECT query. Setting tsdb.orderby automatically creates an implicit min/max sparse index on the orderby column. |
tsdb.segmentby | TEXT | TimescaleDB looks at pg_statspg_stats is not available, TimescaleDB looks for an appropriate column from the existing indexes. | ✖ | Set the list of columns used to segment data in the columnstore for table. An identifier representing the source of the data such as device_id or tags_id is usually a good candidate. |
tsdb.sparse_index | TEXT | TimescaleDB evaluates the columns you already have indexed, checks which data types are a good fit for sparse indexing, then creates a sparse index as an optimization. | ✖ | Configure the sparse indexes for compressed chunks. Requires setting tsdb.orderby. Supported index types include: bloom(<column_name>): a probabilistic index, effective for = filters. Cannot be applied to tsdb.orderby columns.minmax(<column_name>): stores min/max values for each compressed chunk. Setting tsdb.orderby automatically creates an implicit min/max sparse index on the orderby column. |
TimescaleDB returns a simple message indicating success or failure.
Keywords
Found an issue on this page?Report an issue or Edit this page
in GitHub.