Swiftpack.co - Package - timescale/timescaledb

Build Status Coverity Scan Build Status

TimescaleDB

TimescaleDB is an open-source database designed to make SQL scalable for time-series data. It is engineered up from PostgreSQL, providing automatic partitioning across time and space (partitioning key), as well as full SQL support.

TimescaleDB is packaged as a PostgreSQL extension and released under the Apache 2 open-source license. Contributors welcome.

Below is an introduction to TimescaleDB. For more information, please check out these other resources:

Using TimescaleDB

TimescaleDB scales PostgreSQL for time-series data via automatic partitioning across time and space (partitioning key), yet retains the standard PostgreSQL interface.

In other words, TimescaleDB exposes what look like regular tables, but are actually only an abstraction (or a virtual view) of many individual tables comprising the actual data. This single-table view, which we call a hypertable, is comprised of many chunks, which are created by partitioning the hypertable's data in either one or two dimensions: by a time interval, and by an (optional) "partition key" such as device id, location, user id, etc. (Architecture discussion)

Virtually all user interactions with TimescaleDB are with hypertables. Creating tables and indexes, altering tables, inserting data, selecting data, etc., can (and should) all be executed on the hypertable.

From the perspective of both use and management, TimescaleDB just looks and feels like PostgreSQL, and can be managed and queried as such.

Creating a hypertable

-- We start by creating a regular SQL table
CREATE TABLE conditions (
  time        TIMESTAMPTZ       NOT NULL,
  location    TEXT              NOT NULL,
  temperature DOUBLE PRECISION  NULL,
  humidity    DOUBLE PRECISION  NULL
);

-- Then we convert it into a hypertable that is partitioned by time
SELECT create_hypertable('conditions', 'time');

Inserting and querying data

Inserting data into the hypertable is done via normal SQL commands:

INSERT INTO conditions(time, location, temperature, humidity)
  VALUES (NOW(), 'office', 70.0, 50.0);

SELECT * FROM conditions ORDER BY time DESC LIMIT 100;

SELECT time_bucket('15 minutes', time) AS fifteen_min,
    location, COUNT(*),
    MAX(temperature) AS max_temp,
    MAX(humidity) AS max_hum
  FROM conditions
  WHERE time > NOW() - interval '3 hours'
  GROUP BY fifteen_min, location
  ORDER BY fifteen_min DESC, max_temp DESC;

In addition, TimescaleDB includes additional functions for time-series analysis that are not present in vanilla PostgreSQL. (For example, the time_bucket function above.)

Installation

TimescaleDB can be installed via a variety of ways:

We recommend following our detailed installation instructions.

Building from source (Unix-based systems)

Prerequisites:

  • A standard PostgreSQL 9.6 or 10 installation with development environment (header files) (e.g., postgresql-server-dev-9.6package for Linux, Postgres.app for MacOS)
  • C compiler (e.g., gcc or clang)
  • CMake version 3.4 or greater
git clone git@github.com:timescale/timescaledb.git
cd timescaledb

# Bootstrap the build system
./bootstrap

# To build the extension
cd build && make

# To install
make install

Please see our additional configuration instructions.

Building from source (Windows)

Prerequisites:

  • A standard PostgreSQL 9.6 or 10 64-bit installation
  • Microsoft Visual Studio 2017 with CMake and Git components
  • OR Visual Studio 2015/2016 with CMake version 3.4 or greater and Git
  • Make sure all relevant binaries are in your PATH: pg_config, cmake, MSBuild

If using Visual Studio 2017 with the CMake and Git components, you should be able to simply clone the repo and open the folder in Visual Studio which will take care of the rest.

If you are using an earlier version of Visual Studio, then it can be built in the following way:

git clone git@github.com:timescale/timescaledb.git
cd timescaledb

# Bootstrap the build system
./bootstrap.bat

# To build the extension from command line
cd build
MSBuild.exe timescaledb.sln

# To install
MSBuild.exe /p:Configuration=Release INSTALL.vcxproj

# Alternatively, open build/timescaledb.sln in Visual Studio and build

Additional documentation

Support

Github

link
Stars: 3559
Help us keep the lights on

Dependencies

Used By

Total: 0

Releases

0.8.0 - Dec 20, 2017

High-level changes

  • TimescaleDB now builds and runs on Windows! Now in addition to using Docker, users can choose to build the extension from source and install on 64-bit Windows
  • Update functions add_dimension and set_chunk_time_interval to take INTERVAL types
  • Improved tablespace management including detaching tablespaces from hypertables and looking up tablespaces associated with a hypertable
  • Reduced memory usage for INSERTs with out-of-order data
  • Fix inserts on 32-bit architectures, in particular ARM
  • Other correctness improvements including preventing attachment of PG10 partitions to hypertables, improved handling of space dimensions with one partition, and correctly working with pg_upgrade
  • Test and build improvements making those both more robust and easier to do

Notable commits

  • [26971d2] Make tablespace_show function return Name instead of CString
  • [2fe447b] Make TimescaleDB work with pg_upgrade
  • [90c7a6f] Fix logic for one space partition
  • [6cfdd79] Prevent native partitioning attachment of hypertables
  • [438d79d] Fix trigger relcache handling for COPY
  • [cc1ad95] Reduce memory usage for out-of-order inserts
  • [a0f62c5] Improve bootstrap script's robustness
  • [00a096f] Modify tests to make more platform agnostic
  • [0e76b5f] Do not add tablespaces to hypertable objects
  • [176b75e] Add command to show tablespaces attached to a hypertable
  • [6e92383] Add function to detach tablespaces from hypertables
  • [e593876] Refactor tablespace handling
  • [c4a46ac] Add hypertable cache lookup on ID/pkey
  • [f38a578] Fix handling of long constraint names
  • [20c9b28] Unconditionally add pg_config --includedir to src build
  • [12dff61] Fixes insert for 32bit architecture
  • [e44e47e] Update add_dimension to take INTERVAL times
  • [0763e62] Update set_chunk_time_interval to take INTERVAL times
  • [87c4b4f] Fix test generator to work for PG 10.1
  • [51854ac] Fix error message to reflect that drop_chunks can take a DATE interval
  • [66396fb] Add build support for Windows
  • [e1a0e81] Refactor and fix cache invalidation

Thanks

  • @oldgreen for reporting an issue where COPY was warning of relcache reference leaks
  • @campeterson for pointing out some documentation typos
  • @jwdeitch for the PR to prevent attaching PG10 partitions to hypertables
  • @vjpr and @sztanpet for reporting bugs and suggesting improvements to the bootstrap script

0.7.1 - Nov 29, 2017

High-level changes

  • Fix to the migration script for those coming from 0.6.1 (or earlier)
  • Fix edge case in drop_chunks when hypertable uses TIMESTAMP type
  • Query planning improvements & fixes
  • Permission fixes and support SET ROLE functionality

Notable commits

  • [717299f] Change time handling in drop_chunks for TIMESTAMP times
  • [d8ec285] Do not append-optimize plans with result relations (DELETE/UPDATE)
  • [30b72ec] Handle empty append plans in ConstraintAwareAppend
  • [b35509b] Permission fixes and allow SET ROLE

Thanks

  • @shaneodonnell for reporting a bug with empty append plans in ConstraintAwareAppend
  • @ryan-shaw for reporting a bug with query plans involving result relations and reporting an issue with our 0.6.1 to 0.7.0 migration script

0.7.0 - Nov 21, 2017

Please note: This update may take a long time (minutes, even hours) to complete, depending on the size of your database

High-level changes

  • Initial PostgreSQL 10 support. TimescaleDB now should work on both PostgreSQL 9.6 and 10. As this is our first release supporting PG10, we look forward to community feedback and testing. Some release channels, like Ubuntu & RPM-based distros will remain on 9.6 for now
  • Support for CLUSTER on hypertables to recursively apply to chunks
  • Improve constraint handling of edge cases for DATE and TIMESTAMP
  • Fix range_start and range_end to properly handle the full 32-bit int space
  • Allow users to specify their desired partitioning function
  • Enforce NOT NULL constraint on time columns
  • Add testing infrastructure to use Coverity and test PostgreSQL regression tests in TimescaleDB
  • Switch to the CMake build system for better cross-platform support
  • Several other bug fixes, cleanups, and improvements

Notable commits

  • [13e1cb5] Add reindex function
  • [6594018] Handle when create_hypertable is invoked on partitioned table
  • [818bdbc] Add coverity testing
  • [5d0cbc1] Recurse CLUSTER command to chunks
  • [9c7191e] Change TIMESTAMP partitioning to be completely tz-independent
  • [741b256] Mark IMMUTABLE functions as PARALLEL SAFE
  • [2ffb30d] Make aggregate serialize and deserialize functions STRICT
  • [c552410] Add build target to run the standard PostgreSQL regression tests
  • [291050b] Change DATE partitioning to be completely tz-independent
  • [ca0968a] Make all partitioning functions take anyelement argument
  • [a4e1e32] Change range_start and range_end semantics
  • [2dfbc82] Fix off-by-one error on range-end
  • [500563f] Add support for PostgreSQL 10
  • [201a948] Check that time dimensions are set as NOT NULL.
  • [4532650] Allow setting partitioning function
  • [4a0a0d8] Fix column type change on plain tables
  • [cf009cc] Avoid string conversion in hash partitioning
  • [8151098] Improve update testing by adding a rerun test
  • [c420c11] Create a catalog entry for constraint-backed indexes
  • [ec746d1] Add ability to run regression test locally
  • [44f9fec] Add analyze to parallel test for stability
  • [9e0422a] Fix bug with pointer assignment after realloc
  • [114fa8d] Refactor functions used to recurse DDL commands to chunks
  • [b1ec4fa] Refactor build system to use CMake

Thanks

  • @jgraichen for reporting an issue with drop_chunks not accepting BIGINT
  • @nathansgreen for reporting an edge case with constraints for TIMESTAMP
  • @jonmd for reporting a similar edge case for DATE
  • @jwdeitch for a PR to cover an error case in PG10

0.6.1 - Nov 7, 2017

High-level changes

  • Fix several memory bugs that caused segfaults
  • Fix bug when creating expression indexes
  • Plug a memory leak with constraint expressions
  • Several other bug fixes and stability improvements

Notable commits

  • [2799075] Fix EXPLAIN for ConstraintAware and MergeAppend
  • [8084594] Use per-chunk memory context for cached chunks
  • [a13d9de] Do not convert tuples on insert unless needed
  • [da09f24] Limit growth of range table during chunk inserts
  • [85dee79] Fix issue with creating expression indexes
  • [844ff7f] Fix memory leak due to constraint expressions.
  • [e90d3ee] Consider precvious CIS state in copy FROM file to rel
  • [56d632f] Fix bug with pointer assignment after realloc
  • [f97d624] Make event trigger creation idempotent

Thanks

  • @jwdeitch for submitting a patch to correct behavior in the COPY operation
  • @jgraichen for reporting a bug with expression indexes
  • @zixet for reporting a memory leak
  • @djk447 for reporting a bug in EXPLAIN with ConstraintAware and MergeAppend

0.6.0 - Oct 12, 2017

High-level changes

  • Fix bugs where hypertable-specific handlers were affecting normal Postgres tables.
  • Make it so that all TimescaleDB commands can run as a normal user rather than a superuser.
  • Updates to the code to make the extension compileable on Windows; future changes will add steps to properly build.
  • Move time_bucket functions out of public schema (put in schema where extension is).
  • Several other bugs fixes.

Notable commits

  • [1d73fb8] Fix bug with extension starting too early.
  • [fd390ec] Fix chunk index attribute mismatch and locking issue
  • [430ed8a] Fix bug with collected commands in index statement.
  • [614c2b7] Fix permissions bugs and run tests as normal user
  • [ce12104] Fix "ON CONFLICT ON CONSTRAINT" on plain PostgreSQL tables
  • [4c451e0] Fix rename and reindex bugs when objects are not relations
  • [c3ebc67] Fix permission problems with dropping hypertables and chunks
  • [040e815] Remove truncate and hypertable metadata triggers
  • [5c26328] Fix INSERT on hypertables using sub-selects with aggregates
  • [b57e2bf] Prepare C code for compiling on Windows
  • [a2bad2b] Fix constraint validation on regular tables
  • [fb5717f] Remove explicit schema for time_bucket
  • [04d01ce] Split DDL processing into start and end hooks

Thanks

  • @oldgreen for reporting time_bucket being incorrectly put in the public schema and pointing out permission problems
  • @qlandman for reporting a bug with INSERT using sub-selects with aggregates
  • @min-mwei for reporting a deadlock issue during INSERTs
  • @ryan-shaw for reporting a bug where the extension sometimes used pg_cache too soon