MacLochlainns Weblog

Michael McLaughlin's Technical Blog

Site Admin

Archive for the ‘PostgreSQL Developer’ tag

PostgreSQL Identity Columns

without comments

It’s interesting to see the way different databases implement automatic numbering. Oracle Database 12c is the closest to PostgreSQL in some significant ways. However, its probably more accurate to say Oracle Database 12c copied PostgreSQL’s implementation. At least, that’s my conjecture because Oracle added a way to reset the START WITH value of the indirect sequence. However, I prefer the MySQL approach because the automatic numbering sequence is a property of the table and a simple clause of the CREATE TABLE statement.

Both PostgreSQL and Oracle Database 12c implement automatic numbering as indirect sequences. Indirect sequences are those created by a table when you designate a column as an identity column in Oracle or as a serial column in PostgreSQL. The difference is that PostgreSQL doesn’t provide a syntax version inside the CREATE TABLE semantic.

MySQL provides such syntax. You set an auto numbering column in MySQL by appending the AUTO_INCREMENT clause to the table creation statement when you want it to start with a number other than 1, like this:

CREATE TABLE auto
( id           INT UNSIGNED PRIMARY KEY AUTO_INCREMENT
, text_field   VARCHAR(30)  NOT NULL
) ENGINE=InnoDB AUTO_INCREMENT=1001 DEFAULT CHARSET=utf8;

Oracle disallows you to changing a sequence created as a background activity of the CREATE TABLE statement; and Oracle disallows you dropping an indirect sequence without changing the table that created it, which is exactly how they handle indexes created for unique constraints. Unfortunately, Oracle also disallows altering the START WITH value of any sequence.

If you want to change the START WITH value on an Oracle Database 12c indirect sequence, you must export the table, drop the table, and recreate the table with a new START WITH value before importing the data back into the table. The syntax for setting an IDENTITY column value higher than 1 is:

CREATE TABLE auto
( auto_id     NUMBER GENERATED BY DEFAULT AS IDENTITY (START WITH 1001)
, text_field  VARCHAR2(30)
, CONSTRAINT  auto_pk PRIMARY KEY (auto_id));

You can only create a PostgreSQL table with automatic numbering by using the SERIAL data type, which always sets the initial value to 1. You can reset the SERIAL sequence value in PostgreSQL with the ALTER statement. Unlike Oracle Database 12c, PostgreSQL does let you modify the START WITH value of any sequence. The trick is understanding how to find the sequence name. The name is always the combination of the table name, an underscore, an id string, an underscore, and a seq string. This behavior makes a great case for choosing id as the name of any auto numbering columns in a table.

CREATE TABLE auto
( id          SERIAL      CONSTRAINT auto_pk PRIMARY KEY
, text_field  VARCHAR(30));
 
ALTER SEQUENCE auto_id_seq RESTART WITH 1001;

You can see the table and assigned sequence with the following command in PostgreSQL:

\d+ auto

It should display:

                                                      Table "public.auto"
   Column   |         Type          |                     Modifiers                     | Storage  | Stats target | Description 
------------+-----------------------+---------------------------------------------------+----------+--------------+-------------
 id         | integer               | not null default nextval('auto_id_seq'::regclass) | plain    |              | 
 text_field | character varying(30) |                                                   | extended |              | 
Indexes:
    "auto_pk" PRIMARY KEY, btree (id)
Has OIDs: no

As always, I hope this helps those trying to sort through how to start identity columns above the initial value of 1.

Written by maclochlainn

August 4th, 2017 at 12:52 am

PostgreSQL Composites

without comments

PostgreSQL like Oracle supports record data types but unlike Oracle, PostgreSQL doesn’t support collections of record data types. Here’s an example of how to define a PostgreSQL composite data type, and how to use it as a column’s data type.

CREATE TYPE address_type AS
( street_address  VARCHAR
, city            VARCHAR
, state           VARCHAR
, zip_code        VARCHAR );

Then, you define an ADDRESS table, like:

CREATE TABLE address
( address_id      SERIAL
, address_struct  ADDRESS_TYPE );

You can now insert rows like:

-- Insert the first row.
INSERT INTO address
( address_struct )
VALUES
(('52 Hubble Street','Lexington','KY','40511-1225'));
 
-- Insert the second row.
INSERT INTO address
( address_struct )
VALUES
(('54 Hubble Street','Lexington','KY','40511-1225'));

Then, you can query them like this:

SELECT * FROM address;

It returns:

 address_id |                address_struct
------------+----------------------------------------------
          1 | ("52 Hubble Street",Lexington,KY,40511-1225)
          2 | ("54 Hubble Street",Lexington,KY,40511-1225)
(2 rows)

You must use parentheses around the ADDRESS_STRUCT column to query individual items, like:

SELECT   address_id
,       (address_struct).street_address
,       (address_struct).city
,       (address_struct).state
,       (address_struct).zip_code
FROM     address;

It returns output like a table:

 address_id |  street_address  |   city    | state |  zip_code
------------+------------------+-----------+-------+------------
          1 | 52 Hubble Street | Lexington | KY    | 40511-1225
          2 | 54 Hubble Street | Lexington | KY    | 40511-1225
(2 rows)

While you can define a table that holds an array of a composite type, there’s no syntax that appears to work with an array of a composite type. I hope this helps those interested in implementing record structures in PostgreSQL.

Written by maclochlainn

March 16th, 2015 at 2:02 am

PostgreSQL Auto IDs

with 2 comments

PostgreSQL’s approach to automatic numbering is as simple as Oracle but different than MySQL, and Microsoft SQL Server. For example, you have a two-step process with Oracle, PostgreSQL, MySQL, and Microsoft SQL Server. First, you create an Oracle table with the GENERATED AS IDENTITY clause, a PostgreSQL table with the SERIAL data type, a MySQL table with the AUTO_INCREMENT clause, and a Microsoft SQL Server table with the IDENTITY(1,1) clause. Then, you need to write an INSERT statement for Oracle, MySQL, or Microsoft SQL Server like:

  1. Oracle’s INSERT statement excludes the auto-incrementing column from the list of columns or provides a NULL value in the VALUES-list. You can then assign the RETURNING INTO result from an INSERT statement to a session-level (bind) variable.
  2. MySQL’s INSERT statement excludes the auto-incrementing column from the list of columns or provides a NULL value in the VALUES-list. You can then assign the LAST_INSERT_ID() function value to a session-level variable, and populate a foreign key column.
  3. Microsoft SQL Server’s INSERT statement excludes the auto-incrementing column from the list of columns or provides a NULL value in the VALUES-list. You can then assign the SCOPE_IDENTITY() function’s value to a session-level variable, and populate a foreign key column.

PostgreSQL differs because it works differently between the SQL and PL/pgSQL contexts. Let’s look at how you link the insert of data into two tables in both contexts.

The following PostgreSQL syntax creates an ADDRESS table with an auto incrementing ADDRESS_ID column that uses a SERIAL data type, which acts like an auto numbering column:

/* Create a customer table. */
CREATE TABLE customer
( customer_id     SERIAL CONSTRAINT customer_pk PRIMARY KEY
, first_name      VARCHAR(20)
, last_name       VARCHAR(20));
 
/* Create an address table. */
CREATE TABLE address
( address_id      SERIAL CONSTRAINT address_pk PRIMARY KEY
, customer_id     INTEGER
, street_address  VARCHAR(40)
, city            VARCHAR(30)
, state           VARCHAR(8)
, zip_code        VARCHAR(10));

If you want to insert one row into the CUSTOMER table and a related row in the ADDRESS table. You have two possible approaches. One works in both the SQL and PL/pgSQL contexts. That mechanism requires you to use a scalar subquery to capture the foreign key value of the CUSTOMER_ID column in the ADDRESS table, like this:

/* Insert into customer table. */
INSERT INTO customer
( first_name, last_name ) 
VALUES
('F. Scott','Fitzgerald');
 
/* Insert into address table. */
INSERT INTO address
( customer_id
, street_address
, city
, state
, zip_code )
VALUES
((SELECT   customer_id
  FROM     customer
  WHERE    first_name = 'F. Scott'
  AND      last_name = 'Fitzgerald')
,'599 Summit Avenue'
,'St. Paul'
,'Minnesota'
,'55102');

The RETURNING INTO clause of PostgreSQL only works in a PL/pgSQL context, like this:

DO $$
DECLARE lv_customer_id INTEGER;
BEGIN
  /* Insert into customer table. */
  INSERT INTO customer
  ( first_name, last_name ) 
  VALUES
  ('Madeleine','Smith') 
  RETURNING customer_id INTO lv_customer_id;
 
  /* Insert into address table. */
  INSERT INTO address
  ( customer_id
  , street_address
  , city
  , state
  , zip_code )
  VALUES
  ( lv_customer_id
  ,'7 Blythswood Square'
  ,'Glasgow'
  ,'Scotland'
  ,'G2 4BG');
 
  /* Manage any exceptions. */
  EXCEPTION
    WHEN OTHERS THEN
      RAISE NOTICE '% %', SQLERRM, SQLSTATE;
END$$;

You query the auto generated values and data from the INSERT statement to the CUSTOMER table with a scalar subquery against the natural key (the FIRST_NAME and LAST_NAME columns) from the ADDRESS table. The following is an example of such a query:

SELECT   *
FROM     customer c INNER JOIN address a
ON       c.customer_id = a.customer_id;

It returns:

 customer_id | first_name | last_name  | address_id | customer_id |   street_address    |   city   |   state   | zip_code
-------------+------------+------------+------------+-------------+---------------------+----------+-----------+----------
           1 | F. Scott   | Fitzgerald |          1 |           1 | 599 Summit Avenue   | St. Paul | Minnesota | 55102
           2 | Madeleine  | Smith      |          2 |           2 | 7 Blythswood Square | Glasgow  | Scotland  | G2 4BG
(2 rows)

My take is that the RETURNING column_value INTO local_value clause is a better approach than using Oracle’s .NEXTVAL and .CURRVAL values. I also think the RETURNING INTO clause is a better approach than using MySQL’s LAST_INSERT_ID() or Microsoft SQL Server’s SCOPE_IDENTITY().

Initially, I felt it was odd that the PostgreSQL disallows the RETURNING INTO clause in a SQL context, because it allows the syntax in a PL/pgSQL context. After some reflection the choice makes more sense because most developers work within a procedural context when they use transactions across two or more tables. PL/pgSQL is PostgreSQL’s procedural context from managing transactions across two or more tables.

As always, I hope this helps.

Written by maclochlainn

March 15th, 2015 at 9:37 pm

A PL/pgSQL Function

without comments

Somebody wanted to know how to write a basic PostgreSQL PL/pgSQL function that returned a full name whether or not the middle name was provided. That’s pretty simple. There are principally two ways to write that type of concatenation function. One uses formal parameter names and the other uses positional values in lieu of the formal parameter names.

The two ways enjoy two techniques (SQL language and PL/pgSQL language), which gives us four possible solutions. I’ve also provided a conditional drop statement for the full_name function. If you’re new to PostgreSQL the DROP statement might make you scratch your head because you’re wondering why you need to use the formal parameter list. The DROP statement needs the parameter list because PostgeSQL lets you overload schema/database functions and procedures.

The code is for a named parameter lists using the SQL language is:

DROP FUNCTION IF EXISTS full_name
( IN pv_first_name  text
, IN pv_middle_name text
, IN pv_full_name   text);
 
CREATE FUNCTION full_name
( IN  pv_first_name  text
, IN  pv_middle_name text
, IN  pv_last_name   text
, OUT pv_full_name   text) AS
   'SELECT  pv_first_name
    ||      CASE
              WHEN pv_middle_name IS NOT NULL THEN '' '' || pv_middle_name || '' ''
              ELSE '' ''
            END
    ||      pv_last_name'
    LANGUAGE SQL;

The code is for a positional parameter lists using the SQL language is:

DROP FUNCTION IF EXISTS full_name
( IN text
, IN text
, IN text);
 
CREATE FUNCTION full_name
( IN text
, IN text
, IN text
, OUT text) AS
   'SELECT  $1
    ||      CASE
              WHEN $2 IS NOT NULL THEN '' '' || $2 || '' ''
              ELSE '' ''
            END
    ||      $3'
    LANGUAGE SQL;

You would re-write the function in the PL/pgSQL language as follows (please note the named parameter list):

CREATE FUNCTION full_name
( IN  pv_first_name   text
, IN  pv_middle_name  text
, IN  pv_last_name    text) RETURNS text AS
$$
DECLARE
  lv_output text;
BEGIN
  IF pv_middle_name IS NULL THEN
    lv_output = CONCAT(pv_first_name, N' ', pv_last_name);
  ELSE
    lv_output = CONCAT(pv_first_name, N' ', pv_middle_name, N' ', pv_first_name);
  END IF;
  RETURN lv_output;
END
$$
LANGUAGE plpgsql IMMUTABLE;

You can test either version of the program with the following two queries from the pseudo table dual, which isn’t require in the SELECT statement:

SELECT full_name('Henry',NULL,'Pym') AS "Ant-Man"
UNION ALL
SELECT full_name('Henry','''Hank''','Pym') AS "Ant-Man";

It prints:

Ant-Man
text
----------------
Henry Pym
Henry 'Hank' Pym

As always, I hope this helps those looking for how to accomplish a concatenation function in PostgreSQL.

Written by maclochlainn

January 22nd, 2015 at 1:03 am

Fedora PostgreSQL Install

without comments

Somebody asked how to put PostgreSQL on my Fedora image with Oracle Database 11g and MySQL. It’s fairly simple. You can check for the current download at yum.postgresql.org and then download it like this as the root user:

yum localinstall http://yum.postgresql.org/9.3/fedora/fedora-20-x86_64/pgdg-fedora93-9.3-1.noarch.rpm

You should see the following output when the download is successful, don’t forget to type y to complete the download:

Loaded plugins: langpacks, refresh-packagekit
pgdg-fedora93-9.3-1.noarch.rpm                              | 5.1 kB  00:00     
Examining /var/tmp/yum-root-2EPf_J/pgdg-fedora93-9.3-1.noarch.rpm: pgdg-fedora93-9.3-1.noarch
Marking /var/tmp/yum-root-2EPf_J/pgdg-fedora93-9.3-1.noarch.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package pgdg-fedora93.noarch 0:9.3-1 will be installed
--> Finished Dependency Resolution
 
Dependencies Resolved
 
================================================================================
 Package           Arch       Version     Repository                       Size
================================================================================
Installing:
 pgdg-fedora93     noarch     9.3-1       /pgdg-fedora93-9.3-1.noarch     2.1 k
 
Transaction Summary
================================================================================
Install  1 Package
 
Total size: 2.1 k
Installed size: 2.1 k
Is this ok [y/d/N]: y
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction (shutdown inhibited)
  Installing : pgdg-fedora93-9.3-1.noarch                                   1/1 
  Verifying  : pgdg-fedora93-9.3-1.noarch                                   1/1 
 
Installed:
  pgdg-fedora93.noarch 0:9.3-1                                                  
 
Complete!

After downloading the packages, you install with the following command:

yum install postgresql93-server

You should see the following output when the installation is successful, don’t forget to type y to complete the installation:

Loaded plugins: langpacks, refresh-packagekit
pgdg93                                                      | 3.6 kB  00:00     
(1/2): pgdg93/20/x86_64/group_gz                            |  332 B  00:00     
(2/2): pgdg93/20/x86_64/primary_db                          |  84 kB  00:00     
Resolving Dependencies
--> Running transaction check
---> Package postgresql93-server.x86_64 0:9.3.5-1PGDG.f20 will be installed
--> Processing Dependency: postgresql93-libs(x86-64) = 9.3.5-1PGDG.f20 for package: postgresql93-server-9.3.5-1PGDG.f20.x86_64
--> Processing Dependency: postgresql93(x86-64) = 9.3.5-1PGDG.f20 for package: postgresql93-server-9.3.5-1PGDG.f20.x86_64
--> Processing Dependency: postgresql93 = 9.3.5-1PGDG.f20 for package: postgresql93-server-9.3.5-1PGDG.f20.x86_64
--> Processing Dependency: libpq.so.5()(64bit) for package: postgresql93-server-9.3.5-1PGDG.f20.x86_64
--> Running transaction check
---> Package postgresql93.x86_64 0:9.3.5-1PGDG.f20 will be installed
---> Package postgresql93-libs.x86_64 0:9.3.5-1PGDG.f20 will be installed
--> Finished Dependency Resolution
 
Dependencies Resolved
 
================================================================================
 Package                  Arch        Version                 Repository   Size
================================================================================
Installing:
 postgresql93-server      x86_64      9.3.5-1PGDG.f20         pgdg93      3.6 M
Installing for dependencies:
 postgresql93             x86_64      9.3.5-1PGDG.f20         pgdg93      1.0 M
 postgresql93-libs        x86_64      9.3.5-1PGDG.f20         pgdg93      203 k
 
Transaction Summary
================================================================================
Install  1 Package (+2 Dependent packages)
 
Total download size: 4.8 M
Installed size: 22 M
Is this ok [y/d/N]: y
Downloading packages:
(1/3): postgresql93-libs-9.3.5-1PGDG.f20.x86_64.rpm         | 203 kB  00:00     
(2/3): postgresql93-9.3.5-1PGDG.f20.x86_64.rpm              | 1.0 MB  00:01     
(3/3): postgresql93-server-9.3.5-1PGDG.f20.x86_64.rpm       | 3.6 MB  00:02     
--------------------------------------------------------------------------------
Total                                              1.6 MB/s | 4.8 MB  00:02     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction (shutdown inhibited)
  Installing : postgresql93-libs-9.3.5-1PGDG.f20.x86_64                     1/3 
  Installing : postgresql93-9.3.5-1PGDG.f20.x86_64                          2/3 
  Installing : postgresql93-server-9.3.5-1PGDG.f20.x86_64                   3/3 
  Verifying  : postgresql93-server-9.3.5-1PGDG.f20.x86_64                   1/3 
  Verifying  : postgresql93-9.3.5-1PGDG.f20.x86_64                          2/3 
  Verifying  : postgresql93-libs-9.3.5-1PGDG.f20.x86_64                     3/3 
 
Installed:
  postgresql93-server.x86_64 0:9.3.5-1PGDG.f20                                  
 
Dependency Installed:
  postgresql93.x86_64 0:9.3.5-1PGDG.f20                                         
  postgresql93-libs.x86_64 0:9.3.5-1PGDG.f20                                    
 
Complete!

You can confirm the installation with the following command:

rpm -qa | grep postgres

It returns:

postgresql93-9.3.5-1PGDG.f20.x86_64
postgresql93-server-9.3.5-1PGDG.f20.x86_64
postgresql93-libs-9.3.5-1PGDG.f20.x86_64

You’ve now installed PostgreSQL but did you really? If you’re asking that questions you have other questions. Let me try to answer them quickly, here:

You have installed PostgreSQL and created a postgres user. postgres is the owner of the PostgreSQL database. You can connect to the database as the postgres user without credentials because that’s where you administer the database. However, you can’t connect using ssh as the postgres user. You must use sudo to assume the root user’s privileges and then use the su command to become the postgres user.

If you just completed the installation, you are the root user. You can verify that with a call to the whoami utility:

whoami

It should return:

root

You connect as the postgres user with the su utility like this:

su - postgres

If you rerun the whoami command now, you should see:

postgres

You can start the PostgreSQL command-line utility (psql), like this:

psql

At the postgres (or psql) prompt, you can interactively confirm the setup of a database installation:

postgres=# SELECT setting as "Data Location"
postgres-# FROM   pg_settings
postgres-# WHERE  name = 'data_directory';

It should return the following:

      Data Location      
-------------------------
 /var/lib/pgsql/9.3/data
(1 row)

At this point, you should refer to this other blog post that shows you how to setup a new Database or Schema in PostgreSQL. You can find basic Postgres help files in this other blog post. As always, I hope this helps those timid about adding new software.

Written by maclochlainn

September 24th, 2014 at 2:43 am

PostgreSQL New Database

with one comment

How time flies, last March I explained how to install and configure PostgreSQL on Windows. It was my intent to start posting more content on PostgreSQL but I was distracted by another writing commitment on Oracle Database 12c PL/SQL Advanced Programming Techniques, which should be available in November. It tempted me away from PostgreSQL because I got to write about how to use Java inside Oracle Database 12c, which was fun. Having completed that, I’m back on task. Here’s the second entry on PostgreSQL. It shows you howto create your own database, database administrator role, user, and how to connect with psql CLI (Command Line Interface) as the new user.

  1. Create a user-defined video_db tablespace for your database. This requires that you know where the physical files where created when you installed PostgreSQL. You can discover the directory with the following query:
SELECT   setting AS "Data Location"
FROM     pg_settings
WHERE    name = 'data_directory';
            Data Location
--------------------------------------
 C:/Program Files/PostgreSQL/9.3/data
(1 row)

You create the video_db tablespace with the following syntax:

CREATE TABLESPACE video_db
  OWNER postgres
  LOCATION 'C:\Program Files\PostgreSQL\9.3\data';

You can check the presence of the video_db tablespace after creating it with the following query:

SELECT * FROM pg_tablespace;

It should print:

  spcname   | spcowner | spcacl | spcoptions
------------+----------+--------+------------
 pg_default |       10 |        |
 pg_global  |       10 |        |
 video_db   |       10 |        |
(3 rows)

It’s important to note for those new to PostgreSQL that the pg_global and pg_default tablespaces are creating when initializing the database. The pg_global holds shared tables and the pg_default holds everything else.

  1. Create a database that uses your user-defined video_db tablespace with the following two commands:
CREATE DATABASE videodb
  WITH OWNER = postgres
       ENCODING = 'UTF8'
       TABLESPACE = video_db
       LC_COLLATE = 'English_United States.1252'
       LC_CTYPE = 'English_United States.1252'
       CONNECTION LIMIT = -1;
 
COMMENT ON DATABASE videodb
  IS 'VideoDB';
  1. Create a database role, grant the super user privileges to the role, and create a user with the role. You can do that with the following three commands:
CREATE ROLE dba
  WITH SUPERUSER;
 
GRANT ALL PRIVILEGES ON DATABASE videodb TO dba;
 
CREATE USER video
  WITH ROLE dba
       PASSWORD 'video';
  1. Connect to the new videodb database with the psql CLI as the video user. You can do that with the following OS command:
psql -d videodb -U video
  1. Once connected as the new video user, you can use a system information function to determine the current database:
SELECT current_database();

It should display:

 current_database
------------------
 videodb
(1 row)

There are many privilege options, and you should choose wisely which ones you use. As always, I hope this answers questions for other users.

Written by maclochlainn

July 24th, 2014 at 3:22 pm