MacLochlainns Weblog

Michael McLaughlin's Technical Blog

Site Admin

Archive for the ‘PostgreSQL’ tag

Postgres Drop Structures

with one comment

While building my PostgreSQL environment for the class, I had to write a couple utilities. They do the following:

  1. Drops all the tables from a schema.
  2. Drops all the sequences from a schema that aren’t tied to an _id column with a SERIAL data type.
  3. Drops all the functions and procedures (qualified as routines) from a schema.
  4. Drops all the triggers from a schema.

The following gives you the code for all four files: drop_tables.sql, drop_sequences.sql, drop_routines.sql, and drop_triggers.sql.

  • The drop_tables.sql Script:
  • /* Verify all tables present. */
    SELECT table_name
    FROM   information_schema.tables
    WHERE  table_catalog = current_setting('videodb.catalog_name')
    AND    table_schema = 'public';
     
    DO $$
    DECLARE
      /* Declare an indefinite length string and record variable. */
      sql  VARCHAR;
      row  RECORD;
     
      /* Declare a cursor. */
      table_cursor CURSOR FOR
        SELECT table_name
        FROM   information_schema.tables
        WHERE  table_catalog = current_setting('videodb.catalog_name')
        AND    table_schema = 'public';
    BEGIN
      /* Open the cursor. */
      OPEN table_cursor;
      LOOP
        /* Fetch table names. */
        FETCH table_cursor INTO row;
     
        /* Exit when no more records are found. */
        EXIT WHEN NOT FOUND;
     
        /* Concatenate together a DDL to drop the table with prejudice. */
        sql := 'DROP TABLE IF EXISTS '||row.table_name||' CASCADE';
     
        /* Execute the DDL statement. */
        EXECUTE sql;
      END LOOP;
     
      /* Close the cursor. */
      CLOSE table_cursor;
    END;
    $$;
     
    /* Verify all tables are dropped. */
    SELECT table_name
    FROM   information_schema.tables
    WHERE  table_catalog = current_setting('videodb.catalog_name')
    AND    table_schema = 'public';

  • The drop_sequences.sql script:
  • /* Verify all tables present. */
    SELECT sequence_name
    FROM   information_schema.sequences
    WHERE  sequence_catalog = current_setting('videodb.catalog_name')
    AND    sequence_schema = 'public';
     
    DO $$
    DECLARE
      /* Declare an indefinite length string and record variable. */
      sql  VARCHAR;
      row  RECORD;
     
      /* Declare a cursor. */
      sequence_cursor CURSOR FOR
        SELECT sequence_name
        FROM   information_schema.sequences
        WHERE  sequence_catalog = current_setting('videodb.catalog_name')
        AND    sequence_schema = 'public';
    BEGIN
      /* Open the cursor. */
      OPEN sequence_cursor;
      LOOP
        /* Fetch table names. */
        FETCH sequence_cursor INTO row;
     
        /* Exit when no more records are found. */
        EXIT WHEN NOT FOUND;
     
        /* Concatenate together a DDL to drop the table with prejudice. */
        sql := 'DROP SEQUENCE IF EXISTS '||row.sequence_name;
     
        /* Execute the DDL statement. */
        EXECUTE sql;
      END LOOP;
     
      /* Close the cursor. */
      CLOSE sequence_cursor;
    END;
    $$;
     
    /* Verify all tables are dropped. */
    SELECT sequence_name
    FROM   information_schema.sequences
    WHERE  sequence_catalog = current_setting('videodb.catalog_name')
    AND    sequence_schema = 'public';

  • The drop_routines.sql script:
  • /* Verify all tables present. */
    SELECT routine_name
    ,      routine_type
    FROM   information_schema.routines
    WHERE  specific_catalog = current_setting('videodb.catalog_name')
    AND    specific_schema = 'public';
     
    DO $$
    DECLARE
      /* Declare an indefinite length string and record variable. */
      sql  VARCHAR;
      row  RECORD;
     
      /* Declare a cursor. */
      routine_cursor CURSOR FOR
        SELECT routine_name
        ,      routine_type
        FROM   information_schema.routines
        WHERE  specific_catalog = current_setting('videodb.catalog_name')
        AND    routine_schema = 'public';
    BEGIN
      /* Open the cursor. */
      OPEN routine_cursor;
      LOOP
        /* Fetch table names. */
        FETCH routine_cursor INTO row;
     
        /* Exit when no more records are found. */
        EXIT WHEN NOT FOUND;
     
        /* Concatenate together a DDL to drop the table with prejudice. */
        sql := 'DROP '||row.routine_type||' IF EXISTS '||row.routine_name;
     
        /* Execute the DDL statement. */
        EXECUTE sql;
      END LOOP;
     
      /* Close the cursor. */
      CLOSE routine_cursor;
    END;
    $$;
     
    /* Verify all tables are dropped. */
    SELECT routine_name
    ,      routine_type
    FROM   information_schema.routines
    WHERE  specific_catalog = 'videodb'
    AND    specific_schema = 'public';

  • The drop_triggers.sql script:
  • /* Verify all tables present. */
    SELECT trigger_name
    FROM   information_schema.triggers
    WHERE  trigger_catalog = current_setting('videodb.catalog_name')
    AND    trigger_schema = 'public';
     
    DO $$
    DECLARE
      /* Declare an indefinite length string and record variable. */
      sql  VARCHAR;
      row  RECORD;
     
      /* Declare a cursor. */
      trigger_cursor CURSOR FOR
        SELECT trigger_name
        FROM   information_schema.triggers
        WHERE  trigger_catalog = current_setting('videodb.catalog_name')
        AND    trigger_schema = 'public';
    BEGIN
      /* Open the cursor. */
      OPEN trigger_cursor;
      LOOP
        /* Fetch table names. */
        FETCH trigger_cursor INTO row;
     
        /* Exit when no more records are found. */
        EXIT WHEN NOT FOUND;
     
        /* Concatenate together a DDL to drop the table with prejudice. */
        sql := 'DROP TRIGGER IF EXISTS '||row.trigger_name;
     
        /* Execute the DDL statement. */
        EXECUTE sql;
      END LOOP;
     
      /* Close the cursor. */
      CLOSE trigger_cursor;
    END;
    $$;
     
    /* Verify all tables are dropped. */
    SELECT trigger_name
    FROM   information_schema.triggers
    WHERE  trigger_catalog = current_setting('videodb.catalog_name')
    AND    trigger_schema = 'public';

You can create a cleanup_catalog.sql script to call all four in sequence, like the following:

\i /home/student/Data/cit225/postgres/lib/utility/drop_tables.sql
\i /home/student/Data/cit225/postgres/lib/utility/drop_sequences.sql
\i /home/student/Data/cit225/postgres/lib/utility/drop_routines.sql
\i /home/student/Data/cit225/postgres/lib/utility/drop_triggers.sql

The nice thing about this approach is that you won’t see any notices when tables, sequences, routines, or triggers aren’t found. It’s a clean approach to cleaning the schema for a testing environment.

Written by maclochlainn

October 27th, 2019 at 3:58 pm

pgAdmin4 on Fedora 30

without comments

While attempting an install of pgAdmin and updating a Fedora 30 environment, I encountered a conflict on the upgrade of MySQL 8.0.17-1 to 8.0.17.2. The community-mysql-8.0.17-2.fc30.x86_64 had conflicts with:

  • mysql-community-client-8.0.17-1.fc30.x86_64 package
  • mysql-community-server-8.0.17-1.fc30.x86_64 package

I tried to update the system before install pgadmin4 with the following syntax:

dnf -y update && dnf -y install pgadmin4

The dnf utility raise the following MySQL package errors during transaction checking:

Since I’m not sure what’s wrong or how to fix it, I’ve put it in my queue of things to get to later. However, when I figure it out I’ll update this blog page with the solution or work around. If anybody knows the fix and would like to share, please let me know.

I removed the pending update packages with the following command:

dnf clean packages

Then, I simply installed pgadmin4 with the following command:

dnf -y install pgadmin4

The pgadmin4 configuration instructions can be found for several Linux versions at Josphat Mutai’s Computing for Geeks web page. On Fedora 30, you need to do the following:

  • Install, start, and enable Apache as the httpd service unless you already have done that.
  • Copy the /etc/httpd/conf.d/pgadmin4.conf.sample file to /etc/httpd/conf.d/pgadmin4.conf, which is a new file.
  • Restart the httpd service to incorporate the pgadmin4 configuration file.

After that, you create the following new directories as the root or sudo user:

  • /var/lib/pgadmin4
  • /var/log/pgadmin4

You can make both directories with a single mkdir command, like:

mkdir -p /var/lib/pgadmin4 /var/log/pgadmin4

As the root or sudo user, change the ownership of these two directories to the apache user with the following syntax:

chown -R apache:apache /var/lib/pgadmin4 /var/log/pgadmin4

You add the following four statements to the config_distro.py file in the /usr/lib/python3.7/site-packages/pgadmin4-web directory as the root or sudo user:

LOG_FILE = '/var/log/pgadmin4/pgadmin4.log'
SQLITE_PATH = '/var/lib/pgadmin4/pgadmin4.db'
SESSION_DB_PATH = '/var/lib/pgadmin4/sessions'
STORAGE_DIR = '/var/lib/pgadmin4/storage'

You need to setup the pgadmin user with the following python3 command:

python3 /usr/lib/python3.7/site-packages/pgadmin4-web/setup.py

Enter the following values, a real email address and a password twice:

NOTE: Configuring authentication for SERVER mode.
 
Enter the email address and password to use for the initial pgAdmin user account:
 
Email address: admin@example.com   
Password: your_password
Retype password: your_password
pgAdmin 4 - Application Initialisation
======================================

Assuming you have an enabled firewall, you need to issue the following two commands as the root or sudo user:

rirewall-cmd --permanent --add-service=http
firewall-cmd --reload

You invoke pgAdmin4 from within a browser window with the following URL for a stand alone workstation (for a workstation on a DNS network you would enter pgadmin.domain.domain_type in lieu of localhost):

pgadmin/localhost/pgadmin4

You most likely will encounter an Internal Server Error, the recommended fix is reputed to be:

ausearch -c 'httpd' --raw | audit2allow -M my-httpd
semodule -X 300 -i my-httpd.pp

It didn’t work for me. At the end of the process, I have an Internal Server Error. It is something that I’ll try to fix next. The actual error message:

Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
 
Please contact the server administrator at root@localhost to inform them of the time this error occurred, and the actions you performed just before this error.
 
More information about this error may be available in the server error log.

If somebody figures out the last step before I do, that’s great. Let me and everybody else know the mystery.

On a positive note, the pgadmin4 package provided the psycopg2 library. I had looked for it as a psycopg2 package but it is in python3-psycopg2 package.

Written by maclochlainn

September 21st, 2019 at 5:29 pm