MacLochlainns Weblog

Michael McLaughlin's Technical Blog

Site Admin

Archive for the ‘SQL*Plus’ Category

Oracle 23c Free Ext Files

without comments

This is an example of how you would upload data from a flat file, or Comma Separated Value (CSV) file inside Docker Oracle Database 23c Free. It’s important to note that in the file upload you are transferring information that doesn’t have surrogate key values by leveraing joins inside a MERGE statement.

Step #1 : Create a virtual directory

You can create a virtual directory without a physical directory but it won’t work when you try to access it. Therefore, you should create the physical directory first. Assuming you’ve created the Docker Oracle Database 23c Free instance, you should put the code in subdirectories of the /opt/oracle file directory.

  • Connect as the root user with the following Docker command:

    docker exec -it --user root oracle23c bash

    Issue the following commands as the oracle user inside the Docker container to create the necessary physical directories. You may need to refer to my earlier blog post if you haven’t setup the oracle user inside the Docker instance. While this blog post will only use the /opt/oracle/upload/text and /opt/oracle/upload/log directories, a subsequent post will demonstrate the preprocessing module for the external tables.

    mkdir /opt/oracle/upload
    mkdir /opt/oracle/upload/text
    mkdir /opt/oracle/upload/log
    mkdir /opt/oracle/upload/preproc
  • Connect to the Oracle Database 23c Free inside the container as the system user to create a c##studentrole, and do the following three things:

    • Grant privileges to the c##studentrole, and grant the c##studentrole to the c##student user.

      -- Create the role.
      CREATE ROLE c##studentrole;
       
      -- Grant privileges to the role.
      GRANT CREATE CLUSTER, CREATE INDEXTYPE, CREATE PROCEDURE,
      CREATE SEQUENCE, CREATE SESSION, CREATE TABLE, CREATE TRIGGER,
      CREATE TYPE, CREATE VIEW TO c##studentrole;
       
      -- Grant privileges to the user.
      GRANT c##studentrole TO c##student;
    • As the system user, create the necessary virtual directories that map to the physical directories inside the Docker container:

      CREATE DIRECTORY upload AS '/opt/oracle/upload/text';
      CREATE DIRECTORY preproc AS '/opt/oracle/upload/preproc';
      CREATE DIRECTORY LOG AS '/opt/oracle/upload/log';
    • As the system user, grant the necessary privileges on the virtual directories to the c##studentrole role:

      GRANT read ON DIRECTORY upload TO c##studentrole;
      GRANT read, WRITE ON DIRECTORY LOG TO c##studentrole;
      GRANT read, EXECUTE ON DIRECTORY preproc TO c##studentrole;

Step #2 : Position your CSV file in the physical directory

After creating the virtual directory, copy the following contents into a file named kingdom_import.csv in the /opt/oracle/upload/texgt directory or folder. If you attempt to do this in Windows, you need to disable Windows UAC before performing this step.

Place the following in the kingdom_import.csv file. The trailing commas aren’t too meaningful in Oracle but they’re very helpful if you use the file in MySQL. A key element in creating this files requires that you avoid trailing line returns at the bottom of the file because they’re inserted as null values. There should be no lines after the last row of data.

'Narnia',77600,'Peter the Magnificent','20-MAR-1272','19-JUN-1292','The Lion, The Witch and The Wardrobe',
'Narnia',77600,'Edmund the Just','20-MAR-1272','19-JUN-1292','The Lion, The Witch and The Wardrobe',
'Narnia',77600,'Susan the Gentle','20-MAR-1272','19-JUN-1292','The Lion, The Witch and The Wardrobe',
'Narnia',77600,'Lucy the Valiant','20-MAR-1272','19-JUN-1292','The Lion, The Witch and The Wardrobe',
'Narnia',42100,'Peter the Magnificent','12-APR-1531','31-MAY-1328','Prince Caspian',
'Narnia',42100,'Edmund the Just','12-APR-1531','31-MAY-1328','Prince Caspian',
'Narnia',42100,'Susan the Gentle','12-APR-1531','31-MAY-1328','Prince Caspian',
'Narnia',42100,'Lucy the Valiant','12-APR-1531','31-MAY-1328','Prince Caspian',
'Camelot',15200,'King Arthur','10-MAR-0631','12-DEC-0686','The Once and Future King',
'Camelot',15200,'Sir Lionel','10-MAR-0631','12-DEC-0686','The Once and Future King',
'Camelot',15200,'Sir Bors','10-MAR-0631','12-DEC-0635','The Once and Future King',
'Camelot',15200,'Sir Bors','10-MAR-0640','12-DEC-0686','The Once and Future King',
'Camelot',15200,'Sir Galahad','10-MAR-0631','12-DEC-0686','The Once and Future King',
'Camelot',15200,'Sir Gawain','10-MAR-0631','12-DEC-0686','The Once and Future King',
'Camelot',15200,'Sir Tristram','10-MAR-0631','12-DEC-0686','The Once and Future King',
'Camelot',15200,'Sir Percival','10-MAR-0631','12-DEC-0686','The Once and Future King',
'Camelot',15200,'Sir Lancelot','30-SEP-0670','12-DEC-0686','The Once and Future King',

Step #3 : Reconnect as the student user

Disconnect and connect as the c##student user, or reconnect as the c##student user. The reconnect syntax that protects your password is:

CONNECT c##student@free

Step #4 : Run the script that creates tables and sequences

Copy the following into a create_kingdom_upload.sql file within a directory of your choice. I use varchar as the data type because it’s an alias for varchar2 and highlights appropriately with the GeSHi formatting. Then, run it as the student account.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
-- Conditionally drop tables.
DROP TABLE IF EXISTS kingdom;
DROP TABLE IF EXISTS knight;
DROP TABLE IF EXISTS kingdom_knight_import;
 
-- Conditionally drop sequences.
DROP SEQUENCE IF EXISTS kingdom_s1;
DROP SEQUENCE IF EXISTS knight_s1;
 
-- Create normalized kingdom table.
CREATE TABLE kingdom
( kingdom_id    NUMBER
, kingdom_name  VARCHAR(20)
, population    NUMBER
, book          VARCHAR(40));
 
-- Create a sequence for the kingdom table.
CREATE SEQUENCE kingdom_s1;
 
-- Create normalized knight table.
CREATE TABLE knight
( knight_id             NUMBER
, knight_name           VARCHAR(22)
, kingdom_allegiance_id NUMBER
, allegiance_start_date DATE
, allegiance_end_date   DATE
, book                  VARCHAR(40));
 
-- Create a sequence for the knight table.
CREATE SEQUENCE knight_s1;
 
-- Create external import table.
CREATE TABLE kingdom_knight_import
( kingdom_name           VARCHAR(20)
, population             NUMBER
, knight_name            VARCHAR(22)
, allegiance_start_date  DATE
, allegiance_end_date    DATE
, book                   VARCHAR(40))
  ORGANIZATION EXTERNAL
  ( TYPE oracle_loader
    DEFAULT DIRECTORY upload
    ACCESS PARAMETERS
    ( RECORDS DELIMITED BY NEWLINE CHARACTERSET US7ASCII
      BADFILE     'LOG':'kingdom_import.bad'
      DISCARDFILE 'LOG':'kingdom_import.dis'
      LOGFILE     'LOG':'kingdom_import.log'
      FIELDS TERMINATED BY ','
      OPTIONALLY ENCLOSED BY "'"
      MISSING FIELD VALUES ARE NULL )
    LOCATION ('kingdom_import.csv'))
REJECT LIMIT UNLIMITED;

Step #5 : Test your access to the external table

There a number of things that could go wrong with setting up an external table, such as file permissions. Before moving on to the balance of the steps, you should test what you’ve done. Run the following query from the student account to check whether or not you can access the kingdom_import.csv file.

1
2
3
4
5
6
7
8
9
10
11
12
SET PAGESIZE 999
COL kingdom_name  FORMAT A7     HEADING "Kingdom|Name"
COL folks         FORMAT 99999  HEADING "Folks"
COL knight_name   FORMAT A21    HEADING "Knight Name"
COL dates         FORMAT A11    HEADING "Start Date"
COL source_book   FORMAT A38    HEADING "Book"
SELECT   kingdom_name
,        knight_name
,        TO_CHAR(allegiance_start_date,'DD-MON-YYYY')
||       TO_CHAR(allegiance_end_date,'DD-MON-YYYY') AS dates
,        book
FROM     kingdom_knight_import;

Step #6 : Create the upload procedure

Copy the following into a create_upload_procedure.sql file within a virtual directory of your choice. As noted above in the external table definition writes only occur in the log virtual directory. This is important because there are articles out there on the Internet that could misdirect you when you get the following error message on the upload virtual directory.

ORA-06564: Object UPLOAD does not exist or is not accessible to the user.

By the way, you’ll only see that error if you fail to:

  • Designate the procedure as AUTH_ID CURRENT, and
  • Enabled SERVEROUTPUT inside the SQL*Plus command-line interface (CLI) session or inside the glogin.sql file for the Oracle Database 23c Free Docker instance.

Then, run it as the student account.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
-- Create a procedure to wrap the transaction.
CREATE OR REPLACE
  PROCEDURE upload_kingdom AUTHID CURRENT_USER IS 
BEGIN
  -- Set save point for an all or nothing transaction.
  SAVEPOINT starting_point;
 
  -- Insert or update the table, which makes this rerunnable when the file hasn't been updated.  
  MERGE INTO kingdom target
  USING (SELECT   DISTINCT
                  k.kingdom_id
         ,        kki.kingdom_name
         ,        kki.population
         ,        kki.book
         FROM     kingdom_knight_import kki LEFT JOIN kingdom k
         ON       kki.kingdom_name = k.kingdom_name
         AND      kki.population = k.population
         AND      kki.book = k.book) SOURCE
  ON (target.kingdom_id = SOURCE.kingdom_id)
  WHEN MATCHED THEN
  UPDATE SET kingdom_name = SOURCE.kingdom_name
  WHEN NOT MATCHED THEN
  INSERT VALUES
  ( kingdom_s1.nextval
  , SOURCE.kingdom_name
  , SOURCE.population
  , SOURCE.book);
 
  -- Insert or update the table, which makes this rerunnable when the file hasn't been updated.  
  MERGE INTO knight target
  USING (SELECT   kn.knight_id
         ,        kki.knight_name
         ,        k.kingdom_id
         ,        kki.allegiance_start_date AS start_date
         ,        kki.allegiance_end_date AS end_date
         ,        kki.book
         FROM     kingdom_knight_import kki INNER JOIN kingdom k
         ON       kki.kingdom_name = k.kingdom_name
         AND      kki.population = k.population LEFT JOIN knight kn 
         ON       k.kingdom_id = kn.kingdom_allegiance_id
         AND      kki.knight_name = kn.knight_name
         AND      kki.allegiance_start_date = kn.allegiance_start_date
         AND      kki.allegiance_end_date = kn.allegiance_end_date
         AND      kki.book = kn.book) SOURCE
  ON (target.kingdom_allegiance_id = SOURCE.kingdom_id)
  WHEN MATCHED THEN
  UPDATE SET allegiance_start_date = SOURCE.start_date
  ,          allegiance_end_date = SOURCE.end_date
  ,          book = SOURCE.book
  WHEN NOT MATCHED THEN
  INSERT VALUES
  ( knight_s1.nextval
  , SOURCE.knight_name
  , SOURCE.kingdom_id
  , SOURCE.start_date
  , SOURCE.end_date
  , SOURCE.book);
 
  -- Save the changes.
  COMMIT;
 
EXCEPTION
  WHEN OTHERS THEN
    dbms_output.put_line(SQLERRM);
    ROLLBACK TO starting_point;
    RETURN;
END;
/

Step #7 : Run the upload procedure

You can run the file by calling the script above. The procedure ensures that records are inserted or updated into their respective tables.

EXECUTE upload_kingdom;

Step #8 : Test the results of the upload procedure

You can test whether or not it worked by running the following queries.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
-- Format Oracle output.
COLUMN kingdom_id    FORMAT 999      HEADING "Kingdom|ID #"
COLUMN kingdom_name  FORMAT A14      HEADING "Kingdom|Name"
COLUMN population    FORMAT 999,999  HEADING "Population"
COLUMN book          FORMAT A40      HEADING "Source Book"
 
-- Check the kingdom table.
SELECT * FROM kingdom;
 
-- Format Oracle output.
SET PAGESIZE 999
COLUMN knight_id              FORMAT 999  HEADING "Knight|ID #"
COLUMN knight_name            FORMAT A23  HEADING "Knight|Name"
COLUMN kingdom_allegiance_id  FORMAT 999  HEADING "Kingdom|ID #"
COLUMN allegiance_start_date  FORMAT A11 HEADING "Allegiance|Start Date"
COLUMN allegiance_end_date    FORMAT A11 HEADING "Allegiance|End Date"
 
-- Check the knight table.
SELECT   knight_id
,        knight_name
,        kingdom_allegiance_id
,        TO_CHAR(allegiance_start_date,'DD-MON-YYYY') AS allegiance_start_date
,        TO_CHAR(allegiance_end_date,'DD-MON-YYYY') AS allegiance_end_date
FROM     knight;

It should display the following information:

Kingdom Kingdom
   ID # Name	       Population Source Book
------- -------------- ---------- ----------------------------------------
      1 Narnia		   42,100 Prince Caspian
      2 Narnia		   77,600 The Lion, The Witch and The Wardrobe
      3 Camelot 	   15,200 The Once and Future King
 
 
Knight Knight		       Kingdom Allegiance  Allegiance
  ID # Name			  ID # Start Date  End Date
------ ----------------------- ------- ----------- -----------
     1 Peter the Magnificent	     2 20-MAR-1272 19-JUN-1292
     2 Edmund the Just		     2 20-MAR-1272 19-JUN-1292
     3 Susan the Gentle 	     2 20-MAR-1272 19-JUN-1292
     4 Lucy the Valiant 	     2 20-MAR-1272 19-JUN-1292
     5 Peter the Magnificent	     1 12-APR-1531 31-MAY-1328
     6 Edmund the Just		     1 12-APR-1531 31-MAY-1328
     7 Susan the Gentle 	     1 12-APR-1531 31-MAY-1328
     8 Lucy the Valiant 	     1 12-APR-1531 31-MAY-1328
     9 King Arthur		     3 10-MAR-0631 12-DEC-0686
    10 Sir Lionel		     3 10-MAR-0631 12-DEC-0686
    11 Sir Bors 		     3 10-MAR-0631 12-DEC-0635
    12 Sir Bors 		     3 10-MAR-0640 12-DEC-0686
    13 Sir Galahad		     3 10-MAR-0631 12-DEC-0686
    14 Sir Gawain		     3 10-MAR-0631 12-DEC-0686
    15 Sir Tristram		     3 10-MAR-0631 12-DEC-0686
    16 Sir Percival		     3 10-MAR-0631 12-DEC-0686
    17 Sir Lancelot		     3 30-SEP-0670 12-DEC-0686

You can rerun the procedure to check that it doesn’t alter any information, then you could add a new knight to test the insertion portion.

Written by maclochlainn

January 6th, 2024 at 11:53 pm

Oracle 23c Free SQL*Plus

without comments

It’s always frustrated me when using the sqlplus command-line interface (CLI) that you can’t just “up arrow” to through the history. At least, that’s the default case unless you wrap the sqlplus executable.

I like to do my development work as close to the database as possible. The delay from SQL Developer to the database or VSCode to the database is just too long. Therefore, I like the native sqlplus to be as efficient as possible. This post shows you how to install the rlwarp utility to wrap sqlplus and create a sandboxed student user for a local development account inside the Oracle 23c Free container. You should note that the Docker or Podman Container is using Oracle Unbreakable Linux 8 as it’s native OS.

You can connect to your Docker version of Oracle Database 23c Free with the following command:

docker exec -it -u root oracle23c bash

You can’t just use dnf to install rlwrap and get it to magically install all the dependencies. That would be too easy, eh?

Attempting to do so will lock your base OS and eventually force you to kill with prejudice the hung dnf process (at least it forced me to do so). You need to determine the rlwrap dependencies and then install them first. In that process, I noticed that the which utility program wasn’t installed in the container.

Naturally, I installed the which utility first with this command:

dnf install -y which

The rlwrap dependencies are: glibc, ncurses, perl, readline, python, and git. Only the perl, python, and git are missing from the list of formal dependencies but there’s another dependency the epel-release package.

If you want to verify whether a package is installed, you can use the rpm command like this:

rpm -qa | grep package_name

I installed the perl programming environment (a big install) with this command:

dnf install -y perl

I installed the python3 with this command:

dnf install -y python3

I installed the git module with this command:

dnf install -y git

I installed the epel-release container with this command:

dnf install -y epel-release

After installing all of these, you’re now ready to install the core rlwrap utility program. Like the other installations, you use:

dnf install -y rlwrap

At this point, you need to create a sandboxed user account for the Docker instance because as a developer using the root user for simple tasks is a bad idea. While you could do this with a Docker command, the Oracle 23c Free edition raised a lock on the /etc/group file when I tried it. Naturally, that’s not a problem because you can connect as the root user with this syntax:

docker exec -it -u root oracle23c bash

As the root user, create a student account as a developer account in the Oracle 23c Free container:

useradd -u 501 -g dba -G users -d /home/student -s /bin/bash/ -c "Student" -n student

You’ll be unable to leverage the tnsnames.ora file unless you alter the prior command to replace dba with oinstall or add the following command:

usermod -a -G oinstall student

Exit the Oracle 23c Free container as the root user and reconnect as the student user with this syntax:

docker exec -it --user student oracle23c bash

While you’re connected as the root user, you should create an upload directory as a subdirectory of the $ORACLE_BASE directory. The $ORACLE_BASE directory in the Oracle Database 23c Free Docker image is the /opt/oracle directory.

You should use the following syntax to create the upload directory and change its permission to that of the Oracle Database 23c Free installation (for a future blog post on developing external table deployment on the Docker image):

mkdir /opt/oracle
chown -R oracle:install /opt/oracle/upload

You also can add the following student function to the Ubuntu student user’s .bashrc file. It means all you need to type to connect to the Oracle Database 23c Free Docker instance is “student“. I like shortcuts like this one, which let you leverage one-line Python commands.

student () 
{
    # Discover the fully qualified program name. 
    path=`which docker 2>/dev/null`
    file=''
 
    # Parse the program name from the path.
    if [ -n ${path} ]; then
        file=${path##/*/}
    fi
 
    # Wrap when there is a file and it is rewrap.
    if [ -n ${file} ] && [[ ${file} = "docker" ]]; then
        python -c "import subprocess; subprocess.run(['docker exec -it --user student oracle23c bash'], shell=True)" 
    else
        echo "Docker is unavailable: Install the docker package."
    fi
}

Open a Ubuntu Terminal shell and type a student function name to connect to the Docker Oracle Database 23c Free instance where you can now test things like external tables with the SQL*Plus command line without installing it on the Ubuntu local operating system.

student@student-virtual-machine:~$ student
[student@d28375f0c43f ~]$ sqlplus c##student/student@free
 
SQL*Plus: Release 23.0.0.0.0 - Production on Wed Jan 3 02:14:22 2024
Version 23.3.0.23.09
 
Copyright (c) 1982, 2023, Oracle.  All rights reserved.
 
Last Successful login time: Wed Jan 03 2024 01:56:44 +00:00
 
Connected to:
Oracle Database 23c Free Release 23.0.0.0.0 - Develop, Learn, and Run for Free
Version 23.3.0.23.09
 
SQL>

Then, I added this sqlplus function to the /home/student/.bashrc file, which is owned by the student user. However, I also added the instruction to change to the student user’s home directory because the Oracle 23c Free container will connect you to the /home/oracle directory by default. I also added the default long list (ll) alias to the .bashrc file.

sqlplus () 
{
    # Discover the fully qualified program name. 
    path=`which rlwrap 2>/dev/null`
    file=''
 
    # Parse the program name from the path.
    if [ -n ${path} ]; then
        file=${path##/*/}
    fi;
 
    # Wrap when there is a file and it is rewrap.
    if [ -n ${file} ] && [[ ${file} = "rlwrap" ]]; then
        rlwrap sqlplus "${@}"
    else
        echo "Command-line history unavailable: Install the rlwrap package."
        $ORACLE_HOME/bin/sqlplus "${@}"
    fi
}
 
# Change to the user's home directory.
cd ${HOME}
 
# Create a long list alias:
alias ll='ls -l --color=auto'

After you’ve configured your student user, you can configure the oracle user account to work like a regular server. Exit the Docker Oracle Database 23c Free as the student user, then connect as the root user with this command:

docker exec -it -u root oracle23c bash

As the root user you can become the oracle user with the following command:

su - oracle

Now, add the following .bashrc shell in the /home/oracle directory:

# The oracle user's .bashrc
 
# Source global definitions
if [ -f /etc/bashrc ]; then
	. /etc/bashrc
fi
 
# User specific environment
if ! [[ "$PATH" =~ "$HOME/.local/bin:$HOME/bin:" ]]
then
    PATH="$HOME/.local/bin:$HOME/bin:$PATH"
fi
export PATH
 
# Uncomment the following line if you don't like systemctl's auto-paging feature:
# export SYSTEMD_PAGER=
 
# User specific aliases and functions
export ORACLE_SID=FREE
export ORACLE_BASE=/opt/oracle
export ORACLE_HOME=/opt/oracle/product/23c/dbhomeFree
export PATH=$PATH:/$ORACLE_HOME/bin
 
# Change to the user's home directory.
cd ${HOME}
 
# Create a long list alias:
alias ll='ls -l --color=auto'
 
sqlplus () 
{
    # Discover the fully qualified program name. 
    path=`which rlwrap 2>/dev/null`
    file=''
 
    # Parse the program name from the path.
    if [ -n ${path} ]; then
        file=${path##/*/}
    fi;
 
    # Wrap when there is a file and it is rewrap.
    if [ -n ${file} ] && [[ ${file} = "rlwrap" ]]; then
        rlwrap sqlplus "${@}"
    else
        echo "Command-line history unavailable: Install the rlwrap package."
        $ORACLE_HOME/bin/sqlplus "${@}"
    fi
}

You need to manually source the .bashrc for the oracle user because it’s not an externally available user. Use this syntax to connect as the internal user:

sqlplus / as sysdba

It’ll display:

SQL*Plus: RELEASE 23.0.0.0.0 - Production ON Wed Jan 3 07:08:11 2024
Version 23.3.0.23.09
 
Copyright (c) 1982, 2023, Oracle.  ALL rights reserved.
 
 
Connected TO:
Oracle DATABASE 23c Free RELEASE 23.0.0.0.0 - Develop, Learn, AND Run FOR Free
Version 23.3.0.23.09
 
SQL>

After all this, I can now click the “up arrow” to edit any of the sqlplus command history. If you like to work inside sqlplus natively, this should help you.

Written by maclochlainn

December 20th, 2023 at 11:11 pm

AWS EC2 TNS Listener

without comments

Having configured an AlmaLinux 8.6 with Oracle Database 11g XE, MySQL 8.0.30, and PostgreSQL 15, we migrated it to AWS EC2 and provisioned it. We used the older and de-supported Oracle Database 11g XE because it didn’t require any kernel modifications and had a much smaller footprint.

I had to address why attempting to connect with the sqlplus utility raised the following error after provisioning a copy with a new static IP address:

ERROR:
ORA-12514: TNS:listener does NOT currently know OF service requested IN CONNECT descriptor

A connection from SQL Developer raises a more addressable error, like:

ORA-17069

I immediately tried to check the connection with the tnsping utility and found that tnsping worked fine. However, when I tried to connect with the sqlplus utility it raised an ORA-12514 connection error.

There were no diagnostic steps beyond checking the tnsping utility. So, I had to experiment with what might block communication.

I changed the host name from ip-172-58-65-82.us-west-2.compute.internal to a localhost string in both the listener.ora and tnsnames.ora. The listener.ora file:

# listener.ora Network Configuration FILE:
 
SID_LIST_LISTENER =
  (SID_LIST =
    (SID_DESC =
      (SID_NAME = PLSExtProc)
      (ORACLE_HOME = /u01/app/oracle/product/11.2.0/xe)
      (PROGRAM = extproc)
    )
  )
 
LISTENER =
  (DESCRIPTION_LIST =
    (DESCRIPTION =
      (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC_FOR_XE))
      (ADDRESS = (PROTOCOL = TCP)(HOST = localhost)(PORT = 1521))
    )
  )
 
DEFAULT_SERVICE_LISTENER = (XE)

The tnsnames.ora file:

# tnsnames.ora Network Configuration FILE:
 
XE =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = localhost)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = XE)
    )
  )
 
EXTPROC_CONNECTION_DATA =
  (DESCRIPTION =
    (ADDRESS_LIST =
      (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC_FOR_XE))
    )
    (CONNECT_DATA =
      (SID = PLSExtProc)
      (PRESENTATION = RO)
    )
  )

I suspected that it might be related to the localhost value. So, I checked the /etc/hostname and /etc/hosts files.

Then, I modified /etc/hostname file by removing the AWS EC2 damain address. I did it on a memory that Oracle’s TNS raises errors for dots or periods in some addresses.

The /etc/hostname file:

ip-172-58-65-82

The /etc/hosts file:

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4 ip-172-58-65-82
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6 ip-172-58-65-82

Now, we can connect to the Oracle Database 11g XE instance with the sqlplus utility. I believe this type of solution will work for other AWS EC2 provisioned Oracle databases.

Written by maclochlainn

March 22nd, 2023 at 10:09 pm

Wrapping sqlplus

with one comment

After sorting out the failures of Oracle Database 11g (11.2.0) on AlmaLinux, I grabbed the Enterprise Linux 9 rlwrap library. The rlwrap is a ‘readline wrapper’ that uses the GNU readline library to
allow the editing of keyboard input for any other command. Input history is remembered across invocations, separately for each command; history completion and search work as in bash and completion word
lists can be specified on the command line.

Installed it with the dnf utility:

dnf install -y rlwrap

It gave me this log file:

Last metadata expiration check: 0:53:30 ago on Fri 02 Dec 2022 01:07:54 AM EST.
Dependencies resolved.
================================================================================================================================
 Package                      Architecture                 Version                             Repository                  Size
================================================================================================================================
Installing:
 rlwrap                       x86_64                       0.45.2-3.el9                        epel                       132 k
 
Transaction Summary
================================================================================================================================
Install  1 Package
 
Total download size: 132 k
Installed size: 323 k
Downloading Packages:
rlwrap-0.45.2-3.el9.x86_64.rpm                                                                  162 kB/s | 132 kB     00:00    
--------------------------------------------------------------------------------------------------------------------------------
Total                                                                                           117 kB/s | 132 kB     00:01     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Regex version mismatch, expected: 10.40 2022-04-14 actual: 10.37 2021-05-26
Regex version mismatch, expected: 10.40 2022-04-14 actual: 10.37 2021-05-26
  Preparing        :                                                                                                        1/1 
  Installing       : rlwrap-0.45.2-3.el9.x86_64                                                                             1/1 
  Running scriptlet: rlwrap-0.45.2-3.el9.x86_64                                                                             1/1 
  Verifying        : rlwrap-0.45.2-3.el9.x86_64                                                                             1/1 
 
Installed:
  rlwrap-0.45.2-3.el9.x86_64                                                                                                    
 
Complete!

Then, I added this sqlplus function to the student account’s .bashrc file:

sqlplus () 
{
    # Discover the fully qualified program name. 
    path=`which rlwrap 2>/dev/null`
    file=''
 
    # Parse the program name from the path.
    if [ -n ${path} ]; then
        file=${path##/*/}
    fi;
 
    # Wrap when there is a file and it is rewrap.
    if [ -n ${file} ] && [[ ${file} = "rlwrap" ]]; then
        rlwrap sqlplus "${@}"
    else
        echo "Command-line history unavailable: Install the rlwrap package."
        $ORACLE_HOME/bin/sqlplus "${@}"
    fi
}

Then, I connected to the old, but tiny, footprint of Oracle Database 11g XE for testing, which worked:

Yes, I couldn’t resist. After all Version 11 was the last non-pluggable release and it’s been 11 years since its release. A double lucky 11.

Naturally, you can always use vi (or vim) to edit the command history provided you include the following command in your .bashrc file:

set -o vi

Next, I’ll build a new VM instance with the current version of Oracle Database XE for student testing.

As always, I hope this helps those working with Oracle’s database products.

Written by maclochlainn

December 19th, 2022 at 11:28 am

Oracle’s Sparse Lists

without comments

Oracle’s PL/SQL Programming Language is really quite nice. I’ve written 8 books on it and still have fun coding in it. One nasty little detail about Oracle’s lists, introduced in Oracle 8 as PL/SQL Tables according their documentation, is they rely on sequential numeric indexes. Unfortunately, Oracle lists support a DELETE method, which can create gaps in the sequential indexes.

Oracle calls a sequence without gaps densely populated and a sequence with gaps sparsely populated. This can cause problems when PL/SQL code inadvertently removes elements at the beginning, end, or somewhere in the middle of the list. That’s because a program can then pass the sparsely populated list as a parameter to another stored function or procedure where the developer may traverse the list in a for-loop. That traversal may raise an exception in a for-loop, like this when it has gaps in the index sequence:

DECLARE
*
ERROR AT line 1:
ORA-01403: no data found
ORA-06512: AT line 20

Oracle’s myriad built-in libraries don’t offer a function to compact a sparsely populated list into a densely populated list. This post provides a compact stored procedure that converts a sparsely populated list to a densely populated list.

The first step to using the compact stored procedure requires that you create an object type in SQL, like this list of 20-character strings:

DROP TYPE list;
CREATE OR REPLACE
  TYPE list IS TABLE OF VARCHAR2(20);
/

Now, you can implement the compact stored procedure by passing the User-Defined Type as it’s sole parameter.

CREATE OR REPLACE
  PROCEDURE compact ( sparse IN OUT LIST ) IS
    /* Declare local variables. */
    iterator  NUMBER;           -- Leave iterator as null.
 
    /* Declare new list. */
    dense     LIST := list();
  BEGIN
    /*
      Initialize the iterator with the starting value, which is
      necessary because the first element of the original list
      could have been deleted in earlier operations. Setting the
      initial iterator value to the first numeric index value
      ensures you start at the lowest available index value.
    */
    iterator := sparse.FIRST;
 
    /* Convert sparsely populated list to densely populated. */
    WHILE (iterator <= sparse.LAST) LOOP
      dense.EXTEND;
      dense(dense.COUNT) := sparse(iterator);
      iterator := sparse.NEXT(iterator);
    END LOOP;
 
    /* Replace the input parameter with the compacted list. */
    sparse := dense;
  END;
/

Before we test the compact stored procedure, let’s create deleteElement stored procedure for our testing:

CREATE OR REPLACE
  PROCEDURE deleteElement ( sparse   IN OUT LIST
                          , element  IN     NUMBER ) IS
  BEGIN
    /* Delete a value. */
    sparse.DELETE(element);
  END;
/

Now, let’s use an anonymous block to test compacting a sparsely populated list into a densely populated list. The test program will remove the first, last, and one element in the middle before printing the sparsely populated list’s index and string values. This test will show you gaps in the remaining non-sequential index values.

After you see the gaps, the test program compacts the remaining list values into a new densely populated list. It then prints the new index values with the data values.

DECLARE
  /* Declare a four item list. */
  lv_strings  LIST := list('one','two','three','four','five','six','seven');
BEGIN
  /* Check size of list. */
  dbms_output.put_line('Print initial list size:  ['||lv_strings.COUNT||']');
  dbms_output.put_line('===================================');
 
  /* Delete a value. */
  deleteElement(lv_strings,lv_strings.FIRST);
  deleteElement(lv_strings,3);
  deleteElement(lv_strings,lv_strings.LAST);
 
  /* Check size of list. */
  dbms_output.put_line('Print modified list size: ['||lv_strings.COUNT||']');
  dbms_output.put_line('Print max index and size: ['||lv_strings.LAST||']['||lv_strings.COUNT||']');
  dbms_output.put_line('===================================');
  FOR i IN 1..lv_strings.LAST LOOP
    IF lv_strings.EXISTS(i) THEN
      dbms_output.put_line('List list index and item: ['||i||']['||lv_strings(i)||']');
    END IF;
  END LOOP;
 
  /* Call a procedure by passing current sparse collection and
     the procedure returns dense collection. */
  dbms_output.put_line('===================================');
  dbms_output.put_line('Compacting list.');
  compact(lv_strings);
  dbms_output.put_line('===================================');
 
  /* Print the new maximum index value and list size. */
  dbms_output.put_line('Print new index and size: ['||lv_strings.LAST||']['||lv_strings.COUNT||']');
  dbms_output.put_line('===================================');
  FOR i IN 1..lv_strings.COUNT LOOP
    dbms_output.put_line('List list index and item: ['||i||']['||lv_strings(i)||']');
  END LOOP;
  dbms_output.put_line('===================================');
END;
/

It produces output, like:

Print initial list size:  [7]
===================================
Print modified list size: [4]
Print max index and size: [6][4]
===================================
List list index and item: [2][two]
List list index and item: [4][four]
List list index and item: [5][five]
List list index and item: [6][six]
===================================
Compacting list.
===================================
Print new index and size: [4][4]
===================================
List list index and item: [1][two]
List list index and item: [2][four]
List list index and item: [3][five]
List list index and item: [4][six]
===================================

You can extend this concept by creating User-Defined Types with multiple attributes, which are essentially lists of tuples (to draw on Pythonic lingo).

Written by maclochlainn

October 4th, 2021 at 11:49 pm

Wrap Oracle SQL*Plus

without comments

One of the key problems with Oracle’s deployment is that you can not use the up-arrow key to navigate the sqlplus command-line history. Here’s little Bash shell function that you can put in your .bashrc file. It requires you to have your system administrator install the rlwrap package, which wraps the sqlplus command-line history.

You should also set the $ORACLE_HOME environment variable before you put this function in your .bashrc file.

sqlplus () 
{
    # Discover the fully qualified program name. 
    path=`which rlwrap 2>/dev/null`
    file=''
 
    # Parse the program name from the path.
    if [ -n ${path} ]; then
        file=${path##/*/}
    fi;
 
    # Wrap when there is a file and it is rewrap.
    if [ -n ${file} ] && [[ ${file} = "rlwrap" ]]; then
        rlwrap sqlplus "${@}"
    else
        echo "Command-line history unavailable: Install the rlwrap package."
        $ORACLE_HOME/bin/sqlplus "${@}"
    fi
}

If you port this shell script to an environment where rlwrap is not installed, it simply prints the error message and advises you to install the rlwrap package.

As always, I hope this helps those looking for a solution.

Written by maclochlainn

June 29th, 2020 at 10:53 pm

DBeaver for PostgreSQL

without comments

I’m migrating my database classes from the Oracle database to the PostgreSQL database. Using the Oracle Express Edition has always required a virtualized image because students use Windows and Mac OS. Also, the university doesn’t like my use of a virtualized image. Virtualization imposes incremental cost on students to have high end laptops.

The available Docker images don’t typically support the Oracle Express Edition. That means there are licensing implications tied to Oracle.

As a committee, we decided to use PostgreSQL as our new database platform. We opted to use PostgreSQL over MySQL because it supports arrays and stored procedures. PostgreSQL PL/pgSQL are a solid implementation of stored procedures. While MySQL supports a stored procedure language, it effectively discourages using it.

We use sqlplus, mysql, or psql Command-Line Interface (CLI) as the primary interface. That’s because CLI supports production-oriented deployment and DevOps practices. The Open Source DBeaver project provides a clean native GUI management console for PostgreSQL. DBeaver also supports other databases, which pgAdmin doesn’t.

You click the Download button from the DBeaver home page. It support Windows, Mac OS, and Linux distributions. I downloaded the Linux RPM package to my Downloads directory. You can run the RPM with the following syntax:

rpm -ivh dbeaver-ce-6.2.3-stable.x86_64.rpm

It doesn’t check for dependencies. The installation is quick. You can verify that it’s installed with the following command:

which dbeaver

It installs here:

/usr/bin/dbeaver

If you failed to install the PostgreSQL JAR file, you’ll see the following error message:

It will look like the following on Linux:

You should have a user in the pg_hba.conf file, like the following:

host    all             all             127.0.0.1/32            trust

As always, I hope this helps those solving problems.

Written by maclochlainn

November 3rd, 2019 at 5:06 pm

External Tables

with 2 comments

Oracle Database 9i introduced external tables. You can create external tables to load plain text files by using Oracle SQL*Loader. Alternatively, you can create external tables that load and unload files by using Oracle Data Pump. This article demonstrates both techniques.

You choose external tables that use Oracle SQL*Loader when you want to import plain text files. There are three types of plain text files. They are comma-separated value (CSV), tab-separated value (TSV), and position specific text files.

External tables that use Oracle Data Pump don’t work with plain text files. They work with an Oracle proprietary format. That means you load source files previously created by an Oracle Data Pump export. You typically create external tables with Oracle Data Pump when you’re moving large data sets between database instances.

External tables use Oracle’s virtual directories. An Oracle virtual directory is an internal reference in the data dictionary. A virtual directory maps a unique directory name to a physical directory on the local operating system. Virtual directories were simple before Oracle Database 12c gave us the multitenant architecture. In a multitenant database there are two types of virtual directories. One services the schemas of the Container Database (CDB) and it’s in the CDB’s SYS schema. The other services the schemas of a Pluggable Database (PDB) and it’s in the ADMIN schema for the PDB.

You can create a CDB virtual directory as SYSTEM user with the following syntax in Windows:

SQL> CREATE DIRECTORY upload AS 'C:\Data\Upload';

or, like this in Linux or Unix:

SQL> CREATE DIRECTORY upload AS '/u01/app/oracle';

There are some subtle differences between these two statements. Windows directories or folders start with a logical drive letter, like C:\, D:\, and so forth. Linux and Unix directories start with a mount point like /u01.

One of the subtle differences is directory and file ownership. You can change ownership for a directory in Windows as the Administrator account. The change makes the directory publically accessible, and that’s probably fine for a test database. After such a change, the Oracle user can find the external file even when parent directories aren’t navigable. Although, a production database on Windows would requires more skill at setting and restricting file permissions.

Linux and Unix directories require that the oracle user can navigate the tree from the mount point to the target physical directory. Also, you must designate the ownership of external files as the same as the Oracle Database user. Assuming a standard install of the Oracle Database 11g XE instance, you would issue the following shell command as the root user to change file ownership and access privileges:

# chown –R oracle:dba /u01/app/oracle/upload
# chmod –R 755 /u01/app/oracle/upload

After you create the virtual directory, you must grant privileges or a role to the user that defines the external table. While data and log files should be separated, this example assumes they co-exist in the same directory.

The following statement grants read privilege for the data file and write privileges for the log files to a CDB user. You should run this statement as the system user.

SQL> GRANT read, WRITE ON DIRECTORY upload TO c##importer;

or, like this in non-multitenant database or PDB user:

SQL> GRANT read, WRITE ON DIRECTORY upload TO importer;

The last preparation steps require a plain text file in the physical directory. Let’s create a CSV file of key Avenger characters, and name it the avenger.csv file.

The avenger.csv file holds the following values:

1,'Anthony','Stark','Iron Man'
2,'Thor','Odinson','God of Thunder'
3,'Steven','Rogers','Captain America'
4,'Bruce','Banner','Hulk'
5,'Clinton','Barton','Hawkeye'
6,'Natasha','Romanoff','Black Widow'

You create the external table after creating the virtual directory, granting read and write privileges on the virtual directory, and creating an external physical file. The syntax for the CREATE TABLE statement of an external table is very similar to the syntax of an ordinary table. The difference between the two types of tables is a clause. An internal table has a STORAGE clause, while an external table has an ORGANIZATION EXTERNAL clause.

The following creates the avenger table as an external table:

SQL> CREATE TABLE avenger
  2  ( avenger_id      NUMBER
  3  , first_name      VARCHAR2(20)
  4  , last_name       VARCHAR2(20)
  5  , character_name  VARCHAR2(20))
  6    ORGANIZATION EXTERNAL
  7    ( TYPE oracle_loader
  8      DEFAULT DIRECTORY upload
  9      ACCESS PARAMETERS
 10      ( RECORDS DELIMITED BY NEWLINE CHARACTERSET US7ASCII
 11        BADFILE     'UPLOAD':'avenger.bad'
 12        DISCARDFILE 'UPLOAD':'avenger.dis'
 13        LOGFILE     'UPLOAD':'avenger.log'
 14        FIELDS TERMINATED BY ','
 15        OPTIONALLY ENCLOSED BY "'"
 16        MISSING FIELD VALUES ARE NULL)
 17      LOCATION ('avenger.csv'))
 18  REJECT LIMIT UNLIMITED;

Lines 1 through 5 create the columns of the avenger table. Lines 6 through 17 contain the ORGANIZATION EXTERNAL clause. Line 7 designates the external table as managed by the Oracle SQL*Loader utility. Line 8 sets the default virtual directory. Lines 11 through 12 set the bad, discard, and log file location. The bad and discard files keep all that can’t be read. The log file keeps all rows read by a query against the avenger table.

You also have the option of making all reads automatic parallel. You simply add a PARALLEL clause, like this:

19  PARALLEL;

A simple query with SQL*Plus formatting lets us test whether the avenger table works. The query to display all columns of all rows is:

SQL> COLUMN first_name FORMAT A10
SQL> COLUMN last_name  FORMAT A10
SQL> COLUMN character_name FORMAT A15
SQL> SELECT * FROM avenger;

Yields the following formatted output:

AVENGER_ID FIRST_NAME LAST_NAME  CHARACTER_NAME
---------- ---------- ---------- ---------------
         1 Anthony    Stark      Iron Man
         2 Thor       Odinson    God of Thunder
         3 Steven     Rogers     Captain America
         4 Bruce      Banner     Hulk
         5 Clinton    Barton     Hawkeye
         6 Natasha    Romanoff   Black Widow
 
6 rows selected.

It’s possible to redefine the avenger table to use either relative or fixed positional columns. You change the ACCESS PARAMETERS clause on lines 9 through 16 to make this change.
The following ACCESS PARAMETERS clause runs across lines 9 through 19 and creates relative position definition:

  9      ACCESS PARAMETERS
 10      ( RECORDS DELIMITED BY NEWLINE CHARACTERSET US7ASCII
 11        BADFILE     'UPLOAD':'avenger.bad'
 12        DISCARDFILE 'UPLOAD':'avenger.dis'
 13        LOGFILE     'UPLOAD':'avenger.log'
 14        FIELDS
 15        MISSING FIELD VALUES ARE NULL
 16        ( avenger_id      CHAR(4)
 17        , first_name      CHAR(20)
 18        , last_name       CHAR(20)
 19        , character_name  CHAR(4)))

You can change from the relative position, to a fixed position by changing lines 16 through 19. The change for fixed length strings is:

 16        ( avenger_id      POSITION 1:4
 17        , first_name      POSITION 5:24
 18        , last_name       POSITION 25:44
 19        , character_name  POSITION 45:64))

Having worked with the Oracle SQL*Loader version of external tables, lets create one that uses Oracle Data Pump. Assuming we keep the same data structure, drop the avenger table, and create a catalog managed avenger_internal table.

This statement creates the avenger_internal table:

SQL> CREATE TABLE avenger_internal
  2  ( avenger_id      NUMBER
  3  , first_name      VARCHAR2(20)
  4  , last_name       VARCHAR2(20)
  5  , character_name  VARCHAR2(20));

To avoid writing six INSERT statements, you can write one INSERT statement with a query against the SQL*Loader avenger table. The syntax for that INSERT statement is:

SQL> INSERT INTO avenger_internal
  2  SELECT * FROM avenger;

With an internally managed table, you create an avenger_export table that uses Oracle Data Pump like this:

SQL> CREATE TABLE avenger_export
  2  ORGANIZATION EXTERNAL
  3  ( TYPE oracle_datapump
  4    DEFAULT DIRECTORY upload
  5    LOCATION ('avenger_export.dmp')) AS
  6  SELECT   avenger_id
  7  ,        first_name
  8  ,        last_name
  9  ,        character_name
 10  FROM     avenger_internal;

The CREATE TABLE statement exports data to the avenger_export.dmp file immediately. You must drop and recreate the avenger_export table to get a fresh extract of the avenger_internal table’s data. You must also remove the previous avenger_export.dmp file before you try to recreate the avenger_export table.

You raise the following error when you fail to remove the previous export file:

CREATE TABLE avenger_export
*
ERROR AT line 1:
ORA-29913: error IN executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
KUP-11012: FILE avenger_export.dmp IN /u01/... already EXISTS

This is a simple example with only four columns. You might think you can use the SELECT * as the SELECT-list of the query on lines 6 through 10. If you’re running Oracle Database 12c, you can use the shorter syntax, but if you’re running Oracle Database 11g you can’t. If you attempt it in an Oracle Database 11g instance, the CREATE TABLE statement returns the following error:
ERROR at line 6:

ORA-30656: COLUMN TYPE NOT supported ON external organized TABLE

You create an avenger_import table with another twist on this now familiar Oracle SQL syntax. The CREATE TABLE statement is:

SQL> CREATE TABLE avenger_import
  2  ( avenger_id      NUMBER
  3  , first_name      VARCHAR2(20)
  4  , last_name       VARCHAR2(20)
  5  , character_name  VARCHAR2(20))
  6    ORGANIZATION EXTERNAL
  7    ( TYPE oracle_datapump
  8      DEFAULT DIRECTORY up2load
  9      LOCATION ('avenger_export.dmp'));

Like the export process, the import process happens immediately when the CREATE TABLE statement runs. A query against the avenger_import table would show you the original six rows we started with in the plain text files.

This article has introduced Oracle external tables. It has shown you how to import plain text files with SQL*Loader. It has also shown you how to export files from tables.

Written by maclochlainn

November 9th, 2018 at 9:44 am

Substitutable Columns

with 2 comments

Oracle’s substitutable columns are interesting and substantially different than Oracle’s nested tables. The benefit of substitutable columns is that you can create one for an object type or any subtypes of that object type. Unfortunately, you can’t create the same behavior with nested tables because Oracle’s implementation of collection types are always final data types and you can’t extend their behaviors.

The Oracle Database has three types of collections. Two are SQL scoped collection types and the remaining one is a PL/SQL-only collection. You can only use the two SQL scoped collection types as column data types. One of the SQL-scoped collection types is an Attribute Data Type (ADT), which uses a base data type of DATA, NUMBER, or VARCHAR2.

The base data types of a UDT are scalar data types and scalar data types are data types that hold one thing. The other SQL-scoped collection type is a collection of User-Defined Types (UDTs), which are object types that you create like record structures by assembling sets of basic scalar data types. The elements of a UDT are known as members, whereas the instances of a collection are known as elements because they indexed in a set.

You can join a row with any nested table by using a cross join because they match the row with the nested table by using an ID-dependent join. An ID-dependent join is inexpensive because it relies on a structural dependency, the existence of the nested table in a column of a row. Typical joins on the other hand are joins between two tables or two copies of the same table. These non ID-dependent joins use at least matching values in one column of each table or one column of two copies of a table.

Joins between substitutable columns that hold UDTs are unlike joins between nested tables. The following sets up an example to demonstrate how you can join the non-substitutable columns of a row with the substitutable columns.

  1. You need a base UDT object type that you can extend, where extend means you can create a subtype of the base object type. While this is straight forward when you create an Oracle object type with methods, it isn’t necessarily straight forward when you want to simply create a base data structure as a generalized type with subtypes.

    The important clause is overriding the FINAL default by making the base type NOT FINAL. The example use BASE_T as the generalized type or data structure of a substitutable column:

    CREATE OR REPLACE TYPE base_t AS OBJECT
    ( base_id  NUMBER ) NOT FINAL;
    /
  2. After you create your base data structure, you create a specialized subtype. The following example creates a PERSON_T type and accepts the default of FINAL, which means you can’t create another subtype level.

    CREATE OR REPLACE TYPE person_t UNDER base_t
    ( first_name   VARCHAR2(20)
    , middle_name  VARCHAR2(20)
    , last_name    VARCHAR2(20));
    /
  3. With a generalized BASE_T type and a specialized PERSON_T subtype, you create a CUSTOMER table with a substitutable CUSTOMER_NAME column. The CUSTOMER_NAME column uses the generalized BASE_T data type. You should also create a CUSTOMER_S sequence that you can use as a surrogate key column for the table.

    CREATE TABLE customer
    ( customer_id    NUMBER
    , customer_name  BASE_T );
     
    CREATE SEQUENCE customer_s;
  4. You can now populate the table with instances of the BASE_T type or the PERSON_T subtype. The following inserts three rows into the CUSTOMER table. One for Hank Pym the original Ant-Man, one for Scott Lang the succeeding Ant-Man, and another for Darren Cross the original Yellowjacket.

    INSERT INTO customer
    VALUES
    ( customer_s.NEXTVAL
    , person_t( customer_s.CURRVAL
              , first_name => 'Hank'
              , middle_name => NULL
              , last_name => 'Pym'));
     
    INSERT INTO customer
    VALUES
    ( customer_s.NEXTVAL
    , person_t( customer_s.CURRVAL
              , first_name => 'Scott'
              , middle_name => NULL
              , last_name => 'Lang'));
     
    INSERT INTO customer
    VALUES
    ( customer_s.NEXTVAL
    , person_t( customer_s.CURRVAL
              , first_name => 'Darren'
              , middle_name => NULL
              , last_name => 'Cross'));
  5. The significance or problem associated with substitutable columns is that the actual columns of the object data type are hidden, which means you can’t query them like they’re nested elements of the substitutable column. The following query demonstrates what happens when you try to access those hidden member columns:

    SELECT customer_id
    ,      customer_name.base_id
    ,      customer_name.first_name
    ,      customer_name.middle_name
    ,      customer_name.last_name
    FROM   customer;

    It returns the following error message:

    ,      customer_name.last_name
           *
    ERROR at line 5:
    ORA-00904: "CUSTOMER_NAME"."LAST_NAME": invalid identifier
  6. It only raises the last column in the SELECT-list because that’s the first place where it fails to recognize an identifier, which is a valid column name in scope of the query.

  7. This error message may lead you to call the CUSTOMER_NAME column in a subquery and use the TABLE function to convert it to a result set. However, it also fails because a UDT object type by itself is an ordinary object type not a collection of object types. The TABLE function can’t promote the single instance to collection.

    SELECT *
    FROM   TABLE(SELECT TREAT(customer_name AS person_t) FROM customer);

    It returns the following error message:

    FROM   TABLE(SELECT TREAT(customer_name AS person_t) FROM customer)
           *
    ERROR at line 2:
    ORA-22905: cannot access rows from a non-nested table item
  8. The non-nested table error message should lead you to wrap the call to the TREAT function in a call to the COLLECT function, like this:

    COL base_id        FORMAT 9999  HEADING "Base|ID #"
    COL customer_name  FORMAT A38   HEADING "Customer Name"
    COL first_name     FORMAT A6    HEADING "First|Name"
    COL middle_name    FORMAT A6    HEADING "Middle|Name"
    COL last_name      FORMAT A6    HEADING "Last|Name"
    SELECT *
    FROM   TABLE(
             SELECT COLLECT(TREAT(customer_name AS person_t)) AS cte
             FROM customer);

    It returns the substitutable column’s hidden column labels and their values:

     Base First  Middle Last
     ID # Name   Name   Name
    ----- ------ ------ ------
        1 Hank	    Pym
        2 Scott	    Lang
        3 Darren	    Cross
  9. After learning how to unwrap the hidden columns of the substitutable column, you can now join the ordinary columns to the hidden columns like this:

    COL customer_id    FORMAT 9999  HEADING "Customer|ID #"
    COL base_id        FORMAT 9999  HEADING "Base|ID #"
    COL customer_name  FORMAT A38   HEADING "Customer Name"
    COL first_name     FORMAT A6    HEADING "First|Name"
    COL middle_name    FORMAT A6    HEADING "Middle|Name"
    COL last_name      FORMAT A6    HEADING "Last|Name"
    SELECT   c.customer_id
    ,        o.*
    FROM     customer c INNER JOIN
             TABLE(SELECT COLLECT(TREAT(customer_name AS person_t)) AS cte
                   FROM   customer) o
    ON       c.customer_id = o.base_id
    ORDER BY c.customer_id;

    It returns the ordinary column and substitutable column’s hidden column labels and their values:

    Customer  Base First  Middle Last
        ID #  ID # Name   Name   Name
    -------- ----- ------ ------ ------
           1     1 Hank	     Pym
           2     2 Scott	     Lang
           3     3 Darren	     Cross
  10. The preceding query only returns values when the substitutable column holds a value. It fails to return a value when the substitutable column holds a null value. You need to use a LEFT JOIN to ensure you see all ordinary columns whether or not the substitutable column holds a value.

    COL customer_id    FORMAT 9999  HEADING "Customer|ID #"
    COL base_id        FORMAT 9999  HEADING "Base|ID #"
    COL customer_name  FORMAT A38   HEADING "Customer Name"
    COL first_name     FORMAT A6    HEADING "First|Name"
    COL middle_name    FORMAT A6    HEADING "Middle|Name"
    COL last_name      FORMAT A6    HEADING "Last|Name"
    SELECT   c.customer_id
    ,        o.*
    FROM     customer c LEFT JOIN
             TABLE(SELECT COLLECT(TREAT(customer_name AS person_t)) AS cte
                   FROM   customer) o
    ON       c.customer_id = o.base_id
    ORDER BY c.customer_id;

    It returns the ordinary column and substitutable column’s hidden column labels and their values when the substitutable column holds an instance value. However, it only returns the ordinary column when the substitutable column holds a null value, as shown below:

    Customer  Base First  Middle Last
        ID #  ID # Name   Name   Name
    -------- ----- ------ ------ ------
           1     1 Hank	     Pym
           2     2 Scott	     Lang
           3     3 Darren	     Cross
           4
  11. It should be noted that queries like this have a cost, and that cost is high. So, you should only implement substitutable columns when the maintenance coding costs (or sustaining engineering) outweighs the processing cost.

    You can determine the cost like this:

    EXPLAIN PLAN
    SET STATEMENT_ID = 'Strange'
    FOR
    SELECT   c.customer_id
    ,        o.*
    FROM     customer c LEFT JOIN
             TABLE(SELECT COLLECT(TREAT(customer_name AS person_t)) AS cte
                   FROM   customer) o
    ON       c.customer_id = o.base_id
    ORDER BY c.customer_id;

    You can query the cost like this:

    SET LINESIZE 130
    SELECT *
    FROM   TABLE(dbms_xplan.display(NULL,'Strange'));

    It should return something like this for the sample table and solution:

    PLAN_TABLE_OUTPUT
    ---------------------------------------------------------------------------------------------------------
    Plan hash value: 2373055701
     
    ---------------------------------------------------------------------------------------------------------
    | Id  | Operation			     | Name	| Rows	| Bytes |TempSpc| Cost (%CPU)| Time	|
    ---------------------------------------------------------------------------------------------------------
    |   0 | SELECT STATEMENT		     |		|  8168 |   550K|	|   167   (2)| 00:00:03 |
    |   1 |  SORT ORDER BY			     |		|  8168 |   550K|   624K|   167   (2)| 00:00:03 |
    |*  2 |   HASH JOIN OUTER		     |		|  8168 |   550K|	|    32   (4)| 00:00:01 |
    |   3 |    TABLE ACCESS FULL		     | CUSTOMER |     5 |    15 |	|     2   (0)| 00:00:01 |
    |   4 |    VIEW 			     |		|  8168 |   526K|	|    29   (0)| 00:00:01 |
    |   5 |     COLLECTION ITERATOR PICKLER FETCH|		|  8168 |	|	|    29   (0)| 00:00:01 |
    |   6 |      SORT AGGREGATE		     |		|     1 |    14 |	|	     |		|
    |   7 |       TABLE ACCESS FULL 	     | CUSTOMER |     5 |    70 |	|     2   (0)| 00:00:01 |
    ---------------------------------------------------------------------------------------------------------
     
    Predicate Information (identified by operation id):
    ---------------------------------------------------
     
       2 - access("C"."CUSTOMER_ID"="O"."SYS_NC_ROWINFO$"."BASE_ID"(+))

As always, I hope this explains how to insert and query the hidden columns of a substitutable column, and how you join ordinary columns and hidden columns of a substitutable column from a table.

Written by maclochlainn

December 8th, 2017 at 11:17 pm

Finding Direct Indexes

without comments

If you’re not using Toad DBA Suite, it’s sometimes hard to find solutions. Somebody wanted to know how to find indexes that aren’t indirect. Indirect indexes are those created for a primary key because a primary key column or set of columns are both not null and uniquely constrained. Likewise, you create a unique index when you can create a unique constraint. You can’t drop a unique index for a primary key without dropping the primary key or unique constraint that indirectly created it.

The following query returns indexes with one or more columns that are created by a CREATE INDEX statement on a target table. It excludes unique indexes created by a primary key constraint, and it returns the relative position of columns in an index:

COLUMN sequence_name   FORMAT A22 HEADING "Sequence Name"
COLUMN column_position FORMAT 999 HEADING "Column|Position"
COLUMN column_name     FORMAT A22 HEADING "Column|Name"
SELECT   uin.index_name
,        uic.column_position
,        uic.column_name
FROM     user_indexes uin INNER JOIN user_ind_columns uic
ON       uin.index_name = uic.index_name
AND      uin.table_name = uic.table_name
WHERE    uin.table_name = UPPER('&&table_name')
AND NOT  uin.index_name IN (SELECT constraint_name
                            FROM   user_constraints
                            WHERE  table_name = UPPER('&&table_name'))
ORDER BY uin.index_name
,        uic.column_position;

It can be rewritten into a function, which can then drop indexes based on a table name:

CREATE OR REPLACE FUNCTION drop_indexes_on
( pv_table_name  VARCHAR2 ) RETURN NUMBER IS 
 
  /* A return value. */
  lv_return  NUMBER := 0;
 
  /* A query to return only directly created indexes. */
  CURSOR find_indexes_on
  ( cv_table_name  VARCHAR2 ) IS
    SELECT   DISTINCT ui.index_name
    FROM     user_indexes ui INNER JOIN user_ind_columns uic
    ON       ui.index_name = uic.index_name
    AND      ui.table_name = uic.table_name
    WHERE    ui.table_name = UPPER(cv_table_name)
    AND NOT  ui.index_name IN (SELECT constraint_name
                               FROM   user_constraints
                               WHERE  table_name = UPPER(cv_table_name));
 
  /* Declare function autonomous. */
  PRAGMA AUTONOMOUS_TRANSACTION;
 
BEGIN
 
  /* Drop the indexes on a table. */
  FOR i IN find_indexes_on(pv_table_name) LOOP
    EXECUTE IMMEDIATE 'DROP INDEX '||i.index_name;
    lv_return := 1;
  END LOOP;
 
  RETURN lv_return;
END drop_indexes_on;
/

You can call the drop_on_indexes_on function like this:

SELECT   drop_indexes_on(UPPER('address_lab'))
FROM     dual;

Hope this helps those who need to work with dropping indexes.

Written by maclochlainn

November 23rd, 2014 at 8:42 pm