MacLochlainns Weblog

Michael McLaughlin's Technical Blog

Wrapping SQL*Plus

One annoying thing from installing Oracle Database 11g on Fedora, was that the up arrows for command history didn’t work. I decided to fix that today after seeing Lutz Hartmann’s article on rlwrap. Unfortunately, the epel (Extra Packages for Enterprise Linux) package he recommended doesn’t run on Fedora 20. You can read my tale of woe, or skip to the .bashrc function that fixed it when I installed only rlwrap.

Attempting it on yum, gave me these errors:

# yum install http://www.mirrorservice.org/sites/dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm Loaded plugins: langpacks, refresh-packagekit epel-release-6-8.noarch.rpm | 14 kB 00:00 Examining /var/tmp/yum-root-5CLTPa/epel-release-6-8.noarch.rpm: epel-release-6-8.noarch Marking /var/tmp/yum-root-5CLTPa/epel-release-6-8.noarch.rpm to be installed Resolving Dependencies --> Running transaction check ---> Package epel-release.noarch 0:6-8 will be installed --> Processing Conflict: epel-release-6-8.noarch conflicts fedora-release No package matched to upgrade: epel-release --> Finished Dependency Resolution Error: epel-release conflicts with fedora-release-20-3.noarch You could try using --skip-broken to work around the problem You could try running: rpm -Va --nofiles –nodigest

Poking around for an epel fix wasn’t successful, so I chose to install only the rlwrap package. Here’s that command and log file:

[root@localhost ~]# yum install rlwrap Loaded plugins: langpacks, protectbase, refresh-packagekit 0 packages excluded due to repository protections Resolving Dependencies --> Running transaction check ---> Package rlwrap.x86_64 0:0.41-1.fc20 will be installed --> Finished Dependency Resolution   Dependencies Resolved   ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: rlwrap x86_64 0.41-1.fc20 updates 95 k   Transaction Summary ================================================================================ Install 1 Package   Total download size: 95 k Installed size: 204 k Is this ok [y/d/N]: y Downloading packages: rlwrap-0.41-1.fc20.x86_64.rpm | 95 kB 00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction (shutdown inhibited) Installing : rlwrap-0.41-1.fc20.x86_64 1/1 Verifying : rlwrap-0.41-1.fc20.x86_64 1/1   Installed: rlwrap.x86_64 0:0.41-1.fc20 Complete!

The next step was getting it to work. A sqlplus function wrapper inside the .bashrc file seemed the easiest. Here’s the code to the .bashrc file:

# .bashrc   # Source global definitions if [ -f /etc/bashrc ]; then . /etc/bashrc fi   # Uncomment the following line if you don't like systemctl's auto-paging feature: # export SYSTEMD_PAGER=   # User specific aliases and functions . /u01/app/oracle/product/11.2.0/xe/bin/oracle_env.sh   # Wrap sqlplus with rlwrap to edit prior lines. sqlplus() { if [ "$RLWRAP" = "0" ]; then sqlplus "$@" else rlwrap sqlplus "\$@" fi }   # Set vi as a command line editor. set -o vi

As always, I hope this helps some folks.

Written by maclochlainn

August 20th, 2014 at 6:58 pm

Hiding a Java Source

with one comment

The ability to deploy Java inside the Oracle database led somebody to conclude that the source isn’t visible in the data catalog. Then, that person found that they were wrong because the Java source is visible when you use a DDL command to CREATE, REPLACE, and COMPILE the Java source. This post discloses how to find the Java source and how to prevent it from being stored in the data catalog.

You can verify that the Java class and source files exist with the following query:

1 2 3 4 5 6 7 8 COLUMN object_name FORMAT A20 HEADING "Object Name" COLUMN object_type FORMAT A12 HEADING "Object Type" COLUMN STATUS FORMAT A14 HEADING "Object Status" SELECT object_name , object_type , STATUS FROM user_objects WHERE object_name = 'ReadFile';

It displays:

Object Name Object Type Object Status -------------------- ------------ -------------- ReadFile JAVA SOURCE VALID ReadFile JAVA CLASS VALID   2 rows selected.

Then, you can use the following query to discovery a Java library created by a SQL command:

1 2 3 4 5 6 COLUMN line FORMAT 9999 HEADING "Line|#" COLUMN text FORMAT A66 HEADING "Text" SELECT line , text FROM user_source WHERE name = 'ReadFile';

It displays the following:

 Line # Text ------- ------------------------------------------------------------------ 1 // Java library imports. 2 import java.io.File; 3 import java.io.BufferedReader; 4 import java.io.FileNotFoundException; 5 import java.io.IOException; 6 import java.io.FileReader; 7 import java.security.AccessControlException; 8 9 // Class definition. 10 public class ReadFile { 11 // Define class variables. 12 private static File file; 13 private static FileReader inTextFile; 14 private static BufferedReader inTextReader; 15 private static StringBuffer output = new StringBuffer(); 16 private static String outLine, outText; 17 18 // Define readText() method. 19 public static String readText(String fromFile) 20 throws AccessControlException, IOException { 21 // Read file. 22 try { 23 // Initialize File. 24 file = new File(fromFile); 25 26 // Check for valid file. 27 if (file.exists()) { 28 29 // Assign file to a stream. 30 inTextFile = new FileReader(file); 31 inTextReader = new BufferedReader(inTextFile); 32 33 // Read character-by-character. 34 while ((outLine = inTextReader.readLine()) != null) { 35 output.append(outLine + "\n"); } 36 37 // Assing the StringBuffer to a String. 38 outText = Integer.toString(output.toString().length()); 39 40 // Close File. 41 inTextFile.close(); } 42 else { 43 outText = new String("Empty"); }} 44 catch (IOException e) { 45 outText = new String(""); 46 return outText; } 47 return outText; }}   47 rows selected.

You can eliminate the source by compiling the Java library outside the database. Then, you use the loadjava utility to load the only the class file into the data catalog. The syntax would be the following command for an importer user in a video Pluggable Database (PDB):

loadjava -r -f -o -user importer/importer@video ReadFile.class

You should know that this syntax is disallowed by the loadjava utility, notwithstanding it’s found in the Oracle Database 12c documentation:

loadjava -r -f -o -user importer@video/importer ReadFile.class

You can verify that only the Java class file exists with the following query:

1 2 3 4 5 6 7 8 COLUMN object_name FORMAT A20 HEADING "Object Name" COLUMN object_type FORMAT A12 HEADING "Object Type" COLUMN STATUS FORMAT A14 HEADING "Object Status" SELECT object_name , object_type , STATUS FROM user_objects WHERE object_name = 'ReadFile';

It displays:

Object Name Object Type Object Status -------------------- ------------ -------------- ReadFile JAVA CLASS VALID   1 row selected.

Hope this helps those who want to hide the Java source files.

Written by maclochlainn

July 22nd, 2014 at 10:23 pm

External Table Access

I left to chance where students would attempt to place their external files in a Linux or Unix implementation. As frequently occurs, they choose a location in their student user’s home directory. Any attempt to read an external table based on a file in this type of directory fails because it’s not accessible by the Oracle user because the directory within the student user’s account isn’t reachable. You can’t simply chown a directory and the files in a directory.

The failure returns the following result:

SELECT COUNT(*) FROM transaction_upload * ERROR at line 1: ORA-29913: error in executing ODCIEXTTABLEOPEN callout ORA-29400: data cartridge error error opening file /home/student/upload/transaction_upload.log

The reason isn’t readily visible to all, but a virtual directory must reference a physical directory owned by the oracle user and dba group. That also means the oracle user must own all directories from the logical mount point to the physical directory name.

Assuming you’re working in an Oracle Database 11g XE instance, you can create a valid upload directory by navigating to this directory:

/u01/app/oracle

Then, issue this command as the root user to create a new upload directory:

mkdir upload

Now you have the following directory:

/u01/app/oracle/upload

Assuming you’ve created the upload directory as the root user, the root user should issue the following two commands from the /u01/app/oracle directory:

chown -R oracle:dba upload chmod -R 755 upload

Having made that change you should now be able to query the external file source, like a *.csv (comma-separated values) file. Hope this helps those trying to use external tables, which I subsequently wrote about for Toad World – External Tables.

Written by maclochlainn

June 19th, 2014 at 1:39 am

Sequence disallows order by

A call to a PRICE_S1 sequence in a query with an ORDER BY clause is disallowed, which typically occurs in an INSERT statement. Any attempt raises the following exception:

SELECT price_s1.NEXTVAL AS price_id * ERROR at line 1: ORA-02287: SEQUENCE NUMBER NOT allowed here

You need to remove the ORDER BY clause to eliminate the error.

Written by maclochlainn

June 15th, 2014 at 6:20 pm

SQL Insert from Query

Sometimes my students find new errors that I’ve never seen. One student did that this week by including an ORDER BY clause in a subquery that feeds an INSERT statement. It raises an ORA-00907 exception, like:

ORA-00907: missing right parenthesis

You can’t include a subquery with an ORDER BY clause because it generates an error. The reason is simple. A subquery can’t perform a sort operation inside a subquery. Here’s a quick demonstration:

DROP TABLE destination; CREATE TABLE destination ( destination_id NUMBER , destination_name VARCHAR2(20));   INSERT INTO destination ( SELECT 1,'Sample1' FROM dual UNION ALL SELECT 2,'Sample2' FROM dual ORDER BY 1 DESC );

If you remove the ORDER BY clause, the statement works without a problem. For example, here’s the working version:

INSERT INTO destination ( SELECT 1,'Sample1' FROM dual UNION ALL SELECT 2,'Sample2' FROM dual );

Alternatively, you can include an ORDER BY clause when you remove the parentheses from around the subquery. This is an example:

INSERT INTO destination SELECT 1,'Sample1' FROM dual UNION ALL SELECT 2,'Sample2' FROM dual ORDER BY 1 DESC;

I hope this helps anybody who runs into the problem.

Written by maclochlainn

June 14th, 2014 at 10:40 pm

Free Oracle NoSQL Book

It’s seldom that Oracle makes an effort to give an Oracle Press book away for free, but they’ve done it. You can download the Getting Started with Oracle NoSQL Database book from Oracle Press for free.

Getting Started with Oracle NoSQL Database

1. Overview of Oracle NoSQL Database and Big Data
2. Introducing Oracle NoSQL Database
3. Oracle NoSQL Database Architecture
4. Oracle NoSQL Database Installation and Configuration
5. Getting Started with Oracle NoSQL Database Development
7. Advanced Programming Concepts: Avro Schemas and Bindings
8. Capacity Planning and Sizing

Just so you know, knowledge of Java and Hadoop are helpful. It’s a short 71 pages but a nice heads up and here’s what you do starter book.

If you’re opening in iBooks on an iPad, make sure you let the whole book download first as a PDF file. You may only get a partial PDF file when you don’t let it download completely to your browser. It isn’t the speediest download, notwithstanding its small size.

Written by maclochlainn

May 31st, 2014 at 9:34 am

Parsing DBMS_OUTPUT

Testing with DBMS_OUTPUT.PUT_LINE is always a bit of a problem when you have strings longer than 80 characters in length, which occurs more frequently with Oracle Database 12c. An example of managing output occurs when you want to print a string with embedded line breaks. My solution is the following parse_rows procedure:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 CREATE OR REPLACE PROCEDURE parse_rows ( pv_text VARCHAR2 ) IS   /* Declare parsing indexes. */ lv_start NUMBER := 1; lv_end NUMBER := 1; lv_length NUMBER;   BEGIN   /* Assign an end value based on parsing line return or length. */ IF INSTR(pv_text,CHR(10),lv_start) = 0 THEN lv_end := LENGTH(pv_text) + 1; ELSE lv_end := INSTR(pv_text,CHR(10),lv_start); END IF;   /* Assign a length value to the parsed string. */ lv_length := lv_end - lv_start;   /* Print first line. */ dbms_output.put_line(SUBSTR(pv_text,lv_start,lv_length));   /* Print the rows of a multiple line string. */ WHILE (lv_end < LENGTH(pv_text)) LOOP   /* Assign a new start value. */ lv_start := lv_end + 1;   /* Assign a new end value. */ IF INSTR(pv_text,CHR(10),lv_start + 1) = 0 THEN lv_end := LENGTH(pv_text) + 1; ELSE lv_end := INSTR(pv_text,CHR(10),lv_start + 1); END IF;   /* Assign a new length. */ lv_length := lv_end - lv_start;   /* Print the individual rows. */ dbms_output.put_line(SUBSTR(pv_text,lv_start,lv_length));   END LOOP; END; /

Here’s the example of a test program for the procedure:

1 2 3 4 5 6 7 DECLARE /* Declare text. */ lv_text VARCHAR2(200) := 'This is too much'||CHR(10)||'information'||CHR(10)||'on one line.'; BEGIN parse_rows(lv_text); END; /

It prints the following:

1 2 3 This is too much information on one line.

Hope this benefits those looking for quick solution.

Written by maclochlainn

May 21st, 2014 at 12:35 am

Best UTL_FILE Practice

In a post a couple days ago, I promised to provide a best practice approach to reading external files with the UTL_FILE package. My first assumption is that you’re reading unstructured data because structured data is best read by external tables because external tables can read data much faster with the PARALLEL option.

My second assumption is that you’re you don’t know how to use or choose not to use the DBMS_LOB package; specifically, the loadclobfromfile and loadblobfromfile procedures. By the way, Chapter 10 of the Oracle Database 12c PL/SQL Programming has several complete examples using the DBMS_LOB package. My third assumption is that you’d like the external file packaged as a whole, which means you want it returned as a CLOB or BLOB data type.

Here’s a generic function that achieves that and avoids the nonsense with closing the file in the exception handler, or worse yet, wrapping it in another PL/SQL schema-level function:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 CREATE OR REPLACE FUNCTION read_file_to_clob ( pv_location VARCHAR2 , pv_file_name VARCHAR2 ) RETURN CLOB IS   /* Declare local input variables. */ lv_location VARCHAR2(60); lv_file_name VARCHAR2(40);   /* Declare a file reference pointer and buffer. */ lv_file UTL_FILE.FILE_TYPE; -- File reference lv_line VARCHAR2(32767); -- Reading buffer   /* Declare local sizing variables. */ lv_file_size NUMBER; lv_line_size NUMBER; lv_read_size NUMBER :=0;   /* Declare local file attribute data. */ lv_file_exists BOOLEAN := FALSE; lv_block_size BINARY_INTEGER;   /* Declare a control variable and return CLOB variable. */ lv_enable BOOLEAN := FALSE; lv_return CLOB; BEGIN /* Declare local input variables. */ lv_location := pv_location; lv_file_name := pv_file_name;   /* Check for open file and close when open. */ IF utl_file.is_open(lv_file) THEN utl_file.fclose(lv_file); END IF;   /* Read the file attributes to get the physical size. */ utl_file.fgetattr( location => lv_location , filename => lv_file_name , fexists => lv_file_exists , file_length => lv_file_size , block_size => lv_block_size );   /* Open only files that exist. */ IF lv_file_exists THEN   /* Create a temporary CLOB in memory. */ dbms_lob.createtemporary(lv_return, FALSE, dbms_lob.CALL);   /* Open the file for read-only of 32,767 byte lines. */ lv_file := utl_file.fopen( location => lv_location , filename => lv_file_name , open_mode => 'R' , max_linesize => 32767);     /* Read all lines of a text file. */ WHILE (lv_read_size < lv_file_size) LOOP /* Read a line of text until the eof marker. */ utl_file.get_line( file => lv_file , buffer => lv_line );   /* Add the line terminator or 2 bytes to its length. */ lv_line := NVL(lv_line,'')||CHR(10); lv_read_size := lv_read_size + LENGTH(NVL(lv_line,CHR(10))) + 2;   /* Write to an empty CLOB or append to an existing CLOB. */ IF NOT lv_enable THEN /* Write to the temporary CLOB variable. */ dbms_lob.WRITE( lv_return, LENGTH(lv_line), 1, lv_line);   /* Set the control variable. */ lv_enable := TRUE; ELSE /* Append to the temporary CLOB variable. */ dbms_lob.writeappend( lv_return, LENGTH(lv_line),lv_line); END IF; END LOOP;   /* Close the file. */ utl_file.fclose(lv_file); END IF;   /* This line is never reached. */ RETURN lv_return; EXCEPTION WHEN OTHERS THEN utl_file.fclose(lv_file); RAISE NO_DATA_FOUND; END; /

You can test the function with the following:

SET LONG 100000 SET PAGESIZE 999   SELECT read_file_to_clob('SomeVirtualDirectory','TextFile.txt') AS "Output" FROM dual;

If anybody has suggestions for improvements, please pass them along. As always, I hope this helps other developers.

Written by maclochlainn

April 16th, 2014 at 3:07 am

Using UTL_FILE Package

Sometimes I’m surprised. Today, the surprise came when somebody pointed to a potential error in another author’s book. The person who asked the question had to send me a screen shot before I believed it.

The author’s code encounters the following error because the code was designed to loop through a multiple line file, and the code called the UTL_FILE.FOPEN procedure with three instead of four parameters. While it works with only three parameters when the strings are less than or equal to 1,024 (thanks Gary), it throws read errors when a string exceeds the default. You use the fourth parameter when your string exceeds the default length of 1,024.

DECLARE * ERROR at line 1: ORA-29284: file READ error ORA-06512: at "SYS.UTL_FILE", line 106 ORA-06512: at "SYS.UTL_FILE", line 746 ORA-06512: at line 26

As mentioned, you fix the problem by using the fourth parameter like the call on lines 15 through 18 below:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 DECLARE /* Declare local input variables. */ lv_location VARCHAR2(60) := 'C:\Data\Direct'; lv_file_name VARCHAR2(40) := 'TextFile.txt';   /* Declare a file reference pointer and buffer. */ lv_file UTL_FILE.FILE_TYPE; -- File reference lv_line VARCHAR2(32767); -- Reading buffer   BEGIN   /* Check for open file and close when open. */ IF utl_file.is_open(lv_file) THEN utl_file.fclose(lv_file); END IF;   /* Open the file for read-only of 32,767 lines of text. The fourth parameter is required when you want to use the GET_LINE procedure to read a file line-by-line. */ lv_file := utl_file.fopen( location => lv_location , filename => lv_file_name , open_mode => 'R' , max_linesize => 32767);   /* Read all lines of a text file. */ LOOP /* Read a line of text, when the eof marker is found the get_line procedure raises a NO_DATA_FOUND error, which is why there's no explicit loop. */ utl_file.get_line( file => lv_file , buffer => lv_line );   /* Print the line of text or a line return because UTL_FILE doesn't read line returns. */ dbms_output.put_line(NVL(lv_line,CHR(10)));   END LOOP;   EXCEPTION WHEN NO_DATA_FOUND THEN utl_file.fclose(lv_file); END; /

While the foregoing is traditional and uses a physical directory path from the local server’s operating system, you can use a virtual directory from Oracle Database 10g forward. This physical directory is also defined in the utl_file_dir parameter of the database server. A virtual directory hides the physical directory from the application software, which simplifies physical file maintenance when you need to move the files.

3 lv_location VARCHAR2(60) := 'DirectVirtualDirectory';

The preceding example works, and I’ll put one out converting the external to CLOB later in the week.

Written by maclochlainn

April 13th, 2014 at 11:46 pm

DBMS_JAVA Privilege Error?

It’s possible to get an error after granting privileges to an external file system. One of those errors is tedious to resolve until you understand the rules governing Java NIO file permissions.

You grant privileges to external file systems as the sys user with the grant_permission procedure of the dbms_java package, like

SQL> BEGIN 2 DBMS_JAVA.GRANT_PERMISSION('IMPORTER' 3 ,'SYS:java.io.FilePermission' 4 ,'C:\Data\Upload' 5 ,'read'); 6 END; 7 /

After you grant privileges to read, write, and delete files with Oracle’s dbms_java package, you must disconnect any active session to enjoy the new privileges. If you forget to disconnect and re-connect an active session, you get the following error message:

FROM TABLE(list_files(get_directory_path('STAGE'))) * ERROR at line 2: ORA-29532: Java CALL TERMINATED BY uncaught Java exception: java.security.AccessControlException: Directory permissions restricted.

The problem and fix are simple. The permissions aren’t in effect until after you disconnect and reconnect your active sessions. I put this there because when I Google’d it, there wasn’t an answer already.

While I’m on the topic of privilege errors, that aren’t really errors, I thought it would be helpful to qualify how the delete permission works because I couldn’t find it anywhere in the Oracle documentation (if somebody finds it let make a comment that enriches the discussion). Unlike the read and write permissions, the delete permission requires permissions on specific files.

If you only grant permission on like:

SQL> BEGIN 2 DBMS_JAVA.GRANT_PERMISSION('IMPORTER' 3 ,'SYS:java.io.FilePermission' 4 ,'C:\Data\Upload' 5 ,'read,write,delete'); 6 END; 7 /

You would get this message when trying to delete an external file:

BEGIN * ERROR at line 1: ORA-29532: Java CALL TERMINATED BY uncaught Java exception: java.security.AccessControlException: the Permission (java.io.FilePermission c:\DATA\log\item_import.log READ) has NOT been GRANTED TO IMPORTER. The PL/SQL TO GRANT this IS dbms_java.grant_permission( 'IMPORTER', 'SYS:java.io.FilePermission', 'c:\data\log\item_import.log', 'read' ) ORA-06512: at "IMPORTER.DELETE_FILE", line 1 ORA-06512: at line 2

It requires that you grant permissions on individual files to have the privilege of deleting them from within Oracle. That grant would look like the following:

SQL> BEGIN 2 DBMS_JAVA.GRANT_PERMISSION('IMPORTER' 3 ,'SYS:java.io.FilePermission' 4 ,'C:\Data\Upload\item_import.log' 5 ,'read,write,delete'); 6 END; 7 /

As always, I hope this helps other developers.

Written by maclochlainn

March 29th, 2014 at 1:59 pm