Archive for April, 2014
Toad Freeware Page
While I posted how to install Toad for MySQL Freeware five years ago, I’m always surprised how few people know about it there and consistently updated and improved. You can download Toad for MySQL Freeware or Toad Freeware for Oracle, SQL Server, Sybase, or IBM DB2 at this web site.
You can also download Toad Data Modeler Freeware Edition. Just two notes, while Toad for Oracle Freeware is an MSI file, Toad for MySQL Freeware is a zip file and limited to only a Windows install.
A/UX, NeXTSTEP, & OS X
One thing that gets tedious in the IT community and Oracle community is the penchant for Windows only solutions. While Microsoft does an excellent job in certain domains, I remain a loyal Apple customer. By the way, you can install Oracle Client software on Mac OS X and run SQL Developer against any Oracle Database server. You can even run MySQL Workbench and MySQL server natively on the Mac OS X platform, which creates a robust development platform and gives you more testing options with the MySQL monitor (the client software).
Notwithstanding, some Windows users appear to malign Apple and the Mac OS X on compatibility, but they don’t understand that it’s a derivative of the Research Unix, through BSD (Berkeley Software Distribution). This Unix lineage chart illustrates it well:
I’m probably loyal to Apple because in the early 1990’s I worked on Mac OS 6, Mac OS 7, A/UX, NeXTSTEP, and AIX/6000 (Version 3) while working at APL (American President Lines) in Oakland, California. Back then, my desktop was a pricey Macintosh Quadra 950 and today I work on a pricey Mac Pro desktop. The Mac Pro lets me use VMware virtualize development environments for Oracle Linux, Red Hat Enterprise Linux, Fedora, and as you might guess Windows 7/8. My question to those dyed in the wool Microsoft users is simple, why would you choose a single user OS like Windows over a multi-user OS like Mac OS X?
Best UTL_FILE Practice
In a post a couple days ago, I promised to provide a best practice approach to reading external files with the UTL_FILE
package. My first assumption is that you’re reading unstructured data because structured data is best read by external tables because external tables can read data much faster with the PARALLEL
option.
My second assumption is that you’re you don’t know how to use or choose not to use the DBMS_LOB
package; specifically, the loadclobfromfile
and loadblobfromfile
procedures. By the way, Chapter 10 of the Oracle Database 12c PL/SQL Programming has several complete examples using the DBMS_LOB
package. My third assumption is that you’d like the external file packaged as a whole, which means you want it returned as a CLOB
or BLOB
data type.
Here’s a generic function that achieves that and avoids the nonsense with closing the file in the exception handler, or worse yet, wrapping it in another PL/SQL schema-level function:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 | CREATE OR REPLACE FUNCTION read_file_to_clob ( pv_location VARCHAR2 , pv_file_name VARCHAR2 ) RETURN CLOB IS /* Declare local input variables. */ lv_location VARCHAR2(60); lv_file_name VARCHAR2(40); /* Declare a file reference pointer and buffer. */ lv_file UTL_FILE.FILE_TYPE; -- File reference lv_line VARCHAR2(32767); -- Reading buffer /* Declare local sizing variables. */ lv_file_size NUMBER; lv_line_size NUMBER; lv_read_size NUMBER :=0; /* Declare local file attribute data. */ lv_file_exists BOOLEAN := FALSE; lv_block_size BINARY_INTEGER; /* Declare a control variable and return CLOB variable. */ lv_enable BOOLEAN := FALSE; lv_return CLOB; BEGIN /* Declare local input variables. */ lv_location := pv_location; lv_file_name := pv_file_name; /* Check for open file and close when open. */ IF UTL_FILE.is_open(lv_file) THEN UTL_FILE.fclose(lv_file); END IF; /* Read the file attributes to get the physical size. */ UTL_FILE.fgetattr( location => lv_location , filename => lv_file_name , fexists => lv_file_exists , file_length => lv_file_size , block_size => lv_block_size ); /* Open only files that exist. */ IF lv_file_exists THEN /* Create a temporary CLOB in memory. */ DBMS_LOB.createtemporary(lv_return, FALSE, DBMS_LOB.CALL); /* Open the file for read-only of 32,767 byte lines. */ lv_file := UTL_FILE.fopen( location => lv_location , filename => lv_file_name , open_mode => 'R' , max_linesize => 32767); /* Read all lines of a text file. */ WHILE (lv_read_size < lv_file_size) LOOP /* Read a line of text until the eof marker. */ UTL_FILE.get_line( file => lv_file , buffer => lv_line ); /* Add the line terminator or 2 bytes to its length. */ lv_line := NVL(lv_line,'')||CHR(10); lv_read_size := lv_read_size + LENGTH(NVL(lv_line,CHR(10))) + 2; /* Write to an empty CLOB or append to an existing CLOB. */ IF NOT lv_enable THEN /* Write to the temporary CLOB variable. */ DBMS_LOB.WRITE( lv_return, LENGTH(lv_line), 1, lv_line); /* Set the control variable. */ lv_enable := TRUE; ELSE /* Append to the temporary CLOB variable. */ DBMS_LOB.writeappend( lv_return, LENGTH(lv_line),lv_line); END IF; END LOOP; /* Close the file. */ UTL_FILE.fclose(lv_file); END IF; /* This line is never reached. */ RETURN lv_return; EXCEPTION WHEN OTHERS THEN UTL_FILE.fclose(lv_file); RAISE NO_DATA_FOUND; END; / |
You can test the function with the following:
SET LONG 100000 SET PAGESIZE 999 SELECT read_file_to_clob('SomeVirtualDirectory','TextFile.txt') AS "Output" FROM dual; |
If anybody has suggestions for improvements, please pass them along. As always, I hope this helps other developers.
Using UTL_FILE Package
Sometimes I’m surprised. Today, the surprise came when somebody pointed to a potential error in another author’s book. The person who asked the question had to send me a screen shot before I believed it.
The author’s code encounters the following error because the code was designed to loop through a multiple line file, and the code called the UTL_FILE.FOPEN
procedure with three instead of four parameters. While it works with only three parameters when the strings are less than or equal to 1,024 (thanks Gary), it throws read errors when a string exceeds the default. You use the fourth parameter when your string exceeds the default length of 1,024.
DECLARE * ERROR AT line 1: ORA-29284: FILE read error ORA-06512: AT "SYS.UTL_FILE", line 106 ORA-06512: AT "SYS.UTL_FILE", line 746 ORA-06512: AT line 26 |
As mentioned, you fix the problem by using the fourth parameter like the call on lines 15 through 18 below:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 | DECLARE /* Declare local input variables. */ lv_location VARCHAR2(60) := 'C:\Data\Direct'; lv_file_name VARCHAR2(40) := 'TextFile.txt'; /* Declare a file reference pointer and buffer. */ lv_file UTL_FILE.FILE_TYPE; -- File reference lv_line VARCHAR2(32767); -- Reading buffer BEGIN /* Check for open file and close when open. */ IF UTL_FILE.is_open(lv_file) THEN UTL_FILE.fclose(lv_file); END IF; /* Open the file for read-only of 32,767 lines of text. The fourth parameter is required when you want to use the GET_LINE procedure to read a file line-by-line. */ lv_file := UTL_FILE.fopen( location => lv_location , filename => lv_file_name , open_mode => 'R' , max_linesize => 32767); /* Read all lines of a text file. */ LOOP /* Read a line of text, when the eof marker is found the get_line procedure raises a NO_DATA_FOUND error, which is why there's no explicit loop. */ UTL_FILE.get_line( file => lv_file , buffer => lv_line ); /* Print the line of text or a line return because UTL_FILE doesn't read line returns. */ DBMS_OUTPUT.put_line(NVL(lv_line,CHR(10))); END LOOP; EXCEPTION WHEN NO_DATA_FOUND THEN UTL_FILE.fclose(lv_file); END; / |
While the foregoing is traditional and uses a physical directory path from the local server’s operating system, you can use a virtual directory from Oracle Database 10g forward. This physical directory is also defined in the utl_file_dir
parameter of the database server. A virtual directory hides the physical directory from the application software, which simplifies physical file maintenance when you need to move the files.
3 | lv_location VARCHAR2(60) := 'DirectVirtualDirectory'; |
The preceding example works, and I’ll put one out converting the external to CLOB
later in the week.