MacLochlainns Weblog

Michael McLaughlin's Technical Blog

Site Admin

Archive for the ‘sql’ Category

Bash Arrays & MySQL

with 2 comments

Student questions are always interesting! They get me to think and to write. The question this time is: “How do I write a Bash Shell script to process multiple MySQL script files?” This post builds the following model (courtesy of MySQL Workbench) by using a bash shell script and MySQL script files, but there’s a disclaimer on this post. It shows both insecure and secure approaches and you should avoid the insecure ones.

LittleERDModel

It seems a quick refresher on how to use arrays in bash shell may be helpful. While it’s essential in a Linux environment, it’s seems not everyone masters the bash shell.

Especially, since I checked my Learning the Bash Shell (2nd Edition) and found a typo on how you handle arrays in the bash shell, and it’s a mistake that could hang newbies up (on page 161). Perhaps I should update my copy because I bought it in 1998. 😉 It was good then, and the new edition is probably better. The error is probably corrected in the current Learning the Bash Shell, but if not, the following examples show you how to use arrays in loops.

Naturally, these do presume some knowledge of working with bash shell, like the first line always is the same in any bash shell script. That you open an if-statement with an if and close it with a fi, and that you else-if is elif; and that a semicolon between a for-statement and the do statement is required when they’re on the same line because they’re two statements.

If you’re new to bash shell arrays, click on the link below to expand a brief tutorial. It takes you through three progressive examples of working with bash arrays.

Only one more trick needs to be qualified before our main MySQL examples. That trick is how you pass parameters to a bash shell script. For reference, this is the part that’s insecure because user command histories are available inside the Linux OS.

Here’s a hello_whom.sh script to demonstrates the concept of parameter passing:

1
2
3
4
5
6
7
8
9
10
#!/usr/bin/bash
 
# This says hello to the argument while managing no argument.
if [[ ${#} = 1 ]]; then
  echo 'The '${0}' program says: "Hello '${1}'!"'
elif [[ ${#} > 1 ]]; then
  echo 'The '${0}' program wants to know if you have more than one name?'
else
  echo 'The '${0}' program wants to know if you have a name?'
fi

If you need more on how parameters are passed and managed, you can check a prior blob post on Handling bash Parameters, or check the bash help pages. The following leverages bash arrays to run scripts and query the MySQL database from the command line.

You will need the three batch SQL files first, so here they are:

The following list_mysql.sh shell script expects to receive the username, password, database and fully qualified path in that specific order. The script names are entered manually because this should be a unit test script. Naturally, you can extend the script to manage those parameters but as mentioned I see this type of solution as a developer machine only script to simplify unit testing. Anything beyond that is risky!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
#!/usr/bin/bash
 
# Assign user and password
username="${1}"
password="${2}"
database="${3}"
directory="${4}"
 
# List the parameter values passed.
echo "Username:  " ${username}
echo "Password:  " ${password}
echo "Database:  " ${database}
echo "Directory: " ${directory}
echo ""
 
# Define an array.
declare -a cmd
 
# Assign elements to an array.
cmd[0]="actor.sql"
cmd[1]="film.sql"
cmd[2]="movie.sql"
 
# Call the array elements.
for i in ${cmd[*]}; do
  mysql -s -u${username} -p${password} -D${database} < ${directory}/${i} > /dev/null 2>/dev/null
done
 
# Connect and pipe the query result minus errors and warnings to the while loop.
mysql -u${username} -p${password} -D${database} <<<'show tables' 2>/dev/null |
 
# Read through the piped result until it's empty but format the title.
while IFS='\n' read list; do
  if [[ ${list} = "Tables_in_sampledb" ]]; then
    echo $list
    echo "----------------------------------------"
  else
    echo $list
  fi
done
echo ""
 
# Connect and pipe the query result minus errors and warnings to the while loop.
mysql -u${username} -p${password} -D${database} <<<'SELECT CONCAT(a.actor_name," in ",f.film_name) AS "Actors in Films" FROM actor a INNER JOIN movie m ON a.actor_id = m.actor_id INNER JOIN film f ON m.film_id = f.film_id' 2>/dev/null |
 
# Read through the piped result until it's empty but format the title.
while IFS='\n' read actor_name; do
  if [[ ${actor_name} = "Actors in Films" ]]; then
    echo $actor_name
    echo "----------------------------------------"
  else
    echo $actor_name
  fi
done

The IFS (Internal Field Separator) works with whitespace by default. The IFS on lines 33 and 47 sets the IFS to a line return ('\n'). That’s the trick to display the data, and you can read more about the IFS in this question and answer post.

You can run this script with the following input parameters from the local directory where you deploy it. The a parameters are: (1) username, (2) password, (3) database, and (4) a fully qualified path to the SQL setup files.

./list_mysql.sh student student sampledb "/home/student/Code/bash/mysql"

With valid input values, the list_mysql.sh bash script generates the following output, which confirms inputs and verifies actions taken by the scripts with queries:

Username:   student
Password:   student
Database:   sampledb
Directory:  /home/student/Code/bash/mysql
 
Tables_in_sampledb
----------------------------------------
actor
film
movie
 
Actors in Films
----------------------------------------
Chris Hemsworth in Thor
Chris Hemsworth in Thor: The Dark World
Chris Pine in Star Trek
Chris Pine in Star Trek into Darkness
Chris Pine in Guardians of the Galaxy

If you forgot to provide the required inputs to the list_mysql.sh bash script, it alternatively returns the following output:

Username:  
Password:  
Database:  
Directory: 
 
./list_mysql.sh: line 25: /actor.sql: No such file or directory
./list_mysql.sh: line 25: /film.sql: No such file or directory
./list_mysql.sh: line 25: /movie.sql: No such file or directory

The secure way removes the password at a minimum! The refactored program will require you to manually enter the password for all elements of the array (three in this sample), and twice for the two queries. Here’s the refactored code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
#!/usr/bin/bash
 
# Assign user and password
username="${1}"
database="${2}"
directory="${3}"
 
# List the parameter values passed.
echo "Username:  " ${username}
echo "Database:  " ${database}
echo "Directory: " ${directory}
echo ""
 
# Define an array.
declare -a cmd
 
# Assign elements to an array.
cmd[0]="actor.sql"
cmd[1]="film.sql"
cmd[2]="movie.sql"
 
# Call the array elements.
for i in ${cmd[*]}; do
  mysql -s -u${username} -p -D${database} < ${directory}/${i} > /dev/null 2>/dev/null
done
 
# Connect and pipe the query result minus errors and warnings to the while loop.
mysql -u${username} -p -D${database} <<<'show tables' 2>/dev/null |
 
# Read through the piped result until it's empty.
while IFS='\n' read list; do
  if [[ ${list} = "Tables_in_sampledb" ]]; then
    echo $list
    echo "----------------------------------------"
  else
    echo $list
  fi
done
echo ""
 
# Connect and pipe the query result minus errors and warnings to the while loop.
mysql -u${username} -p -D${database} <<<'SELECT CONCAT(a.actor_name," in ",f.film_name) AS "Actors in Films" FROM actor a INNER JOIN movie m ON a.actor_id = m.actor_id INNER JOIN film f ON m.film_id = f.film_id' 2>/dev/null |
 
# Read through the piped result until it's empty.
while IFS='\n' read actor_name; do
  if [[ ${actor_name} = "Actors in Films" ]]; then
    echo $actor_name
    echo "----------------------------------------"
  else
    echo $actor_name
  fi
done

Please let me know if you think there should be any more scaffolding for newbies in this post. As always, I hope this helps those looking for this type of solution.

Written by maclochlainn

May 17th, 2015 at 12:01 pm

Leaf node queries

without comments

A reader posted A dynamic level limiting hierarchical query about Oracle’s hierarchical queries. They wanted to know how to capture only the hierarchy above the level where the first leaf node occurs. They gave me the following hierarchy map as an example:

               1                                    2
        +-------------+                       +-----------+
        |             |                       |           |      
        3             5                       4           6
    +---------+    +-----------+           +-----+    +------+
    |         |    |           |           |     |    |      |
    7         9    11          13          8     10   12     14
+-----+   +-----+  +--+    +-------+                       +-----+ 
|     |   |     |     |    |       |                       |     |
15    17  19    21    23   27      29                     16     18
                                                                 +---+
                                                                     |
                                                                     20

You can find the node values and hierarchical level with the following query:

SELECT   tt.child_id
,        LEVEL
FROM     test_temp tt
WHERE    CONNECT_BY_ISLEAF  = 1
START
WITH     tt.parent_id IS NULL 
CONNECT
BY PRIOR tt.child_id = tt.parent_id
ORDER BY 2;

We really don’t need the node values to solve the problem. We only need the lowest LEVEL value returned by the query, which is 3. The combination of the MIN and CONNECT_BY_ISLEAF functions let us solve this problem without writing a PL/SQL solution. The subquery returns the lowest level value, which is the first level where a leaf node occurs.

SELECT   LPAD(' ', 2*(LEVEL - 1)) || tt.child_id AS child_id
FROM     test_temp tt
WHERE    LEVEL <= (SELECT   MIN(LEVEL)
                   FROM     test_temp tt
                   WHERE    CONNECT_BY_ISLEAF  = 1
                   START
                   WITH     tt.parent_id IS NULL 
                   CONNECT
                   BY PRIOR tt.child_id = tt.parent_id)
START
WITH     tt.parent_id IS NULL
CONNECT
BY PRIOR tt.child_id = tt.parent_id;

It returns:

               1                                    2
        +-------------+                       +-----------+
        |             |                       |           |      
        3             5                       4           6
    +---------+    +-----------+           +-----+    +------+
    |         |    |           |           |     |    |      |
    7         9    11          13          8     10   12     14

While I answered the question in a comment originally, it seemed an important trick that should be shared in its own post.

Written by maclochlainn

April 30th, 2015 at 4:31 pm

MySQL OCP Exams

with 6 comments

Planning out my year, I decided to take the Oracle OCP and MySQL OCP exams. I checked for review books and was pleasantly surprised to find the soon to be released OCP MySQL Database Administrator Exam Guide (Exam 1Z0-883). However, I noticed that the book was actually prepared for the obsolete and discountinued Exams 1Z0-870, 1Z0-873, and 1Z0-874. As it turns out, Steve O’Hearn has informed me that there isn’t a book and that the posting in Amazon.com is in error.

There isn’t an alternative review book for the OCP MySQL 5.6 Developer or Database Administrator Exams. The question that I have is simple: “How relevant is this book because it was prepared for the older exams?” There isn’t a table of content published on the Amazon.com site. If there was a table of contents it could help me determine how close the book’s content is to the new exam.

As a preparation to figure out the value of the book as a study guide, I’ve reviewed the current Oracle MySQL Training Objectives (listed below). The new MySQL OCP Developer and Administrator exams have the following descriptions and objectives:

  • MySQL 5.6 Developer 1Z0-882. Oracle provides the following outline for their MySQL for Developer (Ed 3) training course:

    Course Objectives

    • Describe the MySQL client/server architecture
    • Use MySQL client programs and common options
    • Program MySQL applications with Java and PHP connectors
    • Use a “NoSQL” approach to store and retrieve data
    • Design efficient tables
    • Create and delete database objects
    • Use expressions in SQL statements
    • Examine database metadata
    • Use SQL statements to modify table data
    • Maintain database integrity with transactions
    • Write multiple table queries
    • Create “virtual tables” containing specific data
    • Create user-defined variables, prepared statements, and stored routines
    • Create and manage triggers
    • Identify and deal with errors and exceptions in client programs
    • Write queries that are fast and effective, even under heavy loads
  • MySQL 5.6 Database Administrator 1Z0-883. Oracle provides the following outline for their MySQL for Database Administrators (Ed 3.1) training course:

    Course Objectives

    • Describe the MySQL Architecture
    • Install and Upgrade MySQL
    • Use the INFORMATION_SCHEMA database to access metadata
    • Perform the MySQL start and shutdown operations
    • Configure MySQL server options at runtime
    • Use available tools including MySQL Workbench
    • Evaluate data types and character sets for performance issues
    • Understand data locking in MySQL
    • Understand the use of the InnoDB storage engine with MySQL
    • Maintain integrity of a MySQL installation
    • Use triggers for administration tasks
    • Use Enterprise Audit and Pluggable Authentication
    • Configure advanced replication topologies for high availability
    • Describe introductory performance tuning techniques
    • Perform backup and restore operations
    • Automate administrative tasks with scheduled events

    As always, I hope this helps those who read it; and, in this case I hope it helps you make an effective decision on preparation resources for the MySQL 5.6 OCP exams.

Written by maclochlainn

April 24th, 2015 at 12:39 am

Oracle Cleanup a Schema

with one comment

Back in January 2014, I wrote a script to cleanup an Oracle student schema. It worked well until I started using APEX 4 in my student schema. You create the following 75 objects when you create an APEX 4 schema.

OBJECT TYPE    TOTAL
------------ -------
TABLE		  17
INDEX		  28
SEQUENCE	   5
TRIGGER 	  14
LOB		   9
FUNCTION	   2

Here’s the modified script that ignores the objects created automatically by Oracle APEX when you create a student workspace:

BEGIN
  FOR i IN (SELECT    object_name
            ,         object_type
            ,         last_ddl_time
            FROM      user_objects
            WHERE     object_name NOT IN
                       ('APEX$_WS_WEBPG_SECTION_HISTORY','APEX$_WS_WEBPG_SECTIONS_T1'
                       ,'APEX$_WS_WEBPG_SECTIONS_PK','APEX$_WS_WEBPG_SECTIONS'
                       ,'APEX$_WS_WEBPG_SECHIST_IDX1','APEX$_WS_TAGS_T1'
                       ,'APEX$_WS_TAGS_PK','APEX$_WS_TAGS_IDX2','APEX$_WS_TAGS_IDX1'
                       ,'APEX$_WS_TAGS','APEX$_WS_ROWS_T1','APEX$_WS_ROWS_PK'
                       ,'APEX$_WS_ROWS_IDX','APEX$_WS_ROWS','APEX$_WS_NOTES_T1'
                       ,'APEX$_WS_NOTES_PK','APEX$_WS_NOTES_IDX2','APEX$_WS_NOTES_IDX1'
                       ,'APEX$_WS_NOTES','APEX$_WS_LINKS_T1','APEX$_WS_LINKS_PK'
                       ,'APEX$_WS_LINKS_IDX2','APEX$_WS_LINKS_IDX1','APEX$_WS_LINKS'
                       ,'APEX$_WS_HISTORY_IDX','APEX$_WS_HISTORY','APEX$_WS_FILES_T1'
                       ,'APEX$_WS_FILES_PK','APEX$_WS_FILES_IDX2','APEX$_WS_FILES_IDX1'
                       ,'APEX$_WS_FILES','APEX$_ACL_T1','APEX$_ACL_PK','APEX$_ACL_IDX1'
                       ,'APEX$_ACL','CUSTOM_AUTH','CUSTOM_HASH','DEPT','EMP'
                       ,'UPDATE_ORDER_TOTAL')
            AND NOT ((object_name LIKE 'DEMO%' OR
                      object_name LIKE 'INSERT_DEMO%' OR
                      object_name LIKE 'BI_DEMO%') AND
                      object_type IN ('TABLE','INDEX','SEQUENCE','TRIGGER'))
            AND NOT (object_name LIKE 'SYS_LOB%' AND object_type = 'LOB')
            AND NOT (object_name LIKE 'SYS_C%' AND object_type = 'INDEX')
            ORDER BY object_type DESC) LOOP
 
    /* Drop types in descending order. */
    IF i.object_type = 'TYPE' THEN
 
      /* Drop type and force operation because dependencies may exist. Oracle 12c
         also fails to remove object types with dependents in pluggable databases
         (at least in release 12.1). Type evolution works in container database
         schemas. */
      EXECUTE IMMEDIATE 'DROP '||i.object_type||' '||i.object_name||' FORCE';
 
    /* Drop table tables in descending order. */
    ELSIF i.object_type = 'TABLE' THEN
 
      /* Drop table with cascading constraints to ensure foreign key constraints
         don't prevent the action. */
      EXECUTE IMMEDIATE 'DROP '||i.object_type||' '||i.object_name||' CASCADE CONSTRAINTS PURGE';
 
      /* Oracle 12c ONLY: Purge the recyclebin to dispose of system-generated
         sequence values because dropping the table doesn't automatically 
         remove them from the active session.
         CRITICAL: Remark out the following when working in Oracle Database 11g. */
      EXECUTE IMMEDIATE 'PURGE RECYCLEBIN';
 
    ELSIF i.object_type = 'LOB' OR i.object_type = 'INDEX' THEN
 
      /* A system generated LOB column or INDEX will cause a failure in a
         generic drop of a table because it is listed in the cursor but removed
         by the drop of its table. This NULL block ensures there is no attempt
         to drop an implicit LOB data type or index because the dropping the
         table takes care of it. */
      NULL;
 
    ELSE
 
      dbms_output.put_line('DROP '||i.object_type||' '||i.object_name||';');
      /* Drop any other objects, like sequences, functions, procedures, and packages. */
      EXECUTE IMMEDIATE 'DROP '||i.object_type||' '||i.object_name;
 
    END IF;
  END LOOP;
END;
/

As always, I hope this helps others.

Written by maclochlainn

April 19th, 2015 at 7:13 pm

MySQLdb Manage Columns

without comments

Sometimes trying to keep a post short and to the point raises other questions. Clearly, my Python-MySQL Program post over the weekend did raise a question. They were extending the query example and encountered this error:

TypeError: range() integer end argument expected, got tuple.

That should be a straight forward error message because of two things. First, the Python built-in range() function manages a range of numbers. Second, the row returned from a cursor is actually a tuple (from relational algebra), and it may contain non-numeric data like strings and dates.

The reader was trying to dynamically navigate the number of columns in a row by using the range() function like this (where row was a row from the cursor or result set):

    for j in range(row):

Naturally, it threw the type mismatch error noted above. As promised, the following Python program fixes that problem. It also builds on the prior example by navigatung an unknown list of columns. Lines 16 through 31 contain the verbose comments and programming logic to dynamically navigate the columns of a row.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
#!/usr/bin/python
 
# Import sys library.
import MySQLdb
import sys
 
try:
  # Create new database connection.
  db = MySQLdb.connect('localhost','student','student','studentdb')
  # Create a result set cursor.
  rs = db.cursor()
  rs.execute("SELECT item_title, item_subtitle, item_rating FROM item")
  # Assign the query results to a local variable.
  for i in range(rs.rowcount):
    row = rs.fetchone()
    # Initialize variable for printing row as a string.
    data = ""
    # Address an indefinite number of columns.
    count = 0
    for j in range(len(row)):
      # Initialize column value as an empty string.
      datum = ""
      # Replace column values when they exist.
      if str(row[count]) != 'None':
        datum = str(row[count])
      # Append a comma when another column follows.
      if count == len(row) - 1:
        data += datum
      else:
        data += datum + ", "
      count += 1
    # Print the formatted row as a string.
    print data
except MySQLdb.Error, e:
  # Print the error.
  print "ERROR %d: %s" % (e.args[0], e.args[1])
  sys.exit(1)
finally:
  # Close the connection when it is open.
  if db:
    db.close()

There are a couple Python programming techniques that could be perceived as tricks. Line 24 checks for a not null value by explicitly casting the column’s value to a string and then comparing its value against the string equivalent for a null. The MySQLdb returns a 'None' string for null values by default. The if-block on lines 27 through 30 ensure commas aren’t appended at the end of a row.

While the for-loop with a range works, I’d recommend you write it as a while-loop because its easier to read for most new Python programmers. You only need to replace line 20 with the following to make the change:

20
    while (count < len(row)):

Either approach generates output like:

The Hunt for Red October, Special Collectornulls Edition, PG
Star Wars I, Phantom Menace, PG
Star Wars II, Attack of the Clones, PG
Star Wars II, Attack of the Clones, PG
Star Wars III, Revenge of the Sith, PG-13
The Chronicles of Narnia, The Lion, the Witch and the Wardrobe, PG
RoboCop, , Mature
Pirates of the Caribbean, , Teen
The Chronicles of Narnia, The Lion, the Witch and the Wardrobe, Everyone
MarioKart, Double Dash, Everyone
Splinter Cell, Chaos Theory, Teen
Need for Speed, Most Wanted, Everyone
The DaVinci Code, , Teen
Cars, , Everyone
Beau Geste, , PG
I Remember Mama, , NR
Tora! Tora! Tora!, The Attack on Pearl Harbor, G
A Man for All Seasons, , G
Hook, , PG
Around the World in 80 Days, , G
Harry Potter and the Sorcerer's Stone, , PG
Camelot, , G

As always, I hope this helps those looking for clarity.

Written by maclochlainn

April 13th, 2015 at 10:05 pm

Perl-MySQL Program

with 2 comments

Configuring Perl to work with MySQL is the last part creating a complete Fedora Linux LAMP stack for my students. Perl is already installed on Fedora Linux.

I’ve also shown how to use PHP, Python, and Ruby languages to query a MySQL database on Linux. After installing this additional Perl DBI library, my students will have the opportunity to choose how they implement their LAMP solution.

You can find the Perl version with the following version.pl program:

1
2
3
4
#!/usr/bin/perl -w
 
# Print the version.
print "Perl ".$]."\n";

The first line lets you call the program without prefacing the perl program because it invokes a subshell of perl by default. You just need to ensure the file has read and execute privileges to run. It prints:

Perl 5.018004

You need to install the perl-DBD-MySQL library to enable Perl to work with MySQL. The following command loads the library:

yum install -y perl-DBD-MySQL

It prints the following log file:

Loaded plugins: langpacks, refresh-packagekit
Resolving Dependencies
--> Running transaction check
---> Package perl-DBD-MySQL.x86_64 0:4.024-1.fc20 will be installed
--> Finished Dependency Resolution
 
Dependencies Resolved
 
================================================================================
 Package               Arch          Version                Repository     Size
================================================================================
Installing:
 perl-DBD-MySQL        x86_64        4.024-1.fc20           fedora        142 k
 
Transaction Summary
================================================================================
Install  1 Package
 
Total download size: 142 k
Installed size: 332 k
Downloading packages:
perl-DBD-MySQL-4.024-1.fc20.x86_64.rpm                      | 142 kB  00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction (shutdown inhibited)
  Installing : perl-DBD-MySQL-4.024-1.fc20.x86_64                           1/1 
  Verifying  : perl-DBD-MySQL-4.024-1.fc20.x86_64                           1/1 
 
Installed:
  perl-DBD-MySQL.x86_64 0:4.024-1.fc20                                          
 
Complete!

The following item_query.pl Perl program is consistent with the PHP, Python, and Ruby examples provided in other blog posts. It shows you how to use the Perl DBI library to query and return a data set.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
#!/usr/bin/perl -w
 
# Use the DBI library.
use DBI;
use strict;
use warnings;
 
# Create a connection.
my $dbh = DBI->connect("DBI:mysql:database=studentdb;host=localhost:3306","student","student",{'RaiseError' => 1});
 
# Create SQL statement.
my $sql = "SELECT item_title FROM item";
 
# Prepare SQL statement.
my $sth = $dbh->prepare($sql);
 
# Execute statement and read result set.
$sth->execute() or die $DBI::errstr;
while (my @row = $sth->fetchrow_array()) {
  my $item_title = $row[0];
  print "$item_title\n";
}
 
# Close resources.
$sth->finish();

You call it like this from the present working directory:

./mysql_query.pl

It returns:

The Hunt for Red October
Star Wars I
Star Wars II
Star Wars II
Star Wars III
The Chronicles of Narnia
RoboCop
Pirates of the Caribbean
The Chronicles of Narnia
MarioKart
Splinter Cell
Need for Speed
The DaVinci Code
Cars
Beau Geste
I Remember Mama
Tora! Tora! Tora!
A Man for All Seasons
Hook
Around the World in 80 Days
Harry Potter and the Sorcerer's Stone
Camelot

Alternatively, there’s a different syntax for lines 20 and 21 that you can use when you’re returning multiple columns. It replaces the two statements inside the while loop as follows:

20
21
  my ($item_title, $item_rating) = @row;
  print "$item_title, $item_rating\n";

It returns:

The Hunt for Red October, PG
Star Wars I, PG
Star Wars II, PG
Star Wars II, PG
Star Wars III, PG13
The Chronicles of Narnia, PG
RoboCop, Mature
Pirates of the Caribbean, Teen
The Chronicles of Narnia, Everyone
MarioKart, Everyone
Splinter Cell, Teen
Need for Speed, Everyone
The DaVinci Code, Teen
Cars, Everyone
Beau Geste, PG
I Remember Mama, NR
Tora! Tora! Tora!, G
A Man for All Seasons, G
Hook, PG
Around the World in 80 Days, G
Harry Potter and the Sorcerer's Stone, PG
Camelot, G

As always, I hope this helps those learning how to use Perl and Linux against the MySQL Database. If you want a nice tutorial on Perl and MySQL, check The tutorialspoint.com web site.

Written by maclochlainn

April 13th, 2015 at 2:14 am

MySQL JSON Functions

with 3 comments

What the MySQL team is doing with JSON (JavaScript Object Notation) in MySQL 5.7 is great! The MySQL Server Blog (Rick Hillegas and Dag Wanvik) published two key articles about new JSON functions. If you don’t follow these, let me highlight them as a set:

Most folks know how important JSON is to web development. I like the following visual that highlights it. It was provided as a comment to this earlier Popular Programming Language post by Michael Farmer. Clearly, JavaScript is popular because it’s critical to effective web development. If you’re new to JSON, check out Adam Khoury’s JSON tutorial set on YouTube.

PopularCode2014

If you want the original graphic, you can find it here. It’s always to hard to keep up with the technology, isn’t it? 🙂

Written by maclochlainn

April 11th, 2015 at 11:36 am

Ruby-MySQL Program

with 6 comments

After you install Ruby and build the Rails framework, you need to create the mysql gem. This blog post shows you how to create the mysql gem and how to write a simple Ruby program that queries the MySQL database.

The first step creates the mysql gem for Ruby programming:

gem install mysql

It should show you the following:

Fetching: mysql-2.9.1.gem (100%)
Building native extensions.  This could take a while...
Successfully installed mysql-2.9.1
Parsing documentation for mysql-2.9.1
Installing ri documentation for mysql-2.9.1
Done installing documentation for mysql after 0 seconds
1 gem installed

After you install the mysql Ruby Gem, you can write and test a test.rb Ruby program that tests a MySQL database connection. The simplest complete code looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# Include Ruby Gem libraries.
require 'rubygems'
require 'mysql'
 
begin
  # Create new database connection.
  db = Mysql.new('localhost','student','student','studentdb')
  # Print connected message.
  puts "Connected to the MySQL database server."
rescue Mysql::Error => e
  # Print the error.
  puts "ERROR #{e.errno} (#{e.sqlstate}): #{e.error}"
  puts "Can't connect to the MySQL database specified."
  # Signal an error.
  exit 1
ensure
  # Close the connection when it is open.
  db.close if db
end

You can run the program with the following syntax:

ruby test.rb

The program prints “Connected to the MySQL database server.” when there’s a student user with a student password that’s authorized to connect to the studentdb database. If any of the values are invalid when creating the connection, the program prints “Can’t connect to the MySQL database specified.”

Having tested the connection, the next query.rb program tests the connection by returning values from a query:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# Include Ruby Gem libraries.
require 'rubygems'
require 'mysql'
 
# Begin block.
begin
  # Create a new connection resource.
  db = Mysql.new('localhost','student','student','studentdb1')
  # Create a result set.
  rs = db.query('SELECT item_title FROM item')
  # Read through the result set hash.
  rs.each_hash do | row |
    puts "#{row['item_title']}"
  end
  # Release the result set resources.
  rs.free
rescue Mysql::Error => e
  # Print the error.
  puts "ERROR #{e.errno} (#{e.sqlstate}): #{e.error}"
  puts "Can't connect to MySQL database specified."
  # Signal an error.
  exit 1
ensure
  # Close the connection when it is open.
  db.close if db
end

You can test it with the following command-line syntax:

ruby query.rb

It returns a data set like this from the item table of my video store example:

+---------------------------------------+
| item_title                            |
+---------------------------------------+
| The Hunt for Red October              |
| Star Wars I                           |
| Star Wars II                          |
| Star Wars II                          |
| Star Wars III                         |
| The Chronicles of Narnia              |
| RoboCop                               |
| Pirates of the Caribbean              |
| The Chronicles of Narnia              |
| MarioKart                             |
| Splinter Cell                         |
| Need for Speed                        |
| The DaVinci Code                      |
| Cars                                  |
| Beau Geste                            |
| I Remember Mama                       |
| Tora! Tora! Tora!                     |
| A Man for All Seasons                 |
| Hook                                  |
| Around the World in 80 Days           |
| Harry Potter and the Sorcerer's Stone |
| Camelot                               |
+---------------------------------------+
22 rows in set (0.00 sec)

You need the ruby interpreter to run them. You can make the programs standalone operations by putting the following line as the first line in your Ruby programs.

1
#!/usr/bin/ruby

Then, you can run the program like this if they have read and execute privileges and are located in the present working directory where you issue the following command:

./mysql_query.rb

If you want to work with individual columns, please check this subsequent post that shows how you can access individual columns. As always, I hope this helps those trying to get things working.

After posting this somebody asked for books that could help them learn how to write Ruby programs. While books are nice and listed below, I’d start with the tryruby.org web site.

I’d recommend the following books because …

  • The Ruby Programming Language is 7 years old now and only covers Ruby 1.8 and 1.9, but its written by David Flanagan and the creator of the Ruby Programming Language – Yukihiro Matsumoto.
  • Programming Ruby 1.9 & 2.0: The Pragmatic Programmer’s Guide is more current and a well balanced approach at learning how to write Ruby programs.
  • The Well-Grounded Rubyist is the most current book and teaches you how to think about writing Ruby beyond just the syntax. As a Manning book, you can purchase the physical copy and automatically get a downloadable ebook. It’s certainly the best value for the money option provided you already know how to program in at least one other object-oriented programming language.

Written by maclochlainn

April 11th, 2015 at 2:35 am

APEX Create Table

with 2 comments

The following walks you through how you sign on to a STUDENT Workspace with Oracle’s APEX product. It shows you how to create a new table with the Object Browser tool.

You can find instructions on how to create your own STUDENT Workspace in this blog post. Overall, Oracle APEX is a valuable tool to learn and master.

UseStudentAPEX01

  1. You start the process by accessing the Oracle Database 11g APEX, which you can access at http://localhost:8080/apex by default on the server. If you’ve got a static IP address for your instance, you can replace localhost with the IP address or hostname for the IP address.

    • Workspace: STUDENT
    • Username:  ADMIN
    • Password:  STUDENT

UseStudentAPEX02

  1. After you login to the STUDENT workspace, you have four options. They are the: Application Builder, SQL Workshop, Team Development, and Administration. You start the process by accessing the Oracle Database 11g APEX, which you can access at http://localhost:8080/apex by default on the server. If you’ve got a static IP address for your instance, you can replace localhost with the IP address or hostname for the IP address. Click on the Object Browser icon to proceed.

UseStudentAPEX02A

  1. Clicking the SQL Workshop icon takes you to the second level menu. You click the Object Browser icon to create a database object.

OracleAPEXCT01

  1. After clicking the Object Browser icon, you see the screen at the left. Click the Create button to create a table.

OracleAPEXCT02

  1. After clicking the Create button, you see the screen at the left. Click the type of database object that you want to create. In our case, we click the Table hypertext to start the create table workflow.

OracleAPEXCT03

  1. After clicking the Table hyperlink, you see the Create Table screen at the left. Enter the column names, choose their data types and set the scale and precision. You should also check the Not Null checkbox when you want a column to be mandatory. Click the Next button to continue the create table workflow.

OracleAPEXCT04

  1. After entering the column names, you should choose the data types, enter the scale and precision, and check the NOT NULL checkbox to make appropriate columns mandatory by applying NOT NULL database constraints. If you run out of entry rows, you can click the Add Column button to add new rows. Click the Next button to continue the create table workflow when you’ve defined the columns.

OracleAPEXCT05

  1. After defining the column names, you should choose whether the primary key will use a new sequence or an existing sequence. You also have the ability to not assign a primary key value or simply leave it unpopulated when inserting new rows. The example creates an IMAGE_PK primary key constraint on the IMAGE_ID column, and declares an IMAGE_SEQ sequence value. Click the Next button to continue the create table workflow when you’ve defined the primary key constraint and any new sequence value for the primary key column.

OracleAPEXCTFK1

  1. After defining the primary key constraint, you can define foreign key column constraints. You enter a foreign key constraint name, choose between a Disallow Delete, Cascade Delete, or Set Null on Delete rule, select the foreign key column, the foreign key’s referenced table and column. Click the Add button to continue the create table workflow.

OracleAPEXCTFK2

  1. After defining a foreign key constraint, you can see the constraint that you created. Then, you can define another foreign key column constraints. You repeat the steps from the prior steps to add another foreign key constraint. Click the Add button to create a second foreign key constraint and complete the create table workflow.

OracleAPEXCTFK3

  1. After defining a second foreign key constraint, you see the following two foreign key constraints. Click the Next button to complete the create table workflow.

OracleAPEXTCUK01

  1. After defining all the foreign key constraints, you can create check and unique constraints. You check a radio button for a check or unique constraint, and then you select the columns for the constraint’s key. Click the /Add button to create any check or unique constraints as part of the create table workflow.

OracleAPEXTCUK02

  1. After defining all check and unique key constraints, you can see them in the Constraints box. Click the Next button to complete the create table workflow.

OracleAPEXCTC01

  1. After defining all items about the table, you can see the SQL to create the IMAGE table and its constraints. You can copy the SQL into a file for later use when writing a re-runnable script. Click the Create button to complete the create table workflow and create the table.

     

    The following are the contents of the script for the actions you’ve defined:

    CREATE table "IMAGE" (
        "IMAGE_ID"         NUMBER NOT NULL,
        "FILE_NAME"        VARCHAR2(60) NOT NULL,
        "MIME_TYPE"        NUMBER NOT NULL,
        "ITEM_IMAGE"       BLOB,
        "CREATED_BY"       NUMBER NOT NULL,
        "CREATION_DATE"    DATE NOT NULL,
        "LAST_UPDATED_BY"  NUMBER NOT NULL,
        "LAST_UPDATE_DATE" DATE NOT NULL,
        constraint  "IMAGE_PK" primary key ("IMAGE_ID")
    )
    /
    
    CREATE sequence "IMAGE_SEQ" 
    /
    
    CREATE trigger "BI_IMAGE"  
      before insert on "IMAGE"              
      for each row 
    begin  
      if :NEW."IMAGE_ID" is null then
        select "IMAGE_SEQ".nextval into :NEW."IMAGE_ID" from dual;
      end if;
    end;
    /   
    
    ALTER TABLE "IMAGE" ADD CONSTRAINT "IMAGE_FK1" 
    FOREIGN KEY ("CREATED_BY")
    REFERENCES "SYSTEM_USER" ("SYSTEM_USER_ID")
    
    /
    ALTER TABLE "IMAGE" ADD CONSTRAINT "IMAGE_FK2" 
    FOREIGN KEY ("LAST_UPDATED_BY")
    REFERENCES "SYSTEM_USER" ("SYSTEM_USER_ID")
    
    /
    alter table "IMAGE" add
    constraint "IMAGE_UK1" 
    unique ("FILE_NAME","MIME_TYPE")
    /   
    

OracleAPEXTable

  1. After creating the table, trigger, sequence, and constraints, you can see the table definition. You also have the ability to modify the table. At this point, you can create another structure or you can click the Home or SQL Workshop menu choice.

As always, I hope this helps those looking to learn new things and approaches.

Written by maclochlainn

April 7th, 2015 at 10:21 pm

Open Fedora Port 80

with 4 comments

After installing the LAMP stack on Fedora, you need to open port 80 in the Firewall to access the PHP programs on the Fedora instance from external servers. You can open a firewall port by launching the firewall-config application as the root user with the following syntax:

firewall-config

The firewall-config utility opens the following dialog:

FedoraFirewall1

Click on the Ports tab, and you’ll see the following:

FedoraFirewall2

Skip this step if you only want to set the runtime privilege to the port. Click on the Runtime tab and change it to Permanent if you want the port to be accessible when you reboot your OS.

FedoraFirewallPermanent

Click on Add button to add a port exception, and you’ll see the following:

FedoraFirewall3

Enter Port 80 for the Apache server unless you used a different value for the Apache server’s listener port. If you’re not sure open the /etc/httpd/conf/httpd.conf file and check for the following line (default value shown):

Listen 80

Click the OK button to set the port exception. Then, you can connect to the Linux machine with the IP address, a DNS name, or a name you resolve in your local hosts file, like:

http://192.168.2.1/query.php

You can find the IP address of your Fedora image by inspecting the /etc/hosts file or capture a DHCP assigned address with the following command as the root user (or with sudo as a valid sudoer user):

ifconfig -a

It should return the following image, which is based on the data stored in MySQL’s studentdb database, as qualified in yesterday’s blog post:

ExternalWebPage

I hope this helps those setting up a LAMP instance to work with the MySQL database.

Written by maclochlainn

March 29th, 2015 at 12:35 am