MacLochlainns Weblog

Michael McLaughlin's Technical Blog

Site Admin

Archive for the ‘Python 3.x’ Category

Using Python’s getopt

without comments

A couple of my students wanted me to write a switch and parameter handler for Python scripts. I wrote it just to show them it’s possible but I also show them how to do it correctly with the Python getopt library, which was soft-deprecated in Python 3.13 and replaced by the Python argparse library. The debate is which one I show you first in the blog.

This is the getops.py script that uses Python’s getopt library. There is a small trick to the options and long options values. You append a colon (:) to the option when it has a value, and append an equal (=) to the long option when it has a value.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
#!/usr/bin/python
 
# Import libraries.
import getopt, sys
import mysql.connector
from mysql.connector import errorcode
 
# Define local function.
def help():
 
  # Declare display string.
  display = \
""" Program Help
+---------------+-------------+-------------------+
| -h  --help    |             | Help switch.      |
| -o  --output  | output_file | Output file name. |
| -q  --query   | query_file  | Query file name.  |
| -v  --verbose |             | Verbose switch.   |
+---------------+-------------+-------------------+"""
 
  # Return string.
  return display
 
# ============================================================
#  Set local variables for switch and parameter placeholders.
# ============================================================
display = False
log = []
output_file = ''
query_file = ''
verbose = False
 
opts = "ho:q:v"
long_opts = ["help","output=","query=","verbose"]
 
# ============================================================
#  Capture argument list minus the program name.
# ============================================================
args = sys.argv[1:]
 
# ============================================================
#  Use a try-except block.
# ============================================================
try:
  # Assign the results of the getopt function.
  params, values = getopt.getopt(args, opts, long_opts)
 
  # Loop through the parameters.
  for curr_param, curr_value in params:
    if curr_param in ("-h","--help"):
      print(help())
    elif curr_param in ("-o","--output"):
      output_file = curr_value
    elif curr_param in ("-q","--query"):
      query_file = curr_value
    elif curr_param in ("-v","--verbose"):
      verbose = True
 
    # Append entry to log.
    log.append('[' + curr_param + '][' + curr_value + ']')
 
  # Print verbose parameter handling.
  if verbose:
    print(" Parameter Diagnostics\n-------------------------")
    for i in log: print(i)
 
# Exception block.
except getopt.GetoptError as e:
  # output error, and return with an error code
  print (str(e))

You can run the program in Linux or Unix with the following syntax provided that you’ve already set the parameters to 755. That means granting the file owner with read, write, and execute privileges, and group and other with read and execute privileges.

./getopts.py -h -o output.txt -q query.sql -v

It would return the following:

 Program Help
+---------------+-------------+-------------------+
| -h  --help    |             | Help switch.      |
| -o  --output  | output_file | Output file name. |
| -q  --query   | query_file  | Query file name.  |
| -v  --verbose |             | Verbose switch.   |
+---------------+-------------+-------------------+
 
 Parameter Diagnostics
-------------------------
[-h][]
[-o][output.txt]
[-q][query.sql]
[-v][]

If you didn’t notice, I also took the opportunity to write the help display in such a way that a maintenance programmer could add another switch or parameter easily. This way the programmer only needs to add a new row of text and add an elif statement with the new switch or parameter.

I think using Python’s getopt library is the cleanest and simplest way to implement switch and parameter handling, after all it’s the basis for so many C derived libraries. However, if you must write your own, below is an approach that would work:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
#!/usr/bin/python
 
# Import libraries.
import sys
import mysql.connector
from mysql.connector import errorcode
 
# ============================================================
#  Set local variables for switch and parameter placeholders.
# ============================================================
help = False
display = \
""" Program Help
+---------------+-------------+-------------------+
| -h  --help    |             | Help switch.      |
| -o  --output  | output_file | Output file name. |
| -q  --query   | query_file  | Query file name.  |
| -v  --verbose |             | Verbose switch.   |
+---------------+-------------+-------------------+"""
 
log = []
output = ''
query = ''
verbose = False
 
# ============================================================
#  Capture argument list minus the program name.
# ============================================================
args = sys.argv[1:]
 
# ============================================================
#  If one or more args exists and the first one is an
#  a string that can cast to an int, convert it to an int,
#  assign it to a variable, and ignore any other args
#  in the list.
# ============================================================
if len(args) > 1 and args[0].isdigit():
  powerIn = int(args[0])
 
# Check for switches and parameters.
if isinstance(args,list) and len(args) >= 1:
 
  # Set the limit of switches and parameters.
  argc = len(args)
 
  # Enumerate through switches first and then parameters.
  for i in range(argc):
    if args[i][0] == '-':
 
      # Evaluate switches and ignore any parameter value.
      if   args[i] in ['-h','--help']:
        help = True
 
        # Append entry to log.
        log.append('[' + str(args[i]) + ']')
 
      elif args[i] in ['-v','--verbose']:
        verbose = True
 
        # Append entry to log.
        log.append('[' + str(args[i]) + ']')
 
      # Evaluate parameters.
      elif i < argc and not args[i+1][0] == '-':
        if   args[i] in ['-q','--query']:
          query = args[i+1]
        elif args[i] in ['-o','--output']:
          output = args[i+1]
 
        # Append entry to log.
        log.append('[' + str(args[i]) + '][' + args[i+1] + ']')
    else:
      continue
    continue
 
  # Print the help display when 
  if help: print(display)
 
  # Print the parameter handling collected in the log variable.
  if verbose: 
    for i in log: print(i)

As you can see from the example, I didn’t give it too much effort. I think it should prove you should use the approach adopted by the general Python community.

Written by maclochlainn

December 1st, 2024 at 9:36 pm

SQL Calculations #1

without comments

This was principally written for my SQL students but I thought it might be useful to others. SQL calculation are performed row-by-row in the SELECT-list. In its simplest form without even touching a table, you can add two literal numbers like this:

1
SELECT 2 + 2 AS result;

It will display the result of the addition to the column alias result as a derived table, or the following result:

+--------+
| result |
+--------+
|      4 |
+--------+
1 row in set (0.00 sec)

Unfortunately, the use of literal values as shown above doesn’t really let you see how the calculation is made row-by-row because it only returns one row. You can rewrite the two literal values into one variable by using a Common Table Expressions (CTEs). The CTE creates an struct tuple with only one x element. Another way to describe what the CTE does would say, it creates a derived table named struct with a single x column in the SELECT-list.

The CTE runs first, then a subsequent query may use the CTE’s derived table results. Below is a query that uses the value in the struct.x derived table (or references the struct tuple’s x element) twice while assigning the value to a new column alias, labelled result. The FROM clause places the struct tuple in the queries namespace, which lets you reference it in the SELECT-list.

1
2
3
4
WITH struct AS
 (SELECT 2 AS x)
SELECT struct.x + struct.x AS result
FROM  struct;

Like the literal example, it will display the result of the addition to the column alias result as a derived table of one row:

+--------+
| result |
+--------+
|      4 |
+--------+
1 row in set (0.00 sec)

Having laid a basis for a simple calculation in one row, let’s expand the example and demonstrate how to perform row-by-row calculations. The example requires introducing some new concepts. One uses the UNION ALL set operator to fabricate a CTE derived table with three rows. Another uses a comma within the WITH clause to create two derived tables or CTEs. The last uses the CROSS JOIN to add the single row CTE’s single y column to each of the rows returned by the multiple row CTE.

The CROSS JOIN is a Cartesian product, which multiplies the rows in one table against the rows in another table while adding the columns from each table. That means fabricating a table of one column and one row lets you put a variable into all the rows of another table or set of tables combined through an equijoin or non-equijoin operation.

The query below takes a struct1 derived table of one column and three rows and a struct2 derived table of one column and one row, then uses a CROSS JOIN to create a new derived table, which would be a table of two columns and three rows. The Cartesian product only provides the two columns that we will multiply to create new data.

The SELECT-list lets us fabricate a new column where we multiply the values of column x and column y to create a set of new results in column result.

1
2
3
4
5
6
7
8
9
10
WITH struct1 AS
 (SELECT 1 AS x UNION ALL
  SELECT 2 AS x UNION ALL
  SELECT 3 AS x)
, struct2 AS
 (SELECT 10 AS y)
SELECT struct1.x AS x
,      struct2.y AS y
,      struct1.x * struct2.y AS result
FROM  struct1 CROSS JOIN struct2;

The query returns the following results, which show the values used to calculate the result and the result:

+---+----+--------+
| x | y  | result |
+---+----+--------+
| 1 | 10 |     10 |
| 2 | 10 |     20 |
| 3 | 10 |     30 |
+---+----+--------+
3 rows in set (0.00 sec)

As a rule, the columns x and y would not be displayed in the final derived table. You would only see the result columns’ values.

Let’s use an example from Alan Bwaulieu’s Learning SQL book with a twist. Rather than manually fabricating the ordinal numbers twice, let’s use the scope reference of a subsequent CTE to reference an earlier CTE. That would create two ten row tables of one column each, or a Cartesian product of a 100 row table with two columns. Then, let’s use the SELECT-list lets us fabricate only a new column, which will add 1 to the numbers 0 to 99 to give us the numbers 1 to a 100.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
WITH ones AS
 (SELECT 0 AS x UNION ALL
  SELECT 1 AS x UNION ALL
  SELECT 2 AS x UNION ALL
  SELECT 3 AS x UNION ALL
  SELECT 4 AS x UNION ALL
  SELECT 5 AS x UNION ALL
  SELECT 6 AS x UNION ALL
  SELECT 7 AS x UNION ALL
  SELECT 8 AS x UNION ALL
  SELECT 9 AS x )
, tens AS
 (SELECT x * 10 AS x FROM ones)
SELECT ones.x + tens.x + 1 AS ordinal
FROM   ones CROSS JOIN tens
ORDER BY ordinal;

It returns the following result set:

+---------+
| ordinal |
+---------+
|       1 |
|       2 |
|       3 |
|       4 |
|       5 |
|       6 |
|       7 |
|       8 |
|       9 |
|      10 |
|      11 |
 ...
|      98 |
|      99 |
|     100 |
+---------+
100 rows in set (0.00 sec)

Moving on to more complex math, let’s create a numerals table with the result from our prior query. It will enable calculating the factors of exponents. The easiest way to create the table is shown below (only caveat is that it will build it with a biting rather than an int data type).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
CREATE TABLE numerals AS
WITH ones AS
 (SELECT 0 AS x UNION ALL
  SELECT 1 AS x UNION ALL
  SELECT 2 AS x UNION ALL
  SELECT 3 AS x UNION ALL
  SELECT 4 AS x UNION ALL
  SELECT 5 AS x UNION ALL
  SELECT 6 AS x UNION ALL
  SELECT 7 AS x UNION ALL
  SELECT 8 AS x UNION ALL
  SELECT 9 AS x )
, tens AS
 (SELECT x * 10 AS x FROM ones)
SELECT ones.x + tens.x + 1 AS ordinal
FROM   ones CROSS JOIN tens
ORDER BY ordinal;

Line #15 sets the column alias that determines the name of the column for the numerals table. It can be described after running the foregoing script in MySQL as:

+---------+--------+------+-----+---------+-------+
| Field   | Type   | Null | Key | Default | Extra |
+---------+--------+------+-----+---------+-------+
| ordinal | bigint | NO   |     | 0       |       |
+---------+--------+------+-----+---------+-------+
1 row in set (0.00 sec)

The next query accepts a substitution variable into the WITH clause, which means an external program will call it. (Although, you could use a session level variable, which I would discourage.) This query returns the factors for any given exponent:

1
2
3
4
5
6
7
8
9
WITH magic AS
(SELECT %s AS vkey)
SELECT   CONCAT(magic.vkey,'^',LOG(magic.vkey,n.ordinal)) AS powers
,        n.ordinal AS result
FROM     numerals n CROSS JOIN magic
WHERE    MOD(n.ordinal,magic.vkey) = 0
AND      LOG(magic.vkey,n.ordinal) REGEXP '^[0-9]*$' 
OR       n.ordinal = 1
ORDER BY n.ordinal;

FYI, in the WHERE clause the regular expression is guarantees only rows returning integer values, and the 1 returns the identity property of an exponent raised to the zero power.

Assuming you created the numerals table, put the foregoing query in a query.sql file (because I was to lazy to write the full parameter handling), and you run it in the same directory as this Python program, it’ll take any valid integer as a value.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
#!/usr/bin/python
# ------------------------------------------------------------
#  Name: power.py
#  Date: 19 Oct 2024
# ------------------------------------------------------------
#  Purpose:
#  -------
#    The program shows you how to provide a single agrument
#    to a query and print the formatted output.
#
#    You can call the program:
#
#    ./power.py 3
#
# ------------------------------------------------------------
 
# Import libraries.
import sys
import mysql.connector
from mysql.connector import errorcode
 
# ============================================================
#  Define a local padding function.
# ============================================================
def pad(valueIn):
  # Define local variable.
  padding = ''
 
  # Convert single digit numbers to strings.
  if isinstance(valueIn,int) and len(str(valueIn)) == 1:
    padding = ' '
 
  # Return padding space.
  return padding
# ============================================================
#  End local function defintion.
# ============================================================
 
# Define any local variables.
powerIn = 2
query = ""
 
# ============================================================
#  Capture argument list minus the program name.
# ============================================================
arguments = sys.argv[1:]
 
# ============================================================
#  If one or more arguments exists and the first one is an
#  a string that can cast to an int, convert it to an int,
#  assign it to a variable, and ignore any other arguments
#  in the list.
# ============================================================
if len(arguments) >= 1 and arguments[0].isdigit():
  powerIn = int(arguments[0])
 
# ============================================================
#  Use a try-catch block to read and parse a query from a
#  a file found in the same local directory as the Python
#  program.
# ============================================================
try:
  file = open('query.sql','r')
  query = file.read().replace('\n',' ').replace(';','')
  file.close()
 
except IOError:
  print("Could not read file:", fileName)
 
# ============================================================
#  Attempt connection in a try-catch block.
# ============================================================
# --------------------------------------------------------
#  Open connection, bind variable in query and format
#  query output before closing the cursor.
# --------------------------------------------------------
try:
  # Open connection.
  cnx = mysql.connector.connect(user='student', password='student',
                                host='127.0.0.1',
                                database='studentdb')
 
  # Create cursor.
  cursor = cnx.cursor()
 
  # Execute cursor, and coerce string to tuple.
  cursor.execute(query, (powerIn,))
 
  # Display the rows returned by the query.
  for (powers, result) in cursor:
    print((" {} is: {}").format(powers, pad(result) + str(result)))
 
  # Close cursor.
  cursor.close()
 
# --------------------------------------------------------
#  Handle MySQL exception 
# --------------------------------------------------------
except mysql.connector.Error as e:
  if e.errno == errorcode.ER_ACCESS_DENIED_ERROR:
    print("Something is wrong with your user name or password")
  elif e.errno == errorcode.ER_BAD_DB_ERROR:
    print("Database does not exist")
  else:
    print("Error code:", e.errno)        # error number
    print("SQLSTATE value:", e.sqlstate) # SQLSTATE value
    print("Error message:", e.msg)       # error message
 
# --------------------------------------------------------
#  Close connection after try-catch completes.
# --------------------------------------------------------
# Close the connection when the try block completes.
else:
  cnx.close()

If you forget to call it with a numeric parameter, it uses 2 as the default. You would call it as follows from a Linux prompt from the local directory:

./power.py

It returns:

 2^0 is:  1
 2^1 is:  2
 2^2 is:  4
 2^3 is:  8
 2^4 is: 16
 2^5 is: 32
 2^6 is: 64

If you call it with a numeric parameter, it uses the numeric value. You would call it as follows from a Linux prompt from the local directory:

./power.py 3

It returns:

 3^0 is:  1
 3^1 is:  3
 3^2 is:  9
 3^3 is: 27
 3^4 is: 81

As always, I hope the post helps folks sort out how and why things work.

Written by maclochlainn

November 17th, 2024 at 6:47 pm

SQL Statement Management

without comments

It’s very difficult explaining to students new to relational databases how SQL works. There are many parts that seem intuitive and then there are others that confuse and confound.

For beginners, the idea that a SQL statement is simply a text string that you must dispatch to a SQL statement processing engine is new. That’s because they use an Integrated Development Environment (IDE) that hides, or abstracts the complexity, of how SQL executes.

I start my core SQL class by demonstrating how to run a text literal query without a FROM clause in MySQL Workbench, like this:

SELECT 'Hello World!' AS "Output";

After writing the query, I highlight everything except the semicolon and click the lightening bolt that dispatches the static string to the SQL statement engine. They see this result:

Then, I launch a mysql Monitor session and write the query with a semicolon to dispatch the SQL static string to the SQL statement engine:

SELECT 'Hello World!' AS "Output";

and, with a \g to dispatch the SQL static string to the SQL statement engine:

SELECT 'Hello World!' AS "Output"\g

Both queries return the same output, as shown below:

+--------------+
| output       |
+--------------+
| Hello World! |
+--------------+
1 row in set (0.00 sec)

Rewriting the query with a \G to dispatch the SQL static string to the SQL statement engine:

SELECT 'Hello World!' AS "Output"\G

Both queries return the following output:

*************************** 1. row ***************************
output: Hello World!
1 row in set (0.00 sec)

The next step requires removing the MySQL Workbench and MySQL Monitor from the demonstration. Without either of those tools, a Python program can demonstrate how to run a static SQL string.

The query is now a string literal into a query.sql file. The Python program reads the query.sql file, dispatches the embedded query, and displays the query results.

This is the query.sql file is:

SELECT 'Hello World!' AS "output";

This is the query.py file is:

#!/usr/bin/python
 
# Import libraries.
import sys
import mysql.connector
from mysql.connector import errorcode
 
# ============================================================
#  Use a try-catch block to read and parse a query from a
#  a file found in the same local directory as the Python
#  program.
# ============================================================
try:
  file = open('query.sql','r')
  query = file.read().replace('\n',' ').replace(';','')
  file.close()
 
except IOError:
  print("Could not read file:", fileName)
 
# ============================================================
#  Attempt connection in a try-catch block.
# ============================================================
# --------------------------------------------------------
#  Open connection, bind variable in query and format
#  query output before closing the cursor.
# --------------------------------------------------------
try:
  # Open connection.
  cnx = mysql.connector.connect(user='student', password='student',
                                host='127.0.0.1',
                                database='studentdb')
 
  # Create cursor.
  cursor = cnx.cursor()
 
  # Execute cursor, and coerce string to tuple.
  cursor.execute(query)
 
  # Display the rows returned by the query.
  for row in cursor:
    print(row[0])
 
  # Close cursor.
  cursor.close()
 
# --------------------------------------------------------
#  Handle MySQL exception 
# --------------------------------------------------------
except mysql.connector.Error as e:
  if e.errno == errorcode.ER_ACCESS_DENIED_ERROR:
    print("Something is wrong with your user name or password")
  elif e.errno == errorcode.ER_BAD_DB_ERROR:
    print("Database does not exist")
  else:
    print("Error code:", e.errno)        # error number
    print("SQLSTATE value:", e.sqlstate) # SQLSTATE value
    print("Error message:", e.msg)       # error message
 
# --------------------------------------------------------
#  Close connection after try-catch completes.
# --------------------------------------------------------
# Close the connection when the try block completes.
else:
  cnx.close()

In Linux or Unix from the relative directory where both the query.sql and query.py files are located:

./query.py

It returns:

Hello World!

These examples demonstrate that a query without variable substitution is only a static string. In all the cases, the static SQL strings are dispatched to the SQL engine by a terminator like a semicolon or through an ODBC library call that executes the static SQL string.

Written by maclochlainn

October 20th, 2024 at 1:38 pm

Parametric Queries

without comments

In 2021, I wrote a MySQL example for my class on the usefulness of Common Table Expressions (CTEs). When discussing the original post, I would comment on how you could extend the last example to build a parametric reporting table.

Somebody finally asked for a concrete example. So, this explains how to build a sample MySQL parametric query by leveraging a filter cross join and tests the parameter use with a Python script.

You can build this in any database you prefer but I used a studentdb database with the sakila sample database installed. I’ve granted privileges to both databases to the student user. The following SQL is required for the example:

-- Conditionally drop the levels table.
DROP TABLE IF EXISTS levels;
 
-- Create the levels list.
CREATE TABLE levels
( level_id       int unsigned primary key auto_increment
, parameter_set  enum('Three','Five')
, description    varchar(20)
, min_roles      int
, max_roles      int );
 
-- Insert values into the list table.
INSERT INTO levels
( parameter_set
, description
, min_roles
, max_roles )
VALUES
 ('Three','Hollywood Star', 30, 99999)
,('Three','Prolific Actor', 20, 29)
,('Three','Newcommer',1,19)
,('Five','Newcommer',1,9)
,('Five','Junior Actor',10,19)
,('Five','Professional Actor',20,29)
,('Five','Major Actor',30,39)
,('Five','Hollywood Star',40,99999);

The sample lets you use the three or five value labels while filtering on any partial full_name value as the result of the query below:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
-- Query the data.
WITH actors AS
 (SELECT   a.actor_id
  ,        a.first_name
  ,        a.last_name
  ,        COUNT(*) AS num_roles
  FROM     sakila.actor a INNER JOIN sakila.film_actor fa
  ON       a.actor_id = fa.actor_id
  GROUP BY actor_id)
SELECT   CONCAT(a.last_name,', ',a.first_name) full_name
,        l.description
,        a.num_roles
FROM     actors a CROSS JOIN levels l
WHERE    a.num_roles BETWEEN l.min_roles AND l.max_roles
AND      l.parameter_set = 'Five'
AND      a.last_name LIKE CONCAT('H','%')
ORDER BY a.last_name
,        a.first_name;

They extends a concept exercise found in Chapter 9 on subqueries in Alan Beaulieu’s Learning SQL book.

This is the parametric Python program, which embeds the function locally (to make it easier for those who don’t write a lot of Python). You could set the PYTHONPATH to a relative src directory and import your function if you prefer.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
#!/usr/bin/python
 
# Import the libraries.
import sys
import mysql.connector
from mysql.connector import errorcode
 
# ============================================================
 
# Define function to check and replace arguments.
def check_replace(argv):
 
  # Set defaults for incorrect parameter values.
  defaults = ("Three","_")
 
  # Declare empty list variables.
  inputs = []
  args = ()
 
  # Check whether or not parameters exist after file name.
  if isinstance(argv,list) and len(argv) != 0:
 
    # Check whether there are at least two parameters.
    if len(argv) >= 2:
 
      # Loop through available command-line arguments.
      for element in argv:
 
        # Check first of two parameter values and substitute
        # default value if input value is an invalid option.
        if len(inputs) == 0 and (element in ('Three','Five')) or \
           len(inputs) == 1 and (isinstance(element,str)):
          inputs.append(element)
        elif len(inputs) == 0:
          inputs.append(defaults[0])
        elif len(inputs) == 1:
          inputs.append(defaults[1])
 
      # Assign arguments to parameters.
      args = (inputs)
 
    # Check whether only one parameter value exists.
    elif len(argv) == 1 and (argv[0] in ('Three','Five')):
      args = (argv[0],"_")
 
    # Assume only one parameter is valid and substitute an 
    # empty string as the second parameter.
    else:
      args = (defaults[0],"_")
 
    # Substitute defaults when missing parameters.
  else:
    args = defaults
 
  # Return parameters as a tuple.
  return args
 
# ============================================================
 
# Assign command-line argument list to variable by removing
# the program file name.
# ============================================================
params = check_replace(sys.argv[1:])
# ============================================================
 
#  Attempt the query.
# ============================================================
#  Use a try-catch block to manage the connection.
# ============================================================
try:
  # Open connection.
  cnx = mysql.connector.connect(user='student', password='student',
                                host='127.0.0.1',
                                database='studentdb')
  # Create cursor.
  cursor = cnx.cursor()
 
  # Set the query statement.
  query = ("WITH actors AS "
           "(SELECT   a.first_name "
           " ,        a.last_name "
           " ,        COUNT(*) AS num_roles "
           " FROM     sakila.actor a INNER JOIN sakila.film_actor fa "
           " ON       a.actor_id = fa.actor_id "
           " GROUP BY a.first_name "
           " ,        a.last_name ) "
           " SELECT   CONCAT(a.last_name,', ',a.first_name) AS full_name "
           " ,        l.description "
           " ,        a.num_roles "
           " FROM     actors a CROSS JOIN levels l "
           " WHERE    a.num_roles BETWEEN l.min_roles AND l.max_roles "
           " AND      l.parameter_set = %s "
           " AND      a.last_name LIKE CONCAT(%s,'%') "
           " ORDER BY a.last_name "
           " ,        a.first_name")
 
  # Execute cursor.
  cursor.execute(query, params)
 
  # Display the rows returned by the query.
  for (full_name, description, num_roles) in cursor:
    print('{0} is a {1} with {2} films.'.format( full_name.title()
                                               , description.title()
                                               , num_roles))
 
  # Close cursor.
  cursor.close()
 
# ------------------------------------------------------------
# Handle exception and close connection.
except mysql.connector.Error as e:
  if e.errno == errorcode.ER_ACCESS_DENIED_ERROR:
    print("Something is wrong with your user name or password")
  elif e.errno == errorcode.ER_BAD_DB_ERROR:
    print("Database does not exist")
  else:
    print("Error code:", e.errno)        # error number
    print("SQLSTATE value:", e.sqlstate) # SQLSTATE value
    print("Error message:", e.msg)       # error message
 
# Close the connection when the try block completes.
else:
  cnx.close()

As always, I hope this helps those trying to understand how CTEs can solve problems that would otherwise be coded in external imperative languages like Python.

Written by maclochlainn

March 1st, 2024 at 12:30 am

Sqlite on Ubuntu

without comments

We decided to include some existing Sqlite databases in our AWS Ubuntu learning lab because they’re used by the Data Science courses (specifically, DS 250). Installing Sqlite is quite simple:

sudo apt install -y sqlite

You can check the install by using the which utility, like:

which -a sqlite3

On Ubuntu, it should return:

/usr/bin/sqlite3

There is a friendly help document online that can provide insight in how to use Sqlite. You can create a new student.db database with the following syntax from the Ubuntu CLI (Command-Line Interface):

sqlite3 student.db

It would return the following:

SQLite version 3.37.2 2022-01-06 13:25:41
Enter ".help" FOR usage hints.
sqlite>

A little warning about the simple example and how it opens only a transient in-memory database. If you want a persistent database, you must open sqlite3 without any arguments. Then, you must use the .open method to open a persistent student.db database create a file in the relative directory path where you launched sqlite3 executable. You can read more about persistent in Sqlite in the documentation.

.open student.db

The alternative opens the student.db file in a fully qualified path:

.open /home/student/Code/sqlite/db/student.db

If you type .databases at the sqlite> prompt it would return:

sqlite> .databases
main: /home/student/Code/sqlite/db/student.db r/w

Let’s create a script file that creates two tables, a foreign key reference from one of the tables to the other, and some data with the following create_sample.sql script:

-- Drop knight table if exists.
DROP TABLE IF EXISTS knight;
 
-- Drop kingdom table if exists.
DROP TABLE IF EXISTS kingdom;
 
-- Create normalized table kingdom
CREATE TABLE kingdom
( kingdom_id    INTEGER PRIMARY KEY
, kingdom_name  VARCHAR(20)
, population    INTEGER
, book          VARCHAR(40));
 
-- Insert kingdom into table.
INSERT INTO kingdom
( kingdom_id
, kingdom_name
, population
, book )
VALUES
 ( 1, 'Narnia', 42100, 'Prince Caspian' )
,( 2, 'Narnia', 77600, 'The Lion, The Witch and The Wardrobe' )
,( 3, 'Camelot', 15200, 'The Once and Future King' );
 
-- Create normalized knight table.
CREATE TABLE knight
( knight_id             INTEGER PRIMARY KEY
, knight_name           VARCHAR(22)
, kingdom_allegiance_id INTEGER
, allegiance_start_date text
, allegiance_end_date   text
, book                  VARCHAR(40)
, FOREIGN KEY (kingdom_allegiance_id)
  REFERENCES kingdom(kingdom_id));
 
-- Insert knights into table.  
INSERT INTO knight
( knight_id
, knight_name
, kingdom_allegiance_id
, allegiance_start_date
, allegiance_end_date
, book )
VALUES
 (  1, 'Peter the Magnificent', 2, '1272-03-20', '1292-06-19', 'The Lion, The Witch and The Wardrobe' )
,(  2, 'Edmund the Just', 2, '1272-03-20', '1292-06-19', 'The Lion, The Witch and The Wardrobe' )
,(  3, 'Susan the Gentle', 2, '1272-03-20', '1292-06-19', 'The Lion, The Witch and The Wardrobe' )
,(  4, 'Lucy the Valiant', 2, '1272-03-20', '1292-06-19', 'The Lion, The Witch and The Wardrobe' )
,(  5, 'Peter the Magnificent',	1, '1531-04-12', '1328-05-31', 'Prince Caspian' )
,(  6, 'Edmund the Just', 1, '1531-04-12', '1328-05-31', 'Prince Caspian' )
,(  7, 'Susan the Gentle', 1, '1531-04-12', '1328-05-31', 'Prince Caspian' )
,(  8, 'Lucy the Valiant', 1, '1531-04-12', '1328-05-31', 'Prince Caspian' )
,(  9, 'King Arthur', 3, '0631-03-10', '0686-12-12', 'The Once and Future King' )
,( 10, 'Sir Lionel', 3, '0631-03-10', '0686-12-12', 'The Once and Future King' )
,( 11, 'Sir Bors', 3, '0631-03-10', '0686-12-12', 'The Once and Future King' )
,( 12, 'Sir Bors', 3, '0631-03-10', '0686-12-12', 'The Once and Future King' )
,( 13, 'Sir Galahad', 3, '0631-03-10', '0686-12-12', 'The Once and Future King' )
,( 14, 'Sir Gawain', 3, '0631-03-10', '0686-12-12', 'The Once and Future King' )
,( 15, 'Sir Tristram', 3, '0631-03-10', '0686-12-12', 'The Once and Future King' )
,( 16, 'Sir Percival', 3, '0631-03-10', '0686-12-12', 'The Once and Future King' )
,( 17, 'Sir Lancelot', 3, '0631-03-10', '0686-12-12', 'The Once and Future King' );

You can run the create_sample.sql script with the following syntax using an absolute path:

sqlite> .read /home/student/Code/sqlite/create_tables.sql

Then, you can write a query like this to retrieve the data from two tables:

SELECT k.kingdom_name
,      kn.knight_name
FROM   kingdom k INNER JOIN knight kn
ON     k.kingdom_id = kn.kingdom_allegiance_id
WHERE  k.book = 'Prince Caspian';

It will return the following:

Narnia|Peter the Magnificent
Narnia|Edmund the Just
Narnia|Susan the Gentle
Narnia|Lucy the Valiant

You can exit sqlite3 by entering .quit or Control+D (the system End-Of-File character). If you can determine whether you have a transient or persistent student.db database file with the long list (ll) command.

The following command:

ll /home/student/Code/sqlite/db/student.db

should return the following:

-rw-r--r-- 1 student student 12288 Feb  9 23:01 /home/student/Code/sqlite/db/student.db

If the command returned a 0 sized student.db database file, you created a transient Sqlite table. You’ll need to redo the creation of the student.db database file with the .open command as qualified above.

If you want to detach a database from your active Sqlite session, you can issue the following command to remove it:

sqlite> DETACH DATABASE student.db

Let’s jazz it up a bit with some Python. The first example verifies the ODBC driver’s ability to connect to Sqlite. Please note that it returns the same result for a transient and persistent database file. You can refer to the following documentation for Python examples.

#!/usr/bin/python
 
# Import sqlite3 ODBC library.
import sqlite3
 
try:
  # Open a connection to the student.db database
  db = sqlite3.connect('/home/student/Code/sqlite/db/student.db')
 
  # Print a string to say you've connected to the student.db database.
  print("Sqlite database connection success.")
 
except sqlite3.Error as e:
  print('SQLite error: %s' % (' '.join(e.args)))
  print("Exception class is: ", e.__class__)
  print('SQLite traceback: ')
  exc_type, exc_value, exc_tb = sys.exc_info()
  print(traceback.format_exception(exc_type, exc_value, exc_tb))
  sys.exit(1)
finally:
  # Close the connection when it is open.
  if db:
    db.close()

You can run the sqlite_connection.py script with the following syntax from its local directory:

Sqlite database connection success.

Assuming you have created a persistent Sqlite database, as qualified above with the .open command and fully qualified file name. A fully qualified file name as a path from a Linux mount point to the file.

You must use the fully qualified file name for a persistent Sqlite student.db database as the database parameter for the sqlite3.connect() method, as shown on line #9 of the sqlite_query.py program below.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
#!/usr/bin/python
 
# Import sys library.
import sqlite3
 
try:
  # Open a connection to a persistent database, which should use
  # a fully qualified file name, but may use a relative file
  # name when the Python code is in the same directory as a 
  # persistent student.db sqlite3 database.
  db = sqlite3.connect('/home/student/Code/sqlite/db/student.db')
 
  # Create a cursor.
  cursor = db.cursor()
 
  # Define a query.
  query = "SELECT k.kingdom_name "                           \
          ",      kn.knight_name "                           \
          "FROM   kingdom k INNER JOIN knight kn "           \
          "ON     k.kingdom_id = kn.kingdom_allegiance_id "  \
          "WHERE  k.book = 'Prince Caspian'"
 
  # Execute the cursor with the query.
  cursor.execute( query )
 
  # Display the rows returned by the query.
  for (kingdom_name, knight_name) in cursor:
    print('{0} has {1}'.format( kingdom_name.title(), knight_name.title()))
 
except sqlite3.Error as e:
  print('SQLite error: %s' % (' '.join(e.args)))
  print("Exception class is: ", e.__class__)
  print('SQLite traceback: ')
  exc_type, exc_value, exc_tb = sys.exc_info()
  print(traceback.format_exception(exc_type, exc_value, exc_tb))
  sys.exit(1)
finally:
  # Close the connection when it is open.
  if db:
    db.close()

As always, I hope this helps those trying to get up and running with Sqlite.

Written by maclochlainn

February 9th, 2024 at 8:47 pm

VSCode & $PYTHONPATH

without comments

About 4 years ago, I demonstrated how to develop Python functions with a relative src directory in this old blog post. I thought it might be possible to do with VSCode. Doing a bit of research, it appeared all that was required was adding the PythonPath to VSCode’s Python settings in:

/home/student/.vscode/extensions/ms-python.python-2023.22.0/pythonFiles/.vscode/settings.json

It contained:

{"files.exclude":{"**/__pycache__/**":true,"**/**/*.pyc":true},"python.formatting.provider":"black"}

I added a configuration for the PYTHONPATH, as shown:

{"files.exclude":{"**/__pycache__/**":true,"**/**/*.pyc":true},"python.formatting.provider":"black","python.pythonPath": "/home/student/Lib"}

As you can tell from the embedded VSCode Terminal output below, the PYTHONPATH is not found. You can manually enter it and retest your code successfully. There is no way to use a relative PYTHONPATH like the one you can use from an shell environment file.

This is the hello_whom5.py code:

#!/usr/bin/python
 
# Import the basic sys library.
import sys
from input import parse_input
 
# Assign command-line argument list to variable.
whom = parse_input(sys.argv)
 
# Check if string isn't empty and use dynamic input.  
if len(whom) > 0:
 
  # Print dynamic hello salutation.
  print("Hello " + whom + "!\n")
 
else:
 
  # Print default saluation.
  print("Hello World!")

This is the input.py library module:

# Parse a list and return a whitespace delimited string.
def parse_input(input_list):
 
  # Assign command-line argument list to variable.
  cmd_list = input_list[1:]
 
  # Declare return variable.
  result = ""
 
  # Check whether or not their are parameters beyond the file name.
  if isinstance(input_list,list) and len(input_list) != 0:
 
    # Loop through the command-line argument list and print it. 
    for element in cmd_list:
      if len(result) == 0:
        result = element
      else:
        result = result + " " + element
 
    # Return result variable as string.
    return result

This is the Terminal output from VSCode:

student@student-virtual-machine:~$ /bin/python /home/student/Code/python/hello_whom5.py
Traceback (most recent call last):
  File "/home/student/Code/python/hello_whom5.py", line 5, in <module>
    from input import parse_input
ModuleNotFoundError: No module named 'input'
student@student-virtual-machine:~$ export set PYTHONPATH=/home/student/Lib
student@student-virtual-machine:~$ /bin/python /home/student/Code/python/hello_whom5.py
Hello World!
student@student-virtual-machine:~$ /bin/python /home/student/Code/python/hello_whom5.py Katniss Everdeen
Hello Katniss Everdeen!
 
student@student-virtual-machine:~$

The VSCode image for the test follows below:


As always, I hope this helps somebody working the same issue. However, if somebody has a better solution, please let me know.

Written by maclochlainn

January 14th, 2024 at 11:17 pm

Oracle 23c Free SQL*Plus

without comments

It’s always frustrated me when using the sqlplus command-line interface (CLI) that you can’t just “up arrow” to through the history. At least, that’s the default case unless you wrap the sqlplus executable.

I like to do my development work as close to the database as possible. The delay from SQL Developer to the database or VSCode to the database is just too long. Therefore, I like the native sqlplus to be as efficient as possible. This post shows you how to install the rlwarp utility to wrap sqlplus and create a sandboxed student user for a local development account inside the Oracle 23c Free container. You should note that the Docker or Podman Container is using Oracle Unbreakable Linux 8 as it’s native OS.

You can connect to your Docker version of Oracle Database 23c Free with the following command:

docker exec -it -u root oracle23c bash

You can’t just use dnf to install rlwrap and get it to magically install all the dependencies. That would be too easy, eh?

Attempting to do so will lock your base OS and eventually force you to kill with prejudice the hung dnf process (at least it forced me to do so). You need to determine the rlwrap dependencies and then install them first. In that process, I noticed that the which utility program wasn’t installed in the container.

Naturally, I installed the which utility first with this command:

dnf install -y which

The rlwrap dependencies are: glibc, ncurses, perl, readline, python, and git. Only the perl, python, and git are missing from the list of formal dependencies but there’s another dependency the epel-release package.

If you want to verify whether a package is installed, you can use the rpm command like this:

rpm -qa | grep package_name

I installed the perl programming environment (a big install) with this command:

dnf install -y perl

I installed the python3 with this command:

dnf install -y python3

I installed the git module with this command:

dnf install -y git

I installed the epel-release container with this command:

dnf install -y epel-release

After installing all of these, you’re now ready to install the core rlwrap utility program. Like the other installations, you use:

dnf install -y rlwrap

At this point, you need to create a sandboxed user account for the Docker instance because as a developer using the root user for simple tasks is a bad idea. While you could do this with a Docker command, the Oracle 23c Free edition raised a lock on the /etc/group file when I tried it. Naturally, that’s not a problem because you can connect as the root user with this syntax:

docker exec -it -u root oracle23c bash

As the root user, create a student account as a developer account in the Oracle 23c Free container:

useradd -u 501 -g dba -G users -d /home/student -s /bin/bash/ -c "Student" -n student

You’ll be unable to leverage the tnsnames.ora file unless you alter the prior command to replace dba with oinstall or add the following command:

usermod -a -G oinstall student

Exit the Oracle 23c Free container as the root user and reconnect as the student user with this syntax:

docker exec -it --user student oracle23c bash

While you’re connected as the root user, you should create an upload directory as a subdirectory of the $ORACLE_BASE directory. The $ORACLE_BASE directory in the Oracle Database 23c Free Docker image is the /opt/oracle directory.

You should use the following syntax to create the upload directory and change its permission to that of the Oracle Database 23c Free installation (for a future blog post on developing external table deployment on the Docker image):

mkdir /opt/oracle
chown -R oracle:install /opt/oracle/upload

You also can add the following student function to the Ubuntu student user’s .bashrc file. It means all you need to type to connect to the Oracle Database 23c Free Docker instance is “student“. I like shortcuts like this one, which let you leverage one-line Python commands.

student () 
{
    # Discover the fully qualified program name. 
    path=`which docker 2>/dev/null`
    file=''
 
    # Parse the program name from the path.
    if [ -n ${path} ]; then
        file=${path##/*/}
    fi
 
    # Wrap when there is a file and it is rewrap.
    if [ -n ${file} ] && [[ ${file} = "docker" ]]; then
        python -c "import subprocess; subprocess.run(['docker exec -it --user student oracle23c bash'], shell=True)" 
    else
        echo "Docker is unavailable: Install the docker package."
    fi
}

Open a Ubuntu Terminal shell and type a student function name to connect to the Docker Oracle Database 23c Free instance where you can now test things like external tables with the SQL*Plus command line without installing it on the Ubuntu local operating system.

student@student-virtual-machine:~$ student
[student@d28375f0c43f ~]$ sqlplus c##student/student@free
 
SQL*Plus: Release 23.0.0.0.0 - Production on Wed Jan 3 02:14:22 2024
Version 23.3.0.23.09
 
Copyright (c) 1982, 2023, Oracle.  All rights reserved.
 
Last Successful login time: Wed Jan 03 2024 01:56:44 +00:00
 
Connected to:
Oracle Database 23c Free Release 23.0.0.0.0 - Develop, Learn, and Run for Free
Version 23.3.0.23.09
 
SQL>

Then, I added this sqlplus function to the /home/student/.bashrc file, which is owned by the student user. However, I also added the instruction to change to the student user’s home directory because the Oracle 23c Free container will connect you to the /home/oracle directory by default. I also added the default long list (ll) alias to the .bashrc file.

sqlplus () 
{
    # Discover the fully qualified program name. 
    path=`which rlwrap 2>/dev/null`
    file=''
 
    # Parse the program name from the path.
    if [ -n ${path} ]; then
        file=${path##/*/}
    fi;
 
    # Wrap when there is a file and it is rewrap.
    if [ -n ${file} ] && [[ ${file} = "rlwrap" ]]; then
        rlwrap sqlplus "${@}"
    else
        echo "Command-line history unavailable: Install the rlwrap package."
        $ORACLE_HOME/bin/sqlplus "${@}"
    fi
}
 
# Change to the user's home directory.
cd ${HOME}
 
# Create a long list alias:
alias ll='ls -l --color=auto'

After you’ve configured your student user, you can configure the oracle user account to work like a regular server. Exit the Docker Oracle Database 23c Free as the student user, then connect as the root user with this command:

docker exec -it -u root oracle23c bash

As the root user you can become the oracle user with the following command:

su - oracle

Now, add the following .bashrc shell in the /home/oracle directory:

# The oracle user's .bashrc
 
# Source global definitions
if [ -f /etc/bashrc ]; then
	. /etc/bashrc
fi
 
# User specific environment
if ! [[ "$PATH" =~ "$HOME/.local/bin:$HOME/bin:" ]]
then
    PATH="$HOME/.local/bin:$HOME/bin:$PATH"
fi
export PATH
 
# Uncomment the following line if you don't like systemctl's auto-paging feature:
# export SYSTEMD_PAGER=
 
# User specific aliases and functions
export ORACLE_SID=FREE
export ORACLE_BASE=/opt/oracle
export ORACLE_HOME=/opt/oracle/product/23c/dbhomeFree
export PATH=$PATH:/$ORACLE_HOME/bin
 
# Change to the user's home directory.
cd ${HOME}
 
# Create a long list alias:
alias ll='ls -l --color=auto'
 
sqlplus () 
{
    # Discover the fully qualified program name. 
    path=`which rlwrap 2>/dev/null`
    file=''
 
    # Parse the program name from the path.
    if [ -n ${path} ]; then
        file=${path##/*/}
    fi;
 
    # Wrap when there is a file and it is rewrap.
    if [ -n ${file} ] && [[ ${file} = "rlwrap" ]]; then
        rlwrap sqlplus "${@}"
    else
        echo "Command-line history unavailable: Install the rlwrap package."
        $ORACLE_HOME/bin/sqlplus "${@}"
    fi
}

You need to manually source the .bashrc for the oracle user because it’s not an externally available user. Use this syntax to connect as the internal user:

sqlplus / as sysdba

It’ll display:

SQL*Plus: RELEASE 23.0.0.0.0 - Production ON Wed Jan 3 07:08:11 2024
Version 23.3.0.23.09
 
Copyright (c) 1982, 2023, Oracle.  ALL rights reserved.
 
 
Connected TO:
Oracle DATABASE 23c Free RELEASE 23.0.0.0.0 - Develop, Learn, AND Run FOR Free
Version 23.3.0.23.09
 
SQL>

After all this, I can now click the “up arrow” to edit any of the sqlplus command history. If you like to work inside sqlplus natively, this should help you.

Written by maclochlainn

December 20th, 2023 at 11:11 pm

OracleDB Python Tutorial 1

without comments

This shows you how to get Python working with the Oracle Database 23c in Docker or Podman on Ubuntu. You can find useful connection strings for this in Oracle Database Free Get Started.

  1. First step requires you to install the pip3/span> utility on Ubuntu.

    sudo apt install -y python3-pip

  2. Second step requires that you pip3 install the oracledb library:

    sudo pip3 install oracledb --upgrade

  3. Third step requires you write a Python program to test your connection to Oracle Database 23c Free, like:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    
    #!/usr/bin/python
     
    # Import the Oracle library.
    import oracledb
     
    try:
      # Create a connection to local Docker or Podman installation.
      db = oracledb.connect(user='c##student', password='student', dsn='localhost:51521/FREE')
     
      # Print a connection message.
      print("Connected to the Oracle", db.version, "database.")
     
    except oracledb.DatabaseError as e:
      error, = e.args
      print(sys.stderr, "Oracle-Error-Code:", error.code)
      print(sys.stderr, "Oracle-Error-Message:", error.message)
     
    finally:
      # Close connection. 
      db.close()

    The 51521 port is the recommended port when setting up Docker or Podman services, however, it can be set to any port above 1024.

    It should print:

    Connected to the Oracle 23.3.0.23.9 database.
  4. Fourth step requires you write a Python program to test querying data from an Oracle Database 23c Free instance. I created the following avenger table and seeded it with six Avengers.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    
    /* Conditionally drop the table. */
    DROP TABLE IF EXISTS avenger;
     
    /* Create the table. */
    CREATE TABLE avenger
    ( avenger_id      NUMBER
    , first_name      VARCHAR2(20)
    , last_name       VARCHAR2(20)
    , character_name  VARCHAR2(20));
     
    /* Seed the table with data. */
    INSERT INTO avenger VALUES (1,'Anthony','Stark','Iron Man');
    INSERT INTO avenger VALUES (2,'Thor','Odinson','God of Thunder');
    INSERT INTO avenger VALUES (3,'Steven','Rogers','Captain America');
    INSERT INTO avenger VALUES (4,'Bruce','Banner','Hulk');
    INSERT INTO avenger VALUES (5,'Clinton','Barton','Hawkeye');
    INSERT INTO avenger VALUES (6,'Natasha','Romanoff','Black Widow');

    Then, I extended the program logic to include a cursor and for loop to read the values from the avenger table:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    
    #!/usr/bin/python
     
    # Import the Oracle library.
    import oracledb
     
    try:
      # Create a connection to local Docker or Podman installation.
      db = oracledb.connect(user='c##student', password='student', dsn='localhost:51521/FREE')
     
      # Create a cursor.
      cursor = db.cursor()
     
      # Execute a query.
      cursor.execute("SELECT   character_name " +
                     ",        first_name " +
                     ",        last_name " +
                     "FROM     avenger " +
                     "ORDER BY character_name")
     
      # Read the contents of the cursor.
      for row in cursor:
        print(row[0] + ':',row[2] + ',',row[1])
     
    except oracledb.DatabaseError as e:
      error, = e.args
      print(sys.stderr, "Oracle-Error-Code:", error.code)
      print(sys.stderr, "Oracle-Error-Message:", error.message)
     
    finally:
      # Close cursor and connection.
      cursor.close() 
      db.close()

    The 51521 port is the recommended port when setting up Docker or Podman services, however, it can be set to any port above 1024.

    It should print:

    Black Widow: Romanoff, Natasha
    Captain America: Rogers, Steven
    God of Thunder: Odinson, Thor
    Hawkeye: Barton, Clinton
    Hulk: Banner, Bruce
    Iron Man: Stark, Anthony
  5. Fifth step requires you write a Python program to test querying data filtered by a local variable from an Oracle Database 23c Free instance. This example looks only for the Hulk among the six Avengers.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    
    #!/usr/bin/python
     
    # Import the Oracle library.
    import oracledb
     
    try:
      # Create a connection to local Docker or Podman installation.
      db = oracledb.connect(user='c##student', password='student', dsn='localhost:51521/FREE')
     
      # Create a cursor.
      cursor = db.cursor()
     
      # Execute a query.
      stmt = "SELECT   character_name "            \
             ",        first_name "                \
             ",        last_name "                 \
             "FROM     avenger "                   \
             "WHERE    character_name = :avenger " \
             "ORDER BY character_name"
     
      # Execute with bind variable.
      cursor.execute(stmt, avenger = "Hulk")
     
      # Read the contents of the cursor.
      for row in cursor:
        print(row[0] + ':',row[2] + ',',row[1])
     
    except oracledb.DatabaseError as e:
      error, = e.args
      print(sys.stderr, "Oracle-Error-Code:", error.code)
      print(sys.stderr, "Oracle-Error-Message:", error.message)
     
    finally:
      # Close cursor and connection. 
      cursor.close() 
      db.close()

    It should print:

    Hulk: Banner, Bruce

As always, I hope this puts everything together for setting up Python with Oracle Database 23c Free.

Written by maclochlainn

December 10th, 2023 at 12:27 am

Python3 on PostgreSQL

without comments

The necessary Python 3 driver for connections to the PostgreSQL database is python3-psycopg2, as qualified by this earlier post with full test examples for Red Hat distributions. You can install it on Ubuntu with the following command:

sudo apt-get install -y python3-psycopg2

As always, I hope this helps those looking for a solution. Also, remember the referenced post above provides Linux distribution neutral full solutions.

Written by maclochlainn

December 3rd, 2023 at 11:00 pm

MySQL on Ubuntu

without comments

Working with my students to create an Ubuntu virtual environment for Python development with the MySQL database. After completing the general provisioning covered in this older post, I’d recommend you create a python symbolic link before installing the MySQL-Python driver.

sudo ln -s /usr/bin/python3 /usr/bin/python

You install the Python development driver with the following:

sudo apt-get -y install python3-mysql.connector

Create a python_connect.py file to test your Python deployment’s ability to connect to the MySQL database:

#!/usr/bin/python
 
# Import the library.
import mysql.connector
from mysql.connector import errorcode
 
try:
  # Open connection.
  cnx = mysql.connector.connect(user='student', password='student',
                                host='localhost',
                                database='sakila')
 
  # Print the value.
  print("Database connection resolved.")
 
# Handle exception and close connection.
except mysql.connector.Error as e:
  if e.errno == errorcode.ER_ACCESS_DENIED_ERROR:
    print("Something is wrong with your user name or password")
  elif e.errno == errorcode.ER_BAD_DB_ERROR:
    print("Database does not exist")
  else:
    print(e)
 
# Close the connection when the try block completes.
else:
  cnx.close()

You should change the file permissions of the python_connect.py to read-write-execute as owner, and read-execute as group and other with the following command:

chmod 755 python_connect.py

Then, you can test your python_connect.py program from the local directory with the following command:

./python_connect.py

You see the following output if it works:

Database connection resolved.

As alway, I hope those looking for a code complete solutionk

Written by maclochlainn

July 14th, 2023 at 2:29 pm