MacLochlainns Weblog

Michael McLaughlin's Technical Blog

Site Admin

Archive for the ‘Python’ Category

Parametric Queries

without comments

In 2021, I wrote a MySQL example for my class on the usefulness of Common Table Expressions (CTEs). When discussing the original post, I would comment on how you could extend the last example to build a parametric reporting table.

Somebody finally asked for a concrete example. So, this explains how to build a sample MySQL parametric query by leveraging a filter cross join and tests the parameter use with a Python script.

You can build this in any database you prefer but I used a studentdb database with the sakila sample database installed. I’ve granted privileges to both databases to the student user. The following SQL is required for the example:

-- Conditionally drop the levels table.
DROP TABLE IF EXISTS levels;
 
-- Create the levels list.
CREATE TABLE levels
( level_id       int unsigned primary key auto_increment
, parameter_set  enum('Three','Five')
, description    varchar(20)
, min_roles      int
, max_roles      int );
 
-- Insert values into the list table.
INSERT INTO levels
( parameter_set
, description
, min_roles
, max_roles )
VALUES
 ('Three','Hollywood Star', 30, 99999)
,('Three','Prolific Actor', 20, 29)
,('Three','Newcommer',1,19)
,('Five','Newcommer',1,9)
,('Five','Junior Actor',10,19)
,('Five','Professional Actor',20,29)
,('Five','Major Actor',30,39)
,('Five','Hollywood Star',40,99999);

The sample lets you use the three or five value labels while filtering on any partial full_name value as the result of the query below:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
-- Query the data.
WITH actors AS
 (SELECT   a.actor_id
  ,        a.first_name
  ,        a.last_name
  ,        COUNT(*) AS num_roles
  FROM     sakila.actor a INNER JOIN sakila.film_actor fa
  ON       a.actor_id = fa.actor_id
  GROUP BY actor_id)
SELECT   CONCAT(a.last_name,', ',a.first_name) full_name
,        l.description
,        a.num_roles
FROM     actors a CROSS JOIN levels l
WHERE    a.num_roles BETWEEN l.min_roles AND l.max_roles
AND      l.parameter_set = 'Five'
AND      a.last_name LIKE CONCAT('H','%')
ORDER BY a.last_name
,        a.first_name;

They extends a concept exercise found in Chapter 9 on subqueries in Alan Beaulieu’s Learning SQL book.

This is the parametric Python program, which embeds the function locally (to make it easier for those who don’t write a lot of Python). You could set the PYTHONPATH to a relative src directory and import your function if you prefer.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
#!/usr/bin/python
 
# Import the libraries.
import sys
import mysql.connector
from mysql.connector import errorcode
 
# ============================================================
 
# Define function to check and replace arguments.
def check_replace(argv):
 
  # Set defaults for incorrect parameter values.
  defaults = ("Three","_")
 
  # Declare empty list variables.
  inputs = []
  args = ()
 
  # Check whether or not parameters exist after file name.
  if isinstance(argv,list) and len(argv) != 0:
 
    # Check whether there are at least two parameters.
    if len(argv) >= 2:
 
      # Loop through available command-line arguments.
      for element in argv:
 
        # Check first of two parameter values and substitute
        # default value if input value is an invalid option.
        if len(inputs) == 0 and (element in ('Three','Five')) or \
           len(inputs) == 1 and (isinstance(element,str)):
          inputs.append(element)
        elif len(inputs) == 0:
          inputs.append(defaults[0])
        elif len(inputs) == 1:
          inputs.append(defaults[1])
 
      # Assign arguments to parameters.
      args = (inputs)
 
    # Check whether only one parameter value exists.
    elif len(argv) == 1 and (argv[0] in ('Three','Five')):
      args = (argv[0],"_")
 
    # Assume only one parameter is valid and substitute an 
    # empty string as the second parameter.
    else:
      args = (defaults[0],"_")
 
    # Substitute defaults when missing parameters.
  else:
    args = defaults
 
  # Return parameters as a tuple.
  return args
 
# ============================================================
 
# Assign command-line argument list to variable by removing
# the program file name.
# ============================================================
params = check_replace(sys.argv[1:])
# ============================================================
 
#  Attempt the query.
# ============================================================
#  Use a try-catch block to manage the connection.
# ============================================================
try:
  # Open connection.
  cnx = mysql.connector.connect(user='student', password='student',
                                host='127.0.0.1',
                                database='studentdb')
  # Create cursor.
  cursor = cnx.cursor()
 
  # Set the query statement.
  query = ("WITH actors AS "
           "(SELECT   a.first_name "
           " ,        a.last_name "
           " ,        COUNT(*) AS num_roles "
           " FROM     sakila.actor a INNER JOIN sakila.film_actor fa "
           " ON       a.actor_id = fa.actor_id "
           " GROUP BY a.first_name "
           " ,        a.last_name ) "
           " SELECT   CONCAT(a.last_name,', ',a.first_name) AS full_name "
           " ,        l.description "
           " ,        a.num_roles "
           " FROM     actors a CROSS JOIN levels l "
           " WHERE    a.num_roles BETWEEN l.min_roles AND l.max_roles "
           " AND      l.parameter_set = %s "
           " AND      a.last_name LIKE CONCAT(%s,'%') "
           " ORDER BY a.last_name "
           " ,        a.first_name")
 
  # Execute cursor.
  cursor.execute(query, params)
 
  # Display the rows returned by the query.
  for (full_name, description, num_roles) in cursor:
    print('{0} is a {1} with {2} films.'.format( full_name.title()
                                               , description.title()
                                               , num_roles))
 
  # Close cursor.
  cursor.close()
 
# ------------------------------------------------------------
# Handle exception and close connection.
except mysql.connector.Error as e:
  if e.errno == errorcode.ER_ACCESS_DENIED_ERROR:
    print("Something is wrong with your user name or password")
  elif e.errno == errorcode.ER_BAD_DB_ERROR:
    print("Database does not exist")
  else:
    print("Error code:", e.errno)        # error number
    print("SQLSTATE value:", e.sqlstate) # SQLSTATE value
    print("Error message:", e.msg)       # error message
 
# Close the connection when the try block completes.
else:
  cnx.close()

As always, I hope this helps those trying to understand how CTEs can solve problems that would otherwise be coded in external imperative languages like Python.

Written by maclochlainn

March 1st, 2024 at 12:30 am

Sqlite on Ubuntu

without comments

We decided to include some existing Sqlite databases in our AWS Ubuntu learning lab because they’re used by the Data Science courses (specifically, DS 250). Installing Sqlite is quite simple:

sudo apt install -y sqlite

You can check the install by using the which utility, like:

which -a sqlite3

On Ubuntu, it should return:

/usr/bin/sqlite3

There is a friendly help document online that can provide insight in how to use Sqlite. You can create a new student.db database with the following syntax from the Ubuntu CLI (Command-Line Interface):

sqlite3 student.db

It would return the following:

SQLite version 3.37.2 2022-01-06 13:25:41
Enter ".help" FOR usage hints.
sqlite>

A little warning about the simple example and how it opens only a transient in-memory database. If you want a persistent database, you must open sqlite3 without any arguments. Then, you must use the .open method to open a persistent student.db database create a file in the relative directory path where you launched sqlite3 executable. You can read more about persistent in Sqlite in the documentation.

.open student.db

The alternative opens the student.db file in a fully qualified path:

.open /home/student/Code/sqlite/db/student.db

If you type .databases at the sqlite> prompt it would return:

sqlite> .databases
main: /home/student/Code/sqlite/db/student.db r/w

Let’s create a script file that creates two tables, a foreign key reference from one of the tables to the other, and some data with the following create_sample.sql script:

-- Drop knight table if exists.
DROP TABLE IF EXISTS knight;
 
-- Drop kingdom table if exists.
DROP TABLE IF EXISTS kingdom;
 
-- Create normalized table kingdom
CREATE TABLE kingdom
( kingdom_id    INTEGER PRIMARY KEY
, kingdom_name  VARCHAR(20)
, population    INTEGER
, book          VARCHAR(40));
 
-- Insert kingdom into table.
INSERT INTO kingdom
( kingdom_id
, kingdom_name
, population
, book )
VALUES
 ( 1, 'Narnia', 42100, 'Prince Caspian' )
,( 2, 'Narnia', 77600, 'The Lion, The Witch and The Wardrobe' )
,( 3, 'Camelot', 15200, 'The Once and Future King' );
 
-- Create normalized knight table.
CREATE TABLE knight
( knight_id             INTEGER PRIMARY KEY
, knight_name           VARCHAR(22)
, kingdom_allegiance_id INTEGER
, allegiance_start_date text
, allegiance_end_date   text
, book                  VARCHAR(40)
, FOREIGN KEY (kingdom_allegiance_id)
  REFERENCES kingdom(kingdom_id));
 
-- Insert knights into table.  
INSERT INTO knight
( knight_id
, knight_name
, kingdom_allegiance_id
, allegiance_start_date
, allegiance_end_date
, book )
VALUES
 (  1, 'Peter the Magnificent', 2, '1272-03-20', '1292-06-19', 'The Lion, The Witch and The Wardrobe' )
,(  2, 'Edmund the Just', 2, '1272-03-20', '1292-06-19', 'The Lion, The Witch and The Wardrobe' )
,(  3, 'Susan the Gentle', 2, '1272-03-20', '1292-06-19', 'The Lion, The Witch and The Wardrobe' )
,(  4, 'Lucy the Valiant', 2, '1272-03-20', '1292-06-19', 'The Lion, The Witch and The Wardrobe' )
,(  5, 'Peter the Magnificent',	1, '1531-04-12', '1328-05-31', 'Prince Caspian' )
,(  6, 'Edmund the Just', 1, '1531-04-12', '1328-05-31', 'Prince Caspian' )
,(  7, 'Susan the Gentle', 1, '1531-04-12', '1328-05-31', 'Prince Caspian' )
,(  8, 'Lucy the Valiant', 1, '1531-04-12', '1328-05-31', 'Prince Caspian' )
,(  9, 'King Arthur', 3, '0631-03-10', '0686-12-12', 'The Once and Future King' )
,( 10, 'Sir Lionel', 3, '0631-03-10', '0686-12-12', 'The Once and Future King' )
,( 11, 'Sir Bors', 3, '0631-03-10', '0686-12-12', 'The Once and Future King' )
,( 12, 'Sir Bors', 3, '0631-03-10', '0686-12-12', 'The Once and Future King' )
,( 13, 'Sir Galahad', 3, '0631-03-10', '0686-12-12', 'The Once and Future King' )
,( 14, 'Sir Gawain', 3, '0631-03-10', '0686-12-12', 'The Once and Future King' )
,( 15, 'Sir Tristram', 3, '0631-03-10', '0686-12-12', 'The Once and Future King' )
,( 16, 'Sir Percival', 3, '0631-03-10', '0686-12-12', 'The Once and Future King' )
,( 17, 'Sir Lancelot', 3, '0631-03-10', '0686-12-12', 'The Once and Future King' );

You can run the create_sample.sql script with the following syntax using an absolute path:

sqlite> .read /home/student/Code/sqlite/create_tables.sql

Then, you can write a query like this to retrieve the data from two tables:

SELECT k.kingdom_name
,      kn.knight_name
FROM   kingdom k INNER JOIN knight kn
ON     k.kingdom_id = kn.kingdom_allegiance_id
WHERE  k.book = 'Prince Caspian';

It will return the following:

Narnia|Peter the Magnificent
Narnia|Edmund the Just
Narnia|Susan the Gentle
Narnia|Lucy the Valiant

You can exit sqlite3 by entering .quit or Control+D (the system End-Of-File character). If you can determine whether you have a transient or persistent student.db database file with the long list (ll) command.

The following command:

ll /home/student/Code/sqlite/db/student.db

should return the following:

-rw-r--r-- 1 student student 12288 Feb  9 23:01 /home/student/Code/sqlite/db/student.db

If the command returned a 0 sized student.db database file, you created a transient Sqlite table. You’ll need to redo the creation of the student.db database file with the .open command as qualified above.

If you want to detach a database from your active Sqlite session, you can issue the following command to remove it:

sqlite> DETACH DATABASE student.db

Let’s jazz it up a bit with some Python. The first example verifies the ODBC driver’s ability to connect to Sqlite. Please note that it returns the same result for a transient and persistent database file. You can refer to the following documentation for Python examples.

#!/usr/bin/python
 
# Import sqlite3 ODBC library.
import sqlite3
 
try:
  # Open a connection to the student.db database
  db = sqlite3.connect('/home/student/Code/sqlite/db/student.db')
 
  # Print a string to say you've connected to the student.db database.
  print("Sqlite database connection success.")
 
except sqlite3.Error as e:
  print('SQLite error: %s' % (' '.join(e.args)))
  print("Exception class is: ", e.__class__)
  print('SQLite traceback: ')
  exc_type, exc_value, exc_tb = sys.exc_info()
  print(traceback.format_exception(exc_type, exc_value, exc_tb))
  sys.exit(1)
finally:
  # Close the connection when it is open.
  if db:
    db.close()

You can run the sqlite_connection.py script with the following syntax from its local directory:

Sqlite database connection success.

Assuming you have created a persistent Sqlite database, as qualified above with the .open command and fully qualified file name. A fully qualified file name as a path from a Linux mount point to the file.

You must use the fully qualified file name for a persistent Sqlite student.db database as the database parameter for the sqlite3.connect() method, as shown on line #9 of the sqlite_query.py program below.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
#!/usr/bin/python
 
# Import sys library.
import sqlite3
 
try:
  # Open a connection to a persistent database, which should use
  # a fully qualified file name, but may use a relative file
  # name when the Python code is in the same directory as a 
  # persistent student.db sqlite3 database.
  db = sqlite3.connect('/home/student/Code/sqlite/db/student.db')
 
  # Create a cursor.
  cursor = db.cursor()
 
  # Define a query.
  query = "SELECT k.kingdom_name "                           \
          ",      kn.knight_name "                           \
          "FROM   kingdom k INNER JOIN knight kn "           \
          "ON     k.kingdom_id = kn.kingdom_allegiance_id "  \
          "WHERE  k.book = 'Prince Caspian'"
 
  # Execute the cursor with the query.
  cursor.execute( query )
 
  # Display the rows returned by the query.
  for (kingdom_name, knight_name) in cursor:
    print('{0} has {1}'.format( kingdom_name.title(), knight_name.title()))
 
except sqlite3.Error as e:
  print('SQLite error: %s' % (' '.join(e.args)))
  print("Exception class is: ", e.__class__)
  print('SQLite traceback: ')
  exc_type, exc_value, exc_tb = sys.exc_info()
  print(traceback.format_exception(exc_type, exc_value, exc_tb))
  sys.exit(1)
finally:
  # Close the connection when it is open.
  if db:
    db.close()

As always, I hope this helps those trying to get up and running with Sqlite.

Written by maclochlainn

February 9th, 2024 at 8:47 pm

VSCode & $PYTHONPATH

without comments

About 4 years ago, I demonstrated how to develop Python functions with a relative src directory in this old blog post. I thought it might be possible to do with VSCode. Doing a bit of research, it appeared all that was required was adding the PythonPath to VSCode’s Python settings in:

/home/student/.vscode/extensions/ms-python.python-2023.22.0/pythonFiles/.vscode/settings.json

It contained:

{"files.exclude":{"**/__pycache__/**":true,"**/**/*.pyc":true},"python.formatting.provider":"black"}

I added a configuration for the PYTHONPATH, as shown:

{"files.exclude":{"**/__pycache__/**":true,"**/**/*.pyc":true},"python.formatting.provider":"black","python.pythonPath": "/home/student/Lib"}

As you can tell from the embedded VSCode Terminal output below, the PYTHONPATH is not found. You can manually enter it and retest your code successfully. There is no way to use a relative PYTHONPATH like the one you can use from an shell environment file.

This is the hello_whom5.py code:

#!/usr/bin/python
 
# Import the basic sys library.
import sys
from input import parse_input
 
# Assign command-line argument list to variable.
whom = parse_input(sys.argv)
 
# Check if string isn't empty and use dynamic input.  
if len(whom) > 0:
 
  # Print dynamic hello salutation.
  print("Hello " + whom + "!\n")
 
else:
 
  # Print default saluation.
  print("Hello World!")

This is the input.py library module:

# Parse a list and return a whitespace delimited string.
def parse_input(input_list):
 
  # Assign command-line argument list to variable.
  cmd_list = input_list[1:]
 
  # Declare return variable.
  result = ""
 
  # Check whether or not their are parameters beyond the file name.
  if isinstance(input_list,list) and len(input_list) != 0:
 
    # Loop through the command-line argument list and print it. 
    for element in cmd_list:
      if len(result) == 0:
        result = element
      else:
        result = result + " " + element
 
    # Return result variable as string.
    return result

This is the Terminal output from VSCode:

student@student-virtual-machine:~$ /bin/python /home/student/Code/python/hello_whom5.py
Traceback (most recent call last):
  File "/home/student/Code/python/hello_whom5.py", line 5, in <module>
    from input import parse_input
ModuleNotFoundError: No module named 'input'
student@student-virtual-machine:~$ export set PYTHONPATH=/home/student/Lib
student@student-virtual-machine:~$ /bin/python /home/student/Code/python/hello_whom5.py
Hello World!
student@student-virtual-machine:~$ /bin/python /home/student/Code/python/hello_whom5.py Katniss Everdeen
Hello Katniss Everdeen!
 
student@student-virtual-machine:~$

The VSCode image for the test follows below:


As always, I hope this helps somebody working the same issue. However, if somebody has a better solution, please let me know.

Written by maclochlainn

January 14th, 2024 at 11:17 pm

Oracle 23c Free SQL*Plus

without comments

It’s always frustrated me when using the sqlplus command-line interface (CLI) that you can’t just “up arrow” to through the history. At least, that’s the default case unless you wrap the sqlplus executable.

I like to do my development work as close to the database as possible. The delay from SQL Developer to the database or VSCode to the database is just too long. Therefore, I like the native sqlplus to be as efficient as possible. This post shows you how to install the rlwarp utility to wrap sqlplus and create a sandboxed student user for a local development account inside the Oracle 23c Free container. You should note that the Docker or Podman Container is using Oracle Unbreakable Linux 8 as it’s native OS.

You can connect to your Docker version of Oracle Database 23c Free with the following command:

docker exec -it -u root oracle23c bash

You can’t just use dnf to install rlwrap and get it to magically install all the dependencies. That would be too easy, eh?

Attempting to do so will lock your base OS and eventually force you to kill with prejudice the hung dnf process (at least it forced me to do so). You need to determine the rlwrap dependencies and then install them first. In that process, I noticed that the which utility program wasn’t installed in the container.

Naturally, I installed the which utility first with this command:

dnf install -y which

The rlwrap dependencies are: glibc, ncurses, perl, readline, python, and git. Only the perl, python, and git are missing from the list of formal dependencies but there’s another dependency the epel-release package.

If you want to verify whether a package is installed, you can use the rpm command like this:

rpm -qa | grep package_name

I installed the perl programming environment (a big install) with this command:

dnf install -y perl

I installed the python3 with this command:

dnf install -y python3

I installed the git module with this command:

dnf install -y git

I installed the epel-release container with this command:

dnf install -y epel-release

After installing all of these, you’re now ready to install the core rlwrap utility program. Like the other installations, you use:

dnf install -y rlwrap

At this point, you need to create a sandboxed user account for the Docker instance because as a developer using the root user for simple tasks is a bad idea. While you could do this with a Docker command, the Oracle 23c Free edition raised a lock on the /etc/group file when I tried it. Naturally, that’s not a problem because you can connect as the root user with this syntax:

docker exec -it -u root oracle23c bash

As the root user, create a student account as a developer account in the Oracle 23c Free container:

useradd -u 501 -g dba -G users -d /home/student -s /bin/bash/ -c "Student" -n student

You’ll be unable to leverage the tnsnames.ora file unless you alter the prior command to replace dba with oinstall or add the following command:

usermod -a -G oinstall student

Exit the Oracle 23c Free container as the root user and reconnect as the student user with this syntax:

docker exec -it --user student oracle23c bash

While you’re connected as the root user, you should create an upload directory as a subdirectory of the $ORACLE_BASE directory. The $ORACLE_BASE directory in the Oracle Database 23c Free Docker image is the /opt/oracle directory.

You should use the following syntax to create the upload directory and change its permission to that of the Oracle Database 23c Free installation (for a future blog post on developing external table deployment on the Docker image):

mkdir /opt/oracle
chown -R oracle:install /opt/oracle/upload

You also can add the following student function to the Ubuntu student user’s .bashrc file. It means all you need to type to connect to the Oracle Database 23c Free Docker instance is “student“. I like shortcuts like this one, which let you leverage one-line Python commands.

student () 
{
    # Discover the fully qualified program name. 
    path=`which docker 2>/dev/null`
    file=''
 
    # Parse the program name from the path.
    if [ -n ${path} ]; then
        file=${path##/*/}
    fi
 
    # Wrap when there is a file and it is rewrap.
    if [ -n ${file} ] && [[ ${file} = "docker" ]]; then
        python -c "import subprocess; subprocess.run(['docker exec -it --user student oracle23c bash'], shell=True)" 
    else
        echo "Docker is unavailable: Install the docker package."
    fi
}

Open a Ubuntu Terminal shell and type a student function name to connect to the Docker Oracle Database 23c Free instance where you can now test things like external tables with the SQL*Plus command line without installing it on the Ubuntu local operating system.

student@student-virtual-machine:~$ student
[student@d28375f0c43f ~]$ sqlplus c##student/student@free
 
SQL*Plus: Release 23.0.0.0.0 - Production on Wed Jan 3 02:14:22 2024
Version 23.3.0.23.09
 
Copyright (c) 1982, 2023, Oracle.  All rights reserved.
 
Last Successful login time: Wed Jan 03 2024 01:56:44 +00:00
 
Connected to:
Oracle Database 23c Free Release 23.0.0.0.0 - Develop, Learn, and Run for Free
Version 23.3.0.23.09
 
SQL>

Then, I added this sqlplus function to the /home/student/.bashrc file, which is owned by the student user. However, I also added the instruction to change to the student user’s home directory because the Oracle 23c Free container will connect you to the /home/oracle directory by default. I also added the default long list (ll) alias to the .bashrc file.

sqlplus () 
{
    # Discover the fully qualified program name. 
    path=`which rlwrap 2>/dev/null`
    file=''
 
    # Parse the program name from the path.
    if [ -n ${path} ]; then
        file=${path##/*/}
    fi;
 
    # Wrap when there is a file and it is rewrap.
    if [ -n ${file} ] && [[ ${file} = "rlwrap" ]]; then
        rlwrap sqlplus "${@}"
    else
        echo "Command-line history unavailable: Install the rlwrap package."
        $ORACLE_HOME/bin/sqlplus "${@}"
    fi
}
 
# Change to the user's home directory.
cd ${HOME}
 
# Create a long list alias:
alias ll='ls -l --color=auto'

After you’ve configured your student user, you can configure the oracle user account to work like a regular server. Exit the Docker Oracle Database 23c Free as the student user, then connect as the root user with this command:

docker exec -it -u root oracle23c bash

As the root user you can become the oracle user with the following command:

su - oracle

Now, add the following .bashrc shell in the /home/oracle directory:

# The oracle user's .bashrc
 
# Source global definitions
if [ -f /etc/bashrc ]; then
	. /etc/bashrc
fi
 
# User specific environment
if ! [[ "$PATH" =~ "$HOME/.local/bin:$HOME/bin:" ]]
then
    PATH="$HOME/.local/bin:$HOME/bin:$PATH"
fi
export PATH
 
# Uncomment the following line if you don't like systemctl's auto-paging feature:
# export SYSTEMD_PAGER=
 
# User specific aliases and functions
export ORACLE_SID=FREE
export ORACLE_BASE=/opt/oracle
export ORACLE_HOME=/opt/oracle/product/23c/dbhomeFree
export PATH=$PATH:/$ORACLE_HOME/bin
 
# Change to the user's home directory.
cd ${HOME}
 
# Create a long list alias:
alias ll='ls -l --color=auto'
 
sqlplus () 
{
    # Discover the fully qualified program name. 
    path=`which rlwrap 2>/dev/null`
    file=''
 
    # Parse the program name from the path.
    if [ -n ${path} ]; then
        file=${path##/*/}
    fi;
 
    # Wrap when there is a file and it is rewrap.
    if [ -n ${file} ] && [[ ${file} = "rlwrap" ]]; then
        rlwrap sqlplus "${@}"
    else
        echo "Command-line history unavailable: Install the rlwrap package."
        $ORACLE_HOME/bin/sqlplus "${@}"
    fi
}

You need to manually source the .bashrc for the oracle user because it’s not an externally available user. Use this syntax to connect as the internal user:

sqlplus / as sysdba

It’ll display:

SQL*Plus: RELEASE 23.0.0.0.0 - Production ON Wed Jan 3 07:08:11 2024
Version 23.3.0.23.09
 
Copyright (c) 1982, 2023, Oracle.  ALL rights reserved.
 
 
Connected TO:
Oracle DATABASE 23c Free RELEASE 23.0.0.0.0 - Develop, Learn, AND Run FOR Free
Version 23.3.0.23.09
 
SQL>

After all this, I can now click the “up arrow” to edit any of the sqlplus command history. If you like to work inside sqlplus natively, this should help you.

Written by maclochlainn

December 20th, 2023 at 11:11 pm

Python3 on PostgreSQL

without comments

The necessary Python 3 driver for connections to the PostgreSQL database is python3-psycopg2, as qualified by this earlier post with full test examples for Red Hat distributions. You can install it on Ubuntu with the following command:

sudo apt-get install -y python3-psycopg2

As always, I hope this helps those looking for a solution. Also, remember the referenced post above provides Linux distribution neutral full solutions.

Written by maclochlainn

December 3rd, 2023 at 11:00 pm

Python Objects

without comments

I promised to give my students a full example of how to write and execute a Python object. There were two motivations for this post. The first was driven by my students trying to understand the basics and the second my somebody else saying Python couldn’t deliver objects. Hopefully, this code is simple enough for both audiences. I gave them this other tutorial on writing and mimicking overloaded Python functions earlier.

This defines a Ball object type and a FilledBall object subtype of Ball. It incorporates the following elements:

  • A special __init__ function, which is a C/C++ equivalent to a constructor.
  • A special __str__ function represents a class object instance as a string. It is like the getString() equivalent in the Java programming language.
  • A bounce instance function, which means it acts on any instance of the Ball object type or FilledBall object subtype.
  • A get_direction instance function and it calls the __format local object function, which is intended to mimic a private function call, like other object-oriented programming languages.
  • A private name __format function (Private name mangling: When an identifier that textually occurs in a class definition begins with two or more underscore characters and does not end in two or more underscores, it is considered a private name of that class.)

You can test this code by creating the $PYTHONPATH (Unix or Linux) or %PYTHONPATH% (Windows) as follows with all the code inside the present working directory, like this in Unix or Linux:

export set $PYTHONPATH=.

Then, you create the Ball.py file with this syntax:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
# Creates a Ball object type and FilledBall object subtype.
class Ball:
  # User-defined constructor with required parameters.
  def __init__(self, color = None, radius = None, direction = None):
    # Assign a default color value when the parameter is null.
    if color is None:
      self.color = "Blue"
    else:
      self.color = color.lower()
 
    # Assign a default radius value when the parameter is null.
    if radius is None:
      self.radius = 1
    else:
      self.radius = radius
 
    # Assign a default direction value when the parameter is null.
    if direction is None:
      self.direction = "down"
    else:
      self.direction = direction.lower()
 
    # Set direction switch values.
    self.directions = ("down","up")
 
  # User-defined standard function when printing an object type.
  def __str__(self):
    # Build a default descriptive message of the object.
    msg = "It's a " + self.color + " " + str(self.radius) + '"' + " ball"
 
    # Return the message variable.
    return msg
 
  # Define a bounce function.
  def bounce(self, direction = None):
    # Set direction on bounce.
    if not direction is None:
      self.direction = direction
    else:
      # Switch directions.
      if self.directions[0] == self.direction:
        self.direction = self.directions[1]
      elif self.directions[1] == self.direction:
        self.direction = self.directions[0]
 
  # Define a bounce function.
  def getDirection(self):
    # Return current direction of ball.
    return self.__format(self.direction)
 
  # User-defined pseudo-private function, which is available
  # to instances of the Ball object and any of its subtypes.
  def __format(self, msg):
    return "[" + msg + "]"
 
# This is the object subtype, which takes the parent class as an
# argument.
class FilledBall(Ball):
  def __init__(self, filler = None):
    # Instantiate the parent class and then any incremental 
    # parameter values.
    Ball.__init__(self,"Red",2)
 
    # Add a default value or the constructor filler value.
    if filler is None:
      self.filler = "Air".lower()
    else:
      self.filler = filler
 
  # User-defined standard function when printing an object type, which 
  # uses generalized invocation.
  def __str__(self):
    # Build a default descriptive message of the object.
    msg = Ball.__str__(self) + str(" filled with " + self.filler)
 
    # Return the message variable.
    return msg

Next, let’s test instantiating the Ball object type with the following code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
#!/usr/bin/python
 
# Import the Ball class into its own namespace.
import Ball
 
# Assign an instantiated class to a local variable.
myBall = Ball.Ball()
 
# Check whether the local variable holds a valid Ball instance.
if not myBall is None:
  print(myBall, "instance.")
else:
  print("No Ball instance.")
 
# Loop through 10 times changing bounce direction.
for i in range(1,10):
  # Find dirction of ball.
  print(myBall.getDirection())
 
  # Bounce the ball.
  myBall.bounce()

Next, let’s test instantiating the FilledBall object subtype with the following code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
#!/usr/bin/python
 
# Import the Ball class into its own namespace.
import Ball
 
# Assign an instantiated class to a local variable.
myBall = Ball.FilledBall()
 
# Check whether the local variable holds a valid FilledBall instance.
if not myBall is None:
  print(myBall, "instance.")
else:
  print("No Ball instance.")
 
# Loop through 10 times changing bounce direction.
for i in range(1,10):
  # Find dirction of ball.
  print(myBall.getDirection())
 
  # Bounce the ball.
  myBall.bounce()

As always, I hope this helps those looking to learn and extend their knowledge.

Written by maclochlainn

June 25th, 2023 at 2:07 pm

Python Functions

without comments

It seems a number of my students had some confusion over how to write overloaded Python functions. So, I prepared this little tutorial using Python 3.

The first basic1.py example file is a standalone Python file that:

  • Defines a hello() world function.
  • Calls the local hello() world function.
#!/usr/bin/python
 
# Define a hello() function.
def hello():
  print("Hello World!")
 
# Call the hello() function.
hello()

You can test the basic1.py script as follows:

./basic1.py

It prints:

Hello World!

The second basic2.py example file is also a standalone Python file that:

  • Attempts to define overloaded hello() world functions. One version takes no arguments and the other takes one argument.
  • Attempts to call the overloaded local hello() world function without any arguments and with one argument.
#!/usr/bin/python
 
# Call the hello() function without any arguments.
def hello():
  print("Hello World!")
 
# Call the hello() function with one argument.
def hello(whom):
  print("Hello", whom)
 
# Call the overloaded hello() functions.
hello()
hello("Henry")

You can test the basic2.py script as follows:

./basic2.py

It successfully defines the hello() function and then it replaces it with the hello(whom) function. So, it raises the following runtime error because the call to the hello() world function finds the hello(whom) function and the call lacks a call parameter.

Traceback (most recent call last):
  File "/home/student/Code/python/funct/./basic2.py", line 12, in <module>
    hello()
TypeError: hello() missing 1 required positional argument: 'whom'

The third basic3.py example file is also a standalone Python file that:

  • Defines a function that acts like an overloaded hello(whom=None) world function.
  • Call the local hello(whom=None) world function without any arguments and with one argument. It works because you do two things:
    • You assign a default null value to the whom parameter, which makes the parameter optional in the function’s signature.
    • You use an if-statement to manage the behavior of a null parameter. The None keyword defines a null value. Please note that the is reference comparison operator is necessary to evaluate whether a variable contains a null value.
#!/usr/bin/python
 
# Call the hello() function with an optional parameter; and
# manage the inner workings with and without a parameter.
def hello(whom = None):
  if whom is None:
    print("Hello World!")
  else:
    print("Hello", whom + "!")
 
# Call the overloaded hello() functions.
hello()
hello("Henry")

You can test the basic3.py script as follows:

./basic3.py

It prints:

Hello World!
Hello Henry!

At this point, we need to qualify how you can position a Python library file in a development directory. Development directories aren’t typically in the standard library locations, which means you need to define the directories in the $PYTHONPATH environment variable.

There’s a convenient trick that lets you set the $PYTHONPATH value so that you can use it across multiple test environments. It requires you to create an src directory for your library source code inside the directory where you develop code that will use library functions.

After creating the src directory, you can set the $PYTHONPATH environment variable with a relative src directory in the following syntax:

export set PYTHONPATH=$PYTHONPATH:./src:.

It will now let Python look for libraries in the src subdirectory or the present working directory.

Next, you deploy the following hello(whom=None) function in a lib.py file inside the src subdirectory.

# Call the hello() function with an optional parameter; and
# manage the inner workings with and without a parameter.
def hello(whom = None):
  if whom is None:
    print("Hello World!")
  else:
    print("Hello", whom + "!")

In the parent directory of the src subdirectory create the basic4.py file, like:

#!/usr/bin/python
 
# Import the lib.py file as a lib package.
import lib
 
# Call the hello() function without arguments and
# with one argument within the namespace of the lib 
# library.
lib.hello()
lib.hello("Henry")

An alternate way to write the basic4.py program imports a single namespace element (like a variable, function, or object) and places it in the local namespace of the program. You can redefine hello() function

#!/usr/bin/python
 
# Define the hello namespace element from the lib 
# library in the current program.
from lib import hello
 
# Call the hello() function without arguments and
# with one argument within the namespace of the lib
# library.
hello()
hello("Henry")

The hello() function only prints messages. You can add a return statement to return a value from the hello() function. The modified library returns a string rather than printing a string, as follows:

# Call the hello() function with an optional parameter; and
# manage the inner workings with and without a parameter.
def hello(whom = None):
  if whom is None:
    return "Hello World!"
  else:
    return "Hello " + whom + "!"
 
# Call the goodbye() function with an optional parameter; and
# manage the inner workings with and without a parameter.
def goodbye(whom = None):
  if whom is None:
    return "Goodbye World!"
  else:
    return "Goodbye " + whom + "!"

You would then make the following changes to the basic5.py program that calls the lib.py library file. You could also call the goodbye() function inside the imported lib scope. However, you wouldn’t be able to call the goodbye() function if you had imported only the hello() function from the lib package into the local namespace.

#!/usr/bin/python
 
# Import the lib.py file as a lib package.
import lib
 
# Call the hello() function without arguments and
# with one argument within the namespace of the lib
# library.
print(lib.hello())
print(lib.hello("Henry"))

As always, I hope this helps those looking to understand and use functions in Python.

Written by maclochlainn

June 1st, 2023 at 11:08 pm

Posted in Python,Python 3.x

Tagged with

AlmaLinux Flask

with one comment

This post shows how to install and test Flask with Python on AlmaLinux. You install the flask Python libraries with the following commands as the student user. The student user is in the sudoer group.

pip3 install flask_sslify --user student

It produces the following log file:

You can use the following hello.py test program

# Import libraries.
from flask import Flask, escape, request
from markupsafe import escape
 
# Define the application.
app = Flask(__name__)
 
# Define a base URI route and function.
@app.route('/')
def index():
  return "Hello World!"
 
# Define an application URI route and function.
@app.route("/hello")
def hello():
  name = request.args.get("name","Simon")
  return f'Hello {escape(name)}!'
 
# Define an about URI route and function.
@app.route("/about")
def about():
  return "About Page."
 
# Define an <username> variable rule for a route.
@app.route("/user/<string:username>")
def show_user_profile(username):
  return 'User [%s].' % escape(username)
 
# Define an <username> variable rule for a route.
@app.route("/year/<int:year>")
def show_post(year):
  return 'Year [%d].' % year
 
# Run the file.
if __name__ == "__main__":
  app.run()

You can start the Flask server with the following two commands in a separate shell session. This allows you to monitor activities and writes an activity log:

export FLASK_APP=hello.py
flask run

It also writes a compiled version of the hello.py program to the __pycache__ directory. If you make changes to the base file, you must delete the cached version in the __pycache__ directory.

You can test it by typing any of the following URL in a browser:

  1. The index page without a routing label in the URL:

    http://localhost:5000/hello

    It’ll print the following:

    Hello World!
  2. The index page with a hello routing label in the URL:

    http://localhost:5000/hello

    It’ll print the following:

    Hello Simon!
  3. The index page with a about routing label in the URL:

    http://localhost:5000/about

    It’ll print the following:

    About Page.
  4. The index page with a user routing label in the URL:

    http://localhost:5000/user/Somebody

    It’ll print the following:

    User [Somebody].
  5. The index page with a year routing label in the URL:

    http://localhost:5000/year/1986

    It’ll print the following:

    Year [1986].

The activity log shows the following:

 * Serving Flask app 'hello.py'
 * Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on http://127.0.0.1:5000
Press CTRL+C to quit
127.0.0.1 - - [29/Dec/2022 19:33:12] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [29/Dec/2022 19:33:21] "GET /hello HTTP/1.1" 200 -
127.0.0.1 - - [29/Dec/2022 19:33:27] "GET /about HTTP/1.1" 200 -
127.0.0.1 - - [29/Dec/2022 19:33:48] "GET /year/1986 HTTP/1.1" 200 -
127.0.0.1 - - [29/Dec/2022 19:34:09] "GET /user/Somebody HTTP/1.1" 200 -

If you stop the process with the Ctrl+Z, the process will not stop but not the listener process. As a sudoer user, you can find the open listener process with the following command:

sudo netstat -nlp | grep 5000
[sudo] password for student:

It will return something like this:

tcp        0      0 127.0.0.1:5000          0.0.0.0:*               LISTEN      143363/python3

You kill the process without prejudice by using the following command:

kill -9 143363

As always, I hope this helps those who are looking for step-by-step instructions.

Written by maclochlainn

December 29th, 2022 at 4:35 pm

Posted in flask,Python,Python 3.x

AlmaLinux MySQL+Python

without comments

After installing and configuring MySQL 8.0.30, I installed the Python connector. During that process on AlmaLinux, there were several changes since I last installed the Python’s mysql module. Here are the step-by-step instructions after installing and configuring MySQL Server (blog for those steps).

Using the MySQL Connector/Python X DevAPI Reference, you must install the pip utility before you install the library. You install the pip library as a sudoer user with the following command:

sudo yum install -y pip

Then, using the pip utility as a sudoer user install the mysql-connector-python module with the following command:

sudo pip install mysql-connector-python

Please note that this type of library installation can cause problems when you maintain multiple testing environments. If you plan to maintain multiple testing environments, please install this library in a virtual environment.

You create a bare-bones Python connection test program, like connect.py below:

# Import the library.
import mysql.connector
from mysql.connector import errorcode
 
try:
  # Open connection.
  cnx = mysql.connector.connect(user='student',
                                password='student',
                                host='127.0.0.1',
                                database='studentdb')
 
  # Print the value.
  print("Database connection resolved.")
 
# Handle exception and close connection.
except mysql.connector.Error as e:
  if e.errno == errorcode.ER_ACCESS_DENIED_ERROR:
    print("Something is wrong with your user name or password")
  elif e.errno == errorcode.ER_BAD_DB_ERROR:
    print("Database does not exist")
  else:
    print(e)
 
# Close the connection when the try block completes.
else:
  cnx.close()

You test the connect.py program with this command:

python connect.py

It’ll return the following provided you provisioned the student user and studentdb database:

Database connection resolved.

If you’d like to run it without calling the python executable, you can add the following as the first line:

#/usr/bin/python

and, change the default file permissions from

-rw-rw-r--. 1 student student 717 Oct 30 13:57 connect.py

with the chmod command to

-rwxr-xr-x. 1 student student 717 Oct 30 13:57 connect.py

These instructions should set you up to develop Python code against your AlmaLinux MySQL studentdb database. You use this syntax, assuming a default $PATH environment variable that excludes the present working directory.

./connect.py

As always, I hope this helps those trying to get a complete solution.

Written by maclochlainn

October 30th, 2022 at 12:37 pm

Quick Python Bit #1

without comments

A student asked for a Python example that would find a string starting with a vowel and prepend an “An” or starting with a non-vowel and prepend an “A”. As a rule, this one seems to make sense to most beginning programmers.

#/usr/bin/python
 
# Import the regular expression library.
import re
 
# Create a list of words to check.
haystack = ['anything','something','nothing']
 
# Create a regular expression for a leading vowel.
needle = "^[AEIOUaeiou].*$"
 
# Read through the haystacks and use the right article. 
for i in range(0,len(haystack),1):
  if re.search(needle,haystack[i],re.IGNORECASE):
    print("An",haystack[i]+".")
  else:
    print("A",haystack[i]+".")

However, this loop is much cleaner:

#/usr/bin/python
 
# Import the regular expression library.
import re
 
# Create a list of words to check.
haystack = ['anything','something','nothing']
 
# Create a regular expression for a leading vowel.
needle = "^[AEIOUaeiou].*$"
 
# Read through the haystacks and use the right article. 
for i in haystack:
  if re.search(needle,i,re.IGNORECASE):
    print("An",i+".")
  else:
    print("A",i+".")

Both programs print the following:

An anything.
A something.
A nothing.

As always, happy to help write another example.

Written by maclochlainn

October 26th, 2022 at 11:32 pm

Posted in Python,Python 3.x

Tagged with