MacLochlainns Weblog

Michael McLaughlin's Technical Blog

Site Admin

Archive for the ‘Python 3.x’ Category

Python Functions

without comments

It seems a number of my students had some confusion over how to write overloaded Python functions. So, I prepared this little tutorial using Python 3.

The first basic1.py example file is a standalone Python file that:

  • Defines a hello() world function.
  • Calls the local hello() world function.
#!/usr/bin/python
 
# Define a hello() function.
def hello():
  print("Hello World!")
 
# Call the hello() function.
hello()

You can test the basic1.py script as follows:

./basic1.py

It prints:

Hello World!

The second basic2.py example file is also a standalone Python file that:

  • Attempts to define overloaded hello() world functions. One version takes no arguments and the other takes one argument.
  • Attempts to call the overloaded local hello() world function without any arguments and with one argument.
#!/usr/bin/python
 
# Call the hello() function without any arguments.
def hello():
  print("Hello World!")
 
# Call the hello() function with one argument.
def hello(whom):
  print("Hello", whom)
 
# Call the overloaded hello() functions.
hello()
hello("Henry")

You can test the basic2.py script as follows:

./basic2.py

It successfully defines the hello() function and then it replaces it with the hello(whom) function. So, it raises the following runtime error because the call to the hello() world function finds the hello(whom) function and the call lacks a call parameter.

Traceback (most recent call last):
  File "/home/student/Code/python/funct/./basic2.py", line 12, in <module>
    hello()
TypeError: hello() missing 1 required positional argument: 'whom'

The third basic3.py example file is also a standalone Python file that:

  • Defines a function that acts like an overloaded hello(whom=None) world function.
  • Call the local hello(whom=None) world function without any arguments and with one argument. It works because you do two things:
    • You assign a default null value to the whom parameter, which makes the parameter optional in the function’s signature.
    • You use an if-statement to manage the behavior of a null parameter. The None keyword defines a null value. Please note that the is reference comparison operator is necessary to evaluate whether a variable contains a null value.
#!/usr/bin/python
 
# Call the hello() function with an optional parameter; and
# manage the inner workings with and without a parameter.
def hello(whom = None):
  if whom is None:
    print("Hello World!")
  else:
    print("Hello", whom + "!")
 
# Call the overloaded hello() functions.
hello()
hello("Henry")

You can test the basic3.py script as follows:

./basic3.py

It prints:

Hello World!
Hello Henry!

At this point, we need to qualify how you can position a Python library file in a development directory. Development directories aren’t typically in the standard library locations, which means you need to define the directories in the $PYTHONPATH environment variable.

There’s a convenient trick that lets you set the $PYTHONPATH value so that you can use it across multiple test environments. It requires you to create an src directory for your library source code inside the directory where you develop code that will use library functions.

After creating the src directory, you can set the $PYTHONPATH environment variable with a relative src directory in the following syntax:

export set PYTHONPATH=$PYTHONPATH:./src:.

It will now let Python look for libraries in the src subdirectory or the present working directory.

Next, you deploy the following hello(whom=None) function in a lib.py file inside the src subdirectory.

# Call the hello() function with an optional parameter; and
# manage the inner workings with and without a parameter.
def hello(whom = None):
  if whom is None:
    print("Hello World!")
  else:
    print("Hello", whom + "!")

In the parent directory of the src subdirectory create the basic4.py file, like:

#!/usr/bin/python
 
# Import the lib.py file as a lib package.
import lib
 
# Call the hello() function without arguments and
# with one argument within the namespace of the lib 
# library.
lib.hello()
lib.hello("Henry")

An alternate way to write the basic4.py program imports a single namespace element (like a variable, function, or object) and places it in the local namespace of the program. You can redefine hello() function

#!/usr/bin/python
 
# Define the hello namespace element from the lib 
# library in the current program.
from lib import hello
 
# Call the hello() function without arguments and
# with one argument within the namespace of the lib
# library.
hello()
hello("Henry")

The hello() function only prints messages. You can add a return statement to return a value from the hello() function. The modified library returns a string rather than printing a string, as follows:

# Call the hello() function with an optional parameter; and
# manage the inner workings with and without a parameter.
def hello(whom = None):
  if whom is None:
    return "Hello World!"
  else:
    return "Hello " + whom + "!"
 
# Call the goodbye() function with an optional parameter; and
# manage the inner workings with and without a parameter.
def goodbye(whom = None):
  if whom is None:
    return "Goodbye World!"
  else:
    return "Goodbye " + whom + "!"

You would then make the following changes to the basic5.py program that calls the lib.py library file. You could also call the goodbye() function inside the imported lib scope. However, you wouldn’t be able to call the goodbye() function if you had imported only the hello() function from the lib package into the local namespace.

#!/usr/bin/python
 
# Import the lib.py file as a lib package.
import lib
 
# Call the hello() function without arguments and
# with one argument within the namespace of the lib
# library.
print(lib.hello())
print(lib.hello("Henry"))

As always, I hope this helps those looking to understand and use functions in Python.

Written by maclochlainn

June 1st, 2023 at 11:08 pm

Posted in Python,Python 3.x

Tagged with

AlmaLinux Flask

with one comment

This post shows how to install and test Flask with Python on AlmaLinux. You install the flask Python libraries with the following commands as the student user. The student user is in the sudoer group.

pip3 install flask_sslify --user student

It produces the following log file:

Display detailed console log →

Collecting flask
  Downloading Flask-2.2.2-py3-none-any.whl (101 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 101.5/101.5 kB 737.3 kB/s eta 0:00:00
Collecting student
  Downloading Student-0.0.1-py3-none-any.whl (2.5 kB)
Collecting itsdangerous>=2.0
  Downloading itsdangerous-2.1.2-py3-none-any.whl (15 kB)
Collecting importlib-metadata>=3.6.0
  Downloading importlib_metadata-5.2.0-py3-none-any.whl (21 kB)
Collecting Jinja2>=3.0
  Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.1/133.1 kB 1.9 MB/s eta 0:00:00
Collecting Werkzeug>=2.2.2
  Downloading Werkzeug-2.2.2-py3-none-any.whl (232 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 232.7/232.7 kB 18.5 MB/s eta 0:00:00
Collecting click>=8.0
  Downloading click-8.1.3-py3-none-any.whl (96 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.6/96.6 kB 16.2 MB/s eta 0:00:00
Collecting zipp>=0.5
  Downloading zipp-3.11.0-py3-none-any.whl (6.6 kB)
Collecting MarkupSafe>=2.0
  Downloading MarkupSafe-2.1.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Installing collected packages: zipp, student, MarkupSafe, itsdangerous, click, Werkzeug, Jinja2, importlib-metadata, flask
Successfully installed Jinja2-3.1.2 MarkupSafe-2.1.1 Werkzeug-2.2.2 click-8.1.3 flask-2.2.2 importlib-metadata-5.2.0 itsdangerous-2.1.2 student-0.0.1 zipp-3.11.0

You can use the following hello.py test program

# Import libraries.
from flask import Flask, escape, request
from markupsafe import escape
 
# Define the application.
app = Flask(__name__)
 
# Define a base URI route and function.
@app.route('/')
def index():
  return "Hello World!"
 
# Define an application URI route and function.
@app.route("/hello")
def hello():
  name = request.args.get("name","Simon")
  return f'Hello {escape(name)}!'
 
# Define an about URI route and function.
@app.route("/about")
def about():
  return "About Page."
 
# Define an <username> variable rule for a route.
@app.route("/user/<string:username>")
def show_user_profile(username):
  return 'User [%s].' % escape(username)
 
# Define an <username> variable rule for a route.
@app.route("/year/<int:year>")
def show_post(year):
  return 'Year [%d].' % year
 
# Run the file.
if __name__ == "__main__":
  app.run()

You can start the Flask server with the following two commands in a separate shell session. This allows you to monitor activities and writes an activity log:

export FLASK_APP=hello.py
flask run

It also writes a compiled version of the hello.py program to the __pycache__ directory. If you make changes to the base file, you must delete the cached version in the __pycache__ directory.

You can test it by typing any of the following URL in a browser:

  1. The index page without a routing label in the URL: 

    http://localhost:5000/hello

    It’ll print the following:

    Hello World!
  2. The index page with a hello routing label in the URL: 

    http://localhost:5000/hello

    It’ll print the following:

    Hello Simon!
  3. The index page with a about routing label in the URL: 

    http://localhost:5000/about

    It’ll print the following:

    About Page.
  4. The index page with a user routing label in the URL: 

    http://localhost:5000/user/Somebody

    It’ll print the following:

    User [Somebody].
  5. The index page with a year routing label in the URL: 

    http://localhost:5000/year/1986

    It’ll print the following:

    Year [1986].

The activity log shows the following:

 * Serving Flask app 'hello.py'
 * Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on http://127.0.0.1:5000
Press CTRL+C to quit
127.0.0.1 - - [29/Dec/2022 19:33:12] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [29/Dec/2022 19:33:21] "GET /hello HTTP/1.1" 200 -
127.0.0.1 - - [29/Dec/2022 19:33:27] "GET /about HTTP/1.1" 200 -
127.0.0.1 - - [29/Dec/2022 19:33:48] "GET /year/1986 HTTP/1.1" 200 -
127.0.0.1 - - [29/Dec/2022 19:34:09] "GET /user/Somebody HTTP/1.1" 200 -

If you stop the process with the Ctrl+Z, the process will not stop but not the listener process. As a sudoer user, you can find the open listener process with the following command:

sudo netstat -nlp | grep 5000
[sudo] password for student:

It will return something like this:

tcp        0      0 127.0.0.1:5000          0.0.0.0:*               LISTEN      143363/python3

You kill the process without prejudice by using the following command:

kill -9 143363

As always, I hope this helps those who are looking for step-by-step instructions.

Written by maclochlainn

December 29th, 2022 at 4:35 pm

Posted in flask,Python,Python 3.x

AlmaLinux Install & Configuration

without comments

This is a collection of blog posts for installing and configuring AlmaLinux with the Oracle, PostgreSQL, MySQL databases and several programming languages. Sample programs show how to connect PHP and Python to the MySQL database.

I used Oracle Database 11g XE in this instance to keep the footprint as small as possible. It required a few tricks and discovering the missing library that caused folks grief eleven years ago. I build another with a current Oracle Database XE after the new year.

If you see something that I missed or you’d like me to add, let me know. As time allows, I’ll try to do that. Naturally, the post will get updates as things are added later.

Written by maclochlainn

December 21st, 2022 at 5:29 pm

Posted in AlmaLinux,Apache,Database,DBA,Java,Java 17,LAMP,Linux,MySQL,MySQL 8,MySQL Connect/J,Oracle,Oracle 11g,Oracle DBA,Oracle Developer,Oracle XE,pdo,Perl,Perl DBI,pgAdmin4,PHP,Postgres,PostgreSQL,PostgreSQL 15,Python 3.x,Rust,VMWare,VSCode

Tagged with , , , , , , , ,

AlmaLinux MySQL+Python

without comments

After installing and configuring MySQL 8.0.30, I installed the Python connector. During that process on AlmaLinux, there were several changes since I last installed the Python’s mysql module. Here are the step-by-step instructions after installing and configuring MySQL Server (blog for those steps).

Using the MySQL Connector/Python X DevAPI Reference, you must install the pip utility before you install the library. You install the pip library as a sudoer user with the following command:

sudo yum install -y pip

Then, using the pip utility as a sudoer user install the mysql-connector-python module with the following command:

sudo pip install mysql-connector-python

Please note that this type of library installation can cause problems when you maintain multiple testing environments. If you plan to maintain multiple testing environments, please install this library in a virtual environment.

You create a bare-bones Python connection test program, like connect.py below:

# Import the library.
import mysql.connector
from mysql.connector import errorcode
 
try:
  # Open connection.
  cnx = mysql.connector.connect(user='student',
                                password='student',
                                host='127.0.0.1',
                                database='studentdb')
 
  # Print the value.
  print("Database connection resolved.")
 
# Handle exception and close connection.
except mysql.connector.Error as e:
  if e.errno == errorcode.ER_ACCESS_DENIED_ERROR:
    print("Something is wrong with your user name or password")
  elif e.errno == errorcode.ER_BAD_DB_ERROR:
    print("Database does not exist")
  else:
    print(e)
 
# Close the connection when the try block completes.
else:
  cnx.close()

You test the connect.py program with this command:

python connect.py

It’ll return the following provided you provisioned the student user and studentdb database:

Database connection resolved.

If you’d like to run it without calling the python executable, you can add the following as the first line:

#/usr/bin/python

and, change the default file permissions from

-rw-rw-r--. 1 student student 717 Oct 30 13:57 connect.py

with the chmod command to

-rwxr-xr-x. 1 student student 717 Oct 30 13:57 connect.py

These instructions should set you up to develop Python code against your AlmaLinux MySQL studentdb database. You use this syntax, assuming a default $PATH environment variable that excludes the present working directory.

./connect.py

As always, I hope this helps those trying to get a complete solution.

Written by maclochlainn

October 30th, 2022 at 12:37 pm

Posted in AlmaLinux,Linux,MySQL,MySQL 8,pip,Python,Python 3.x

Tagged with , , ,

Quick Python Bit #1

without comments

A student asked for a Python example that would find a string starting with a vowel and prepend an “An” or starting with a non-vowel and prepend an “A”. As a rule, this one seems to make sense to most beginning programmers.

#/usr/bin/python
 
# Import the regular expression library.
import re
 
# Create a list of words to check.
haystack = ['anything','something','nothing']
 
# Create a regular expression for a leading vowel.
needle = "^[AEIOUaeiou].*$"
 
# Read through the haystacks and use the right article. 
for i in range(0,len(haystack),1):
  if re.search(needle,haystack[i],re.IGNORECASE):
    print("An",haystack[i]+".")
  else:
    print("A",haystack[i]+".")

However, this loop is much cleaner:

#/usr/bin/python
 
# Import the regular expression library.
import re
 
# Create a list of words to check.
haystack = ['anything','something','nothing']
 
# Create a regular expression for a leading vowel.
needle = "^[AEIOUaeiou].*$"
 
# Read through the haystacks and use the right article. 
for i in haystack:
  if re.search(needle,i,re.IGNORECASE):
    print("An",i+".")
  else:
    print("A",i+".")

Both programs print the following:

An anything.
A something.
A nothing.

As always, happy to help write another example.

Written by maclochlainn

October 26th, 2022 at 11:32 pm

Posted in Python,Python 3.x

Tagged with

MySQL JSON Tricks

without comments

Are they really tricks or simply basic techniques combined to create a solution. Before writing these mechanics for using native MySQL to create a compound JSON object, let me point out that the easiest way to get one is to use the MySQL Node.js library, as shown recently in my “Is SQL Programming” blog post.

Moving data from a relational model output to a JSON structure isn’t as simple as a delimited list of columns in a SQL query. Let’s look at it in stages based on the MySQL Server 12.18.2 Functions that create JSON values.

Here’s how you return single row as a JSON object, which is quite straightforward:

SELECT JSON_OBJECT('first_name',c.first_name,'last_name',c.last_name) AS json_result
FROM   contact c
WHERE  first_name = 'Harry'
AND    last_name = 'Potter';

It returns:

+------------------------------------------------+
| json_result                                         |
+------------------------------------------------+
| {"last_name": "Potter", "first_name": "Harry"} |
+------------------------------------------------+
1 row in set (0.00 sec)

With a GROUP_CONCAT function, let’s capture a JSON array of all three Potter family members:

SELECT CONCAT('['
             , GROUP_CONCAT(
                 JSON_OBJECT('first_name',first_name
                            ,'last_name',last_name ) SEPARATOR ',')
             ,']') AS json_result 
FROM   contact c
WHERE  c.last_name = 'Potter';

It returns an array of JSON objects:

+-----------------------------------------------------------------------------------------------------------------------------------------------+
| [{"last_name": "Potter", "first_name": "Harry"},{"last_name": "Potter", "first_name": "Ginny"},{"last_name": "Potter", "first_name": "Lily"}] |
+-----------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.01 sec)

Next, let’s put a 1:many relationship between the member and contact table into a JSON structure with a single account number and an array of contact. It requires a second call to the JSON_OBJECT function and the addition of a GROUP BY clause in the query.

SELECT   JSON_OBJECT(
            'account_number', account_number
           ,'contact', CONCAT('['
                         , GROUP_CONCAT(
                              JSON_OBJECT('first_name',first_name
                             ,'last_name',last_name ) SEPARATOR ',')
                             ,']')
         ) AS json_result 
FROM     member m INNER JOIN contact c
ON       m.member_id = c.member_id
WHERE    c.last_name = 'Potter'
GROUP BY m.account_number;

It returns the following string with an annoying set of backslashes. It also inverts the column order, which appears unavoidable but it shouldn’t matter because the order of name-value pairs in JSON is immaterial.

+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| {"contact": "[{\"last_name\": \"Potter\", \"first_name\": \"Harry\"},{\"last_name\": \"Potter\", \"first_name\": \"Ginny\"},{\"last_name\": \"Potter\", \"first_name\": \"Lily\"}]", "account_number": "US00011"} |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

The following quick little Python code cleans up the JSON string by removing the backslashes and extraneous quotes around the array of contacts.

# Import the library.
import mysql.connector
from mysql.connector import errorcode
 
try:
  # Open connection.
  cnx = mysql.connector.connect(user='student', password='student',
                                host='127.0.0.1',
                                database='studentdb')
  # Create cursor.
  cursor = cnx.cursor()
 
  # Set the query statement.
  query = ("SELECT JSON_OBJECT( "
           "'account_number', m.account_number "
           ",'contact', CONCAT('[' "
           "              , GROUP_CONCAT( "
           "                   JSON_OBJECT('first_name', c.first_name "
           "                  ,'last_name', c.last_name ) SEPARATOR ',') "
           "                  ,']')) AS json_result "
           "FROM   contact c INNER JOIN member m "
           "ON     c.member_id = m.member_id "
           "WHERE  c.last_name = %s "
           "GROUP BY account_number")
 
  # Execute cursor.
  cursor.execute(query,["Potter"])
 
  # Display the column returned by the query stripped of backslashes and
  # extraneous quotes.
  for (row) in cursor:
    for column in range(len(row)):
      print(row[column].replace("\\","").replace("\"[","[").replace("]\"","]"))
 
  # Close cursor.
  cursor.close()
 
# ------------------------------------------------------------
# Handle exception and close connection.
except mysql.connector.Error as e:
  if e.errno == errorcode.ER_ACCESS_DENIED_ERROR:
    print("Something is wrong with your user name or password")
  elif e.errno == errorcode.ER_BAD_DB_ERROR:
    print("Database does not exist")
  else:
    print("Error code:", e.errno)        # error number
    print("SQLSTATE value:", e.sqlstate) # SQLSTATE value
    print("Error message:", e.msg)       # error message
 
# Close the connection when the try block completes.
else:
  cnx.close()

It returns:

{"contact": [{"last_name": "Potter", "first_name": "Harry"},{"last_name": "Potter", "first_name": "Ginny"},{"last_name": "Potter", "first_name": "Lily"}], "account_number": "US00011"}

I hope this helps exhibit less well known MySQL syntax. Check this post to see how to insert a result set without Python as an intermediary.

Written by maclochlainn

June 23rd, 2022 at 12:46 am

Posted in JSON,MySQL,MySQL 8,Python,Python 3.x,sql

Tagged with , ,

Fedora for macOS ARM64

without comments

I’m always updating VMs, and I was gratified to notice that there’s a Fedora arm64 ISO. If you’re interested in it, you can download the Live Workstation from here or the Fedora Server from here.

Unfortunately, I only have macOS running on i7 and i9 Intel Processors. It would be great to hear back how it goes for somebody one of the new Apple M1 chip.

I typically install the workstation version because it meets my needs to run MySQL and other native Linux development tools. However, the server version is also available. Fedora is a wonderful option, as a small footprint for testing things on my MacBookPro.

Written by maclochlainn

May 7th, 2022 at 1:34 pm

Posted in Fedora,Linux,macOS,MySQL,MySQL 8,Python 3.x

Tagged with , ,

Install Python on Windows

without comments

A number of my students want to run the databases and development environments on Windows rather than Linux. Some of the students have various problems configuring a virtual machine or Docker environment on their laptops.

Installing Python on Windows is quite straightforward. You simply open a Windows Command Line Interface (CLI) with Administrator privileges and type python at the command line. It will launch a dialog that lets you download and install Python.

After the installation, you are returned to the Windows CLI where you can type python to launch the Python IDE. You’ll learn that it’s an older version when it opens.

Python 3.10.2 (tags/v3.10.2:a58ebcc, Jan 17 2022, 14:12:15) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>

After installing Python, I started to install drivers with the Python package-management utility – PIP. I loaded the psycopg2 driver first and discovered that Windows installed an older version of PIP, as qualified when I installed the Python library:

pip install psycopg2
Collecting psycopg2
  Downloading psycopg2-2.9.3-cp310-cp310-win_amd64.whl (1.2 MB)
     |████████████████████████████████| 1.2 MB 1.6 MB/s
Installing collected packages: psycopg2
Successfully installed psycopg2-2.9.3
WARNING: You are using pip version 21.2.4; however, version 22.0.3 is available.
You should consider upgrading via the 'C:\Users\mclaughlinm\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\python.exe -m pip install --upgrade pip' command.

So, I immediately installed the new version of PIP and discovered that I have to manually reset the %PATH% environment variable.

python -m pip install --upgrade pip
Requirement already satisfied: pip in c:\program files\windowsapps\pythonsoftwarefoundation.python.3.10_3.10.752.0_x64__qbz5n2kfra8p0\lib\site-packages (21.2.4)
Collecting pip
  Downloading pip-22.0.3-py3-none-any.whl (2.1 MB)
     |████████████████████████████████| 2.1 MB 656 kB/s
Installing collected packages: pip
  WARNING: The scripts pip.exe, pip3.10.exe and pip3.exe are installed in 'C:\Users\mclaughlinm\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pip-22.0.3

The default location for the PIP.exe and PIP3.exe are found in the:

C:\Users\mclaughlinm\AppData\Local\Microsoft\WindowsApps

Rather than put the newer directory in the System Environment %PATH% variable, I created a batch file that lets me set it only when I need it.

SET PATH=C:\Users\mclaughlinm\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\Scripts;%PATH%

After installing the psycopg2 driver, I tested the new_hire.sql and new_hire.py files from this earlier Linux post successfully. I hope this helps those looking to work with Python on Windows.

Written by maclochlainn

March 2nd, 2022 at 11:48 pm

Posted in PostgreSQL,PostgreSQL 14,Python,Python 3.x,Windows 10

Tagged with

Python on PostgreSQL

without comments

The ODBC library you use when connecting Python to PostgreSQL is the psycopg2 Python library. This blog post will show use how to use it in Python and install it on your Fedora Linux installation. It leverages a videodb database that I show you how to build in this earlier post on configuring PostgreSQL 14.

You would import psycopg2 as follows in your Python code:

import psycopg2

Unfortunately, that only works on Linux servers when you’ve installed the library. That library isn’t installed with generic Python libraries. You get the following error when the psycopg2 library isn’t installed on your server.

Traceback (most recent call last):
  File "python_new_hire.sql", line 1, in <module>
    import psycopg2
ModuleNotFoundError: No module named 'psycopg2'

You can install it on Fedora Linux with the following command:

yum install python3-psycopg2

It will install:

====================================================================================
 Package                  Architecture   Version               Repository      Size
====================================================================================
Installing:
 python3-psycopg2         x86_64         2.7.7-1.fc30          fedora         160 k
 
Transaction Summary
====================================================================================
Install  1 Package
 
Total download size: 160 k
Installed size: 593 k
Is this ok [y/N]: y
Downloading Packages:
python3-psycopg2-2.7.7-1.fc30.x86_64.rpm            364 kB/s | 160 kB     00:00    
------------------------------------------------------------------------------------
Total                                               167 kB/s | 160 kB     00:00     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                            1/1 
  Installing       : python3-psycopg2-2.7.7-1.fc30.x86_64                       1/1 
  Running scriptlet: python3-psycopg2-2.7.7-1.fc30.x86_64                       1/1 
  Verifying        : python3-psycopg2-2.7.7-1.fc30.x86_64                       1/1 
 
Installed:
  python3-psycopg2-2.7.7-1.fc30.x86_64                                              
 
Complete!

Here’s a quick test case that you can run in PostgreSQL and Python to test all the pieces. The first SQL script creates a new_hire table and inserts two rows, and the Python program queries data from the new_hire table.

The new_hire.sql file creates the new_hire table and inserts two rows:

-- Environment settings for the script.
SET SESSION "videodb.table_name" = 'new_hire';
SET CLIENT_MIN_MESSAGES TO ERROR;
 
--  Verify table name.
SELECT current_setting('videodb.table_name');
 
-- ------------------------------------------------------------------
--  Conditionally drop table.
-- ------------------------------------------------------------------
DROP TABLE IF EXISTS new_hire CASCADE;
 
-- ------------------------------------------------------------------
--  Create table.
-- -------------------------------------------------------------------
CREATE TABLE new_hire
( new_hire_id  SERIAL
, first_name   VARCHAR(20)  NOT NULL
, middle_name  VARCHAR(20)
, last_name    VARCHAR(20)  NOT NULL
, hire_date    TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP
, PRIMARY KEY (new_hire_id));
 
-- Alter the sequence by restarting it at 1001.
ALTER SEQUENCE new_hire_new_hire_id_seq RESTART WITH 1001;
 
-- Display the table organization.
SELECT   tc.table_catalog || '.' || tc.constraint_name AS constraint_name
,        tc.table_catalog || '.' || tc.table_name AS table_name
,        kcu.column_name
,        ccu.table_catalog || '.' || ccu.table_name AS foreign_table_name
,        ccu.column_name AS foreign_column_name
FROM     information_schema.table_constraints AS tc JOIN information_schema.key_column_usage AS kcu
ON       tc.constraint_name = kcu.constraint_name
AND      tc.table_schema = kcu.table_schema JOIN information_schema.constraint_column_usage AS ccu
ON       ccu.constraint_name = tc.constraint_name
AND      ccu.table_schema = tc.table_schema
WHERE    tc.constraint_type = 'FOREIGN KEY'
AND      tc.table_name = current_setting('videodb.table_name')
ORDER BY 1;
 
SELECT c1.table_name
,      c1.ordinal_position
,      c1.column_name
,      CASE
         WHEN c1.is_nullable = 'NO' AND c2.column_name IS NOT NULL THEN 'PRIMARY KEY'
         WHEN c1.is_nullable = 'NO' AND c2.column_name IS NULL THEN 'NOT NULL'
       END AS is_nullable
,      CASE
         WHEN data_type = 'character varying' THEN
           data_type||'('||character_maximum_length||')'
         WHEN data_type = 'numeric' THEN
           CASE
             WHEN numeric_scale != 0 AND numeric_scale IS NOT NULL THEN
               data_type||'('||numeric_precision||','||numeric_scale||')'
             ELSE
               data_type||'('||numeric_precision||')'
             END
         ELSE
           data_type
        END AS data_type
FROM    information_schema.columns c1 LEFT JOIN
          (SELECT trim(regexp_matches(column_default,current_setting('videodb.table_name'))::text,'{}')||'_id' column_name
           FROM   information_schema.columns) c2
ON       c1.column_name = c2.column_name
WHERE    c1.table_name = current_setting('videodb.table_name')
ORDER BY c1.ordinal_position;
 
-- Display primary key and unique constraints.
SELECT constraint_name
,      lower(constraint_type) AS constraint_type
FROM   information_schema.table_constraints
WHERE  table_name = current_setting('videodb.table_name')
AND    constraint_type IN ('PRIMARY KEY','UNIQUE');
 
-- Insert two test records.
INSERT INTO new_hire
( first_name, middle_name, last_name, hire_date )
VALUES
 ('Malcolm','Jacob','Lewis','2018-2-14')
,('Henry',null,'Chabot','1990-07-31');

You can put it into a local directory, connect as the student user to a videodb database, and run the following command (or any database you’ve created).

\i new_hire.sql

The new_hire.py file creates the new_hire table and inserts two rows:

# Import the PostgreSQL connector library.
import psycopg2
 
try:
  # Open a connection to the database.
  connection = psycopg2.connect( user="student"
                               , password="student"
                               , port="5432"
                               , dbname="videodb")
 
  # Open a cursor.
  cursor = connection.cursor()
 
  # Assign a static query.
  query = "SELECT new_hire_id, first_name, last_name " \
          "FROM new_hire"
 
  # Parse and execute the query.
  cursor.execute(query)
 
  # Fetch all rows from a table.
  records = cursor.fetchall()
 
  # Read through and print the rows as tuples.
  for row in range(0, len(records)):
    print(records[row]) 
 
except (Exception, psycopg2.Error) as error :
  print("Error while fetching data from PostgreSQL", error)
 
finally:
  # Close the database connection.
  if (connection):
    cursor.close()
    connection.close()

You run it from the command line, like:

python3 ./new_hire.py

It should print:

(1001, 'Malcolm', 'Lewis')
(1002, 'Henry', 'Chabot')

As always, I hope this helps those trying to sort out how to connect Python to PostgreSQL.

Written by maclochlainn

March 2nd, 2022 at 1:06 am

Posted in Database,DBA,DevOps,Fedora,LAMP,Linux,PL/pgSQL,Postgres,PostgreSQL,PostgreSQL 11,psql,Python,Python 3.x

Tagged with ,

PostgreSQL CLI Error

without comments

Problems get reported to me all the time on installations for my students, this one was interesting. They got an error complaining about a missing libpq.so.5 library.

psql: /usr/pgsql-11/lib/libpq.so.5: no version information available (required by psql)
psql: /usr/pgsql-11/lib/libpq.so.5: no version information available (required by psql)
could not change directory to "/root": Permission denied
psql (11.7, server 11.8)
Type "help" for help.
 
postgres=#

It appeared as a mismatch of libraries but it’s not that. For reference, this was a Fedora instance. I ran the rpm utility:

rpm -qa | grep postgres

It returned:

postgresql11-libs-11.8-1PGDG.f30.x86_64
postgresql-11.7-2.fc30.x86_64
postgresql-server-11.7-2.fc30.x86_64

Then, I had them run the rpm utility again looking for the Python driver for PostgreSQL:

rpm -qa | grep psycopg2

It returned:

python3-psycopg2-2.7.7-1.fc30.x86_64

Then, it was easy to explain. The Python psycopg2 library uses both PostgreSQL 11.7 and 11.8 dependent libraries and the libpq.so.5 library is missing version information. You must ignore the error, which is really only a warning message, when you want to work on Fedora, PostgreSQL 11, and Python 3.

Written by maclochlainn

March 2nd, 2022 at 12:41 am

Posted in Database,DBA,LAMP,Linux,PL/pgSQL,Postgres,psql,Python,Python 2.7,Python 3.x,Unix

« Older Entries
Newer Entries »