Archive for the ‘Python 2.7’ Category
PostgreSQL CLI Error
Problems get reported to me all the time on installations for my students, this one was interesting. They got an error complaining about a missing libpq.so.5 library.
psql: /usr/pgsql-11/lib/libpq.so.5: no version information available (required by psql) psql: /usr/pgsql-11/lib/libpq.so.5: no version information available (required by psql) could not change directory to "/root": Permission denied psql (11.7, server 11.8) Type "help" for help. postgres=# |
It appeared as a mismatch of libraries but it’s not that. For reference, this was a Fedora instance. I ran the rpm utility:
rpm -qa | grep postgres |
It returned:
postgresql11-libs-11.8-1PGDG.f30.x86_64 postgresql-11.7-2.fc30.x86_64 postgresql-server-11.7-2.fc30.x86_64 |
Then, I had them run the rpm utility again looking for the Python driver for PostgreSQL:
rpm -qa | grep psycopg2 |
It returned:
python3-psycopg2-2.7.7-1.fc30.x86_64 |
Then, it was easy to explain. The Python psycopg2 library uses both PostgreSQL 11.7 and 11.8 dependent libraries and the libpq.so.5 library is missing version information. You must ignore the error, which is really only a warning message, when you want to work on Fedora, PostgreSQL 11, and Python 3.
Wrap Oracle’s tnsping
If you’ve worked with the Oracle database a while, you probably noticed that some utilities write to stdout for both standard output and what should be standard error (stderr). One of those commands is the tnsping utility.
You can wrap the tnsping command to send the TNS-03505 error to stdout with the following code. I put Bash functions like these in a library.sh script, which I can source when automating tasks.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | #!/usr/bin/bash tnsping() { if [ ! -z ${1} ]; then # Set default return value. stdout=`$ORACLE_HOME/bin/tnsping ${1} | tail -1` # Check stdout to return 0 for success and 1 for failure. if [[ `echo ${stdout} | cut -c1-9` = 'TNS-03505' ]]; then python -c 'import os, sys; arg = sys.argv[1]; os.write(2,arg + "\n")' "${stdout}" else echo "${1}" fi fi } |
You should notice that the script uses a Python call to redirect the error message to standard out (stdout) but you can redirect in Bash shell with the following:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | #!/usr/bin/bash tnsping() { if [ ! -z ${1} ]; then # Set default return value. stdout=`$ORACLE_HOME/bin/tnsping ${1} | tail -1` # Check stdout to return 0 for success and 1 for failure. if [[ `echo ${stdout} | cut -c1-9` = 'TNS-03505' ]]; then echo ${stdout} 1>&2 else echo "${1}" fi fi } |
Interactively, we can now test a non-existent service name like wrong with this syntax:
tnsping wrong |
It’ll print the standard error to console, like:
TNS-03505: Failed to resolve name |
or, you can suppress standard error (stderr) by redirecting it to the traditional black hole, like:
tnsping wrong 2>/dev/null |
After redirecting standard error (stderr), you simply receive nothing back. That lets you evaluate in another script whether or not the utility raises an error.
In an automating Bash shell script, you use the source command to put the Bash function in scope, like this:
source library.sh |
As always, I hope this helps those looking for a solution.
Developing Python Libraries
I put this together to show my students how to simplify writing and testing Python library files. The trick requires that you learn how to set a relative $PYTHONPATH environment file.
export set PYTHONPATH=./lib |
After setting the $PYTHONPATH environment variable, connect to Python’s IDLE environment and run the following code:
import os print(os.environ['PYTHONPATH']) |
It prints the following:
./lib |
You can also discover all the standard libraries and your $PYTHONPATH value in your environment with the following command:
for i in sys.path: print(i) |
It prints the following, which lists the one set by the $PYTHONPATH first:
/home/student/Code/python/path/lib /usr/lib64/python37.zip /usr/lib64/python3.7 /usr/lib64/python3.7/lib-dynload /home/student/.local/lib/python3.7/site-packages /usr/lib64/python3.7/site-packages /usr/lib/python3.7/site-packages |
You create a test my_module.py library file in the relative ./lib directory, like the following:
# Define a hello function that accept a name and prints a salutation. def hello(whom): return "Hello " + whom + "!" |
Next, you can create a testlib.py program:
# Import the hello function into the local namesapce from the my_module. from my_module import hello # Call the module hello, which returns a formatted string. print(hello("Suzie Q")) |
It imports the hello(whom) function into the local namespace and then calls the hello(whom) function with the string literal "Susie". It prints:
Hello Suzie Q! |
If you import the my_module module, you must refer to the hello(whom) function by prefacing it with my_module., like the following example:
# Import the hello function into the local namesapce from the my_module. import my_module # Call the module hello, which returns a formatted string. print(my_module.hello("Suzie Q")) |
A direct import doesn’t add the method to the local namespace. It remains in the my_module‘s namespace.
It’s probably important to note where my_module.pyc files are written for the those migrating from Python 2.7 to Python 3. In Python 2.7 they would be written to the ./lib directory, but in Python 3 they’re written to the ./lib/__pycache__ directory.
As always, I hope this helps those who find it and read it.
pgAdmin4 on Fedora 30
While attempting an install of pgAdmin and updating a Fedora 30 environment, I encountered a conflict on the upgrade of MySQL 8.0.17-1 to 8.0.17.2. The community-mysql-8.0.17-2.fc30.x86_64 had conflicts with:
mysql-community-client-8.0.17-1.fc30.x86_64packagemysql-community-server-8.0.17-1.fc30.x86_64package
I tried to update the system before install pgadmin4 with the following syntax:
dnf -y update && dnf -y install pgadmin4 |
The dnf utility raise the following MySQL package errors during transaction checking:
Display detailed console log →
Error: Transaction check error: file /usr/bin/mysql conflicts between attempted installs of community-mysql-8.0.17-2.fc30.x86_64 and mysql-community-client-8.0.17-1.fc30.x86_64 file /usr/bin/mysql_config_editor conflicts between attempted installs of community-mysql-8.0.17-2.fc30.x86_64 and mysql-community-client-8.0.17-1.fc30.x86_64 file /usr/bin/mysqladmin conflicts between attempted installs of community-mysql-8.0.17-2.fc30.x86_64 and mysql-community-client-8.0.17-1.fc30.x86_64 file /usr/bin/mysqlbinlog conflicts between attempted installs of community-mysql-8.0.17-2.fc30.x86_64 and mysql-community-client-8.0.17-1.fc30.x86_64 file /usr/bin/mysqlcheck conflicts between attempted installs of community-mysql-8.0.17-2.fc30.x86_64 and mysql-community-client-8.0.17-1.fc30.x86_64 file /usr/bin/mysqldump conflicts between attempted installs of community-mysql-8.0.17-2.fc30.x86_64 and mysql-community-client-8.0.17-1.fc30.x86_64 file /usr/bin/mysqlimport conflicts between attempted installs of community-mysql-8.0.17-2.fc30.x86_64 and mysql-community-client-8.0.17-1.fc30.x86_64 file /usr/bin/mysqlpump conflicts between attempted installs of community-mysql-8.0.17-2.fc30.x86_64 and mysql-community-client-8.0.17-1.fc30.x86_64 file /usr/bin/mysqlshow conflicts between attempted installs of community-mysql-8.0.17-2.fc30.x86_64 and mysql-community-client-8.0.17-1.fc30.x86_64 file /usr/bin/mysqlslap conflicts between attempted installs of community-mysql-8.0.17-2.fc30.x86_64 and mysql-community-client-8.0.17-1.fc30.x86_64 file /usr/bin/ibd2sdi conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/bin/innochecksum conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/bin/my_print_defaults conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/bin/myisam_ftdump conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/bin/myisamchk conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/bin/myisamlog conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/bin/myisampack conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/bin/mysql_secure_installation conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/bin/mysql_ssl_rsa_setup conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/bin/mysql_tzinfo_to_sql conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/bin/mysql_upgrade conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/bin/mysqld_pre_systemd conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/bin/perror conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib/systemd/system/mysqld.service conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib/systemd/system/mysqld@.service conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib64/mysql/plugin/adt_null.so conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib64/mysql/plugin/auth_socket.so conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib64/mysql/plugin/component_audit_api_message_emit.so conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib64/mysql/plugin/component_log_filter_dragnet.so conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib64/mysql/plugin/component_log_sink_json.so conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib64/mysql/plugin/component_log_sink_syseventlog.so conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib64/mysql/plugin/component_validate_password.so conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib64/mysql/plugin/connection_control.so conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib64/mysql/plugin/ddl_rewriter.so conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib64/mysql/plugin/group_replication.so conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib64/mysql/plugin/ha_example.so conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib64/mysql/plugin/ha_mock.so conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib64/mysql/plugin/innodb_engine.so conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib64/mysql/plugin/keyring_file.so conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib64/mysql/plugin/keyring_udf.so conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib64/mysql/plugin/libmemcached.so conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib64/mysql/plugin/locking_service.so conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib64/mysql/plugin/mypluglib.so conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib64/mysql/plugin/mysql_clone.so conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib64/mysql/plugin/mysql_no_login.so conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib64/mysql/plugin/rewrite_example.so conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib64/mysql/plugin/rewriter.so conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib64/mysql/plugin/semisync_master.so conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib64/mysql/plugin/semisync_slave.so conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib64/mysql/plugin/validate_password.so conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib64/mysql/plugin/version_token.so conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/sbin/mysqld conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /var/lib/mysql conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /var/lib/mysql-keyring conflicts between attempted installs of community-mysql-server-8.0.17-2.fc30.x86_64 and mysql-community-server-8.0.17-1.fc30.x86_64 file /usr/lib64/mysql/libmysqlclient.so.21.1.17 conflicts between attempted installs of community-mysql-libs-8.0.17-2.fc30.x86_64 and mysql-community-libs-8.0.17-1.fc30.x86_64 |
Since I’m not sure what’s wrong or how to fix it, I’ve put it in my queue of things to get to later. However, when I figure it out I’ll update this blog page with the solution or work around. If anybody knows the fix and would like to share, please let me know.
I removed the pending update packages with the following command:
dnf clean packages |
Then, I simply installed pgadmin4 with the following command:
dnf -y install pgadmin4 |
Display detailed console log →
The pgadmin4 configuration instructions can be found for several Linux versions at Josphat Mutai’s Computing for Geeks web page. On Fedora 30, you need to do the following:
- Install, start, and enable Apache as the
httpdservice unless you already have done that. - Copy the
/etc/httpd/conf.d/pgadmin4.conf.samplefile to/etc/httpd/conf.d/pgadmin4.conf, which is a new file. - Restart the
httpdservice to incorporate thepgadmin4configuration file.
After that, you create the following new directories as the root or sudo user:
/var/lib/pgadmin4/var/log/pgadmin4
You can make both directories with a single mkdir command, like:
mkdir -p /var/lib/pgadmin4 /var/log/pgadmin4 |
As the root or sudo user, change the ownership of these two directories to the apache user with the following syntax:
chown -R apache:apache /var/lib/pgadmin4 /var/log/pgadmin4 |
You add the following four statements to the config_distro.py file in the /usr/lib/python3.7/site-packages/pgadmin4-web directory as the root or sudo user:
LOG_FILE = '/var/log/pgadmin4/pgadmin4.log' SQLITE_PATH = '/var/lib/pgadmin4/pgadmin4.db' SESSION_DB_PATH = '/var/lib/pgadmin4/sessions' STORAGE_DIR = '/var/lib/pgadmin4/storage' |
You need to setup the pgadmin user with the following python3 command:
python3 /usr/lib/python3.7/site-packages/pgadmin4-web/setup.py |
Enter the following values, a real email address and a password twice:
NOTE: Configuring authentication for SERVER mode. Enter the email address and password to use for the initial pgAdmin user account: Email address: admin@example.com Password: your_password Retype password: your_password pgAdmin 4 - Application Initialisation ====================================== |
Assuming you have an enabled firewall, you need to issue the following two commands as the root or sudo user:
rirewall-cmd --permanent --add-service=http firewall-cmd --reload |
You invoke pgAdmin4 from within a browser window with the following URL for a stand alone workstation (for a workstation on a DNS network you would enter pgadmin.domain.domain_type in lieu of localhost):
pgadmin/localhost/pgadmin4 |
You most likely will encounter an Internal Server Error, the recommended fix is reputed to be:
ausearch -c 'httpd' --raw | audit2allow -M my-httpd semodule -X 300 -i my-httpd.pp |
It didn’t work for me. At the end of the process, I have an Internal Server Error. It is something that I’ll try to fix next. The actual error message:
Internal Server Error The server encountered an internal error or misconfiguration and was unable to complete your request. Please contact the server administrator at root@localhost to inform them of the time this error occurred, and the actions you performed just before this error. More information about this error may be available in the server error log. |
If somebody figures out the last step before I do, that’s great. Let me and everybody else know the mystery.
On a positive note, the pgadmin4 package provided the psycopg2 library. I had looked for it as a psycopg2 package but it is in python3-psycopg2 package.
MySQL Python Connector
While building my student image on Fedora 30, I installed the MySQL PHP Connector (php-mysqlndrp) but neglected to install the Python Connector. This adds the installation and basic test of the Python Connector to the original blog post.
You use the following command with a wildcard as a privileged user. The wildcard is necessary because you need to load two libraries to support Python 2.7 and 3.7, which are installed on Fedora 30. You also need to be the root user or a user that is found in the sudoer’s list:
yum install -y mysql-connector-python* |
Display detailed console log →
Last metadata expiration check: 0:35:46 ago on Tue 20 Aug 2019 05:36:29 PM MDT. Dependencies resolved. ===================================================================================================================================== Package Architecture Version Repository Size ===================================================================================================================================== Installing: mysql-connector-python x86_64 8.0.17-1.fc30 mysql-connectors-community 435 k mysql-connector-python-cext x86_64 8.0.17-1.fc30 mysql-connectors-community 7.7 M mysql-connector-python3 x86_64 8.0.17-1.fc30 mysql-connectors-community 429 k mysql-connector-python3-cext x86_64 8.0.17-1.fc30 mysql-connectors-community 7.7 M Installing dependencies: python2-protobuf noarch 3.6.1-3.fc30 fedora 563 k python3-protobuf noarch 3.6.1-3.fc30 fedora 568 k Transaction Summary ===================================================================================================================================== Install 6 Packages Total download size: 17 M Installed size: 89 M Downloading Packages: (1/6): python3-protobuf-3.6.1-3.fc30.noarch.rpm 1.0 MB/s | 568 kB 00:00 (2/6): python2-protobuf-3.6.1-3.fc30.noarch.rpm 994 kB/s | 563 kB 00:00 (3/6): mysql-connector-python-8.0.17-1.fc30.x86_64.rpm 481 kB/s | 435 kB 00:00 (4/6): mysql-connector-python3-8.0.17-1.fc30.x86_64.rpm 612 kB/s | 429 kB 00:00 (5/6): mysql-connector-python-cext-8.0.17-1.fc30.x86_64.rpm 3.8 MB/s | 7.7 MB 00:02 (6/6): mysql-connector-python3-cext-8.0.17-1.fc30.x86_64.rpm 4.2 MB/s | 7.7 MB 00:01 ------------------------------------------------------------------------------------------------------------------------------------- Total 5.4 MB/s | 17 MB 00:03 warning: /var/cache/dnf/mysql-connectors-community-8bcc2bd350b53f70/packages/mysql-connector-python-8.0.17-1.fc30.x86_64.rpm: Header V3 DSA/SHA1 Signature, key ID 5072e1f5: NOKEY MySQL Connectors Community 7.0 MB/s | 27 kB 00:00 Importing GPG key 0x5072E1F5: Userid : "MySQL Release Engineering <mysql-build@oss.oracle.com>" Fingerprint: A4A9 4068 76FC BD3C 4567 70C8 8C71 8D3B 5072 E1F5 From : /etc/pki/rpm-gpg/RPM-GPG-KEY-mysql Key imported successfully Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Installing : python3-protobuf-3.6.1-3.fc30.noarch 1/6 Installing : python2-protobuf-3.6.1-3.fc30.noarch 2/6 Installing : mysql-connector-python-8.0.17-1.fc30.x86_64 3/6 Installing : mysql-connector-python3-8.0.17-1.fc30.x86_64 4/6 Installing : mysql-connector-python3-cext-8.0.17-1.fc30.x86_64 5/6 Installing : mysql-connector-python-cext-8.0.17-1.fc30.x86_64 6/6 Running scriptlet: mysql-connector-python-cext-8.0.17-1.fc30.x86_64 6/6 Verifying : python2-protobuf-3.6.1-3.fc30.noarch 1/6 Verifying : python3-protobuf-3.6.1-3.fc30.noarch 2/6 Verifying : mysql-connector-python-8.0.17-1.fc30.x86_64 3/6 Verifying : mysql-connector-python-cext-8.0.17-1.fc30.x86_64 4/6 Verifying : mysql-connector-python3-8.0.17-1.fc30.x86_64 5/6 Verifying : mysql-connector-python3-cext-8.0.17-1.fc30.x86_64 6/6 Installed: mysql-connector-python-8.0.17-1.fc30.x86_64 mysql-connector-python-cext-8.0.17-1.fc30.x86_64 mysql-connector-python3-8.0.17-1.fc30.x86_64 mysql-connector-python3-cext-8.0.17-1.fc30.x86_64 python2-protobuf-3.6.1-3.fc30.noarch python3-protobuf-3.6.1-3.fc30.noarch Complete! |
Leveraging the MySQL Connector/Python Coding Examples documentation, Section 5.1 Connecting to MySQL Using Connector/Python here’s a test of the connection to MySQL 8.
# Import the library. import mysql.connector from mysql.connector import errorcode try: # Open connection. cnx = mysql.connector.connect(user='student', password='student', host='127.0.0.1', database='studentdb') # Print the value. print("Database connection resolved.") # Handle exception and close connection. except mysql.connector.Error as e: if e.errno == errorcode.ER_ACCESS_DENIED_ERROR: print("Something is wrong with your user name or password") elif e.errno == errorcode.ER_BAD_DB_ERROR: print("Database does not exist") else: print(e) # Close the connection when the try block completes. else: cnx.close() |
Leveraging the MySQL Connector/Python Coding Examples documentation, Section 5.4 Querying Data Using Connector/Python here’s a test of the connection to MySQL 8.
# Import the library. import datetime import mysql.connector from mysql.connector import errorcode try: # Open connection. cnx = mysql.connector.connect(user='student', password='student', host='127.0.0.1', database='studentdb') # Create cursor. cursor = cnx.cursor() # Set the query statement. query = ("SELECT " "CASE " " WHEN item_subtitle IS NULL THEN item_title " " ELSE CONCAT(item_title,': ',item_subtitle) " "END AS title, " "release_date " "FROM item " "WHERE release_date BETWEEN %s AND %s " "ORDER BY item_title") # Set the start and end date. start_date = datetime.date(1991, 1, 1) end_date = datetime.date(2004, 12, 31) # Execute cursor. cursor.execute(query, (start_date, end_date)) # Display the rows returned by the query. for (item_name, release_date) in cursor: print("{}, {:%d %b %Y}".format(item_name, release_date)) # Close cursor. cursor.close() # ------------------------------------------------------------ # Handle exception and close connection. except mysql.connector.Error as e: if e.errno == errorcode.ER_ACCESS_DENIED_ERROR: print("Something is wrong with your user name or password") elif e.errno == errorcode.ER_BAD_DB_ERROR: print("Database does not exist") else: print "Error code:", e.errno # error number print "SQLSTATE value:", e.sqlstate # SQLSTATE value print "Error message:", e.msg # error message # Close the connection when the try block completes. else: cnx.close() |
If you run the above in Python 2.7 it works fine. It fails to parse successfully in Python 3.x because the print() function requires the parentheses all the time. You would need to re-write the except block, like this with the parentheses:
# Handle exception and close connection. except mysql.connector.Error as e: if e.errno == errorcode.ER_ACCESS_DENIED_ERROR: print("Something is wrong with your user name or password") elif e.errno == errorcode.ER_BAD_DB_ERROR: print("Database does not exist") else: print("Error code:", e.errno) # error number print("SQLSTATE value:", e.sqlstate) # SQLSTATE value print("Error message:", e.msg) # error message |
While it works without the parentheses in Python 2.7, it also works with the parentheses. That means the best practice is to write cross compatible code by always using the parentheses with the print() function.
As always, I hope this helps somebody.j
Python & Oracle 1
While Python is an interpreted language, Python is a very popular programming language. You may ask yourself why it is so popular? The consensus answers to why it’s so popular points to several factors. For example, Python is a robust high-level programming language that lets you:
- Get complex things done quickly
- Automate system and data integration tasks
- Solve complex analytical problems
You find Python developers throughout the enterprise. Development, release engineering, IT operations, and support teams all use Python to solve problems. Business intelligence and data scientists use Python because it’s easy to use.
Developers don’t generally write end-user applications in Python. They mostly use Python as scripting language. Python provides a simple syntax that lets developers get complex things done quickly. Python also provides you with a vast set of libraries that let you can leverage to solve problems. Those libraries often simplify how you analyze complex data or automate repetitive tasks.
This article explains how to use the Python programming language with the Oracle database. It shows you how to install and use the cx_Oracle library to query data. Part 2 will cover how you insert, update, and delete data in the Oracle database, and how you call and use PL/SQL stored functions and procedures.
The article has two parts:
- How you install and use
cx_Oraclewith the Oracle database - How you query data statically or dynamically
This audience for this article should know the basics of writing a Python program. If you’re completely new to Python, you may want to get a copy of Eric Matthes’ Python Crash Course: A Hands-On, Project-Based Introduction to Programming. More experienced developers with shell scripting backgrounds may prefer Al Sweigart’s Automate the Boring Stuff with Python.
This article uses Python 2.7, which appears to be the primary commercial version of Python in most organizations. At least, it’s what most vendors ship with Linux distros. It also happens to be the Python distro on Fedora Linux.
How you install and use cx_Oracle with the Oracle database
The first step requires that you test the current version of Python on your Operating System (OS). For the purpose of this paper, you use the student user account. The student user is in the sudoer list, which gives the account super user privileges.
You can find the Python version by opening a Terminal session and running the following command:
[student@localhost ~]$ python -V |
It displays:
Python 2.7.5 |
You can download the current version of the cx_Oracle library at the Python Software Foundation’s web site. At the time of writing, the current version of the cx_Oracle is the cx_Oracle 5.2.1 version. The cx_Oracle library is available for download as a Red Hat Package Manager (RPM) module.
You download the cx_Oracle-5.2.1-11g-py26-1.x86_64.rpm to the /tmp directory or to a sudoer-enabled user’s downloads directory. Let’s assume you download the RPM into the /tmp directory. After you download the RPM, you can install it with the yum utility with this syntax:
yum install -y /tmp/cx_Oracle-5.2.1-11g-py27-1.x86_64.rpm |
However, the most current version is now 7.0. You want the following file on Fedora 64-bit Linux, which can be found at the Python Package Index web site:
cx_Oracle-7.0.0-cp27-cp27mu-manylinux1_x86_64.whl |
A wheel file requires that you use the pip utility (make sure to upgrade to the current version), like:
sudo pip install cx_Oracle-7.0.0-cp27-cp27mu*.whl |
It should print the following to the console:
Processing ./cx_Oracle-7.0.0-cp27-cp27mu-manylinux1_x86_64.whl Installing collected packages: cx-Oracle Successfully installed cx-Oracle-7.0.0 |
The cx_Oracle library depends on the Oracle Client software, which may or may not be installed. It installs without a problem but would raise a runtime error when using the Python software. You can check whether cx_Oracle is installed with the following syntax:
rpm –qa oracle-instantclient11.2-basic |
If the oracle-instantclient11.2-basic library isn’t installed, the command returns nothing. If the oracle-instantclient11.2-basic library is installed it returns the following:
oracle-instantclient11.2-basic-11.2.0.4.0-1.x86_64 |
Assuming you don’t have the Oracle Client software installed, you should download it from Oracle’s Instant Client Downloads web page. After you download the RPM, you install the Oracle 11g Release 2 Client software with the following syntax:
yum install -y /tmp/oracle-instantclient11.2-basic-11.2.0.4.0-1.x86_64.rpm |
You now have the necessary software installed and configured to run and test Python programs that work with the Oracle database. Python uses a standard path configuration to look for Python modules or libraries. You can see that set of path values by connecting to the Python IDLE environment, which is the runtime environment. The IDLE environment is very much like the SQL*Plus environment.
You connect to the Python IDLE environment by typing the following:
python |
It opens the Python IDLE environment. It should display the following:
Python 2.7.5 (default, Apr 10 2015, 08:09:05) [GCC 4.8.3 20140911 (Red Hat 4.8.3-7)] on linux2 Type "help", "copyright", "credits" or "license" for more information. |
You import the sys library and then you can print the path elements with the following command:
>>> import sys print sys.path |
It should print the following for Python 2.7 in Fedora Linux:
['', '/usr/lib64/python27.zip', '/usr/lib64/python2.7', '/usr/lib64/python2.7/plat-linux2', '/usr/lib64/python2.7/lib-tk', '/usr/lib64/python2.7/lib-old', '/usr/lib64/python2.7/lib-dynload', '/usr/lib64/python2.7/site-packages', '/usr/lib64/python2.7/site-packages/gtk-2.0', '/usr/lib/python2.7/site-packages'] |
You can now test whether the environment works by typing the following commands in the IDLE environment:
>>> import cx_Oracle db = cx_Oracle.connect("student/student@xe") print db.version |
It prints:
11.2.0.2.0 |
The other two sections require you to test components inside Python files. That means you need to supplement the default Python path variable. You do that by adding values to the Python environment variable, which is $PYTHONPATH.
The following adds the /home/student/Code/python directory to the Python path variable:
export set PYTHONPATH=/home/student/Code/python |
Next, we create an connection.py file, which holds the following:
# Import the Oracle library. import cx_Oracle try: # Create a connection. db = cx_Oracle.connect("student/student@xe") # Print a message. print "Connected to the Oracle " + db.version + " database." except cx_Oracle.DatabaseError, e: error, = e.args print >> sys.stderr, "Oracle-Error-Code:", error.code print >> sys.stderr, "Oracle-Error-Message:", error.message  finally # Close connection. db.close() |
The import statement adds the cx_Oracle library to the program scope. The cx_Oracle library’s connect function takes either the user name and password, or the user name, password, and TNS alias.
The except block differs from what you typically see. The code value maps to the SQLCODE value and the message value maps to the SQLERRM value.
You can test the connection.py file as follows in the /home/student/Code/python directory:
python connection.py |
It prints the following:
Connected to the Oracle 11.2.0.2.0 database. |
This section has shown you how to setup the cx_Oracle library, and how you can test the cx_Oracle library with Python programs.
How you query data statically or dynamically
The prior section shows you how to connect to an Oracle instance and how to verify the driver version of the cx_Oracle library. Like most ODBC and JDBC software, Python first creates a connection. Then, you need to create a cursor inside a connection.
The basicCursor.py program creates a connection and a cursor. The cursor holds a static SQL SELECT statement. The SELECT statement queries a string literal from the pseudo dual table.
# Import the Oracle library. import sys import cx_Oracle try: # Create a connection. db = cx_Oracle.connect("student/student@xe") # Create a cursor. cursor = db.cursor() # Execute a query. cursor.execute("SELECT 'Hello world!' FROM dual") # Read the contents of the cursor. for row in cursor: print (row[0]) except cx_Oracle.DatabaseError, e: error, = e.args print >> sys.stderr, "Oracle-Error-Code:", error.code print >> sys.stderr, "Oracle-Error-Message:", error.message finally: # Close cursor and connection. cursor.close() } db.close() |
The connect function assigns a database connection to the local db variable. The cursor function returns a cursor and assigns it to the local cursor variable. The execute function dispatches the query to Oracle’s SQL*Plus and returns the result set into a row element of the local cursor variable. The for-each loop reads the row element from the cursor variable and prints one row at a time. Since the cursor only returns a string literal, there’s only one row to return.
You test the program with this syntax:
python basicConnection.py |
It prints:
Hello world! |
The next basicTable.py program queries the item table. The item table holds a number of rows of data. The code returns each row inside a set of parentheses.
# Import the Oracle library. import cx_Oracle try: # Create a connection. db = cx_Oracle.connect("student/student@xe") # Create a cursor. cursor = db.cursor() # Execute a query. cursor.execute("SELECT item_title " + ", item_rating " + "FROM item " + "WHERE item_type = " " (SELECT common_lookup_id " + " FROM common_lookup " + " WHERE common_lookup_type = 'DVD_WIDE_SCREEN')") # Read the contents of the cursor. for row in cursor: print (row[0], row[1]) except cx_Oracle.DatabaseError, e: error, = e.args print >> sys.stderr, "Oracle-Error-Code:", error.code print >> sys.stderr, "Oracle-Error-Message:", error.message finally: # Close cursor and connection. cursor.close() db.close() |
The SQL query is split across several lines by using the + operator. The + operator concatenates strings, and it lets you format a long query statement. The range for loop returns tuples from the cursor. The tuples are determined by the SELECT-list of the query.
The query returns the following type of results:
('Casino Royale', 'PG-13') ... ('Star Wars - Episode I', 'PG') ('Star Wars - Episode II', 'PG') ('Star Wars - Episode III', 'PG-13') ('Star Wars - Episode IV', 'PG') ('Star Wars - Episode V', 'PG') ('Star Wars - Episode VI', 'PG') |
At this point, you know how to work with static queries. The next example shows you how to work with dynamic queries. The difference between a static and dynamic query is that an element of the string changes.
You have two options for creating dynamic strings. The first lets you glue a string inside a query. The second lets you embed one or more bind variables in a string. As a rule, you should use bind variables because they avoid SQL injection risks.
The following is the basicDynamicTable.py script
# Import the Oracle library. import cx_Oracle sRate = 'PG-13' try: # Create a connection. db = cx_Oracle.connect("student/student@xe") # Define a dynamic statment. stmt = "SELECT item_title, item_rating FROM item WHERE item_rating = :rating" # Create a cursor. cursor = db.cursor() # Execute a statement with a bind variable. cursor.execute(stmt, rating = sRate) # Read the contents of the cursor. for row in cursor: print (row[0], row[1]) except cx_Oracle.DatabaseError, e: error, = e.args print >> sys.stderr, "Oracle-Error-Code:", error.code print >> sys.stderr, "Oracle-Error-Message:", error.message finally: # Close cursor and connection. cursor.close() db.close() |
You need to assign a dynamic SQL statement to a local string variable. The bind variable is preceded with a colon (:). The execute function takes a string variable with the dynamic SQL statement. Then, you provide a name and value pair. The name needs to match the bind variable in the dynamic SQL statement. The value needs to map to a local Python variable.
The query should return a full list from the item table for the two item_title and item_rating columns:
('Casino Royale', 'PG-13')
...
('Harry Potter and the Goblet of Fire', 'PG-13')
('Harry Potter and the Order of the Phoenix', 'PG-13')
('The Lord of the Rings - Fellowship of the Ring', 'PG-13')
('The Lord of the Rings - Two Towers', 'PG-13')
('The Lord of the Rings - The Return of the King', 'PG-13')
('The Lord of the Rings - The Return of the King', 'PG-13') |
This article should have shown you how to effectively work static and dynamic queries. You can find the scripts on the github.com server.
Read list of a dictionaries
My students wanted a quick example of how to read a list of a dictionaries in Python. So, here it is:
#!/usr/bin/python # Declare list of dictionaries. cakes = [{'cake':"vanilla",'frosting':"chocolate"} ,{'cake':"chocolate",'frosting':"vanilla"}] # Read the list of dictionaries. for lkey, lvalue in enumerate(cakes): print lvalue['cake'] + " with " + lvalue['frosting'] + " frosting." |
Naturally, a list can contain many things and you should ensure each value you read is a dictionary before trying to read it as a dictionary. At least, I’d suggest you check.
Hope this answers the how.
Installing PIP for Python
If you’re on a Mac running macOS Sierra, you can install PIP to add packages. PIP stands for either of the following:
- PIP installs Packages
- PIP installs Python
You use the following to install the PIP utility:
sudo easy_install pip |
It should return the following:
Searching for pip Reading https://pypi.python.org/simple/pip/ Best match: pip 9.0.1 Downloading https://pypi.python.org/packages/11/b6/abcb525026a4be042b486df43905d6893fb04f05aac21c32c638e939e447/pip-9.0.1.tar.gz#md5=35f01da33009719497f01a4ba69d63c9 Processing pip-9.0.1.tar.gz Writing /tmp/easy_install-ryxjDg/pip-9.0.1/setup.cfg Running pip-9.0.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-ryxjDg/pip-9.0.1/egg-dist-tmp-l6_Jjt /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires' warnings.warn(msg) warning: no previously-included files found matching '.coveragerc' warning: no previously-included files found matching '.mailmap' warning: no previously-included files found matching '.travis.yml' warning: no previously-included files found matching '.landscape.yml' warning: no previously-included files found matching 'pip/_vendor/Makefile' warning: no previously-included files found matching 'tox.ini' warning: no previously-included files found matching 'dev-requirements.txt' warning: no previously-included files found matching 'appveyor.yml' no previously-included directories found matching '.github' no previously-included directories found matching '.travis' no previously-included directories found matching 'docs/_build' no previously-included directories found matching 'contrib' no previously-included directories found matching 'tasks' no previously-included directories found matching 'tests' creating /Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg Extracting pip-9.0.1-py2.7.egg to /Library/Python/2.7/site-packages Adding pip 9.0.1 to easy-install.pth file Installing pip script to /usr/local/bin Installing pip2.7 script to /usr/local/bin Installing pip2 script to /usr/local/bin Installed /Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg Processing dependencies for pip Finished processing dependencies for pip |
After you install PIP, you can use PIP to add custom packages to the Python environment. The
sudo pip install easygui |
You get the following warning and installation:
The directory '/Users/michaelmclaughlin/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. The directory '/Users/michaelmclaughlin/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. Collecting easygui Downloading easygui-0.98.1-py2.py3-none-any.whl (90kB) 100% |████████████████████████████████| 92kB 1.0MB/s Installing collected packages: easygui Successfully installed easygui-0.98.1 |
After installing the easygui Python library, you can change to the root directory to confirm the installation of the easygui Python library with the following command:
find . -name easygui* 2>/dev/null |
It returns the following:
./Library/Python/2.7/site-packages/easygui ./Library/Python/2.7/site-packages/easygui/easygui.py ./Library/Python/2.7/site-packages/easygui/easygui.pyc ./Library/Python/2.7/site-packages/easygui-0.98.1.dist-info |
You can connect to Python 2.7 in a Terminal session. Then, you use the easygui library to run a Hello World! message box with the following commands in the Python shell:
import easygui easy gui.msgbox("Hello World!") |
It will raise the following image:
Hopefully, this helps a few folks.
Python variable not defined
While working with a programming example for my students, I ran into an interesting run-time error when I changed their approach to importing Python’s random module. Here’s the raised error message:
Traceback (most recent call last): File "windowBouncingBalls.py", line 84, in <module> speed = [choice([-2,2]), choice([-2,2])] NameError: name 'choice' is not defined |
You raise the missing choice identifier when two things occur. The first thing requires you to use a standard import statement, like the following example, and the second thing requires you to continue to reference the identifier as “choice“.
import random |
You can avoid the error by making the import of random like this:
from random import * |
Or, you can leave the ordinary import statement and fully qualify the choice identifier with the random module name, like this:
speed = [random.choice([-2,2]), random.choice([-2,2])] |
As always, I hope this helps those who encounter a similar problem.
Install PyGame on Fedora
The PyGame library is a wonderful tool for building games with Python. It lets you accomplish a great deal by simply managing events. You need to understand how to use Python functions, modules, and events to build games with this Python library.
You can download and install the PyGame library with the yum utility like this:
yum install -y pygame |
It should generate the following list when you install it as the root user:
Loaded plugins: langpacks, refresh-packagekit Available Packages pygame.x86_64 1.9.1-14.fc20 fedora [root@localhost ~]# yum install -y pygame Loaded plugins: langpacks, refresh-packagekit Resolving Dependencies --> Running transaction check ---> Package pygame.x86_64 0:1.9.1-14.fc20 will be installed --> Processing Dependency: numpy for package: pygame-1.9.1-14.fc20.x86_64 --> Processing Dependency: libportmidi.so.0()(64bit) for package: pygame-1.9.1-14.fc20.x86_64 --> Processing Dependency: libSDL_ttf-2.0.so.0()(64bit) for package: pygame-1.9.1-14.fc20.x86_64 --> Processing Dependency: libSDL_mixer-1.2.so.0()(64bit) for package: pygame-1.9.1-14.fc20.x86_64 --> Processing Dependency: libSDL_image-1.2.so.0()(64bit) for package: pygame-1.9.1-14.fc20.x86_64 --> Running transaction check ---> Package SDL_image.x86_64 0:1.2.12-7.fc20 will be installed ---> Package SDL_mixer.x86_64 0:1.2.12-5.fc20 will be installed --> Processing Dependency: libmikmod for package: SDL_mixer-1.2.12-5.fc20.x86_64 ---> Package SDL_ttf.x86_64 0:2.0.11-4.fc20 will be installed ---> Package numpy.x86_64 1:1.8.2-2.fc20 will be installed --> Processing Dependency: python-nose for package: 1:numpy-1.8.2-2.fc20.x86_64 ---> Package portmidi.x86_64 0:217-9.fc20 will be installed --> Running transaction check ---> Package libmikmod.x86_64 0:3.3.6-3.fc20 will be installed ---> Package python-nose.noarch 0:1.3.0-1.fc20 will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: pygame x86_64 1.9.1-14.fc20 fedora 2.1 M Installing for dependencies: SDL_image x86_64 1.2.12-7.fc20 fedora 41 k SDL_mixer x86_64 1.2.12-5.fc20 fedora 91 k SDL_ttf x86_64 2.0.11-4.fc20 fedora 22 k libmikmod x86_64 3.3.6-3.fc20 updates 142 k numpy x86_64 1:1.8.2-2.fc20 updates 3.0 M portmidi x86_64 217-9.fc20 fedora 26 k python-nose noarch 1.3.0-1.fc20 fedora 272 k Transaction Summary ================================================================================ Install 1 Package (+7 Dependent packages) Total download size: 5.7 M Installed size: 21 M Downloading packages: (1/8): SDL_image-1.2.12-7.fc20.x86_64.rpm | 41 kB 00:00 (2/8): SDL_mixer-1.2.12-5.fc20.x86_64.rpm | 91 kB 00:00 (3/8): portmidi-217-9.fc20.x86_64.rpm | 26 kB 00:00 (4/8): SDL_ttf-2.0.11-4.fc20.x86_64.rpm | 22 kB 00:00 (5/8): libmikmod-3.3.6-3.fc20.x86_64.rpm | 142 kB 00:00 (6/8): numpy-1.8.2-2.fc20.x86_64.rpm | 3.0 MB 00:02 (7/8): pygame-1.9.1-14.fc20.x86_64.rpm | 2.1 MB 00:01 (8/8): python-nose-1.3.0-1.fc20.noarch.rpm | 272 kB 00:00 -------------------------------------------------------------------------------- Total 1.7 MB/s | 5.7 MB 00:03 Running transaction check Running transaction test Transaction test succeeded Running transaction (shutdown inhibited) Installing : SDL_ttf-2.0.11-4.fc20.x86_64 1/8 Installing : SDL_image-1.2.12-7.fc20.x86_64 2/8 Installing : portmidi-217-9.fc20.x86_64 3/8 Installing : libmikmod-3.3.6-3.fc20.x86_64 4/8 Installing : SDL_mixer-1.2.12-5.fc20.x86_64 5/8 Installing : python-nose-1.3.0-1.fc20.noarch 6/8 Installing : 1:numpy-1.8.2-2.fc20.x86_64 7/8 Installing : pygame-1.9.1-14.fc20.x86_64 8/8 Verifying : pygame-1.9.1-14.fc20.x86_64 1/8 Verifying : SDL_mixer-1.2.12-5.fc20.x86_64 2/8 Verifying : python-nose-1.3.0-1.fc20.noarch 3/8 Verifying : libmikmod-3.3.6-3.fc20.x86_64 4/8 Verifying : 1:numpy-1.8.2-2.fc20.x86_64 5/8 Verifying : portmidi-217-9.fc20.x86_64 6/8 Verifying : SDL_image-1.2.12-7.fc20.x86_64 7/8 Verifying : SDL_ttf-2.0.11-4.fc20.x86_64 8/8 Installed: pygame.x86_64 0:1.9.1-14.fc20 Dependency Installed: SDL_image.x86_64 0:1.2.12-7.fc20 SDL_mixer.x86_64 0:1.2.12-5.fc20 SDL_ttf.x86_64 0:2.0.11-4.fc20 libmikmod.x86_64 0:3.3.6-3.fc20 numpy.x86_64 1:1.8.2-2.fc20 portmidi.x86_64 0:217-9.fc20 python-nose.noarch 0:1.3.0-1.fc20 Complete! |
I hope this helps folks install the software.
Subscribe via RSS