MacLochlainns Weblog

Michael McLaughlin's Technical Blog

Site Admin

Oracle23ai Ubuntu Install

without comments

What to do with a Late 2015 iMac with an i7 Quad CPU running at 3.4 GHz, 32 GB or RAM, a 5K Display and an almost warn out hybrid 1 TB hard disk? You could sell it to Apple for pennies, but why enrich them. I opted to upgrade it with an OWC kit that had a 2 TB SSD Disk. Then, I installed Ubuntu 22.0.4 and built a DaaS (Database as a Service) machine with Oracle Database 23ai in a Docker container, and MySQL 8 and PostgreSQL 14 natively.

I’ve posted on installing MySQL 8 and PostgreSQL 14 on Ubuntu before when I repurposed my late 2014 MacBook Pro. This post covers the installation of Docker and Oracle Database 23ai.

Install Docker

Contrary to the instructions, you should do the following as a sudoer user:

sudo apt install -y docker.io

Install all dependency packages using the following command:

sudo snap install docker

You should see the following:

docker 20.10.24 from Canonical✓ installed

You can verify the Docker install with the following command:

sudo docker --version

It should show something like this:

Docker version 24.0.5, build 24.0.5-0ubuntu1~22.04.1

You can check the pulled containers with the following command but at this point there should be no pulled containers.

sudo docker images

At this point, a docker group already exists but you need to add your user to the docker group with the following command:

sudo usermod -aG docker $USER

Using the Docker Commands:

  • To activate the logging, utilize the -f parameter.
  • To divide JSON, use Docker by default; to extract individual keys, use JQ.
  • In your Container file, there are quite a few areas where commands may be specified.
  • Posting to the volumes could be more effective while the picture is being built.
  • Docker offers a highly efficient way to create an alias for its own built-in commands. This makes it easier to set up and handle lengthy and enormous orders. These alias values are stored in the directories /.bashrc or and /.bash_aliases.
  • Docker offers further assistance to remove unused code fragments from the installation of the container.
  • Docker always favors reading statements from the container file that have not changed. Therefore, time savings may be realized by arranging what is shown in the container file in a way that ensures the elements that are susceptible to change are shown towards the end of the document and those that are most likely to undergo change are shown at the top.

Install Oracle Database 23ai Free in a Docker container

Use the following command to pull and install the Oracle Database 23ai container:

sudo docker run --name oracle23ai -p 1521:1521 -p 5500:5500 -e ORACLE_PWD=cangetin container-registry.oracle.com/database/free:latest

After installing the Oracle Database 23ai Free container, you can access it as the root user by default with this syntax:

docker exec -it -u root oracle23ai bash

At the root prompt, you can connect to the system schema with the following command:

sqlplus system/cangetin@FREE

You should see the following:

SQL*Plus: RELEASE 23.0.0.0.0 - Production ON Thu May 9 03:56:57 2024
Version 23.4.0.24.05
 
Copyright (c) 1982, 2024, Oracle.  ALL rights reserved.
 
LAST SUCCESSFUL login TIME: Wed Apr 24 2024 21:23:00 +00:00
 
Connected TO:
Oracle DATABASE 23ai Free RELEASE 23.0.0.0.0 - Develop, Learn, AND Run FOR Free
Version 23.4.0.24.05
 
SQL>

Create a c##student as a sandbox user:

After you create and provision the Oracle Database 21ai Free, you can create a c##student sand-boxed user with the following two step process.

  1. Create a c##student Oracle user account with the following command as the system user:

    CREATE USER c##student IDENTIFIED BY student
    DEFAULT TABLESPACE users QUOTA 200M ON users
    TEMPORARY TABLESPACE temp;

  2. Grant necessary privileges to the newly created c##student user:

    GRANT CREATE CLUSTER, CREATE INDEXTYPE, CREATE OPERATOR
    ,     CREATE PROCEDURE, CREATE SEQUENCE, CREATE SESSION
    ,     CREATE TABLE, CREATE TRIGGER, CREATE TYPE
    ,     CREATE VIEW TO c##student;

  3. Connect to the sandboxed user with the following syntax (by the way it’s a pluggable user account as qualified in Oracle Database 12c forward):

    SQL> CONNECT c##student/student@FREE

    or, disconnect and reconnect with this syntax:

    sqlplus system/cangetin@FREE

Set Docker Oracle 23ai to start always

Assuming that your container name was oracle23ai, as qualified above, you can run the following command to automatically restart the Docker container:

docker update --restart=always `docker ps -aqf "name=oracle23ai"`

The docker command inside the backquotes uses the Docker instance’s name to return the Docker container_id value, which can also be seen when you run the following command:

docker ps

which returns:

CONTAINER ID   IMAGE                                                COMMAND                  CREATED       STATUS                    PORTS                                                                                  NAMES
b211f494e692   container-registry.oracle.com/database/free:latest   "/bin/bash -c $ORACL…"   13 days ago   Up 18 minutes (healthy)   0.0.0.0:1521->1521/tcp, :::1521->1521/tcp, 0.0.0.0:5500->5500/tcp, :::5500->5500/tcp   oracle23ai

The Docker container_id value is required when you perform a Docker update operation.

Configuring your Docker Oracle 23ai environment

Unless you like memorizing the Docker command-line, you may automate connecting as the root user or add a sand boxed user. The root user typically has more power than you need to perform ordinary development and use-case testing tasks.

A sand boxed user has narrow access, can’t start and stop the database instance or perform Oracle Datasbase 23ai administration. In this segment, you’ll learn how to create a couple local Bash functions to simplify your use of the Oracle Database 23ai container; and how to extend the configuration of Oracle’s Docker container:

  • Adding a student user to the Docker container and configuring it to access the Oracle Database 23ai locally from within the Docker container using a direct sqlplus connection.
  • Configuring the Docker container to support external files and leverage a shared directory with your base operating system.

Automating Docker instance connections:

The following shows you how to add a local Bash function to automate access to the Docker container from the Linux command-line. You put the following Bash function in your base Linux operating system’s user .bashrc file:

  1. Create the following Bash function:

    # User defined function to launch Oracle 23 ai container
    # as the root user.
    admin () 
    {
        # Discover the fully qualified program name. 
        path=`which docker 2>/dev/null`
        file=''
     
        # Parse the program name from the path.
        if [ -n ${path} ]; then
            file=${path##/*/}
        fi
     
        # Wrap when there is a file and it is rewrap.
        if [ -n ${file} ] && [[ ${file} = "docker" ]]; then
            python -c "import subprocess; subprocess.run(['docker exec -it --user root oracle23ai bash'], shell=True)" 
        else
            echo "Docker is unavailable: Install the docker package."
        fi
    }

  2. After you source the .bashrc file or simply reconnect as to the terminal as your user, which resources the .bashrc file, you can access the oracle23ai Docker instance with this command:

    admin

    It will display a new prompt with the root user and the Docker container_id value, like:

    [root@b211f494e692 oracle]#

    You can exit the Docker container by typing exit at the Linux command line. If you curious what version of Linux you’re using inside the Docker instance, you can’t use the uname command because it returns the hosting Linux distribution (distro). You must use the following when inside the Docker instance:

    cat /etc/os-release

    or, outside the Docker instance you can use the following docker command:

    docker exec oracle23ai cat /etc/os-release

    Either way, for an Oracle Database 23ai container, it should return:

    NAME="Oracle Linux Server"
    VERSION="8.9"
    ID="ol"
    ID_LIKE="fedora"
    VARIANT="Server"
    VARIANT_ID="server"
    VERSION_ID="8.9"
    PLATFORM_ID="platform:el8"
    PRETTY_NAME="Oracle Linux Server 8.9"
    ANSI_COLOR="0;31"
    CPE_NAME="cpe:/o:oracle:linux:8:9:server"
    HOME_URL="https://linux.oracle.com/"
    BUG_REPORT_URL="https://github.com/oracle/oracle-linux"
     
    ORACLE_BUGZILLA_PRODUCT="Oracle Linux 8"
    ORACLE_BUGZILLA_PRODUCT_VERSION=8.9
    ORACLE_SUPPORT_PRODUCT="Oracle Linux"
    ORACLE_SUPPORT_PRODUCT_VERSION=8.9

    Unfortunately, Oracle has appeared to block updates to the Oracle Unbreakable Linux 8 instance inside the container, which makes native SQL*Plus use more difficult. That’s because you’ll need to install the Oracle SQL*Plus client in the hosting Operating System.

    I’ve written a separate blog post that instructs you on how to install and use Oracle SQL*Plus client on Ubuntu.

Install SQL Developer in the base Linux operating system

The first steps are installing the Java Runtime Environment and Java Development Kit, and then downloading, installing and configuring SQL Developer. These are the required steps:

  1. Install the Java Runtime Environment:

    sudo apt install default-jre

    The log file for this is:

  2. Install the Java Runtime Environment:

    sudo apt install -y default-idk

    The log file for this is:

  3. Download SQL Developer from here; and then install SQL Developer to the /opt directory on your Ubuntu local instance:

    Use the following command to unzip the SQL Developer files to the /opt directory:

    sudo unzip ~/Downloads/sqldeveloper-23.1.0.097.1607-no-jre.zip
  4. Create the following /usr/local/bin/sqldeveloper symbolic link:

    sudo ln -s /opt/sqldeveloper/sqldeveloper.sh /usr/local/bin/sqldeveloper
  5. Edit the /opt/sqldeveloper/sqldeveloper.sh file by replacing the following line:

    cd "`dirname $0`"/sqldeveloper/bin && bash sqldeveloper $*

    with this version:

    /opt/sqldeveloper/sqldeveloper/bin/sqldeveloper $*
  6. Now, you can launch SQL Developer from any location on your local Ubuntu operating system, like:

    sqldeveloper
  7. You can now connect as the system user through SQL Developer to the Oracle Database 23ai Free Docker instance with the following connection information:

    (Excuse recycling the version from 21c but I didn’t see any utility in making a new screen shot.)

  8. You can also create a Desktop shortcut by creating the sqldeveloper.desktop file in the /usr/share/applications directory. The SQL Developer icon is provided in the sqldeveloper base directory.

    You should create the following sqldeveloper.desktop file to use a Desktop shortcut:

    [Desktop Entry]
    Name=Oracle SQL Developer
    Comment=SQL Developer from Oracle
    GenericName=SQL Tool
    Exec=/usr/local/bin/sqldeveloper
    Icon=/opt/sqldeveloper/icon.png
    Type=Application
    StartupNotify=true
    Categories=Utility;Oracle;Development;SQL;

As always, I hope this helps those trying to accomplish this task.

Written by maclochlainn

May 8th, 2024 at 10:12 pm

Learning SQL Exercise

without comments

I’ve been using Alan Beaulieu’s Learning SQL to teach my SQL Development class with MySQL 8. It’s a great book overall but Chapter 12 lacks a complete exercise. Here’s all that the author provides to the reader. This is inadequate for most readers to work with to solve the concept of a transaction.

Exercise 12-1

Generate a unit of work to transfer $50 from account 123 to account 789. You will need to insert two rows into the transaction table and update two rows in the account table. Use the following table definitions/data:

                      Account:
account_id     avail_balance    last_activity_date
-----------    --------------   ------------------
       123               450    2019-07-10 20:53:27
       789               125    2019-06-22 15:18:35
 
                      Transaction:
txn_id    txn_date      account_id    txn_type_cd    amount
------    ----------    -------+--    -----------    ------
  1001    2019-05-15           123    C                 500
  1002    2019-06-01           789    C                  75

Use txn_type_cd = ‘C” to indicate a credit (addition), and use txn_type_cd = ‘D’ to indicate a debit (substraction).

New Exercise 12-1

The problem with the exercise description is that the sakila database, which is used for most of the book, doesn’t have transaction or account tables. Nor, are there any instructions about general accounting practices or principles. These missing components make it hard for students to understand how to build the transaction.

The first thing the exercise’s problem defintion should qualify is how to create the account and transaction tables, like:

  1. Create the account table, like this with an initial auto incrementing value of 1001:

    -- +--------------------+--------------+------+-----+---------+----------------+
    -- | Field              | Type         | Null | Key | Default | Extra          |
    -- +--------------------+--------------+------+-----+---------+----------------+
    -- | account_id         | int unsigned | NO   | PRI | NULL    | auto_increment |
    -- | avail_balance      | double       | NO   |     | NULL    |                |
    -- | last_activity_date | datetime     | NO   |     | NULL    |                |
    -- +--------------------+--------------+------+-----+---------+----------------+
  2. Create the transaction table, like this with an initial auto incrementing value of 1001:

    -- +----------------+--------------+------+-----+---------+----------------+
    -- | Field          | Type         | Null | Key | Default | Extra          |
    -- +----------------+--------------+------+-----+---------+----------------+
    -- | txn_id         | int unsigned | NO   | PRI | NULL    | auto_increment |
    -- | txn_date       | datetime     | YES  |     | NULL    |                |
    -- | account_id     | int unsigned | YES  |     | NULL    |                |
    -- | txn_type_cd    | varchar(1)   | NO   |     | NULL    |                |
    -- | amount         | double       | YES  |     | NULL    |                |
    -- +----------------+--------------+------+-----+---------+----------------+

Checking accounts are liabilities to banks, which means you credit a liability account to increase its value and debit a liability to decrease its value. You should insert the initial rows into the account table with a zero avail_balance. Then, make these iniitial deposits:

  1. Credit transaction table with an account_id column value of 123 with $500 and a txn_type_cd column value of ‘C’.
  2. Credit transaction table with an account_id column value of 789 with $75 and a txn_type_cd column value of ‘C’.

Write an update statement to set the avail_balance column values equal to the aggregate sum of the transaction table’s rows, which treats credit transacctions (those with a ‘C’ in the txn_type_cd column as a positive number and thos with a ‘D’ in the txn_type_cd column as a negative number).

Generate a unit of work to transfer $50 from account 123 to account 789. You will need to insert two rows into the transaction table and update two rows in the account table. Use the following table definitions/data:

  1. Debit transaction table with an account_id column value of 123 with $50 and a txn_type_cd column value of ‘D’.
  2. Credit transaction table with an account_id column value of 789 with $50 and a txn_type_cd column value of ‘C’.

Apply the prior update statement to set the avail_balance column values equal to the aggregate sum of the transaction table’s rows, which treats credit transacctions (those with a ‘C’ in the txn_type_cd column as a positive number and thos with a ‘D’ in the txn_type_cd column as a negative number).

Here’s the solution to the problem:

-- +--------------------+--------------+------+-----+---------+----------------+
-- | Field              | Type         | Null | Key | Default | Extra          |
-- +--------------------+--------------+------+-----+---------+----------------+
-- | account_id         | int unsigned | NO   | PRI | NULL    | auto_increment |
-- | avail_balance      | double       | NO   |     | NULL    |                |
-- | last_activity_date | datetime     | NO   |     | NULL    |                |
-- +--------------------+--------------+------+-----+---------+----------------+
 
DROP TABLE IF EXISTS account, transaction;
 
CREATE TABLE account
( account_id          int unsigned PRIMARY KEY AUTO_INCREMENT
, avail_balance       double       NOT NULL
, last_activity_date  datetime     NOT NULL )
 ENGINE=InnoDB 
 AUTO_INCREMENT=1001 
 DEFAULT CHARSET=utf8mb4 
 COLLATE=utf8mb4_0900_ai_ci;
 
-- +----------------+--------------+------+-----+---------+----------------+
-- | Field          | Type         | Null | Key | Default | Extra          |
-- +----------------+--------------+------+-----+---------+----------------+
-- | txn_id         | int unsigned | NO   | PRI | NULL    | auto_increment |
-- | txn_date       | datetime     | YES  |     | NULL    |                |
-- | account_id     | int unsigned | YES  |     | NULL    |                |
-- | txn_type_cd    | varchar(1)   | NO   |     | NULL    |                |
-- | amount         | double       | YES  |     | NULL    |                |
-- +----------------+--------------+------+-----+---------+----------------+
 
CREATE TABLE transaction
( txn_id         int unsigned  PRIMARY KEY AUTO_INCREMENT
, txn_date       datetime      NOT NULL
, account_id     int unsigned  NOT NULL
, txn_type_cd    varchar(1)
, amount         double
, CONSTRAINT transaction_fk1 FOREIGN KEY (account_id)
 REFERENCES account(account_id))
 ENGINE=InnoDB
 AUTO_INCREMENT=1001
 DEFAULT CHARSET=utf8mb4
 COLLATE=utf8mb4_0900_ai_ci;
 
-- Insert initial accounts.
INSERT INTO account
( account_id
, avail_balance
, last_activity_date )
VALUES
( 123
, 0
,'2019-07-10 20:53:27');
 
INSERT INTO account
( account_id
, avail_balance
, last_activity_date )
VALUES
( 789
, 0
,'2019-06-22 15:18:35');
 
-- Insert initial deposits.
INSERT INTO transaction
( txn_date
, account_id
, txn_type_cd
, amount )
VALUES
( CAST(NOW() AS DATE)
, 123
,'C'
, 500 );
 
INSERT INTO transaction
( txn_date
, account_id
, txn_type_cd
, amount )
VALUES
( CAST(NOW() AS DATE)
, 789
,'C'
, 75 );
 
UPDATE account a
SET    a.avail_balance = 
 (SELECT  SUM(
            CASE
              WHEN t.txn_type_cd = 'C' THEN amount
              WHEN t.txn_type_cd = 'D' THEN amount * -1
            END) AS amount
 FROM     transaction t
 WHERE    t.account_id = a.account_id
 AND      t.account_id IN (123,789)
 GROUP BY t.account_id);
 
SELECT * FROM account;
SELECT * FROM transaction;
 
-- Insert initial deposits.
INSERT INTO transaction
( txn_date
, account_id
, txn_type_cd
, amount )
VALUES
( CAST(NOW() AS DATE)
, 123
,'D'
, 50 );
 
INSERT INTO transaction
( txn_date
, account_id
, txn_type_cd
, amount )
VALUES
( CAST(NOW() AS DATE)
, 789
,'C'
, 50 );
 
UPDATE account a
SET    a.avail_balance = 
 (SELECT  SUM(
            CASE
              WHEN t.txn_type_cd = 'C' THEN amount
              WHEN t.txn_type_cd = 'D' THEN amount * -1
            END) AS amount
 FROM     transaction t
 WHERE    t.account_id = a.account_id
 AND      t.account_id IN (123,789)
 GROUP BY t.account_id);
 
SELECT * FROM account;
SELECT * FROM transaction;

The results are:

+------------+---------------+---------------------+
| account_id | avail_balance | last_activity_date  |
+------------+---------------+---------------------+
|        123 |           450 | 2019-07-10 20:53:27 |
|        789 |           125 | 2019-06-22 15:18:35 |
+------------+---------------+---------------------+
2 rows in set (0.00 sec)
 
+--------+---------------------+------------+-------------+--------+
| txn_id | txn_date            | account_id | txn_type_cd | amount |
+--------+---------------------+------------+-------------+--------+
|   1001 | 2024-04-01 00:00:00 |        123 | C           |    500 |
|   1002 | 2024-04-01 00:00:00 |        789 | C           |     75 |
|   1003 | 2024-04-01 00:00:00 |        123 | D           |     50 |
|   1004 | 2024-04-01 00:00:00 |        789 | C           |     50 |
+--------+---------------------+------------+-------------+--------+
4 rows in set (0.00 sec)

As always, I hope this helps those trying to understand how CTEs can solve problems that would otherwise be coded in external imperative languages like Python.

Written by maclochlainn

April 1st, 2024 at 12:32 am

Ubuntu Pro Upgrade?

without comments

There wasn’t a choice when I chose to update the Ubuntu instance. I was compelled to upgrade to Ubuntu Pro. According to the upgrade I have five free installations. You can read more about Ubuntu Pro on this web page, and find their pricing schedule on this page.

Written by maclochlainn

March 13th, 2024 at 9:04 pm

MongoDB on Ubuntu

without comments

This post shows how to install, configure, and use MongoDB with JavaScript programs. You need to complete each section in the order provided (based on Cherry Server post).

Step #1: MongoDB Installation

Install the prerequisite packages with the following command:

sudo apt install -y software-properties-common gnupg apt-transport-https ca-certificates

Import the public key for MongoDB on your system using the curl command:

curl -fsSL https://pgp.mongodb.com/server-7.0.asc |  sudo gpg -o /usr/share/keyrings/mongodb-server-7.0.gpg --dearmor

Add MongoDB 7.0 APT repository to the /etc/apt/sources.list.d directory:

echo "deb [ arch=amd64,arm64 signed-by=/usr/share/keyrings/mongodb-server-7.0.gpg ] https://repo.mongodb.org/apt/ubuntu jammy/mongodb-org/7.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-7.0.list

Reload the local package index, which refreshes the local repositories and makes Ubuntu aware of the newly added MongoDB repository:

sudo apt update

Install the mongodb-org meta-package:

sudo apt install -y mongodb-org

Verify the installed version of MongoDB with this command:

mongod --version

It should display:

db version v7.0.6
Build Info: {
    "version": "7.0.6",
    "gitVersion": "66cdc1f28172cb33ff68263050d73d4ade73b9a4",
    "openSSLVersion": "OpenSSL 3.0.2 15 Mar 2022",
    "modules": [],
    "allocator": "tcmalloc",
    "environment": {
        "distmod": "ubuntu2204",
        "distarch": "x86_64",
        "target_arch": "x86_64"
    }
}

Step #2: Start MongoDB Service & Shell

You can verify that the installed mongodb is disabled after initial installation with this command:

sudo systemctl status mongod

It should display:

○ mongod.service - MongoDB Database Server
     Loaded: loaded (/lib/systemd/system/mongod.service; disabled; vendor preset: enabled)
     Active: inactive (dead)
       Docs: https://docs.mongodb.org/manual

Exit the output display from the systemctl utility by typing the escape key, a colon (:) and a q in sequence.

You can start the MongoDB service with this command:

sudo systemctl start mongod

Then, check the MongoDB service:

sudo systemctl status mongod

It displays:

● mongod.service - MongoDB Database Server
     Loaded: loaded (/lib/systemd/system/mongod.service; disabled; vendor preset: enabled)
     Active: active (running) since Thu 2024-03-07 16:38:17 MST; 2s ago
       Docs: https://docs.mongodb.org/manual
   Main PID: 33795 (mongod)
     Memory: 79.2M
        CPU: 706ms
     CGroup: /system.slice/mongod.service
             └─33795 /usr/bin/mongod --config /etc/mongod.conf
Mar 07 16:38:17 student-virtual-machine systemd[1]: Started MongoDB Database Server.
Mar 07 16:38:17 student-virtual-machine mongod[33795]: {"t":{"$date":"2024-03-07T23:38:17.642Z"},"s">

You can confirm that the database is up and running by checking if the server is listening on its default port, which is port 27017. Run the ss command to check the port number.

sudo ss -pnltu | grep 27017

It will display:

tcp   LISTEN 0      4096       127.0.0.1:27017      0.0.0.0:*    users:(("mongod",pid=33795,fd=14))

You can enable the mongodb service at startup with the following command:

sudo systemctl enable mongod

It raised the following error:

Created symlink /etc/systemd/system/multi-user.target.wants/mongod.service → /lib/systemd/system/mongod.service.

Now, start the MongoDB Shell (mongosh) by typing either the explicit or implicit MongoDB Shell command. The explicit one uses the port and database path, which are unnecessary when you’ve successfully started the mongosh service. (Please note that at the time of writing this blog post there is erroneous, or obsolete, content on the MongoDB Documentation Enable Access Control web page.

Explicit connection:

mongosh  --port 27017 --db /var/lib/mongodb --help

This version of the command will display most of the options available in MongoDB but it will suppress warning messages.

$ mongosh [options] [db address] [file names (ending in .js or .mongodb)]
 
  Options:
 
    -h, --help                                 Show this usage information
    -f, --file [arg]                           Load the specified mongosh script
        --host [arg]                           Server to connect to
        --port [arg]                           Port to connect to
        --build-info                           Show build information
        --version                              Show version information
        --quiet                                Silence output from the shell during the connection process
        --shell                                Run the shell after executing files
        --nodb                                 Don't connect to mongod on startup - no 'db address' [arg] expected
        --norc                                 Will not run the '.mongoshrc.js' file on start up
        --eval [arg]                           Evaluate javascript
        --json[=canonical|relaxed]             Print result of --eval as Extended JSON, including errors
        --retryWrites[=true|false]             Automatically retry write operations upon transient network errors (Default: true)
 
  Authentication Options:
 
    -u, --username [arg]                       Username for authentication
    -p, --password [arg]                       Password for authentication
        --authenticationDatabase [arg]         User source (defaults to dbname)
        --authenticationMechanism [arg]        Authentication mechanism
        --awsIamSessionToken [arg]             AWS IAM Temporary Session Token ID
        --gssapiServiceName [arg]              Service name to use when authenticating using GSSAPI/Kerberos
        --sspiHostnameCanonicalization [arg]   Specify the SSPI hostname canonicalization (none or forward, available on Windows)
        --sspiRealmOverride [arg]              Specify the SSPI server realm (available on Windows)
 
  TLS Options:
 
        --tls                                  Use TLS for all connections
        --tlsCertificateKeyFile [arg]          PEM certificate/key file for TLS
        --tlsCertificateKeyFilePassword [arg]  Password for key in PEM file for TLS
        --tlsCAFile [arg]                      Certificate Authority file for TLS
        --tlsAllowInvalidHostnames             Allow connections to servers with non-matching hostnames
        --tlsAllowInvalidCertificates          Allow connections to servers with invalid certificates
        --tlsCertificateSelector [arg]         TLS Certificate in system store (Windows and macOS only)
        --tlsCRLFile [arg]                     Specifies the .pem file that contains the Certificate Revocation List
        --tlsDisabledProtocols [arg]           Comma separated list of TLS protocols to disable [TLS1_0,TLS1_1,TLS1_2]
        --tlsUseSystemCA                       Load the operating system trusted certificate list
        --tlsFIPSMode                          Enable the system TLS library's FIPS mode
 
  API version options:
 
        --apiVersion [arg]                     Specifies the API version to connect with
        --apiStrict                            Use strict API version mode
        --apiDeprecationErrors                 Fail deprecated commands for the specified API version
 
  FLE Options:
 
        --awsAccessKeyId [arg]                 AWS Access Key for FLE Amazon KMS
        --awsSecretAccessKey [arg]             AWS Secret Key for FLE Amazon KMS
        --awsSessionToken [arg]                Optional AWS Session Token ID
        --keyVaultNamespace [arg]              database.collection to store encrypted FLE parameters
        --kmsURL [arg]                         Test parameter to override the URL of the KMS endpoint
 
  DB Address Examples:
 
        foo                                    Foo database on local machine
        192.168.0.5/foo                        Foo database on 192.168.0.5 machine
        192.168.0.5:9999/foo                   Foo database on 192.168.0.5 machine on port 9999
        mongodb://192.168.0.5:9999/foo         Connection string URI can also be used
 
  File Names:
 
        A list of files to run. Files must end in .js and will exit after unless --shell is specified.
 
  Examples:
 
        Start mongosh using 'ships' database on specified connection string:
        $ mongosh mongodb://192.168.0.5:9999/ships
 
  For more information on usage: https://docs.mongodb.com/mongodb-shell.

Implicit connection:

mongosh

You should see the following message with any warning messages:

Current Mongosh Log ID:	65ea502a97f4c1e2b7e12af4
Connecting to:		mongodb://127.0.0.1:27017/?directConnection=true&serverSelectionTimeoutMS=2000&appName=mongosh+2.1.5
Using MongoDB:		7.0.6
Using Mongosh:		2.1.5
 
For mongosh info see: https://docs.mongodb.com/mongodb-shell/
 
To help improve our products, anonymous usage data is collected and sent to MongoDB periodically (https://www.mongodb.com/legal/privacy-policy).
You can opt-out by running the disableTelemetry() command.
 
------
   The server generated these startup warnings when booting
   2024-03-07T16:38:17.818-07:00: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine. See http://dochub.mongodb.org/core/prodnotes-filesystem
   2024-03-07T16:38:18.350-07:00: Access control is not enabled for the database. Read and write access to data and configuration is unrestricted
   2024-03-07T16:38:18.350-07:00: vm.max_map_count is too low
------

You can run opt out of the data collection by running the disableTelemetry() command from the Linux command line. Use the following command (a broader explanation is in the MongoDB Telemetry documentation):

mongosh --nodb --eval "disableTelemetry()"

It should return:

Current Mongosh Log ID:	65eab2df3e663bde3711fa2f
Using Mongosh:		2.1.5
 
For mongosh info see: https://docs.mongodb.com/mongodb-shell/
 
Telemetry is now disabled.

You still have three warning messages to deal with at this point. You should fix the vm.max_map_count warning first. This is a Linux kernel issue. You can determine the current value of the vm.max_map_count value with this command:

cat /proc/sys/vm/max_map_count

It should return the system default value:

65530

You can change it at runtime with this command:

sudo sysctl -w vm.max_map_count=262144

However, you must restart the mongod service to see the change in the mongosh shell. There won’t be a warning message for the kernel parameter value being too low until you reboot your operating system. You can restart your mongod service with this command:

sudo service mongod restart

You can make a change to the /etc/sysctl.conf file to ensure the parameter is set to the correct value each time the system reboots. Simply add the following line as the root user or by using the sudo prefacing a text editor or your choice (like vim or nano) to your /etc/sysctl.conf file:

# Adding vm.max_map_count to sysctl.conf defaults.
vm.max_map_count=262144

At this point, you’ve eliminated two of the warning messages. The next step shows you how to enable Access Control. If you want to check the general server status, run the following command from the Linux Command-Line Interface (CLI):

mongosh --eval "db.serverStatus()" > server_status.log

You can inspect the log file, which should be slightly less than 2,000 lines of output with MongoDB a 7.0.6 installation. Using the command from the Linux CLI is generally the easiest way to inspect the output from the db.serverStatus() function, which is just too long to scroll from the console output.

Step #3: MongoDB Enabling Access Control

Connect to the mongosh …

Step #4: MongoDB Installing Node.js and React.js

Install Node.js with the following command:

sudo apt install -y nodejs

You can check the Node.js version with this command:

node -v

v12.22.9

Install the Node.js package manager npm with the following command:

sudo apt install -y npm

You can check the Node.js version with this command:

npm -v

8.5.1

As always, I hope this helps those looking for concise and complete free answer.

Written by maclochlainn

March 7th, 2024 at 11:10 pm

Parametric Queries

without comments

In 2021, I wrote a MySQL example for my class on the usefulness of Common Table Expressions (CTEs). When discussing the original post, I would comment on how you could extend the last example to build a parametric reporting table.

Somebody finally asked for a concrete example. So, this explains how to build a sample MySQL parametric query by leveraging a filter cross join and tests the parameter use with a Python script.

You can build this in any database you prefer but I used a studentdb database with the sakila sample database installed. I’ve granted privileges to both databases to the student user. The following SQL is required for the example:

-- Conditionally drop the levels table.
DROP TABLE IF EXISTS levels;
 
-- Create the levels list.
CREATE TABLE levels
( level_id       int unsigned primary key auto_increment
, parameter_set  enum('Three','Five')
, description    varchar(20)
, min_roles      int
, max_roles      int );
 
-- Insert values into the list table.
INSERT INTO levels
( parameter_set
, description
, min_roles
, max_roles )
VALUES
 ('Three','Hollywood Star', 30, 99999)
,('Three','Prolific Actor', 20, 29)
,('Three','Newcommer',1,19)
,('Five','Newcommer',1,9)
,('Five','Junior Actor',10,19)
,('Five','Professional Actor',20,29)
,('Five','Major Actor',30,39)
,('Five','Hollywood Star',40,99999);

The sample lets you use the three or five value labels while filtering on any partial full_name value as the result of the query below:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
-- Query the data.
WITH actors AS
 (SELECT   a.actor_id
  ,        a.first_name
  ,        a.last_name
  ,        COUNT(*) AS num_roles
  FROM     sakila.actor a INNER JOIN sakila.film_actor fa
  ON       a.actor_id = fa.actor_id
  GROUP BY actor_id)
SELECT   CONCAT(a.last_name,', ',a.first_name) full_name
,        l.description
,        a.num_roles
FROM     actors a CROSS JOIN levels l
WHERE    a.num_roles BETWEEN l.min_roles AND l.max_roles
AND      l.parameter_set = 'Five'
AND      a.last_name LIKE CONCAT('H','%')
ORDER BY a.last_name
,        a.first_name;

They extends a concept exercise found in Chapter 9 on subqueries in Alan Beaulieu’s Learning SQL book.

This is the parametric Python program, which embeds the function locally (to make it easier for those who don’t write a lot of Python). You could set the PYTHONPATH to a relative src directory and import your function if you prefer.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
#!/usr/bin/python
 
# Import the libraries.
import sys
import mysql.connector
from mysql.connector import errorcode
 
# ============================================================
 
# Define function to check and replace arguments.
def check_replace(argv):
 
  # Set defaults for incorrect parameter values.
  defaults = ("Three","_")
 
  # Declare empty list variables.
  inputs = []
  args = ()
 
  # Check whether or not parameters exist after file name.
  if isinstance(argv,list) and len(argv) != 0:
 
    # Check whether there are at least two parameters.
    if len(argv) >= 2:
 
      # Loop through available command-line arguments.
      for element in argv:
 
        # Check first of two parameter values and substitute
        # default value if input value is an invalid option.
        if len(inputs) == 0 and (element in ('Three','Five')) or \
           len(inputs) == 1 and (isinstance(element,str)):
          inputs.append(element)
        elif len(inputs) == 0:
          inputs.append(defaults[0])
        elif len(inputs) == 1:
          inputs.append(defaults[1])
 
      # Assign arguments to parameters.
      args = (inputs)
 
    # Check whether only one parameter value exists.
    elif len(argv) == 1 and (argv[0] in ('Three','Five')):
      args = (argv[0],"_")
 
    # Assume only one parameter is valid and substitute an 
    # empty string as the second parameter.
    else:
      args = (defaults[0],"_")
 
    # Substitute defaults when missing parameters.
  else:
    args = defaults
 
  # Return parameters as a tuple.
  return args
 
# ============================================================
 
# Assign command-line argument list to variable by removing
# the program file name.
# ============================================================
params = check_replace(sys.argv[1:])
# ============================================================
 
#  Attempt the query.
# ============================================================
#  Use a try-catch block to manage the connection.
# ============================================================
try:
  # Open connection.
  cnx = mysql.connector.connect(user='student', password='student',
                                host='127.0.0.1',
                                database='studentdb')
  # Create cursor.
  cursor = cnx.cursor()
 
  # Set the query statement.
  query = ("WITH actors AS "
           "(SELECT   a.first_name "
           " ,        a.last_name "
           " ,        COUNT(*) AS num_roles "
           " FROM     sakila.actor a INNER JOIN sakila.film_actor fa "
           " ON       a.actor_id = fa.actor_id "
           " GROUP BY a.first_name "
           " ,        a.last_name ) "
           " SELECT   CONCAT(a.last_name,', ',a.first_name) AS full_name "
           " ,        l.description "
           " ,        a.num_roles "
           " FROM     actors a CROSS JOIN levels l "
           " WHERE    a.num_roles BETWEEN l.min_roles AND l.max_roles "
           " AND      l.parameter_set = %s "
           " AND      a.last_name LIKE CONCAT(%s,'%') "
           " ORDER BY a.last_name "
           " ,        a.first_name")
 
  # Execute cursor.
  cursor.execute(query, params)
 
  # Display the rows returned by the query.
  for (full_name, description, num_roles) in cursor:
    print('{0} is a {1} with {2} films.'.format( full_name.title()
                                               , description.title()
                                               , num_roles))
 
  # Close cursor.
  cursor.close()
 
# ------------------------------------------------------------
# Handle exception and close connection.
except mysql.connector.Error as e:
  if e.errno == errorcode.ER_ACCESS_DENIED_ERROR:
    print("Something is wrong with your user name or password")
  elif e.errno == errorcode.ER_BAD_DB_ERROR:
    print("Database does not exist")
  else:
    print("Error code:", e.errno)        # error number
    print("SQLSTATE value:", e.sqlstate) # SQLSTATE value
    print("Error message:", e.msg)       # error message
 
# Close the connection when the try block completes.
else:
  cnx.close()

As always, I hope this helps those trying to understand how CTEs can solve problems that would otherwise be coded in external imperative languages like Python.

Written by maclochlainn

March 1st, 2024 at 12:30 am

Add PostGIS to PostgreSQL

without comments

The following blog post shows you how to add PostGIS and PgRouting to your existing install of PostgeSQL 14 on the Ubuntu Desktop, Version 22.0.4. This blog post relies on information in this earlier Install and Configure PostgreSQL on Ubuntu post. Generalized documentation on PostGIS exists at this URL.

You install the postgis libraries:

sudo apt install -y postgis

You install the postgresql-14-pgrouting libraries:

sudo apt install -y postgresql-14-pgrouting

You should also install ogr2ogr program, which is a command-line utility for converting data between GIS data formats, including common file formats and common spatial databases. You install the ogr2ogr libraries:

sudo apt install -y gdal-bin

Verify the installation by using the which utility, like

which -a ogr2ogr

It should return:

/usr/bin/ogr2ogr

You can qualify the installed PostGIS packages with the following command:

dpkg -l | grep -i postgis

It should display:

ii  postgis                                    3.2.0+dfsg-1ubuntu1                     amd64        Geographic objects support for PostgreSQL
ii  postgis-doc                                3.2.0+dfsg-1ubuntu1                     all          Geographic objects support for PostgreSQL -- documentation
ii  postgresql-14-pgrouting                    3.3.0-2                                 amd64        Routing functionality support for PostgreSQL/PostGIS
ii  postgresql-14-pgrouting-scripts            3.3.0-2                                 all          Routing functionality support for PostgreSQL/PostGIS - SQL scripts
ii  postgresql-14-postgis-3                    3.2.0+dfsg-1ubuntu1                     amd64        Geographic objects support for PostgreSQL 14
ii  postgresql-14-postgis-3-scripts            3.2.0+dfsg-1ubuntu1                     all          Geographic objects support for PostgreSQL 14 -- SQL scripts

Connect as the postgres user by becoming the root user with this command:

sudo sh

Then, assume the role of the postgres user with this command:

su - postgres

Connect to the PostgreSQL database as the privileged postgres owner/user:

psql postgres

You will see the following prompt after connecting to the PostgreSQL database:

postgres@student-virtual-machine:~$ psql
psql (14.10 (Ubuntu 14.10-0ubuntu0.22.04.1))
Type "help" for help.
 
postgres=#

As the the privileged postgres owner/user issue the following commands to create the gisdb database and set a new search path for it:

CREATE DATABASE gisdb;
ALTER DATABASE gisdb SET search_path=public,postgis,contrib,tiger;

Connect to the gisdb database:

\connect gisdb

You are now connected to database gisdb as the postgres user. You change to the postgis schema, and create the following extensions in this schema.

CREATE SCHEMA postgis;
CREATE EXTENSION postgis SCHEMA postgis;
CREATE EXTENSION postgis_raster SCHEMA postgis;
CREATE EXTENSION fuzzystrmatch SCHEMA postgis;
CREATE EXTENSION address_standardizer_data_us SCHEMA postgis;

You query the modified catalog with this query:

SELECT n.nspname AS "Name"
,      pg_catalog.pg_get_userbyid(n.nspowner) AS "Owner"
FROM   pg_catalog.pg_namespace n
WHERE  n.nspname !~ '^pg_'
AND n.nspname <> 'information_schema'
ORDER BY 1;

It should return:

  Name   |  Owner   
---------+----------
 postgis | postgres
 public  | postgres
(2 rows)

You must assign the postgis_tiger_geocoder and postgis_topology without a schema assignment. The
postgis_tiger_eeocoder must be assigned by default to the tiger schema, and the postgis_topology schema.

CREATE EXTENSION postgis_tiger_geocoder;
CREATE EXTENSION postgis_topology;

You reuse the same above referenced query to see the modified catalog with this query:

It should return:

    Name    |  Owner   
------------+----------
 postgis    | postgres
 public     | postgres
 tiger      | postgres
 tiger_data | postgres
 topology   | postgres
(5 rows)

Connect as the gisdb database with this command:

\connect gisdb

You can see the active PostGIS extension with this command:

\dx postgis

It shows:

                               List of installed extensions
  Name   | Version | Schema  |                        Description                         
---------+---------+---------+------------------------------------------------------------
 postgis | 3.2.0   | postgis | PostGIS geometry and geography spatial types and functions
(1 row)

Now, you can use this query:

SELECT postgis_full_version();

to discover what PostGIS version is installed:

                                                                       postgis_full_version                                                                        
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
 POSTGIS="3.2.0 c3e3cc0" [EXTENSION] PGSQL="140" GEOS="3.10.2-CAPI-1.16.0" PROJ="8.2.1" LIBXML="2.9.12" LIBJSON="0.15" LIBPROTOBUF="1.3.3" WAGYU="0.5.0 (Internal)"
(1 row)

Connect back to as the privileged postgres owner/user with this command:

\connect postgres

Next, check the available databases with this command:

\l

It should display the following:

                                  List of databases
   Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges   
-----------+----------+----------+-------------+-------------+-----------------------
 gisdb     | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
 videodb   | student  | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =Tc/student          +
           |          |          |             |             | student=CTc/student  +
           |          |          |             |             | dba=CTc/student
(5 rows)

As the privileged postgres owner/user make the following grants with these commands:

GRANT TEMPORARY, CONNECT ON DATABASE gisdb TO PUBLIC;
GRANT ALL PRIVILEGES ON DATABASE gisdb TO postgres;
GRANT ALL PRIVILEGES ON DATABASE gisdb TO dba;

After making the grants, check the available databases access with this \l command:

\l

It should display the following:

                                  List of databases
   Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges   
-----------+----------+----------+-------------+-------------+-----------------------
 gisdb     | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =Tc/postgres         +
           |          |          |             |             | postgres=CTc/postgres+
           |          |          |             |             | dba=CTc/postgres
 postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
 videodb   | student  | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =Tc/student          +
           |          |          |             |             | student=CTc/student  +
           |          |          |             |             | dba=CTc/student
(5 rows)

At this point, you can exit psql, the postgres user’s account, and the root user’s account. This should return you to your sudoer account, which in my case is the student user.

Connect to the gisdb with the following command-line interface command:

psql -U student -W -d gisdb

You can create the following, as per instructions in PostGIS Chapter 9 instructions:

-- Conditionally drop table.
DROP TABLE IF EXISTS geometries;
 
-- Create table with geometry column in table.
CREATE TABLE geometries
( geometries_id  INT
, name           VARCHAR
, geometry_obj   GEOMETRY);
 
 
INSERT INTO geometries
( name
, geometry_obj )
VALUES
 ('Point', 'POINT(0 0)')
,('Linestring', 'LINESTRING(0 0, 1 1, 2 1, 2 2)')
,('Polygon', 'POLYGON((0 0, 1 0, 1 1, 0 1, 0 0))')
,('PolygonWithHole', 'POLYGON((0 0, 10 0, 10 10, 0 10, 0 0),(1 1, 1 2, 2 2, 2 1, 1 1))')
,('Collection', 'GEOMETRYCOLLECTION(POINT(2 0),POLYGON((0 0, 1 0, 1 1, 0 1, 0 0)))');
 
SELECT name
,      ST_AsText(geometry_obj)
FROM   geometries;

Unfortunately, it raises the following error when attempting to create the geometries table:

psql:/home/student/Code/postgis/geometry.sql:7: ERROR:  type "geometry" does not exist
LINE 4: , geometry_obj   GEOMETRY);

As always, I hope the solutions presented helps move forward implementations of the technology. You can also find an excellent tutorial to learning PostGIS in the Introduction to PostGIS tutorial.

Written by maclochlainn

February 15th, 2024 at 10:57 pm

Sqlite on Ubuntu

without comments

We decided to include some existing Sqlite databases in our AWS Ubuntu learning lab because they’re used by the Data Science courses (specifically, DS 250). Installing Sqlite is quite simple:

sudo apt install -y sqlite

You can check the install by using the which utility, like:

which -a sqlite3

On Ubuntu, it should return:

/usr/bin/sqlite3

There is a friendly help document online that can provide insight in how to use Sqlite. You can create a new student.db database with the following syntax from the Ubuntu CLI (Command-Line Interface):

sqlite3 student.db

It would return the following:

SQLite version 3.37.2 2022-01-06 13:25:41
Enter ".help" FOR usage hints.
sqlite>

A little warning about the simple example and how it opens only a transient in-memory database. If you want a persistent database, you must open sqlite3 without any arguments. Then, you must use the .open method to open a persistent student.db database create a file in the relative directory path where you launched sqlite3 executable. You can read more about persistent in Sqlite in the documentation.

.open student.db

The alternative opens the student.db file in a fully qualified path:

.open /home/student/Code/sqlite/db/student.db

If you type .databases at the sqlite> prompt it would return:

sqlite> .databases
main: /home/student/Code/sqlite/db/student.db r/w

Let’s create a script file that creates two tables, a foreign key reference from one of the tables to the other, and some data with the following create_sample.sql script:

-- Drop knight table if exists.
DROP TABLE IF EXISTS knight;
 
-- Drop kingdom table if exists.
DROP TABLE IF EXISTS kingdom;
 
-- Create normalized table kingdom
CREATE TABLE kingdom
( kingdom_id    INTEGER PRIMARY KEY
, kingdom_name  VARCHAR(20)
, population    INTEGER
, book          VARCHAR(40));
 
-- Insert kingdom into table.
INSERT INTO kingdom
( kingdom_id
, kingdom_name
, population
, book )
VALUES
 ( 1, 'Narnia', 42100, 'Prince Caspian' )
,( 2, 'Narnia', 77600, 'The Lion, The Witch and The Wardrobe' )
,( 3, 'Camelot', 15200, 'The Once and Future King' );
 
-- Create normalized knight table.
CREATE TABLE knight
( knight_id             INTEGER PRIMARY KEY
, knight_name           VARCHAR(22)
, kingdom_allegiance_id INTEGER
, allegiance_start_date text
, allegiance_end_date   text
, book                  VARCHAR(40)
, FOREIGN KEY (kingdom_allegiance_id)
  REFERENCES kingdom(kingdom_id));
 
-- Insert knights into table.  
INSERT INTO knight
( knight_id
, knight_name
, kingdom_allegiance_id
, allegiance_start_date
, allegiance_end_date
, book )
VALUES
 (  1, 'Peter the Magnificent', 2, '1272-03-20', '1292-06-19', 'The Lion, The Witch and The Wardrobe' )
,(  2, 'Edmund the Just', 2, '1272-03-20', '1292-06-19', 'The Lion, The Witch and The Wardrobe' )
,(  3, 'Susan the Gentle', 2, '1272-03-20', '1292-06-19', 'The Lion, The Witch and The Wardrobe' )
,(  4, 'Lucy the Valiant', 2, '1272-03-20', '1292-06-19', 'The Lion, The Witch and The Wardrobe' )
,(  5, 'Peter the Magnificent',	1, '1531-04-12', '1328-05-31', 'Prince Caspian' )
,(  6, 'Edmund the Just', 1, '1531-04-12', '1328-05-31', 'Prince Caspian' )
,(  7, 'Susan the Gentle', 1, '1531-04-12', '1328-05-31', 'Prince Caspian' )
,(  8, 'Lucy the Valiant', 1, '1531-04-12', '1328-05-31', 'Prince Caspian' )
,(  9, 'King Arthur', 3, '0631-03-10', '0686-12-12', 'The Once and Future King' )
,( 10, 'Sir Lionel', 3, '0631-03-10', '0686-12-12', 'The Once and Future King' )
,( 11, 'Sir Bors', 3, '0631-03-10', '0686-12-12', 'The Once and Future King' )
,( 12, 'Sir Bors', 3, '0631-03-10', '0686-12-12', 'The Once and Future King' )
,( 13, 'Sir Galahad', 3, '0631-03-10', '0686-12-12', 'The Once and Future King' )
,( 14, 'Sir Gawain', 3, '0631-03-10', '0686-12-12', 'The Once and Future King' )
,( 15, 'Sir Tristram', 3, '0631-03-10', '0686-12-12', 'The Once and Future King' )
,( 16, 'Sir Percival', 3, '0631-03-10', '0686-12-12', 'The Once and Future King' )
,( 17, 'Sir Lancelot', 3, '0631-03-10', '0686-12-12', 'The Once and Future King' );

You can run the create_sample.sql script with the following syntax using an absolute path:

sqlite> .read /home/student/Code/sqlite/create_tables.sql

Then, you can write a query like this to retrieve the data from two tables:

SELECT k.kingdom_name
,      kn.knight_name
FROM   kingdom k INNER JOIN knight kn
ON     k.kingdom_id = kn.kingdom_allegiance_id
WHERE  k.book = 'Prince Caspian';

It will return the following:

Narnia|Peter the Magnificent
Narnia|Edmund the Just
Narnia|Susan the Gentle
Narnia|Lucy the Valiant

You can exit sqlite3 by entering .quit or Control+D (the system End-Of-File character). If you can determine whether you have a transient or persistent student.db database file with the long list (ll) command.

The following command:

ll /home/student/Code/sqlite/db/student.db

should return the following:

-rw-r--r-- 1 student student 12288 Feb  9 23:01 /home/student/Code/sqlite/db/student.db

If the command returned a 0 sized student.db database file, you created a transient Sqlite table. You’ll need to redo the creation of the student.db database file with the .open command as qualified above.

If you want to detach a database from your active Sqlite session, you can issue the following command to remove it:

sqlite> DETACH DATABASE student.db

Let’s jazz it up a bit with some Python. The first example verifies the ODBC driver’s ability to connect to Sqlite. Please note that it returns the same result for a transient and persistent database file. You can refer to the following documentation for Python examples.

#!/usr/bin/python
 
# Import sqlite3 ODBC library.
import sqlite3
 
try:
  # Open a connection to the student.db database
  db = sqlite3.connect('/home/student/Code/sqlite/db/student.db')
 
  # Print a string to say you've connected to the student.db database.
  print("Sqlite database connection success.")
 
except sqlite3.Error as e:
  print('SQLite error: %s' % (' '.join(e.args)))
  print("Exception class is: ", e.__class__)
  print('SQLite traceback: ')
  exc_type, exc_value, exc_tb = sys.exc_info()
  print(traceback.format_exception(exc_type, exc_value, exc_tb))
  sys.exit(1)
finally:
  # Close the connection when it is open.
  if db:
    db.close()

You can run the sqlite_connection.py script with the following syntax from its local directory:

Sqlite database connection success.

Assuming you have created a persistent Sqlite database, as qualified above with the .open command and fully qualified file name. A fully qualified file name as a path from a Linux mount point to the file.

You must use the fully qualified file name for a persistent Sqlite student.db database as the database parameter for the sqlite3.connect() method, as shown on line #9 of the sqlite_query.py program below.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
#!/usr/bin/python
 
# Import sys library.
import sqlite3
 
try:
  # Open a connection to a persistent database, which should use
  # a fully qualified file name, but may use a relative file
  # name when the Python code is in the same directory as a 
  # persistent student.db sqlite3 database.
  db = sqlite3.connect('/home/student/Code/sqlite/db/student.db')
 
  # Create a cursor.
  cursor = db.cursor()
 
  # Define a query.
  query = "SELECT k.kingdom_name "                           \
          ",      kn.knight_name "                           \
          "FROM   kingdom k INNER JOIN knight kn "           \
          "ON     k.kingdom_id = kn.kingdom_allegiance_id "  \
          "WHERE  k.book = 'Prince Caspian'"
 
  # Execute the cursor with the query.
  cursor.execute( query )
 
  # Display the rows returned by the query.
  for (kingdom_name, knight_name) in cursor:
    print('{0} has {1}'.format( kingdom_name.title(), knight_name.title()))
 
except sqlite3.Error as e:
  print('SQLite error: %s' % (' '.join(e.args)))
  print("Exception class is: ", e.__class__)
  print('SQLite traceback: ')
  exc_type, exc_value, exc_tb = sys.exc_info()
  print(traceback.format_exception(exc_type, exc_value, exc_tb))
  sys.exit(1)
finally:
  # Close the connection when it is open.
  if db:
    db.close()

As always, I hope this helps those trying to get up and running with Sqlite.

Written by maclochlainn

February 9th, 2024 at 8:47 pm

Git Hub Desktop on Ubuntu

without comments

I need to install Git Hub Desktop on Ubuntu. It was quite straightforward:

bash -c "$(curl -fsSL https://raw.githubusercontent.com/kontr0x/github-desktop-install/main/installGitHubDesktop.sh)"

You can find it in the 9-dot menu’s second page or launch it from the command line, like:

github-desktop

Either will launch the GitHub Desktop, as shown below:


As always, I hope this helps those looking for concise and complete free answer on installing GitHub Desktop.

Written by maclochlainn

February 9th, 2024 at 6:40 pm

Posted in github,Linux,Ubuntu

Tagged with

Ruby+PostgreSQL on Ubuntu

without comments

This extends the earlier post on installing and configuring Ruby 3.3.0 on Ubuntu 22.0.4. Please refer to that earlier post to install Ruby. This post shows you how to install the necessary libraries and Ruby Gems for PostgreSQL.

You need to install the libra-dev package, as shown:

sudo apt install postgresql libpq-dev

Next, you need to install the PG Gem:

gem install pg

You can now write a postgres_version.rb program to verify a connection to the PostgreSQL database, like:

# Include Ruby Gem libraries.
require 'rubygems'
require 'pg'
 
# Begin block.
begin
  # Create a new connection resource.
  db = PG::connect( 'localhost', 5432, '', '', 'videodb', 'student', 'student')
 
  # Create a result set.
  stmt = db.query('SELECT version() AS version')
 
  # Read through the result set hash.
  stmt.each do | row |
    puts "#{row['version']}"
  end
  # Release the result set resources.
  stmt.freeze
 
rescue PG::Error => e
  # Print the error.
  puts "ERROR #{e.error} (#{e.sqlstate})"
  puts "Can't connect to the PostgreSQL database specified."
  # Signal an error.
  exit 1
 
ensure
  # Close the connection when it is open.
  db.close if db
end

Call the postgres_version.rb program with this syntax:

ruby mysql_version.rb

It should return:

PostgreSQL 14.10 (Ubuntu 14.10-0ubuntu0.22.04.1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, 64-bit

The postgres_columns.rb script returns a couple columns concatenated into a single column:

# Include Ruby Gem libraries.
require 'rubygems'
require 'pg'
 
# Begin block.
begin
  # Create a new connection resource.
  db = PG::connect( 'localhost', 5432, '', '', 'videodb', 'student', 'student')
 
  # Create a result set.
  stmt = db.query("SELECT   CONCAT(nh.last_name, ', ', nh.first_name) AS name " + \
                  "FROM     new_hire nh " +                                       \
                  "ORDER BY nh.last_name")
 
  # Read through the result set hash.
  stmt.each do | row |
    out = ""
    i = 0
    while i < stmt.fields.count()
      # Check when not last column and use the:
      #   - Hash returned by the result set for the value, and
      #   - String array value returned by the statement object
      #     as the name value of the hash by leveraging its 
      #     numeric index.
      if i < stmt.fields.count() - 1
        out += "#{row[stmt.fields[i]]}"
        out += ", "
      else
        out += "#{row[stmt.fields[i]]}"
      end
      i += 1
    end
    puts "#{out}"
  end
 
  # Release the result set resources.
  stmt.freeze
 
rescue PG::Error => e
  # Print the error.
  puts "ERROR #{e.error} (#{e.sqlstate})"
  puts "Can't connect to PostgreSQL database specified."
  # Signal an error.
  exit 1
 
ensure
  # Close the connection when it is open.
  db.close if db
end

Call the postgres_columns.rb program with this syntax:

ruby mysql_columns.rb

It should return:

Chabot, Henry
Lewis, Malcolm

As always, I hope this helps those looking to learn and solve a problem. You can find the PG Gem documentation here.

Written by maclochlainn

February 7th, 2024 at 12:15 am

VSCode Package Error

without comments

While running an update on Ubuntu 22.0.4 with the following syntax I got an error on finding the VSCode Package. I ran this to update before adding Ruby and Rails to an Ubuntu virtual machine instance.

sudo apt-get update

I manually moved the vscode.list file to my student user’s home directory and removed the file from /etc/apt/sources.list.d directory. This allowed me to update all other packages.

Don’t forget to replace the vscode.list file in the /etc/apt/sources.list.d directory.

Written by maclochlainn

February 5th, 2024 at 1:06 pm

Posted in Linux,Unix,VSCode

Tagged with