Archive for the ‘Unix’ Category
Postgres 11 Video DB
Installing PostgreSQL 11 on Fedora, Version 30, requires an update to my previous instructions to create a sandboxed user. A sandboxed user can only access a non-data dictionary database with a password. In the real world, rather than a personal test instance you would configure users to include aspects of networking. However, this post is only showing you how to connect from the local server.
This post builds on my PostgreSQL Installation blog post and shows you how to create a tablespace, database, role, and user. It also shows you how to change the default configuration for how users connect to the database.
The following steps create a tablespace, database, role, and user:
- Create tablespace
The directory for the data dictionary changes with PostgreSQL has changed. You can find it with the following command:
postgres=# show data_directory; |
This will return the following:
data_directory ------------------------ /var/lib/pgsql/11/data (1 row) |
You need to create a physical videoDB subdirectory in the /var/lib/pgsql/11 directory. You can use the following syntax from the /var/lib/pgsql/11 directory to create the videoDB subdirectory:
mkdir videoDB |
- Create Tablespace
You can create a video_db tablespace with the following syntax:
CREATE TABLESPACE video_db OWNER postgres LOCATION '/var/lib/pgsql/11/videoDB'; |
This will return the following:
CREATE TABLESPACE |
You can query whether you successfully create the video_db tablespace with the following:
SELECT * FROM pg_tablespace; |
It should return the following:
spcname | spcowner | spcacl | spcoptions ------------+----------+--------+------------ pg_default | 10 | | pg_global | 10 | | video_db | 10 | | (3 rows) |
- Create a Database
You can create a videodb database with the following syntax:
CREATE DATABASE videodb WITH OWNER = postgres ENCODING = 'UTF8' TABLESPACE = video_db LC_COLLATE = 'en_US.UTF-8' LC_CTYPE = 'en_US.UTF-8' CONNECTION LIMIT = -1; |
You can verify the creation of the videodb with the following command:
postgres# \l |
It should show you a display like the following:
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+----------+----------+-------------+-------------+-----------------------
postgres | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
template0 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
videodb | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =Tc/postgres +
| | | | | postgres=CTc/postgres+
| | | | | dba=CTc/postgres
(4 rows) |
Then, you can assign comment to the database with the following syntax:
COMMENT ON DATABASE videodb IS 'Video Database'; |
- Create a Role, Grant, and User
In this section you create a dba role, grant privileges on a videodb database to a role, and create a user with the role that you created previously with the following three statements. There are three steps in this sections.
- The first step creates a
dbarole:CREATE ROLE dba WITH SUPERUSER;
- The second step grants all privileges on a
videodbdatabase to adbarole:GRANT ALL PRIVILEGES ON DATABASE videodb TO dba;
- The third step creates a
studentuser with thedbarole:CREATE USER student WITH ROLE dba ENCRYPTED PASSWORD 'student';
- It is possible that you may (and should if this is a new instance you are building) encounter an error when you try to connect as a sandboxed user. The syntax to connect as the student user is:
psql -d videodb -U student -W
You may encounter this error:
psql: FATAL: Peer authentication failed for user "student"
You can fix this in PostgreSQL 11 by changing the user access parameters in the
pg_hba.confconfiguration file. The file is found in the/var/lib/pgsql/11/datadirectory and you need to edit it as therootorpostgresuser.The default entry is:
# TYPE DATABASE USER ADDRESS METHOD # "local" is for Unix domain socket connections only local all all peer
You should replace it with the following:
# TYPE DATABASE USER ADDRESS METHOD # "local" is for Unix domain socket connections only local postgres all peer local videodb student md5
After you save those changes in the
pg_hba.conffile, you need to restart the PostgreSQL (postgresql-11) service. You can do that with the following command as therootuser:service postgresql-11 restart
- Connect to the
videodbas thestudentuser
Once the postgresql-11 service is restarted, you can connect with the sandboxed student user with this syntax:
psql -d videodb -U student -W |
If you did everything correctly, you should see the following after correctly providing the student password for the student user:
psql (11.4, server 11.5) Type "help" for help. videodb=> |
After connecting to the videodb database, you can query the current database, like
SELECT current_database(); |
It should return the following:
current_database ------------------ videodb (1 row) |
This has shown you how to create a videodb database, dba role, and student user.
MongoDB on Fedora 30
I found that MongoDB instructions changed significantly from a year ago. More or less, there was no pre-configured yum repository. Fortunately, the following web page was very helpful though not specific about Fedora.
Installing MongoDB 4.0 on Fedora 30 requires updating the yum repository. You need to create the mongodb.repo file as the root user in the /etc/yum.repos.d directory. The mongodb.repo file should contain the following:
[Mongodb] name=MongoDB Repository baseurl=https://repo.mongodb.org/yum/amazon/2013.03/mongodb-org/4.0/x86_64/ gpgcheck=1 enabled=1 gpgkey=https://www.mongodb.org/static/pgp/server-4.0.asc |
After you create the appropriate mongodb.repo file, you can run the following command to install the MongoDB database.
def install mongodb-org |
It will produce the following log:
MongoDB Repository 31 kB/s | 21 kB 00:00 Dependencies resolved. ============================================================================= Package Arch Version Repository Size ============================================================================= Installing: mongodb-org x86_64 4.0.12-1.amzn1 Mongodb 5.8 k Installing dependencies: mongodb-org-mongos x86_64 4.0.12-1.amzn1 Mongodb 11 M mongodb-org-server x86_64 4.0.12-1.amzn1 Mongodb 20 M mongodb-org-shell x86_64 4.0.12-1.amzn1 Mongodb 13 M mongodb-org-tools x86_64 4.0.12-1.amzn1 Mongodb 29 M Transaction Summary ============================================================================= Install 5 Packages Total download size: 73 M Installed size: 213 M Is this ok [y/N]: y Downloading Packages: (1/5): mongodb-org-4.0.12-1.amzn1.x86_64.rpm 18 kB/s | 5.8 kB 00:00 (2/5): mongodb-org-shell-4.0.12-1.amzn1.x86_ 4.6 MB/s | 13 MB 00:02 (3/5): mongodb-org-mongos-4.0.12-1.amzn1.x86 1.8 MB/s | 11 MB 00:06 (4/5): mongodb-org-tools-4.0.12-1.amzn1.x86_ 6.6 MB/s | 29 MB 00:04 (5/5): mongodb-org-server-4.0.12-1.amzn1.x86 2.5 MB/s | 20 MB 00:08 ----------------------------------------------------------------------------- Total 9.0 MB/s | 73 MB 00:08 warning: /var/cache/dnf/Mongodb-f722cd88d61a4e38/packages/mongodb-org-4.0.12-1.amzn1.x86_64.rpm: Header V3 RSA/SHA1 Signature, key ID e52529d4: NOKEY MongoDB Repository 3.0 kB/s | 1.7 kB 00:00 Importing GPG key 0xE52529D4: Userid : "MongoDB 4.0 Release Signing Key <packaging@mongodb.com>" Fingerprint: 9DA3 1620 334B D75D 9DCB 49F3 6881 8C72 E525 29D4 From : https://www.mongodb.org/static/pgp/server-4.0.asc Is this ok [y/N]: y Key imported successfully Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Installing : mongodb-org-tools-4.0.12-1.amzn1.x86_64 1/5 Installing : mongodb-org-shell-4.0.12-1.amzn1.x86_64 2/5 Running scriptlet: mongodb-org-server-4.0.12-1.amzn1.x86_64 3/5 Installing : mongodb-org-server-4.0.12-1.amzn1.x86_64 3/5 Running scriptlet: mongodb-org-server-4.0.12-1.amzn1.x86_64 3/5 Installing : mongodb-org-mongos-4.0.12-1.amzn1.x86_64 4/5 Installing : mongodb-org-4.0.12-1.amzn1.x86_64 5/5 Running scriptlet: mongodb-org-4.0.12-1.amzn1.x86_64 5/5 Verifying : mongodb-org-4.0.12-1.amzn1.x86_64 1/5 Verifying : mongodb-org-mongos-4.0.12-1.amzn1.x86_64 2/5 Verifying : mongodb-org-server-4.0.12-1.amzn1.x86_64 3/5 Verifying : mongodb-org-shell-4.0.12-1.amzn1.x86_64 4/5 Verifying : mongodb-org-tools-4.0.12-1.amzn1.x86_64 5/5 Installed: mongodb-org-4.0.12-1.amzn1.x86_64 mongodb-org-mongos-4.0.12-1.amzn1.x86_64 mongodb-org-server-4.0.12-1.amzn1.x86_64 mongodb-org-shell-4.0.12-1.amzn1.x86_64 mongodb-org-tools-4.0.12-1.amzn1.x86_64 Complete! |
You create a MongoDB service with the following syntax as a privileged user in the sudoer list:
sudo systemctl enable mongodb.service |
You can then start the mongod service with the following command as a privileged user in the sudoer list:
sudo systemctl start mongod.service |
You confirm that it started with the following command as the same privileged user:
sudo service mongod status |
It should produce a log file like this:
● mongod.service - SYSV: Mongo is a scalable, document-oriented database. Loaded: loaded (/etc/rc.d/init.d/mongod; generated) Active: active (running) since Fri 2019-08-16 14:57:22 MDT; 2min 57s ago Docs: man:systemd-sysv-generator(8) Process: 128115 ExecStart=/etc/rc.d/init.d/mongod start (code=exited, stat> Tasks: 27 (limit: 4661) Memory: 47.0M CGroup: /system.slice/mongod.service └─128131 /usr/bin/mongod -f /etc/mongod.conf Aug 16 14:57:21 localhost.localdomain systemd[1]: Starting SYSV: Mongo is a > Aug 16 14:57:21 localhost.localdomain runuser[128127]: pam_unix(runuser:sess> Aug 16 14:57:22 localhost.localdomain runuser[128127]: pam_unix(runuser:sess> Aug 16 14:57:22 localhost.localdomain mongod[128115]: [30B blob data] Aug 16 14:57:22 localhost.localdomain systemd[1]: Started SYSV: Mongo is a s> lines 1-15/15 (END) |
You close the service log with a “q“. You can determine your version with the following command:
mongod --version |
It should show you something like this:
db version v4.0.12 git version: 5776e3cbf9e7afe86e6b29e22520ffb6766e95d4 OpenSSL version: OpenSSL 1.0.0-fips 29 Mar 2010 allocator: tcmalloc modules: none build environment: distmod: amazon distarch: x86_64 target_arch: x86_64 |
You can connect to the MongoDB shell with the following command:
mongo |
Inside the MongoDB shell, you can run standard MongoDB commands, like:
> use mydb; switched to db mydb > db.version() 4.0.12 > db.stats() { "db" : "mydb", "collections" : 0, "views" : 0, "objects" : 0, "avgObjSize" : 0, "dataSize" : 0, "storageSize" : 0, "numExtents" : 0, "indexes" : 0, "indexSize" : 0, "fileSize" : 0, "fsUsedSize" : 0, "fsTotalSize" : 0, "ok" : 1 } > quit() |
As always, I hope this helps those looking for the missing steps.
Apache on Fedora 30
There was an option during the Fedora 30 Workstation installation to add the Apache Web Server, but you need to set it to start automatically. Unfortunately, there was no option to install PHP, which I thought odd because of how many web developers learn the trade first on PHP with a LAMP (Linux, Apache, MySQL, Perl/PHP/Python) stack. You see how to fix that shortcoming in this post and how to install and test PHP, mysqli, and pdo to support MySQL 8.
Before you do that make sure you install MySQL 8. You can find my prior blog post on that here.
You set Apache to start automatically, on the next boot of the operating system, with the following command:
chkconfig httpd on |
It creates a symbolic link:
Created symlink /etc/systemd/system/multi-user.target.wants/httpd.service → /usr/lib/systemd/system/httpd.service. |
However, that command only starts the Apache server the next time you boot the server. You use the following command as the root user to start the Apache server:
apachectl start |
You can verify the installation with the following command as the root user:
ps -ef | grep httpd | grep -v grep |
It should return:
root 5433 1 0 17:03 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND apache 5434 5433 0 17:03 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND apache 5435 5433 0 17:03 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND apache 5436 5433 0 17:03 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND apache 5437 5433 0 17:03 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND apache 5438 5433 0 17:03 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND apache 5442 5433 0 17:03 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND |
and, then verify the listening port with the following command as the root user:
netstat -tulpn | grep :80 |
It should return the following when both the Apache server is listening on port 80 and the Oracle multi-protocol server is listening on port 8080:
tcp6 0 0 :::80 :::* LISTEN 119810/httpd tcp6 0 0 :::8080 :::* LISTEN 1403/tnslsnr |
You can also enter the following URL in the browser to see the Apache Test Page:
http://localhost |
It should display the test page, like this:
You can also create a hello.htm file in the /var/www/html directory to test the ability to read an HTML file. I would suggest the traditional hello.htm file:
<html> <body> Hello World! </body> </html> |
You can call it by using this URL in the browser:
http://localhost/hello.htm |
It should display the test page, like this:
Now, let’s install PHP. You use the following command as a privileged user, which is one found in the sudoer’s list:
yum install -y php |
Display detailed console log →
Last metadata expiration check: 0:37:02 ago on Fri 16 Aug 2019 11:03:54 AM MDT. Dependencies resolved. ============================================================================= Package Arch Version Repository Size ============================================================================= Installing: php x86_64 7.3.8-1.fc30 updates 2.8 M Installing dependencies: nginx-filesystem noarch 1:1.16.0-3.fc30 updates 11 k php-cli x86_64 7.3.8-1.fc30 updates 4.3 M php-common x86_64 7.3.8-1.fc30 updates 1.1 M Installing weak dependencies: php-fpm x86_64 7.3.8-1.fc30 updates 1.5 M Transaction Summary ============================================================================= Install 5 Packages Total download size: 9.6 M Installed size: 43 M Downloading Packages: (1/5): nginx-filesystem-1.16.0-3.fc30.noarch 34 kB/s | 11 kB 00:00 (2/5): php-common-7.3.8-1.fc30.x86_64.rpm 1.1 MB/s | 1.1 MB 00:00 (3/5): php-7.3.8-1.fc30.x86_64.rpm 2.0 MB/s | 2.8 MB 00:01 (4/5): php-fpm-7.3.8-1.fc30.x86_64.rpm 2.2 MB/s | 1.5 MB 00:00 (5/5): php-cli-7.3.8-1.fc30.x86_64.rpm 1.7 MB/s | 4.3 MB 00:02 ----------------------------------------------------------------------------- Total 3.0 MB/s | 9.6 MB 00:03 Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Installing : php-common-7.3.8-1.fc30.x86_64 1/5 Installing : php-cli-7.3.8-1.fc30.x86_64 2/5 Running scriptlet: nginx-filesystem-1:1.16.0-3.fc30.noarch 3/5 Installing : nginx-filesystem-1:1.16.0-3.fc30.noarch 3/5 Installing : php-fpm-7.3.8-1.fc30.x86_64 4/5 Running scriptlet: php-fpm-7.3.8-1.fc30.x86_64 4/5 Installing : php-7.3.8-1.fc30.x86_64 5/5 Running scriptlet: php-7.3.8-1.fc30.x86_64 5/5 Running scriptlet: php-fpm-7.3.8-1.fc30.x86_64 5/5 Verifying : nginx-filesystem-1:1.16.0-3.fc30.noarch 1/5 Verifying : php-7.3.8-1.fc30.x86_64 2/5 Verifying : php-cli-7.3.8-1.fc30.x86_64 3/5 Verifying : php-common-7.3.8-1.fc30.x86_64 4/5 Verifying : php-fpm-7.3.8-1.fc30.x86_64 5/5 Installed: php-7.3.8-1.fc30.x86_64 php-fpm-7.3.8-1.fc30.x86_64 nginx-filesystem-1:1.16.0-3.fc30.noarch php-cli-7.3.8-1.fc30.x86_64 php-common-7.3.8-1.fc30.x86_64 Complete! |
Before you test the installation of PHP in a browser, you must restart the Apache HTTP Server. You can do that with the following command as a privileged user:
sudo apachectl restart |
After verifying the connection, you can test it by creating the traditional info.php program file in the /var/www/http directory. The file should contain the following:
1 2 3 | <?php phpinfo(); ?> |
It should display the PHP Version 7.3.8 web page, which ships with Fedora 30:
The next step shows you how to install mysqli and pdo with the yum utility. While it’s unnecessary to check for the older mysql library (truly deprecated), its good practice to know how to check for a conflicting library before installing a new one. Also, I’d prefer newbies get exposed to using the yum utility’s shell environment.
You start the yum shell, as follows:
yum shell |
With the yum shell, you would remove a mysql package with the following command:
> remove php-mysql |
The command will remove the package or tell you that there is no package to remove. Next, you install the php-mysqli package with this command:
install php-mysqli |
You will then be prompted to confirm the installation of the php-mysqli library. Finally, you exit the yum shell with this command:
> quit |
If you want to see the whole interactive shell, click on the link below.
Display detailed console log →
Last metadata expiration check: 0:53:05 ago on Fri 16 Aug 2019 11:03:54 AM MDT. > remove php-mysql No match for argument: php-mysql No packages marked for removal. > install php-mysqlnd > run ============================================================================= Package Architecture Version Repository Size ============================================================================= Installing: php-mysqlnd x86_64 7.3.8-1.fc30 updates 195 k Installing dependencies: php-pdo x86_64 7.3.8-1.fc30 updates 91 k Transaction Summary ============================================================================= Install 2 Packages Total download size: 286 k Installed size: 1.4 M Is this ok [y/N]: y Downloading Packages: (1/2): php-pdo-7.3.8-1.fc30.x86_64.rpm 136 kB/s | 91 kB 00:00 (2/2): php-mysqlnd-7.3.8-1.fc30.x86_64.rpm 183 kB/s | 195 kB 00:01 ----------------------------------------------------------------------------- Total 24 kB/s | 286 kB 00:11 Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Installing : php-pdo-7.3.8-1.fc30.x86_64 1/2 Installing : php-mysqlnd-7.3.8-1.fc30.x86_64 2/2 Running scriptlet: php-mysqlnd-7.3.8-1.fc30.x86_64 2/2 Verifying : php-mysqlnd-7.3.8-1.fc30.x86_64 1/2 Verifying : php-pdo-7.3.8-1.fc30.x86_64 2/2 Installed: php-mysqlnd-7.3.8-1.fc30.x86_64 php-pdo-7.3.8-1.fc30.x86_64 Last metadata expiration check: 0:53:54 ago on Fri 16 Aug 2019 11:03:54 AM MDT. > quit Leaving Shell The downloaded packages were saved in cache until the next successful transaction. You can remove cached packages by executing 'dnf clean packages'. |
You need to restart the Apache HTTP listener for these changes to take place, which you do with the same command as shown earlier:
sudo apachectl restart |
I wrote the mysqli_check.php script to verify installation of both the mysqli and pdo libraries. The full code should be put in a mysqli_check.php file in the /var/www/html directory for testing.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | <html>
<header>
<title>Static Query Object Sample</title>
<style type="text/css">
/* HTML element styles. */
table {background:white;border-style:solid;border-width:3px;border-color:black;border-collapse:collapse;}
th {text-align:center;font-style:bold;background:lightgray;border:solid 1px gray;}
td {border:solid 1px gray;}
/* Class tag element styles. */
.ID {min-width:50px;text-align:right;}
.Label {min-width:200px;text-align:left;}
</style>
</header>
<body>
<?php
if (!function_exists('mysqli_init') && !extension_loaded('mysqli')) {
print 'mysqli not installed.'; }
else {
print 'mysqli installed.'; }
if (!function_exists('pdo_init') && !extension_loaded('pdo')) {
print '<p>pdo not installed.</p>'; }
else {
print '<p>pdo installed.</p>'; }
?>
</script>
</body>
</html> |
You can test it with the following URL from the local browser:
http://localhost/mysqli_check.php |
It should print the following to the web page when you’ve successfully install the mysqli and pdo libraries:
mysqli installed. pdo installed. |
If you plan to use PHP to display and render graphics, you need to install php-gd library. You can do that with the yum utility and this prior blog post explains it. Don’t forget to restart the Apache HTTP Server after you add the php-gd library.
For example, one of my sample PHP programs loads a PNG image into a BLOB column as raw binary text. Then, the program reads it and renders it with PHP to produce the following web page.
As always, I hope this helps those looking for a complete solution without cost.
Oracle Error Bash f(x)
My students always struggle initially with basic Linux skills. I wrote little function for their .bashrc file to help them avoid the frustration. It finds and displays all errors by file name, line number and error message for a collection of log files in a single directory (or folder).
errors() { # Determine if any log files exist and check for errors. label="File Name:Line Number:Error Code" list=`ls ./*.$1 | wc -l` if [[ $list} -eq 1 ]]; then echo ${label} echo "--------------------------------------------------" filename=`ls *.txt` echo ${filename}:`find . -type f | grep -in *.txt -e ora\- -e pls\- -e sp2\-` elif [[ ${list} -gt 1 ]]; then echo ${label} echo "--------------------------------------------------" find . -type f | grep -in *.txt -e ora\- -e pls\- -e sp2\- fi } |
Let’s say you name your log files with a file extension of .txt, then you would call the function like this:
errors txt |
It would return output like the following:
common_lookup_lab.txt:229:ORA-02275: such a referential constraint already exists in the table common_lookup_lab.txt:239:ORA-02275: such a referential constraint already exists in the table |
As always, I hope this helps those looking for a solution.
Add user as sudoer
Somebody asked why adding a user to the wheel group in didn’t enable them as a sudoer, as qualified in my earlier Fedora post. The reason is that you also need to modify the primary group in the /etc/passwd file to specify the Group ID value for the wheel group as the primary group of the designated student user.
You can identify the Group ID with the following command:
cat /etc/group | grep wheel |
It should return the following for the wheel group:
wheel:x:10:student |
You need to check the target student user in the /etc/passwd file, which you can do with the following command:
cat /etc/passwd | grep student |
It should return the following for the student user, which has a default group value equal to the user of the same name:
student:x:1000:1000:Student:/home/student:/bin/bash |
As the root user, edit the /etc/passwd file to correct the student user’s primary group ID, as follows:
student:x:1000:10:Student:/home/student:/bin/bash |
You should see the following two lines. If you want authorized sudoers to provide a password (recommended), then modify the first line by removing the # comment. If you don’t want authorized sudoers to provide a password, modify the second line by removing the # comment. Open the /etc/sudoers file with vi or gedit if you’d like a GUI editor.
# %wheel ALL=(ALL) ALL # %wheel ALL=(ALL) NOPASSWD: ALL |
Hope this helps. It’s a quick update for Fedora 30, you su to root and add your user to the sudoers list with the following syntax:
usermod someusername -a -G wheel |
By the way, don’t forget to log off and then back on to the account.
Find files with errors
My students wanted a quick solution on how to find the log files that contain errors. That’s a simple line of code in Linux if you want any Oracle errors that start with ORA-:
find $HOME/lab2 -type f | xargs grep -i ora\- |
It takes only a moment more to look for errors starting with ORA- or PLS-, like:
find $HOME/lab2 -type f | xargs grep -i -e ora\- -e pls\- |
The latter might return something like this:
contact_lab.txt:ORA-00904: "MEMBER_LAB_ID": invalid identifier contact_lab.txt:ORA-00942: table or view does not exist contact_lab.txt:ORA-00942: table or view does not exist member_lab.txt:ORA-02264: name already used by an existing constraint member_lab.txt:ORA-00955: name is already used by an existing object |
You can improve the error identification by identifying line numbers by adding -n option, like:
find $HOME/lab2 -type f | xargs grep -in -e ora\- -e pls\- |
The latter might return something like this when there are two or more files:
contact_lab.txt:76:ORA-00904: "MEMBER_LAB_ID": invalid identifier contact_lab.txt:150:ORA-00942: table or view does not exist contact_lab.txt:157:ORA-00942: table or view does not exist member_lab.txt:75:ORA-02264: name already used by an existing constraint member_lab.txt:149:ORA-00955: name is already used by an existing object |
Unfortunately, the command raises an error when there aren’t any files found of with a qualified extension. It also fails to prepend the file name when there’s only one qualified file name. As a result of these deficiencies, I’ve written the following Bash shell script. I’ve opted to call it the .findErrors.bashrc file name and deploy it in the user’s $HOME directory.
#!/bin/bash # Assign any file filter to the ext variable. ext=${1} # Assign the extension or simply use a wildcard for all files. if [ ! -z ${ext} ]; then ext="*.${ext}" else ext="*" fi # Assign the number of qualifying files to a variable. fileNum=$(ls -l ${ext} 2>/dev/null | grep -v ^l | wc -l) # Evaluate the number of qualifying files and process. if [ ${fileNum} -eq "0" ]; then echo "[0] files exist." elif [ ${fileNum} -eq "1" ]; then fileName=$(ls ${ext}) find `pwd` -type f | grep -in ${ext} -e ora\- -e pls\- | while IFS='\n' read list; do echo "${fileName}:${list}" done else find `pwd` -type f | grep -in ${ext} -e ora\- -e pls\- | while IFS='\n' read list; do echo "${list}" done fi |
You can modify the errors() function with or without a file extension to identify errors beginning with ORA- or PLS- in their log files. As always, I hope this helps those looking for a solution.
MongoDB Update Statement
While discussing the pros and cons of MongoDB, my students wanted to know how to update a specific element in a collection. Collections are like tables in relational databases.
You create the users collection by inserting rows like this:
db.users.insert(
[
{ contact_account: "CA_20170321_0001"
, first_name: "Jonathan"
, middle_name: "Eoin"
, last_name: "MacGregor"
, addresses:
{
street_address: ["1111 Broadway","Suite 101"]
, city: "Oakland"
, state: "CA"
, zip: "94607"
}
}
, { contact_account: "CA_20170328_0001"
, first_name: "Remington"
, middle_name: "Alain"
, last_name: "Dennison"
, addresses:
{
street_address: ["2222 Broadway","Suite 121"]
, city: "Oakland"
, state: "CA"
, zip: "94607"
}
}
]) |
You can query the results with the db.users.find({}) command, or you can query the formatted results with the following command:
db.users.find({}).pretty() |
You can provide a simple update of middle_name element of a given collection element with the findAndModify() function. The following queries the users collection to find the JSON middle_name element where the contact_account value is equal to the “CA_20170330_0001” string.
db.users.findAndModify(
{ query: { contact_account: "CA_20170328_0001" }
, update: { $set: { middle_name: "Alan" }}
, upsert: false }) |
After changing the middle_name value from “Alain” to “Alan”, you can query the single element of the collection with the following:
db.users.find({ contact_account: "CA_20170328_0001" }).pretty() |
It should return the following:
{
"_id" : ObjectId("5bd7f69ba135dda917665de7"),
"contact_account" : "CA_20170328_0001",
"first_name" : "Remington",
"middle_name" : "Alan",
"last_name" : "Dennison",
"addresses" : {
"street_address" : [
"2222 Broadway",
"Suite 121"
],
"city" : "Oakland",
"state" : "CA",
"zip" : "94607"
}
} |
You can replace the addresses string element value a collection of elements with the following findAndModify() function:
db.users.findAndModify(
{ query: { contact_account: "CA_20170328_0001" }
, update:
{ $set:
{ addresses:
[
{
active_status: true
, start_date : new Date("2018-10-30")
, street_address: ["2222 Broadway","Suite 121"]
, city: "Oakland"
, state: "CA"
, zip: "94607"
}
, {
active_status: false
, start_date: new Date("2017-10-01")
, end_date : new Date("2018-10-29")
, street_address: ["2222 Broadway","Suite 121"]
, city: "Oakland"
, state: "CA"
, zip: "94607"
}
]
}
}
, upsert: false }) |
You can re-query the modified result set with find() function with the same query syntax as used previously. This looks for a specific member element in the collection by matching the contact_account name’s value pair. It is the same as the one used earlier in this blog post.
db.users.find({ contact_account: "CA_20170328_0001" }).pretty() |
It should return the following:
{
"_id" : ObjectId("5bd7f69ba135dda917665de7"),
"contact_account" : "CA_20170328_0001",
"first_name" : "Remington",
"middle_name" : "Alan",
"last_name" : "Dennison",
"addresses" : [
{
"active_status" : true,
"start_date" : ISODate("2018-10-30T00:00:00Z"),
"street_address" : [
"2222 Broadway",
"Suite 121"
],
"city" : "Oakland",
"state" : "CA",
"zip" : "94607"
},
{
"active_status" : false,
"start_date" : ISODate("2017-10-01T00:00:00Z"),
"end_date" : ISODate("2018-10-29T00:00:00Z"),
"street_address" : [
"2222 Broadway",
"Suite 121"
],
"city" : "Oakland",
"state" : "CA",
"zip" : "94607"
}
]
} |
As always, I hope this helps someone.
Linux mongod Service
The installation of MongoDB doesn’t do everything for you. In fact, the first time you start the mongod service, like this as the root user or sudoer user with the command:
service mongod start |
A sudoer user will be prompted for their password, like
A typical MongoDB instance raises the following errors:
Redirecting to /bin/systemctl start mongod.service [student@localhost cit425]$ mongo MongoDB shell version v3.4.11 connecting to: mongodb://127.0.0.1:27017 MongoDB server version: 3.4.11 Server has startup warnings: 2018-10-29T10:51:57.515-0600 I STORAGE [initandlisten] 2018-10-29T10:51:57.515-0600 I STORAGE [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine 2018-10-29T10:51:57.515-0600 I STORAGE [initandlisten] ** See http://dochub.mongodb.org/core/prodnotes-filesystem 2018-10-29T10:51:58.264-0600 I CONTROL [initandlisten] 2018-10-29T10:51:58.264-0600 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database. 2018-10-29T10:51:58.264-0600 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted. 2018-10-29T10:51:58.264-0600 I CONTROL [initandlisten] 2018-10-29T10:51:58.265-0600 I CONTROL [initandlisten] 2018-10-29T10:51:58.265-0600 I CONTROL [initandlisten] ** WARNING: soft rlimits too low. rlimits set to 15580 processes, 64000 files. Number of processes should be at least 32000 : 0.5 times number of files. |
You can fix this by following the MongoDB instructions for the Unix ulimit Settings, which will tell you to create a mongod file in the /etc/systemd/system directory. You should create this file as the root superuser. This is what you should put in the file:
[Unit] Description=MongoDB Documentation=man:mongo [Service] # Other directives omitted # (file size) LimitFSIZE=infinity # (cpu time) LimitCPU=infinity # (virtual memory size) LimitAS=infinity # (locked-in-memory size) LimitMEMLOCK=infinity # (open files) LimitNOFILE=64000 # (processes/threads) LimitNPROC=64000 |
Then, you should be able to restart the mongod service without any warnings with this command:
service mongod restart |
As always, I hope this helps somebody.
Docker on Fedora 27
This walks you through the steps to install the Community Edition of Docker on Fedora 27. If you’ve been living under a rock for a few years, Docker is an open source container virtualization software.
Docker or Docker Community Edition is a necessary step if you want to install something like Microsoft’s SQL Server on Fedora because only these are supported platforms:
- Red Hat Enterprise Linux 7.3 or 7.4
- SUSE Linux Enterprise Server V12 SP2
- Ubuntu 16.04
- Docker Engine 1.8+
The first step requires you to add the docker-ce repo to your instance with he curl utility, which should already be installed in most cases. You can check whether the curl utility is available with the which command, like:
which -a curl |
If installed it should return the following:
/usr/bin/curl /bin/curl |
You add the docker-ce repo with the following curl command:
sudo curl -o /etc/yum.repos.d/docker-ce.repo https://download.docker.com/linux/fedora/docker-ce.repo |
You should see something like the following if it was successful:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 2544 100 2544 0 0 2544 0 0:00:01 --:--:-- 0:00:01 12913 |
Next, you can install docker-ce with the following yum command or if you prefer use the dnf utility:
sudo yum install -y docker-ce |
It should produce a log file like the one provided, which you can see by clicking the Display detailed console link that expands the page to show you the console details.
Display detailed console →
Docker CE Stable - x86_64 30 kB/s | 5.9 kB 00:00
Last metadata expiration check: 0:00:00 ago on Thu 05 Jul 2018 06:56:36 PM MDT.
Dependencies resolved.
================================================================================================
Package Arch Version Repository Size
================================================================================================
Installing:
docker-ce x86_64 18.03.1.ce-1.fc27 docker-ce-stable 35 M
Installing dependencies:
container-selinux noarch 2:2.65-1.gitbf5b26b.fc27 updates 41 k
Transaction Summary
================================================================================================
Install 2 Packages
Total download size: 35 M
Installed size: 150 M
Downloading Packages:
(1/2): container-selinux-2.65-1.gitbf5b26b.fc27.noarch.rpm 133 kB/s | 41 kB 00:00
(2/2): docker-ce-18.03.1.ce-1.fc27.x86_64.rpm 20 MB/s | 35 MB 00:01
------------------------------------------------------------------------------------------------
Total 14 MB/s | 35 MB 00:02
warning: /var/cache/dnf/docker-ce-stable-8cf73e63b8c72e93/packages/docker-ce-18.03.1.ce-1.fc27.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 621e9f35: NOKEY
Importing GPG key 0x621E9F35:
Userid : "Docker Release (CE rpm) <docker@docker.com>"
Fingerprint: 060A 61C5 1B55 8A7F 742B 77AA C52F EB6B 621E 9F35
From : https://download.docker.com/linux/fedora/gpg
Key imported successfully
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : container-selinux-2:2.65-1.gitbf5b26b.fc27.noarch 1/2
Running scriptlet: container-selinux-2:2.65-1.gitbf5b26b.fc27.noarch 1/2
neverallow check failed at /var/lib/selinux/targeted/tmp/modules/100/base/cil:9006
(neverallow base_typeattr_7 unlabeled_t (file (entrypoint)))
<root>
allow at /var/lib/selinux/targeted/tmp/modules/200/container/cil:1489
(allow spc_t unlabeled_t (file (entrypoint)))
<root>
allow at /var/lib/selinux/targeted/tmp/modules/100/sandboxX/cil:866
(allow sandbox_x_domain exec_type (file (entrypoint)))
<root>
allow at /var/lib/selinux/targeted/tmp/modules/100/virt/cil:1669
(allow virtd_lxc_t exec_type (file (entrypoint)))
<root>
allow at /var/lib/selinux/targeted/tmp/modules/100/virt/cil:2060
(allow svirt_sandbox_domain exec_type (file (entrypoint)))
Failed to generate binary
/usr/sbin/semodule: Failed!
Running scriptlet: docker-ce-18.03.1.ce-1.fc27.x86_64 2/2
Installing : docker-ce-18.03.1.ce-1.fc27.x86_64 2/2
Running scriptlet: docker-ce-18.03.1.ce-1.fc27.x86_64 2/2
Running as unit: run-rb873740b51a44061abb2c9e197871e4b.service
Verifying : docker-ce-18.03.1.ce-1.fc27.x86_64 1/2
Verifying : container-selinux-2:2.65-1.gitbf5b26b.fc27.noarch 2/2
Installed:
docker-ce.x86_64 18.03.1.ce-1.fc27 container-selinux.noarch 2:2.65-1.gitbf5b26b.fc27
Complete! |
After installing docker-ce, you can check it with the following systemctl command:
sudo systemctl status docker |
It should show service is disabled (default) and inactive:
● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled; vendor preset: disabled)
Active: inactive (dead)
Docs: https://docs.docker.com |
This indicates everything installed correctly, now you need to enable the docker service with this systemctl command:
sudo systemctl enable docker |
It should show that the docker service is enabled and inactive:
● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
Active: inactive (dead)
Docs: https://docs.docker.com |
You start the docker service with this systemctl command:
sudo systemctl start docker |
It should show that the service is enabled and inactive:
● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
Active: active (running) since Thu 2018-07-05 19:22:25 MDT; 4s ago
Docs: https://docs.docker.com
Main PID: 107768 (dockerd)
Tasks: 16
Memory: 35.8M
CPU: 140ms
CGroup: /system.slice/docker.service
├─107768 /usr/bin/dockerd
└─107772 docker-containerd --config /var/run/docker/containerd/containerd.toml
Jul 05 19:22:25 localhost.localdomain dockerd[107768]: time="2018-07-05T19:22:25.350914727-06:00
Jul 05 19:22:25 localhost.localdomain dockerd[107768]: time="2018-07-05T19:22:25.352583385-06:00
Jul 05 19:22:25 localhost.localdomain dockerd[107768]: time="2018-07-05T19:22:25.352718174-06:00
Jul 05 19:22:25 localhost.localdomain dockerd[107768]: time="2018-07-05T19:22:25.353419088-06:00
Jul 05 19:22:25 localhost.localdomain dockerd[107768]: time="2018-07-05T19:22:25.632193965-06:00
Jul 05 19:22:25 localhost.localdomain dockerd[107768]: time="2018-07-05T19:22:25.823732094-06:00
Jul 05 19:22:25 localhost.localdomain dockerd[107768]: time="2018-07-05T19:22:25.893601936-06:00
Jul 05 19:22:25 localhost.localdomain dockerd[107768]: time="2018-07-05T19:22:25.894351396-06:00
Jul 05 19:22:25 localhost.localdomain systemd[1]: Started Docker Application Container Engine.
Jul 05 19:22:25 localhost.localdomain dockerd[107768]: time="2018-07-05T19:22:25.907087735-06:00
lines 1-22/22 (END) |
A CNTL+C dismisses the open log file. As usual, I hope this helps those trying to accomplish the task for the first time.
Fedora User Manager
As one of my students pointed out, Fedora doesn’t install a GUI user management tool installed by default. That’s true. It’s easy to install. You can install it as the root user, like:
yum install -y system-config-users |
It should generate a screen console output like this when successfully installed:
Loaded plugins: langpacks, refresh-packagekit http://yum.postgresql.org/9.3/fedora/fedora-20-x86_64/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found Trying other mirror. Resolving Dependencies --> Running transaction check ---> Package system-config-users.noarch 0:1.3.6-1.fc20 will be installed --> Processing Dependency: libuser-python >= 0.56 for package: system-config-users-1.3.6-1.fc20.noarch --> Processing Dependency: python-pwquality for package: system-config-users-1.3.6-1.fc20.noarch --> Running transaction check ---> Package libuser-python.x86_64 0:0.60-3.fc20 will be installed ---> Package python-pwquality.x86_64 0:1.2.3-1.fc20 will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: system-config-users noarch 1.3.6-1.fc20 updates 332 k Installing for dependencies: libuser-python x86_64 0.60-3.fc20 fedora 51 k python-pwquality x86_64 1.2.3-1.fc20 fedora 11 k Transaction Summary ================================================================================ Install 1 Package (+2 Dependent packages) Total download size: 395 k Installed size: 1.9 M Downloading packages: (1/3): libuser-python-0.60-3.fc20.x86_64.rpm | 51 kB 00:00 (2/3): python-pwquality-1.2.3-1.fc20.x86_64.rpm | 11 kB 00:00 (3/3): system-config-users-1.3.6-1.fc20.noarch.rpm | 332 kB 00:00 -------------------------------------------------------------------------------- Total 212 kB/s | 395 kB 00:01 Running transaction check Running transaction test Transaction test succeeded Running transaction (shutdown inhibited) Installing : python-pwquality-1.2.3-1.fc20.x86_64 1/3 Installing : libuser-python-0.60-3.fc20.x86_64 2/3 Installing : system-config-users-1.3.6-1.fc20.noarch 3/3 Verifying : system-config-users-1.3.6-1.fc20.noarch 1/3 Verifying : libuser-python-0.60-3.fc20.x86_64 2/3 Verifying : python-pwquality-1.2.3-1.fc20.x86_64 3/3 Installed: system-config-users.noarch 0:1.3.6-1.fc20 Dependency Installed: libuser-python.x86_64 0:0.60-3.fc20 python-pwquality.x86_64 0:1.2.3-1.fc20 Complete! |
You can verify the GUI user management tool is present with the following command:
which system-config-users |
It should return this:
/bin/system-config-users |
You can run the GUI user management tool from the root user account or any sudoer account. The following shows how to launch the GUI User Manager from a sudoer account:
sudo system-config-users |
It will generate the following GUI User Manager screen:
As always, I hope this helps those trying to do some that should be a basic task that seems to get lost form the books and manuals beginners use.
Subscribe via RSS