Archive for the ‘Linux’ Category
C Shared Libraries
I wrote a shared C library example to demonstrate external procedures in the Oracle Database 11g PL/SQL Programming book. I also reused the same example to demonstrate Oracle’s external procedures in the Oracle Database 12c PL/SQL Advanced Programming Techniques book last year. The example uses a C Shared Library but a PL/SQL wrapper and PL/SQL test case.
One of my students asked me to simplify the unit test case example by writing the complete unit test in the C Progamming Language. The student request seemed like a good idea, and while poking around on the web it appears there’s a strong case for a set of simple shared C library examples. This blog post isn’t meant to replace the C Programming web site and C Programming Tutorial web site, which I recommend as a great reference point.
Like most things, the best place to start is with basics of C programming because some readers may be very new to C programming. I’ll start with basic standalone programs and how to use the gcc compiler before showing you how to use shared C libraries.
The most basic program is a hello.c program that serves as a “Hello World!” program:
1 2 3 4 5 | #include <stdio.h> int main() { printf("Hello World!\n"); return(0); } |
Assuming you put the C source files in a src subdirectory and the executable files in a bin subdirectory. You compile the program with the gcc program from the parent directory of the src and bin subdirectories, as follows:
gcc -o bin/hello src/hello.c |
Then, you execute the hello executable program from the parent directory as follows:
bin/hello |
It prints:
Hello World! |
You can modify the basic Hello World! program to accept a single input word, like this hello_whom.c program:
1 2 3 4 5 6 7 8 9 10 11 12 | #include <stdio.h> /* The executable main method. */ int main() { // Declare a character array to hold an input value. char whom[30]; /* Print a question and read a string input. */ printf("Who are you? "); scanf("%s", whom); printf("Hello %s!\n", whom); return(0); } |
You can compile the hello_whom.c program as follows:
gcc -o bin/hello_whom src/hello_whom.c |
Then, you execute the hello_whom executable program from the parent directory as follows:
bin/hello_whom
Who are you? Stuart |
It prints:
Hello Stuart! |
Alternatively, you can modify the hello_whom.c program to accept a stream of text, like the following hello_string.c program:
1 2 3 4 5 6 7 8 9 10 11 12 | #include <stdio.h> /* The executable main method. */ int main() { // Declare a character array to hold an input name. char phrase[4000]; /* Print a question and read a string input. */ printf("Hello? "); scanf("%[^\n]%*c", phrase); printf("Hello %s!\n", phrase); return(0); } |
The [] is the scan set character. The [^\n] on line 10 defines the input as not a newline with a white space, and the %*c reads the newline character from the input buffer. After you compile the program you can call it like this:
bin/hello_string
Hello? there, it reads like a C++ stream |
It would print:
Hello there, it reads like a C++ stream! |
These example, like the previous examples, assume the source files are in a src subdirectory and the executable files are in the bin subdirectory. All compilation commands are run from the parent directory of the src and bin subdirectories.
The first example puts everything into a single writingstr.c file. It defines a writestr() function prototype before the main() function and the writestr() function after the main() function.
The code follows below:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | #include <stdio.h> /* Declare two integer variables. */ char path[255], message[4000]; /* Define a prototype for the writestr() function. */ void writestr(char *path, char *message); /* The executable main method. */ int main() { printf("Enter file name and message: "); scanf("%s %[^\n]%*c", &path, &message); printf("File name: %s\n", path); printf("File content: %s\n", message); writestr(path, message); return(0); } void writestr(char *path, char *message) { FILE *file_name; file_name = fopen(path,"w"); fprintf(file_name,"%s\n",message); fclose(file_name); } |
You can compile the writingstr.c function with the following syntax:
gcc -o bin/writingstr src/writingstr.c |
You can run the writingstr executable file with the following syntax:
bin/writingstr Enter file name and message: /home/student/Code/c/test.txt A string for a file. File name: /home/student/Code/c/test.txt File content: A string for a file. |
You’ll find a test.txt file written to the /home/student/Code/C directory. The file contains only the single sentence fragment entered above.
Now, let’s create a writestr.h header file, a writestr.c shared object file, and a main.c testing file. You should note a pattern between the self-contained code and the approach for using shared libraries. The prototype of the writestr() function becomes the definition of the writestr.h file, and the implementation of the writestr() function becomes the writestr.so shared library.
The main.c file contains the only the main() function from the writingstr.c file. The main() function uses the standard scanf() function to read a fully qualified file name (also known as a path) as a string and then a text stream for the content of the file.
You define the writestr.h header file as:
1 2 3 4 5 6 | #ifndef writestr_h__ #define writestr_h__ extern void writestr(char *path, char *message); #endif |
You define the writestr.c shared library, which differs from the example in the book. The difference is the #include statement of the writestr.h header file. The source code follows:
1 2 3 4 5 6 7 8 | #include <stdio.h> #include "writestr.h" void writestr(char *path, char *message) { FILE *file_name; file_name = fopen(path,"w"); fprintf(file_name,"%s\n",message); fclose(file_name); } |
You define the main.c testing program as:
1 2 3 4 5 6 7 8 9 10 11 12 | #include <stdio.h> #include "writestr.h" /* Declare two integer variables. */ char path[255], message[4000]; /* The executable main method. */ int main() { printf("Enter file name and message: "); scanf("%s %[^\n]%*c", &path, &message); writestr(path, message); return(0); } |
Before you begin the process to compile these, you should create an environment file that sets the $LD_LIBRARY_PATH environment variable or add it to your .bashrc file. You should point the $LD_LIBRARY_PATH variable to the directory where you’ve put your shared libraries.
# Set the LD_LIBRARY_PATH environment variable. export LD_LIBRARY_PATH=/home/student/Code/c/trylib/libfile |
With programs defined, you need to first compile the writestr.c shared library first. You use the following syntax from the parent directory of the src and bin subdirectories.
gcc -shared -fPIC -o bin/writestr.so src/writestr.c |
If you haven’t set the $LD_LIBRARY_PATH, you may raise an exception. There’s also an alternative to setting the $LD_LIBRARY_PATH before you call the gcc executable. You can use the -L option set the $LD_LIBRARY_PATH for a given all to the gcc executable, like:
gcc -L /home/student/Code/c/trylib/libfile -shared -fPIC -o bin/writestr.so src/writestr.c |
Then, you compile the main.c program. You must put the writestr.so shared library before you designate the main target object and main.c source files, like this:
gcc bin/writestr.so -o bin/main src/main.c |
Now, you can perform a C-only unit test case by calling the main executable. However, you must have set the $LD_LIBRARY_PATH environment variable at runtime too. You see the following reply to the “Enter file name and message” question when you run the main program unit:
bin/main Enter file name and message: /home/student/Code/c/trylib/libfile/test.txt A long native string is the second input to this C program. |
You can now see that the a new test.txt file has been written to the target directory, and that it contains the following string:
A long native string is the second input to this C program. |
As always, I hope this helps those you want to write shared libraries in the C programming language.
Netbeans 8 – Fedora
Some of my students want to use the Fedora image that I built for my database classes in my Java software development life cycle course. As a result, they wanted a Java development environment installed. I examined JDeveloper 11g (11.1.1.7.0) and 12c (12.1.3) but resolved on the more generic Netbeans 8 (8.0.2) IDE.
JDK 7 with Netbeans 8 Download
You can download the generic Netbeans 8 IDE, the JDK 7 with Netbeans, or the JDK 8 with Netbeans for the Linux installation. After you download the executable program, you should follow these instructions to install the Netbeans 8 IDE on Fedora.
As the student user, you can download the file to your ~student/Downloads directory and then run these two commands:
chmod +x ./jdk-7u80-nb-8_0_2-linux-x64.sh sudo ./jdk-7u80-nb-8_0_2-linux-x64.sh |
It produces the following output log:
Configuring the installer... Searching for JVM on the system... Preparing bundled JVM ... Extracting installation data... Running the installer wizard... |
Then, it launches the installer. These screens show you how to install and create your first Java project.
JDK 7 with Netbeans 8 Installation
- The first installation dialog welcomes you to the JDK 7 Update and NetBeans 8 Installer. Click the Next button to proceed.
- The second installation dialog asks you to accept the terms in the license agreement. Click the Next button to proceed.
- The third installation dialog asks you to install Netbeans 8. Click the Browse button if you would like to install it in a different area. Click the Next button to proceed.
- The fourth installation dialog asks you to install another Java JDK 7 that supports the current release of Netbeans 8. Click the Browse button if you would like to install it in a different area. Click the Next button to proceed.
- The fifth installation dialog shows you the progress bar for installing Java JDK 7 that supports the current release of Netbeans 8. You may not need to click the Next button to proceed because it should progress to the Netbeans progress dialog. Click the Next button to proceed when it doesn’t do it automatically.
- The sixth installation dialog shows you the progress bar for installing Netbeans 8. Click the Next button to proceed when it doesn’t do it automatically.
- The next screen is the final screen of the Java SE Development Kit and NetBeans IDE Installer. Click the Finish button to complete the installation.
After the installation, you need to check if the netbeans program can be found by users. It shouldn’t be found at this point because it isn’t in the default $PATH environment variable.
Configuring the student user
You can set the $PATH variable dynamically like this:
export PATH=$PATH:/usr/local/netbeans-8.0.2/bin |
The netbeans program location was set in Step #4 of the Netbeans installation. After setting the $PATH environment variable, you can run netbeans with this syntax:
./netbeans & |
However, the better approach is to put the following lines in your .bashrc file. This change ensures that you can access the netbeans program anytime you launch a Terminal session.
# Add netbeans to the user's PATH variable. export PATH=$PATH:/usr/local/netbeans-8.0.2/bin |
After you have configured the student user’s .bashrc file, you can now use Netbeans to create a Java project.
Create a new Netbeans project
- The next screen is the Netbeans 8 Start Page. This is where you can create your first Java development project.
- You click the File menu and then the New Project menu option to open a new project.
- It launches the New Project dialog at Step #1 – Choose Project, where you choose Java from your Categories list and Java Application from the Projects list. You click the Next button to continue.
- It launches the New Project dialog at Step #2 – Name and Location, where you enter a Project Name. The example uses
MySQLJavaas the project name. You click the Next button to continue.
- It launches the
MySQLJava.javatab in the Netbeans 8 application. This is where you can enter your code.
After you successfully download the Java 7 SE and Netbeans 8, you should download JDK 8 with Netbeans 8 because Java 7 EOL (End-of-Life) is April 30th, 2015. You may think that you need to uninstall the JDK 7 with Netbeans 8 before you install the JDK 8 with Netbeans 8, but you don’t have to do so. When you install JDK 8 with Netbeans 8 into an environment with a preinstalled JDK 7 with Netbeans 8, the installer only adds the JDK 8.
The following segments of the post show you how to download and install JDK 8 with Netbeans 8, and how to configure Netbeans to work with the JDK 7 and JDK 8 as interchangeable libraries.
JDK 8 with Netbeans 8 Download
You can now download the JDK 8 with Netbeans for the Linux installation. After you download the executable program, you should follow these instructions to install it on Fedora.
As the student user, you can download the file to your ~student/Downloads directory and then run these two commands:
chmod +x ./jdk-8u45-nb-8_0_2-linux-x64.sh sudo ./jdk-8u45-nb-8_0_2-linux-x64.sh |
It produces the following output log:
Configuring the installer... Searching for JVM on the system... Preparing bundled JVM ... Extracting installation data... Running the installer wizard... |
Then, it launches the installer, which will be very similar to the steps you went through earlier. There are differences. There are only five screens that you navigate as opposed to the seven from the earlier JDK 7 with Netbeans 8 installation, as you’ll see below.
JDK 8 with Netbeans 8 Installation
- The first installation dialog welcomes you to the JDK 8 Update and NetBeans 8 Installer. Click the Next button to proceed.
- The second installation dialog installs the JDK 8. Click the Next button to proceed.
- The third installation dialog is a summary of what you’ll install. Click the Install button to proceed.
- The fourth installation dialog shows you a progress bar. You don’t need to do anything but watch the progress.
- The fifth installation dialog shows you the installation is complete. Click the Finish button to proceed when it doesn’t do it automatically.
After you have installed the JDK 8 SE, you can use Netbeans to add the JDK 8 platform.
Add the JDK 8 Platform to Netbeans 8
- After you open Netbeans 8, you choose the Tools menu choice. Then, you select the Java Platforms menu option.
- It launches the Java Platform Manager dialog. You click the Add Platform button to add the JDK 8 platform.
- It launches the Add Java Platform dialog. Leave the Java Standard Edition radio button checked. You click the Next button to proceed.
- It launches the Add Java Platform file chooser dialog. Here you navigate to find the JDK 8 software, which is located in
/usr/local/jdk1.8.0_45directory.
- After selecting the
/usr/local/jdk1.8.0_45directory as the platform folder, click the Next button to proceed.
- After setting the directory, you’re asked to verify the Java Platform information. If it’s correct, click the Finish button to proceed.
- After finishing the installation, you’ll see that you have two installed Java Platforms. Unfortunately, the first one installed is the default unless you modify the
netbeans.conffile. You click the Close button to complete the process.
Set JDK 8 Platform as the Default Java Platform for Netbeans 8
After adding the JDK 8 Java Platform, you can change the default setting my manually editing the /usr/local/netbeans-8.0.2/etc/netbeans.conf file. You simply remark out the line for JDK 7 and replace it with one for JDK 8, as shown below. The next time you boot the Netbeans application it uses Java 1.8.
# netbeans_jdkhome="/usr/local/jdk1.7.0_80" netbeans_jdkhome="/usr/local/jdk1.8.0_45" |
The next time you launch Netbeans 8, it will use JDK 8 because you set that as the default Java Platform
As always, I hope this helps those looking for information like this.
Add zsh to Fedora
One of my students requested an option to the bash shell. It was interesting to hear that he wanted me to instal the zsh in my Fedora image. There’s only one book that I’m aware of that’s been published on the Z Shell, and it is From Bash to Z Shell.
This post shows how to add the zsh to my Fedora image because I already release a new one for the term without the zsh shell. You use the yum utility as the root user to install the zsh library:
yum install -y zsh |
It should produce an output stream like the following, which required accessing the alternate mirror site:
Loaded plugins: langpacks, refresh-packagekit mysql-connectors-community | 2.5 kB 00:00 mysql-tools-community | 2.5 kB 00:00 mysql56-community | 2.5 kB 00:00 pgdg93 | 3.6 kB 00:00 updates/20/x86_64/metalink | 14 kB 00:00 updates | 4.9 kB 00:00 (1/3): mysql-connectors-community/20/x86_64/primary_db | 8.8 kB 00:00 (2/3): pgdg93/20/x86_64/primary_db | 83 kB 00:01 (3/3): updates/20/x86_64/primary_db | 13 MB 00:13 updates/20/x86_64/pkgtags FAILED http://mirror.utexas.edu/fedora/linux/updates/20/x86_64/repodata/1ea83dc402a2bcba53f9b0011ecfa0d579b5a316e4c7f01ec5f1166dcdca138f-pkgtags.sqlite.gz: [Errno 14] HTTP Error 404 - Not Found Trying other mirror. (1/2): updates/20/x86_64/updateinfo | 1.9 MB 00:07 (2/2): updates/20/x86_64/pkgtags | 1.4 MB 00:01 Resolving Dependencies --> Running transaction check ---> Package zsh.x86_64 0:5.0.7-6.fc20 will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: zsh x86_64 5.0.7-6.fc20 updates 2.5 M Transaction Summary ================================================================================ Install 1 Package Total download size: 2.5 M Installed size: 5.9 M Downloading packages: zsh-5.0.7-6.fc20.x86_64.rpm | 2.5 MB 00:03 Running transaction check Running transaction test Transaction test succeeded Running transaction (shutdown inhibited) Installing : zsh-5.0.7-6.fc20.x86_64 1/1 Verifying : zsh-5.0.7-6.fc20.x86_64 1/1 Installed: zsh.x86_64 0:5.0.7-6.fc20 Complete! |
Once you’ve installed the zsh, you can configure like you would the bash shell. You make edits to individual .zshrc files and generic changes to the /etc/zshrc file. You can find the documentation to edit the zsh in the User’s Guide to the Z-Shell.
The following is a modified .zshrc file. The changes enable the up-arrow in Oracle’s sqlplus and provides you with a color prompt, like this:
# Source global definitions if [ -f /etc/zshrc ]; then . /etc/zshrc fi # Uncomment the following line if you don't like systemctl's auto-paging feature: # export SYSTEMD_PAGER= # Set the JAVA_HOME path. export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.75-2.5.4.2.fc20.x86_64 # Set the CLASSPATH path. export CLASSPATH=/usr/share/java/mysql-connector-java.jar:. # User specific aliases and functions . /u01/app/oracle/product/11.2.0/xe/bin/oracle_env.sh # Wrap sqlplus with rlwrap to edit prior lines with the # up, down, left and right keys. sqlplus() { if [ "$RLWRAP" = "0" ]; then sqlplus "$@" else rlwrap sqlplus "$@" fi } # Set the bindkey. bindkey -v bindkey "^R" history-incremental-search-backward export EDITOR="vim" # history stuff HISTFILE=~/.zsh-histfile HISTSIZE=2000 # Set vi as a command line editor. set -o vi autoload -U colors && colors PS1="[%{$fg[red]%}%n%{$reset_color%}@%{$fg[blue]%}%m %{$fg[yellow]%}%~%{$reset_color%}% ]$ " |
The zsh prompt looks like the following:
[student@localhost ~]$
If you’re configured with a bash shell, you can change your shell to a zsh with the following command:
chsh -s /bin/zsh |
As always, I hope this helps those looking for this type of information.
Ruby Thin Web Server
Somebody suggested that I try out thin, “A fast and very simple Ruby web server.” So, I thought it might be interesting to test, and a simplification over Rails to demonstrate an small Ruby MVC pattern.
Installing thin seemed straight forward as a gem installation, like
gem install thin |
The initial install didn’t work out of the box because I’d neglected to install the gcc-c++ library. It raised the following errors:
Fetching: eventmachine-1.0.7.gem (100%)
Building native extensions. This could take a while...
ERROR: Error installing thin:
ERROR: Failed to build gem native extension.
/usr/bin/ruby extconf.rb
checking for main() in -lssl... no
checking for rb_trap_immediate in ruby.h,rubysig.h... no
checking for rb_thread_blocking_region()... yes
checking for ruby/thread.h... yes
checking for rb_thread_call_without_gvl() in ruby/thread.h... yes
checking for inotify_init() in sys/inotify.h... yes
checking for writev() in sys/uio.h... yes
checking for rb_thread_fd_select()... yes
checking for rb_fdset_t in ruby/intern.h... yes
checking for rb_wait_for_single_fd()... yes
checking for rb_enable_interrupt()... no
checking for rb_time_new()... yes
checking for sys/event.h... no
checking for epoll_create() in sys/epoll.h... yes
checking for clock_gettime()... yes
checking for CLOCK_MONOTONIC_RAW in time.h... yes
checking for CLOCK_MONOTONIC in time.h... yes
creating Makefile
make "DESTDIR="
g++ -I. -I/usr/include -I/usr/include/ruby/backward -I/usr/include -I. -DWITHOUT_SSL -DBUILD_FOR_RUBY -DHAVE_RB_THREAD_BLOCKING_REGION -DHAVE_TBR -DHAVE_RUBY_THREAD_H -DHAVE_RB_THREAD_CALL_WITHOUT_GVL -DHAVE_RB_THREAD_CALL_WITHOUT_GVL -DHAVE_INOTIFY_INIT -DHAVE_INOTIFY -DHAVE_WRITEV -DHAVE_WRITEV -DHAVE_RB_THREAD_FD_SELECT -DHAVE_RB_THREAD_FD_SELECT -DHAVE_TYPE_RB_FDSET_T -DHAVE_RB_FDSET_T -DHAVE_RB_WAIT_FOR_SINGLE_FD -DHAVE_RB_TIME_NEW -DOS_UNIX -DHAVE_EPOLL_CREATE -DHAVE_EPOLL -DHAVE_CLOCK_GETTIME -DHAVE_CONST_CLOCK_MONOTONIC_RAW -DHAVE_CONST_CLOCK_MONOTONIC -fPIC -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -mtune=generic -m64 -o ed.o -c ed.cpp
make: g++: Command not found
make: *** [ed.o] Error 127
Gem files will remain installed in /usr/local/share/gems/gems/eventmachine-1.0.7 for inspection.
Results logged to /usr/local/share/gems/gems/eventmachine-1.0.7/ext/gem_make.out |
Naturally, I installed the gcc-c++ library with the yum utility, like this:
yum list gcc-c++ |
It displayed the following log output:
Loaded plugins: langpacks, refresh-packagekit mysql-connectors-community | 2.5 kB 00:00 mysql-tools-community | 2.5 kB 00:00 mysql56-community | 2.5 kB 00:00 pgdg93 | 3.6 kB 00:00 updates/20/x86_64/metalink | 14 kB 00:00 updates | 4.9 kB 00:00 updates/20/x86_64/primary_db | 13 MB 00:04 (1/2): updates/20/x86_64/updateinfo | 1.9 MB 00:02 (2/2): updates/20/x86_64/pkgtags | 1.4 MB 00:00 Resolving Dependencies --> Running transaction check ---> Package gcc-c++.x86_64 0:4.8.3-7.fc20 will be installed --> Processing Dependency: libstdc++-devel = 4.8.3-7.fc20 for package: gcc-c++-4.8.3-7.fc20.x86_64 --> Running transaction check ---> Package libstdc++-devel.x86_64 0:4.8.3-7.fc20 will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: gcc-c++ x86_64 4.8.3-7.fc20 updates 7.2 M Installing for dependencies: libstdc++-devel x86_64 4.8.3-7.fc20 updates 1.5 M Transaction Summary ================================================================================ Install 1 Package (+1 Dependent package) Total download size: 8.7 M Installed size: 25 M Downloading packages: (1/2): gcc-c++-4.8.3-7.fc20.x86_64.rpm | 7.2 MB 00:05 (2/2): libstdc++-devel-4.8.3-7.fc20.x86_64.rpm | 1.5 MB 00:01 -------------------------------------------------------------------------------- Total 1.3 MB/s | 8.7 MB 00:06 Running transaction check Running transaction test Transaction test succeeded Running transaction (shutdown inhibited) Installing : libstdc++-devel-4.8.3-7.fc20.x86_64 1/2 Installing : gcc-c++-4.8.3-7.fc20.x86_64 2/2 Verifying : gcc-c++-4.8.3-7.fc20.x86_64 1/2 Verifying : libstdc++-devel-4.8.3-7.fc20.x86_64 2/2 Installed: gcc-c++.x86_64 0:4.8.3-7.fc20 Dependency Installed: libstdc++-devel.x86_64 0:4.8.3-7.fc20 Complete! |
After installing the gcc-c++ libraries, I reran the gem utility to install the thin utility. It created three Ruby Gems: eventmachine-1.0.7, daemons-1.2.2.gem, and thin-1.6.3.gem, as shown:
Building native extensions. This could take a while... Successfully installed eventmachine-1.0.7 Fetching: daemons-1.2.2.gem (100%) Successfully installed daemons-1.2.2 Fetching: thin-1.6.3.gem (100%) Building native extensions. This could take a while... Successfully installed thin-1.6.3 Parsing documentation for daemons-1.2.2 Installing ri documentation for daemons-1.2.2 Parsing documentation for eventmachine-1.0.7 Installing ri documentation for eventmachine-1.0.7 Parsing documentation for thin-1.6.3 Installing ri documentation for thin-1.6.3 Done installing documentation for daemons, eventmachine, thin after 11 seconds 3 gems installed |
Having created the Ruby Gems, I followed the thin instruction on the web site, and created the following Ruby web server:
app = proc do |env| [ 200, # Status code { # Response headers 'Content-Type' => 'text/html', 'Content-Length' => '12', }, ['Hello World!'] # Response body ] end # You can install Rack middlewares # to do some crazy stuff like logging, # filtering, auth or build your own. use Rack::CommonLogger run app |
Then, I tested the Ruby web server with the following command:
thin start -R thin.ru |
It displayed the following test server as the result of localhost:3000 URL:
As always, I hope this helps those who land on this page.
Find a string in files
From time to time, folks ask questions about how to solve common problems in Linux or Unix. Today, the question is: “How do I find a list of files that contain a specific string?” There are two alternatives with the find command, and the following sample searches look for files that contain a sqlite3 string literal.
- Search for only the file names:
find . -type f | xargs grep -li sqlite3 |
Or, the more verbose:
find . -type f -exec grep -li sqlite3 /dev/null {} + |
- Search for the file names and text line:
find . -type f | xargs grep -i sqlite3 |
Or, the more verbose:
find . -type f -exec grep -i sqlite3 /dev/null {} + |
Don’t exclude the /dev/null from the verbose syntax or you’ll get the things you lack permissions to inspect or that raise other errors. I don’t post a lot of Linux or Unix tips and techniques, and you may find this site more useful to answer these types of questions:
Unix & Linux Stack Exchange web site
As always, I hope this helps those you land on the blog page.
Ruby-MySQL Columns
Last week I posted how to configure and test Ruby and MySQL. Somebody asked me how to handle a dynamic list of columns. So, here’s a quick little program to show you how to read the dynamic list of column (and this updated blog post has the 2024 update for the new Mysql2 ODBC driver):
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 | require 'rubygems' require 'mysql' # Begin block. begin # Create a new connection resource. db = Mysql.new('localhost','student','student','studentdb') # Create a result set. rs = db.query('SELECT item_title, item_rating FROM item') # Read through the result set hash. rs.each do | row | out = "" i = 0 while i < db.field_count # Check if not last column. if i < db.field_count - 1 out += "#{row[i]}, " else out += "#{row[i]}" end i += 1 end puts "#{out}" end # Release the result set resources. rs.free rescue Mysql::Error => e # Print the error. puts "ERROR #{e.errno} (#{e.sqlstate}): #{e.error}" puts "Can't connect to MySQL database specified." # Signal an error. exit 1 ensure # Close the connection when it is open. db.close if db end |
The new logic on lines 13 through 22 reads the list of columns into a comma delimited list of values. The if-block checks to make sure it doesn’t append a comma to the last column in the list. It prints output like:
The Hunt for Red October, PG Star Wars I, PG Star Wars II, PG Star Wars II, PG Star Wars III, PG13 The Chronicles of Narnia, PG RoboCop, Mature Pirates of the Caribbean, Teen The Chronicles of Narnia, Everyone MarioKart, Everyone Splinter Cell, Teen Need for Speed, Everyone The DaVinci Code, Teen Cars, Everyone Beau Geste, PG I Remember Mama, NR Tora! Tora! Tora!, G A Man for All Seasons, G Hook, PG Around the World in 80 Days, G Harry Potter and the Sorcerer's Stone, PG Camelot, G |
As always, I hope this helps those looking for a solution.
MySQLdb Manage Columns
Sometimes trying to keep a post short and to the point raises other questions. Clearly, my Python-MySQL Program post over the weekend did raise a question. They were extending the query example and encountered this error:
TypeError: range() integer end argument expected, got tuple. |
That should be a straight forward error message because of two things. First, the Python built-in range() function manages a range of numbers. Second, the row returned from a cursor is actually a tuple (from relational algebra), and it may contain non-numeric data like strings and dates.
The reader was trying to dynamically navigate the number of columns in a row by using the range() function like this (where row was a row from the cursor or result set):
for j in range(row): |
Naturally, it threw the type mismatch error noted above. As promised, the following Python program fixes that problem. It also builds on the prior example by navigatung an unknown list of columns. Lines 16 through 31 contain the verbose comments and programming logic to dynamically navigate the columns of a row.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 | #!/usr/bin/python # Import sys library. import MySQLdb import sys try: # Create new database connection. db = MySQLdb.connect('localhost','student','student','studentdb') # Create a result set cursor. rs = db.cursor() rs.execute("SELECT item_title, item_subtitle, item_rating FROM item") # Assign the query results to a local variable. for i in range(rs.rowcount): row = rs.fetchone() # Initialize variable for printing row as a string. data = "" # Address an indefinite number of columns. count = 0 for j in range(len(row)): # Initialize column value as an empty string. datum = "" # Replace column values when they exist. if str(row[count]) != 'None': datum = str(row[count]) # Append a comma when another column follows. if count == len(row) - 1: data += datum else: data += datum + ", " count += 1 # Print the formatted row as a string. print data except MySQLdb.Error, e: # Print the error. print "ERROR %d: %s" % (e.args[0], e.args[1]) sys.exit(1) finally: # Close the connection when it is open. if db: db.close() |
There are a couple Python programming techniques that could be perceived as tricks. Line 24 checks for a not null value by explicitly casting the column’s value to a string and then comparing its value against the string equivalent for a null. The MySQLdb returns a 'None' string for null values by default. The if-block on lines 27 through 30 ensure commas aren’t appended at the end of a row.
While the for-loop with a range works, I’d recommend you write it as a while-loop because its easier to read for most new Python programmers. You only need to replace line 20 with the following to make the change:
20 | while (count < len(row)): |
Either approach generates output like:
The Hunt for Red October, Special Collectornulls Edition, PG Star Wars I, Phantom Menace, PG Star Wars II, Attack of the Clones, PG Star Wars II, Attack of the Clones, PG Star Wars III, Revenge of the Sith, PG-13 The Chronicles of Narnia, The Lion, the Witch and the Wardrobe, PG RoboCop, , Mature Pirates of the Caribbean, , Teen The Chronicles of Narnia, The Lion, the Witch and the Wardrobe, Everyone MarioKart, Double Dash, Everyone Splinter Cell, Chaos Theory, Teen Need for Speed, Most Wanted, Everyone The DaVinci Code, , Teen Cars, , Everyone Beau Geste, , PG I Remember Mama, , NR Tora! Tora! Tora!, The Attack on Pearl Harbor, G A Man for All Seasons, , G Hook, , PG Around the World in 80 Days, , G Harry Potter and the Sorcerer's Stone, , PG Camelot, , G |
As always, I hope this helps those looking for clarity.
Perl-MySQL Program
Configuring Perl to work with MySQL is the last part creating a complete Fedora Linux LAMP stack for my students. Perl is already installed on Fedora Linux.
I’ve also shown how to use PHP, Python, and Ruby languages to query a MySQL database on Linux. After installing this additional Perl DBI library, my students will have the opportunity to choose how they implement their LAMP solution.
You can find the Perl version with the following version.pl program:
1 2 3 4 | #!/usr/bin/perl -w # Print the version. print "Perl ".$]."\n"; |
The first line lets you call the program without prefacing the perl program because it invokes a subshell of perl by default. You just need to ensure the file has read and execute privileges to run. It prints:
Perl 5.018004 |
You need to install the perl-DBD-MySQL library to enable Perl to work with MySQL. The following command loads the library:
yum install -y perl-DBD-MySQL |
It prints the following log file:
Loaded plugins: langpacks, refresh-packagekit Resolving Dependencies --> Running transaction check ---> Package perl-DBD-MySQL.x86_64 0:4.024-1.fc20 will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: perl-DBD-MySQL x86_64 4.024-1.fc20 fedora 142 k Transaction Summary ================================================================================ Install 1 Package Total download size: 142 k Installed size: 332 k Downloading packages: perl-DBD-MySQL-4.024-1.fc20.x86_64.rpm | 142 kB 00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction (shutdown inhibited) Installing : perl-DBD-MySQL-4.024-1.fc20.x86_64 1/1 Verifying : perl-DBD-MySQL-4.024-1.fc20.x86_64 1/1 Installed: perl-DBD-MySQL.x86_64 0:4.024-1.fc20 Complete! |
The following item_query.pl Perl program is consistent with the PHP, Python, and Ruby examples provided in other blog posts. It shows you how to use the Perl DBI library to query and return a data set.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | #!/usr/bin/perl -w # Use the DBI library. use DBI; use strict; use warnings; # Create a connection. my $dbh = DBI->connect("DBI:mysql:database=studentdb;host=localhost:3306","student","student",{'RaiseError' => 1}); # Create SQL statement. my $sql = "SELECT item_title FROM item"; # Prepare SQL statement. my $sth = $dbh->prepare($sql); # Execute statement and read result set. $sth->execute() or die $DBI::errstr; while (my @row = $sth->fetchrow_array()) { my $item_title = $row[0]; print "$item_title\n"; } # Close resources. $sth->finish(); |
You call it like this from the present working directory:
./mysql_query.pl |
It returns:
The Hunt for Red October Star Wars I Star Wars II Star Wars II Star Wars III The Chronicles of Narnia RoboCop Pirates of the Caribbean The Chronicles of Narnia MarioKart Splinter Cell Need for Speed The DaVinci Code Cars Beau Geste I Remember Mama Tora! Tora! Tora! A Man for All Seasons Hook Around the World in 80 Days Harry Potter and the Sorcerer's Stone Camelot |
Alternatively, there’s a different syntax for lines 20 and 21 that you can use when you’re returning multiple columns. It replaces the two statements inside the while loop as follows:
20 21 | my ($item_title, $item_rating) = @row; print "$item_title, $item_rating\n"; |
It returns:
The Hunt for Red October, PG Star Wars I, PG Star Wars II, PG Star Wars II, PG Star Wars III, PG13 The Chronicles of Narnia, PG RoboCop, Mature Pirates of the Caribbean, Teen The Chronicles of Narnia, Everyone MarioKart, Everyone Splinter Cell, Teen Need for Speed, Everyone The DaVinci Code, Teen Cars, Everyone Beau Geste, PG I Remember Mama, NR Tora! Tora! Tora!, G A Man for All Seasons, G Hook, PG Around the World in 80 Days, G Harry Potter and the Sorcerer's Stone, PG Camelot, G |
As always, I hope this helps those learning how to use Perl and Linux against the MySQL Database. If you want a nice tutorial on Perl and MySQL, check The tutorialspoint.com web site.
Python-MySQL Program
This post works through the Python configuration of Fedora instance, and continues the configuration of my LAMP VMware instance. It covers how you add the MySQL-python libraries to the Fedora instance, and provides the students with one more language opportunity for their capstone lab in the database class.
A standard Fedora Linux distribution installs Python 2.7 by default. Unfortunately, the MySQL-python library isn’t installed by default. You can verify the Python version by writing and running the following version.py program before installing the MySQL-python library:
1 2 3 4 5 | # Import sys library. import sys # Print the Python version. print sys.version |
You can run the version.py program dynamically like this from the current working directory:
python version.py |
It will print the following:
2.7.5 (default, Nov 3 2014, 14:26:24) [GCC 4.8.3 20140911 (Red Hat 4.8.3-7)] |
If you modify the program by adding the following first line
1 2 3 4 5 6 7 | #!/usr/bin/python # Import sys library. import sys # Print the Python version. print sys.version |
Provided you’ve set the file permissions to read and execute, you can run the program by simply calling version.py like this from the present working directory:
./version.py |
You can install the MySQL-python library with the yum utility like this:
yum install -y MySQL-python |
It shows you the following output:
Loaded plugins: langpacks, refresh-packagekit mysql-connectors-community | 2.5 kB 00:00 mysql-tools-community | 2.5 kB 00:00 mysql56-community | 2.5 kB 00:00 pgdg93 | 3.6 kB 00:00 updates/20/x86_64/metalink | 12 kB 00:00 updates | 4.9 kB 00:00 updates/20/x86_64/primary_db | 13 MB 00:04 (1/2): updates/20/x86_64/updateinfo | 1.9 MB 00:02 (2/2): updates/20/x86_64/pkgtags | 1.4 MB 00:02 Resolving Dependencies --> Running transaction check ---> Package MySQL-python.x86_64 0:1.2.3-8.fc20 will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: MySQL-python x86_64 1.2.3-8.fc20 fedora 82 k Transaction Summary ================================================================================ Install 1 Package Total download size: 82 k Installed size: 231 k Downloading packages: MySQL-python-1.2.3-8.fc20.x86_64.rpm | 82 kB 00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction (shutdown inhibited) Installing : MySQL-python-1.2.3-8.fc20.x86_64 1/1 Verifying : MySQL-python-1.2.3-8.fc20.x86_64 1/1 Installed: MySQL-python.x86_64 0:1.2.3-8.fc20 Complete! |
After installing the MySQL-python library, you can call the following mysql_connect.py program from the local directory:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | #!/usr/bin/python # Import sys library. import MySQLdb import sys try: # Create new database connection. db = MySQLdb.connect('localhost','student','student','studentdb') # Query the version of the MySQL database. db.query("SELECT version()") # Assign the query results to a local variable. result = db.use_result() # Print the results. print "MySQL Version: %s " % result.fetch_row()[0] except MySQLdb.Error, e: # Print the error. print "ERROR %d: %s" % (e.args[0], e.args[1]) sys.exit(1) finally: # Close the connection when it is open. if db: db.close() |
Like the version.py program, set the file permissions to read and execute and call , you can run the program by simply calling mysql_connect.py program like this from the present working directory:
./mysql_connect.py |
The mysql_connect.py program displays:
MySQL Version: 5.6.24 |
After verifying the MySQL connection, you can query actual data with the following mysql_queryset.py program:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | #!/usr/bin/python # Import sys library. import MySQLdb import sys try: # Create new database connection. db = MySQLdb.connect('localhost','student','student','studentdb') # Create a result set cursor. rs = db.cursor() rs.execute("SELECT item_title FROM item") # Assign the query results to a local variable. rows = rs.fetchall() # Print the results. for row in rows: print row except MySQLdb.Error, e: # Print the error. print "ERROR %d: %s" % (e.args[0], e.args[1]) sys.exit(1) finally: # Close the connection when it is open. if db: db.close() |
You call the mysql_queryset.py file from the present working directory like this:
./mysql_queryset.py |
It prints the following:
('The Hunt for Red October',)
('Star Wars I',)
('Star Wars II',)
('Star Wars II',)
('Star Wars III',)
('The Chronicles of Narnia',)
('RoboCop',)
('Pirates of the Caribbean',)
('The Chronicles of Narnia',)
('MarioKart',)
('Splinter Cell',)
('Need for Speed',)
('The DaVinci Code',)
('Cars',)
('Beau Geste',)
('I Remember Mama',)
('Tora! Tora! Tora!',)
('A Man for All Seasons',)
('Hook',)
('Around the World in 80 Days',)
("Harry Potter and the Sorcerer's Stone",)
('Camelot',) |
You can substantially improve on the behavior of the prior example by handling each row one at a time. The following mysql_query.py program reads through the cursor result set one row at a time:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | #!/usr/bin/python # Import sys library. import MySQLdb import sys try: # Create new database connection. db = MySQLdb.connect('localhost','student','student','studentdb') # Create a result set cursor. rs = db.cursor() rs.execute("SELECT item_title FROM item") # Assign the query results to a local variable. for i in range(rs.rowcount): row = rs.fetchone() print row[0] except MySQLdb.Error, e: # Print the error. print "ERROR %d: %s" % (e.args[0], e.args[1]) sys.exit(1) finally: # Close the connection when it is open. if db: db.close() |
You call the mysql_query.py with the following syntax:
./mysql_query.py |
It returns the following result set:
The Hunt for Red October Star Wars I Star Wars II Star Wars II Star Wars III The Chronicles of Narnia RoboCop Pirates of the Caribbean The Chronicles of Narnia MarioKart Splinter Cell Need for Speed The DaVinci Code Cars Beau Geste I Remember Mama Tora! Tora! Tora! A Man for All Seasons Hook Around the World in 80 Days Harry Potter and the Sorcerer's Stone Camelot |
As always, I hope this helps those looking for this type of solution. The Python tutorial web site teaches you more about the Python Programming Language. You may also find the TutorialsPoint.com site useful while you’re learning and using Python. The MySQLdb User’s Guide teaches more about working writing Python-MySQL library. The MySQLdb implements the Python Database API Specification v2.0.
Ruby-MySQL Program
After you install Ruby and build the Rails framework, you need to create the mysql gem. This blog post shows you how to create the mysql gem and how to write a simple Ruby program that queries the MySQL database.
The first step creates the mysql gem for Ruby programming:
gem install mysql |
It should show you the following:
Fetching: mysql-2.9.1.gem (100%) Building native extensions. This could take a while... Successfully installed mysql-2.9.1 Parsing documentation for mysql-2.9.1 Installing ri documentation for mysql-2.9.1 Done installing documentation for mysql after 0 seconds 1 gem installed |
After you install the mysql Ruby Gem, you can write and test a test.rb Ruby program that tests a MySQL database connection. The simplest complete code looks like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | # Include Ruby Gem libraries. require 'rubygems' require 'mysql' begin # Create new database connection. db = Mysql.new('localhost','student','student','studentdb') # Print connected message. puts "Connected to the MySQL database server." rescue Mysql::Error => e # Print the error. puts "ERROR #{e.errno} (#{e.sqlstate}): #{e.error}" puts "Can't connect to the MySQL database specified." # Signal an error. exit 1 ensure # Close the connection when it is open. db.close if db end |
You can run the program with the following syntax:
ruby test.rb |
The program prints “Connected to the MySQL database server.” when there’s a student user with a student password that’s authorized to connect to the studentdb database. If any of the values are invalid when creating the connection, the program prints “Can’t connect to the MySQL database specified.”
Having tested the connection, the next query.rb program tests the connection by returning values from a query:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | # Include Ruby Gem libraries. require 'rubygems' require 'mysql' # Begin block. begin # Create a new connection resource. db = Mysql.new('localhost','student','student','studentdb1') # Create a result set. rs = db.query('SELECT item_title FROM item') # Read through the result set hash. rs.each_hash do | row | puts "#{row['item_title']}" end # Release the result set resources. rs.free rescue Mysql::Error => e # Print the error. puts "ERROR #{e.errno} (#{e.sqlstate}): #{e.error}" puts "Can't connect to MySQL database specified." # Signal an error. exit 1 ensure # Close the connection when it is open. db.close if db end |
You can test it with the following command-line syntax:
ruby query.rb |
It returns a data set like this from the item table of my video store example:
+---------------------------------------+ | item_title | +---------------------------------------+ | The Hunt for Red October | | Star Wars I | | Star Wars II | | Star Wars II | | Star Wars III | | The Chronicles of Narnia | | RoboCop | | Pirates of the Caribbean | | The Chronicles of Narnia | | MarioKart | | Splinter Cell | | Need for Speed | | The DaVinci Code | | Cars | | Beau Geste | | I Remember Mama | | Tora! Tora! Tora! | | A Man for All Seasons | | Hook | | Around the World in 80 Days | | Harry Potter and the Sorcerer's Stone | | Camelot | +---------------------------------------+ 22 rows in set (0.00 sec) |
You need the ruby interpreter to run them. You can make the programs standalone operations by putting the following line as the first line in your Ruby programs.
1 | #!/usr/bin/ruby |
Then, you can run the program like this if they have read and execute privileges and are located in the present working directory where you issue the following command:
./mysql_query.rb |
If you want to work with individual columns, please check this subsequent post that shows how you can access individual columns. As always, I hope this helps those trying to get things working.
After posting this somebody asked for books that could help them learn how to write Ruby programs. While books are nice and listed below, I’d start with the tryruby.org web site.
I’d recommend the following books because …
- The Ruby Programming Language is 7 years old now and only covers Ruby 1.8 and 1.9, but its written by David Flanagan and the creator of the Ruby Programming Language – Yukihiro Matsumoto.
- Programming Ruby 1.9 & 2.0: The Pragmatic Programmer’s Guide is more current and a well balanced approach at learning how to write Ruby programs.
- The Well-Grounded Rubyist is the most current book and teaches you how to think about writing Ruby beyond just the syntax. As a Manning book, you can purchase the physical copy and automatically get a downloadable ebook. It’s certainly the best value for the money option provided you already know how to program in at least one other object-oriented programming language.
Subscribe via RSS