MacLochlainns Weblog

Michael McLaughlin’s Technical Blog

Site Admin

Archive for the ‘Linux’ Category

What a VMWare Thrill …

with 2 comments

I’d upgraded from VMWare Fusion 2 to 3 and taken care of most instances. A NASTY surprise awaited me when I tried to upgrade my Linux installations this morning.

You MUST to uninstall VMWare tools from your repository of Linux operating systems before upgrading your VMWare Fusion or ESX server. If you don’t, you can’t access the console because the drivers make the console look like this:

Perhaps I missed this note when, as an early adopted I opted to move straight to VMWare Fusion 3. I would have complied with these instructions to avoid this headache.

All that’s required now, is that: (a) I export 500 GBs worth of virtual machines to another machine running VMWare Fusion 2; (b) Individually start each machine and run the uninstall VMWare Tools command individually; and, (c) Shutdown and reposition all virtual machines on the original server.

As stated in the note, the command to remove it is:

/usr/bin/vmware-tools-uninstall.pl

Click on the note in the event the link fails to resolve, which would mean the note vanishes into oblivion some day in the future …

While I’ve another machine that’s not yet upgraded, this is a major inconvenience. It’s a shame that the Linux components don’t install automatically. It’s a shame that the install didn’t say something like, “Don’t do this if you’ve Linux virtual machines, unless you’ve removed their VMWare Tools installation.”

Another word to the wise, you don’t get the Linux Tools automatically when you download the product. The software prompts you to download the additional components when you attempt to launch a Linux environment. A cruel irony since by the time you see the prompt, you can’t! This is a change from the prior upgrade process.

Yes, haste makes waste but now I know. In the future, treat all VMWare upgrades like those from Microsoft test, re-test, re-authenticate in a small way before upgrading. Do you think VMWare really want to send that message to its customer base?

I got back to this tonight, and thanks to Red Hat’s Session Manager I was able to fix the Red Hat VMs. Launching it, I simply switched to the Failsafe Terminal and ran the following command, as per the note:

# /usr/bin/vmware-uninstall.pl

After that, I rebooted. Then, installed (mounted the VMWare Tools disk) from the VMWare Fusion menu. Opening a terminal as root, I then re-installed and configured VMWare Fusion. Those are done. As more time allows, I’ll update about the others.

I’m now reconfiguring the network since the VMWare Fusion 2 bridged at a 172.16.153 subdomain and VMWare Fusion 3 bridges at a 172.16.123 subdomain.

Written by maclochlainn

February 6th, 2010 at 11:49 am

The ereg() function is gone

with one comment

Alas, poor ereg() I abused you well. PHP 5.3 has deprecated ereg() and now we must move forward with preg_match(). Along with that change, somebody asked me to show how to upload images to the file system as opposed to the database. Personally, I think they should be stored in the database.

With my bias toward databases, I threw in a virtual directory mapping in a MySQL database because it doesn’t natively support an Oracle equivalent BFILE data type. You can see this older post how to use the DBA_DIRECTORIES view in Oracle to mimic this behavior.

Naturally, MySQL is the preferred database of the person asking the question. You could also implement this exactly the same in Oracle but you really don’t want to do so. Using Oracle’s virtual directories has it’s own pre-built set of security features. They provide a more robust solution.

The code is presented as follows (setup for MySQL instructions):

  1. Create and seed the DIRECTORY table in MySQL:
-- Create a directory table.
CREATE TABLE directory
( directory_id   int PRIMARY KEY AUTO_INCREMENT
, virtual_name   varchar(30)
, directory_name varchar(60));
 
-- Seed the table with a virtual directory mapping.
INSERT INTO directory VALUES ( NULL,'CMS_IMAGES','C:\\Data' );
  1. Create a MySQLCredentials.inc credentails file for inclusion in the PHP program:
1
2
3
4
5
6
7
<?php
  // Connection variables.
  define('HOSTNAME',"localhost");
  define('USERNAME',"student");
  define('PASSWORD',"student");
  define('DATABASE',"sampledb");
?>
  1. Create the PHP uploading program, named MySQLFileUpload.php:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
<?php
  // Set database credentials.
  include_once("MySQLCredentials.inc");
 
  // Declare input variables.
  $id = (isset($_POST['id'])) ? $_POST['id'] : 1021;
 
  // Upload a file to server in a mapped physical drive location.
  if (process_uploaded_file(map_virtual_directory($id)))
    print "Successfully Uploaded the file.<br />";
 
  // Map a virtual directory to a physical directory.
  function map_virtual_directory($virtual) {
 
    // Return successful attempt to connect to the database.
    if (!$c = @mysqli_connect(HOSTNAME,USERNAME,PASSWORD,DATABASE)) {
 
      // Print user message.
      print "Sorry! The connection to the database failed. Please try again later.";
 
      // Return error message.
      print mysqli_error();
 
      // Kill the resource.
      die();
    }
    else {
 
      // Initialize a statement in the scope of the connection.
      $stmt = mysqli_stmt_init($c);
 
      // Declare a case insensitive dynamic SQL statement.
      $sql = "SELECT directory_name FROM directory WHERE virtual_name = UCASE(?)";
 
      // Prepare the statement.
      if (mysqli_stmt_prepare($stmt,$sql)) {
 
        // Bind the input parameter to the prepared statement.
        mysqli_stmt_bind_param($stmt,'s',$virtual);
 
        // Execute the prepared statement.
        if (mysqli_stmt_execute($stmt)) {
 
          // Bind the result to a local variable.
          mysqli_stmt_bind_result($stmt,$directory);
 
          // FetchPrepare statement and link it to a connection.
          while (mysqli_stmt_fetch($stmt))
            return $directory;
        }
        else
          // Return error message.
          print mysqli_error();
      }
      else
        // Return error message.
        print mysqli_error();
 
          // Disconnect from database.
      mysqli_close($c);
    }
  }
 
  // Manage file upload.
  function process_uploaded_file($directory) {
 
    /* Assume the application may allow a virtual directory with a trailing backslash or forward
       slash to be stored in the database, and manage both scenarios across Windows and Linux. */
    if (preg_match(".Win32.",$_SERVER["SERVER_SOFTWARE"]))
      if (preg_match("/\b\\\/",$directory));
      else if (preg_match("/\b\//",$directory)) {
        $directory = substr($directory,0,strlen($directory)-1);
        $directory = $directory."\\";
      }
      else $directory = $directory."\\";
    else
      if (preg_match("/\b\//",$directory))
        $directory = substr($directory,0,strlen($directory)-1);
      else
        $directory = $directory."/";  
 
    // Check for, move uploaded file, and confirm processing.
    if (is_uploaded_file($_FILES['userfile']['tmp_name'])) {
 
      // Move temporary cache into a file directory with the uploaded file name.
      move_uploaded_file($_FILES['userfile']['tmp_name'],$directory.$_FILES['userfile']['name']);
 
      // Remove this from real code, it's here for example only. ;-)
      print "Uploaded [".$_FILES['userfile']['name']."] to".$directory."<br />";
 
      // Return true to encapsulate the functional logic on success.
	    return true;
    }
    else
      // Return false to encapsulate the functional logic on failure.
	    return false;
  }
?>
  1. Create a web page to test it:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
<html>
<head>
<title>
  UploadFileFormMySQL.htm
</title>
</head>
<body>
<form id="uploadForm"
      action="MySQLFileUpload.php"
      enctype="multipart/form-data"
      method="post">
  <table border=0 cellpadding=0 cellspacing=0>
    <tr>
      <td width=125>Item Number</td>
      <td>
        <input id="id" name="id" type="text">
      </td>
    </tr>
    <tr>
      <td width=125>Select File</td>
      <td>
        <input id="uploadfilename" name="userfile" type="file">
      </td>
    </tr>
    <tr>
      <td width=125>Click Button to</td>
      <td><input type="submit" value="Upload File"></td>
    </tr>
  </table>
</form>
</body>
</html>

Hope this helps a few folks. I imagine that the prepared statement with bound variables may help a few folks because it’s not found (at writing) on the php.net web site.

Written by maclochlainn

December 29th, 2009 at 2:03 am

Posted in LAMP, Linux, MAMP, PHP, Zend

VMWare nabs me again …

without comments

When I run into failures on VMWare Fusion, they’re always a bit tedious. This one happened on my iMac (OS X Leopard) running VMWare 2 (both constrained to old releases by university governance policies). The VM is Microsoft Vista in an IDE partition, it hung after running too long. I had to force quit the application. On reboot the socket file was still there, and it gave the following error message when trying to start it.

VMwarefusionvirtualdevice0

Here’s the error in plain text, so search engines can find it for others.

Virtual device serial0: File "/var/folders/Sf/SfvoJITAHMq1Vp8bNI7QZU+++TM/-Tmp-//vmware-mmclaugh/thnuclnt-641/socket" exists, but no server is listening to it.
 
There are three possible causes for this:
 - The server is alive but not ready yet, and you can retry later.
 - The server is busy communicating with another client, so you cannot run this client at the same time.
 - A previous server exited abruptly, and you can remove the file and try again.
 
The device will be disconnected.

How to fix it?

Delete the file, right? Yes, but there’s a trick. Navigating through the -Tmp- directory required a Unix shell trick because the - (dash) is a switch and backquoting it with a \ (backslash) didn’t work. Jeff Yoder, told me the trick to change directory into a dash leading directory name. It was this:

cd -- -Tmp-

The -- is how most shells mark the end of options to a command. After a -- all - (dashes) are treated as ordinary characters.

Mark Olaveson reminded me that using the present working directory before the directory name also worked. It demotes the dash to an ordinary character too.

cd ./-Tmp-

When I got to the directory, there was the socket file. I deleted it and everything worked like a charm.

srwxrwxrwx  1 mmclaugh  staff     0 Dec  8 13:04 socket

Written by maclochlainn

December 8th, 2009 at 3:43 pm

Posted in Linux, Mac OS X, VMWare, bash

sudo conferred powers

with 3 comments

Coming from Solaris Unix and Red Hat Linux to Ubuntu, Mac OS X and other distributions of Linux was interesting a couple years ago. The fact that you couldn’t su to the root account was tedious. It didn’t take too long (a couple momentes) to recall that I could assume those privileges in a shell, like:

admin_user@machine_name:~$ sudo sh
#

Naturally, this avoids entering sudo before a series of commands and makes administering any box simpler. It should work on all distributions but I’ve not checked ALL of them because they’re too numerous anymore. I know it works on the Mac OS X, Ubuntu, and now Fedora distributions.

Today, I got a kick from the message provided by Fedora 10 when you assume root permissions. It’s been over 20 years since I got that lecture on an AT&T box at First Interstate Bank. I imagine that any equivalent box to that is in a museum, while that bank was acquired by Wells Fargo in the early 1990s. The message from Fedora is just too funny to pass on making a comment. Here’s the screen shot:

SuperUserPowers

Hope it brought a smile to some faces …

Written by maclochlainn

December 6th, 2009 at 12:59 pm

Posted in Linux, Mac, Ubuntu

Toad for MySQL Freeware

without comments

While SQL Developer is a nice tool and free, Toad is still an awesome development platform, albeit for Windows. In fact, it was annoying to have to install the Microsoft .NET Framework before installing it. It is free for MySQL!

Since my students have to do all their work in Oracle and then port it to MySQL, I demonstrate Quest’s Toad for MySQL’s at the end of the term. I don’t want them to leverage the automatic ERD diagramming while they’re learning how to do it.

There’s only one real trick to making automatic ERD diagramming work. That trick requires that you write your loading scripts for the Inno DB and use referential integrity constraints. My sample Video Store scripts for my database class are updated for MySQL referential integrity.

ToadERDModel

Unlike the friendly CASCADE CONSTRAINTS clause you can use in Oracle, MySQL won’t let you create a re-runnable script with only DDL statements. Actually, the constraint comes from the InnoDB engine. You must issue a specialized InnoDB command before running your script:

11
12
13
-- This enables dropping tables with foreign key dependencies.
-- It is specific to the InnoDB Engine.
SET FOREIGN_KEY_CHECKS = 0;

Primary keys are a bit different from Oracle and it appears you can’t name them, at least I couldn’t see how to do it. Here’s an example of primary and foreign key constraints in MySQL. The primary key is inline and the foreign key constraints are out of line. This example from the downloadable scripts uses self referencing foreign key constraints.

24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
CREATE TABLE system_user
( system_user_id              INT UNSIGNED PRIMARY KEY AUTO_INCREMENT
, system_user_name            CHAR(20)     NOT NULL
, system_user_group_id        INT UNSIGNED NOT NULL
, system_user_type            INT UNSIGNED NOT NULL
, first_name                  CHAR(20)
, middle_name                 CHAR(20)
, last_name                   CHAR(20)
, created_by                  INT UNSIGNED NOT NULL
, creation_date               DATE         NOT NULL
, last_updated_by             INT UNSIGNED NOT NULL
, last_update_date            DATE         NOT NULL
, KEY system_user_fk1 (created_by)
, CONSTRAINT system_user_fk1 FOREIGN KEY (created_by)
  REFERENCES system_user (system_user_id)
, KEY system_user_fk2 (last_updated_by)
, CONSTRAINT system_user_fk2 FOREIGN KEY (last_updated_by)
  REFERENCES system_user (system_user_id)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

Once installed (instructions are here) and connected to the MySQL database, you simply click the ERD icon in the top panel and drag the tables onto the canvas. You’ll see something like this (by the way click on the image to see its full size):

ToadQuery

Have fun with it. It’ll be interesting to see how Oracle positions MySQL when they own it. My hunch is that they’ll continue to sell it and provide it as an open source product.

Written by maclochlainn

July 11th, 2009 at 5:03 pm

Configuring Zend Server

with one comment

I got all the screen shots out last night, then I realized that the configuration instructions were missing. They’ve been added now, and you can check if you’re interested.

In the process, I noticed that Zend Server Community Edition is using connection pooling by default with Oracle. This means that the TNS alias must be qualified by a hostname.

For example, on your local PC with Oracle Database 10g Express Edition, the default TNS alias is XE. You can’t use that anymore when you’re running the Zend Server. You must qualify it as localhost/XE or hostname/XE as the connection string. A natural alternative is to insert a fully qualified TNS network map

Here are three possible connection patterns:

I’m sure it was in the documents but then again, I didn’t read them. ;)

Written by maclochlainn

July 2nd, 2009 at 11:39 pm

PHP, LOBs, and Oracle

without comments

I finally got around to summarizing how to use PHP to store, retrieve, and display CLOBs and BLOBs from an Oracle database. I think too often we default to BFILEs. I put all the code in zip files with instructions and suggestions for locations. This is really the second entry that goes with configuring Zend Server Community Edition or the DEPRECATED Zend Core for Oracle.

If you’re new to PHP, check out the Underground PHP and Oracle book from Christopher Jones and Alison Holloway. It’s FREE!

The Oracle LOB Processing entry is in this blog page. I know it means another click, but I’ll probably add and modify it over time. If you’ve got time and interest, take a look and let me know what you think and what improvements you’d like to see. Thanks.

Written by maclochlainn

June 29th, 2009 at 8:35 pm

Posted in LAMP, Linux, MAMP, OPAL, Oracle, PHP, pl/sql, sql

Excel date conversion

with 2 comments

I put together a post on how to upload to MySQL from a CSV file with dates. It was more or less for my students but one of them was curious how the mega formula worked. As commented, the easier solution is to create a custom format. Oddly, Open Office does support the MySQL default format natively.

Excel doesn’t support the native MySQL date format, which is YYYYMMDD, or 20090602 for June 2, 2009. That means you have to convert it from a scalar date to a string. If you just shook your head at the term scalar date, let me explain that Excel supports only three data types. They’re a string literal, a numeric literal, and a formula. Dates in Excel are merely formatted numbers. When the numbers are integers, the date is a date, but when the number has a fractional component, the date is really a timestamp.

Here’s a brief description of the process required to convert a date in Excel into a MySQL date format string literal in a CSV file. You need the following Excel functions:

Date Functions

  • The DAY(date) function returns a 1 or 2 digit numeric value for the day of the month, with ranges of 1 to 28, 1 to 29, 1 to 30, or 1 to 31 dependent on the month and year.
  • The MONTH(date) function returns a 1 or 2 digit numeric value for the month of the year.
  • The YEAR(date) function returns a 4 digit numeric value for the year.

Logical Functions

  • The IF(logical_expression,truth_action,false_action) function returns the truth action when the expression is true, and the false action when the expression isn’t true.

MySQL Server

  • CONCATENATE(string_1, string_2, ...) glues strings together.
  • LEN(numeric_value) function returns the length of a string or number.

MySQL requires that you return an eight character string of numbers. The first four numbers must be a valid year, the fifth and sixth numbers a valid month, and the seventh and eigth numbers a valid day in the context of the year and month provided. Unfortunately, the DAY() and MONTH() functions may return a 1 or 2 digit value. That can’t happen in the CSV file’s string for a date, so you use the IF() and LEN() functions to guarantee a 2 digit return value.

Here are the examples that guarantee 2 digit day and month values, assuming that the base date is in the A1 cell. The concatenation of a "0" (zero between two double quotes) or the "" (two double quotes or a string null) ensures the number data types become strings.

=IF(LEN(DAY(A1)) = 1,CONCATENATE("0",DAY(A1)),CONCATENATE("",DAY(A1)))
=IF(LEN(MONTH(A1)) = 1,CONCATENATE("0",MONTH(A1)),MONTH(A1))

A zero is placed before the day or month when the logical condition is met, which means the day or month value is a single digit string. A null is place before the day or month when the logical condition isn’t met, which means the day or month value is a two digit string.
There’s only one problem with these mega functions. They return a number.

The year calculation doesn’t require the explicit casting when you concatenate it with the other strings because it is implicity cast as a string. However, it’s a better practice to include it for clarity (most folks don’t know about the implicit casting behaviors in Excel).

=CONCATENATE(CONCATENATE("",YEAR(A1)),IF(LEN(MONTH(A1)),CONCATENATE("0",MONTH(A1))),IF(LEN(DAY(A1)) = 1,CONCATENATE("0",DAY(A1)),CONCATENATE("",DAY(A1))))

You can see the full MySQL import from CSV in the previous post. Naturally, you may want to copy and paste special the value before creating the CSV file. Also, don’t forget to delete any unused columns to the right or rows beneath because if you don’t your file won’t map to your table definition.

Written by maclochlainn

June 16th, 2009 at 8:15 pm

MySQL MSI Service Fails

with 3 comments

While installing the MySQL 6.0 Alpha release, I encountered a failure running the configuration component. It shows the following dialog, which hangs until you cancel it. By the way, I’ve encountered this on other MySQL 5.0 and 5.1 installs from time to time.

mysql6servicefailure

Don’t uninstall and reinstall because nothing will change. The only problem appears to be setting the root password. This show you how to verify it and fix the missing configuration step. While the service says it failed, it actually started. You can check that by launching services.msc from Start and Run.

You can verify the problem by attempting to connect to the MySQL server. My server is setup on localhost with port 3308 because there are multiple MySQL servers running on my virtual machine. A typically connection would look like this if your password was cangetin (the old Solaris training password):

C:\>mysql -uroot -pcangetin -P3308

If you get the following error message, it’s most likely a missing root password.

ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES)

Since my machine is running multiple MySQL servers and it’s my preference to associate their execution to their binaries, the paths to the installations aren’t loaded automatically on installation. A quick caution, my path statements are from the Windows XP 64-bit installation and they’ll differ from a 32-bit installation path. Specifically, the executable programs are in C:\Program Files (x86) directory not C:\Program Files. You can set the path like this:

C:\>set PATH=C:\Program Files (x86)\MySQL 6.0\MySQL Server 6.0\bin;%PATH%

To verify and fix the problem requires you login without a password, connect to the mysql database, and query the user table. All those steps follow below, unless you’re on Microsoft Vista. If you’re running Microsoft Vista follow these instructions.

C:\>mysql -uroot -P3308
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 6.0.10-alpha-community MySQL Community Server (GPL)
 
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
 
mysql> USE mysql;
Database changed
 
mysql> SELECT host, user, passowrd
    -> FROM user WHERE user='root' AND host='localhost'\G
*************************** 1. row ***************************
                  Host: localhost
                  User: root
              Password:
1 row in set (0.00 sec)

You fix this problem by running the following grant of privileges to the root user:

mysql> GRANT ALL ON *.* TO 'root'@'localhost'
    -> IDENTIFIED BY 'cangetin' WITH GRANT OPTION;
Query OK, 0 rows affected (0.00 sec)

I learned this technique by attending the MySQL for Database Administrator’s course. I hope it solves a mystery for somebody along the way. I also hope that Oracle Education maintains the excellent folks that Sun Microsystems acquired when they snagged MySQL.

Written by maclochlainn

May 7th, 2009 at 11:40 pm

VMWare and Fedora 10

without comments

It sometimes gets tiresome to sort the VMWare compatibility issues with Linux release-by-release but at the end of the cycle it’s fine. I downloaded the current Fedora 10 Live release because it is so much smaller than the install disk. It uses a kernel of 2.6.27.5-117.fc10.i686. I encountered a critical errors when I tried to install the VMWare Toolkit from VMWare Fusion, version 2.0.3 (156731). The only error guidance you’ll get is that there isn’t any make file.

There’s a twist here if you’re most familiar with Ubuntu and sudo command execution. You need to su to the root account and run everything as root user. An alternative would be to edit the /etc/inittab file to allow the root user to boot the instance. The default is 5 (X11). You enable the root user by changing it to 3 (full multiuser mode). Don’t forget to change it back, you really shouldn’t log in as the root user. Anyway, you’ll have to do it to run the VMWare Toolkit successfully as shown later in the post.

You fix the incompatibility of the VMWare Fusion Toolkit by downloading the following two missing RPMs. You should be able to find them here. As to why they’re not in the Live DVD distribution, who knows.

binutils-2.18.50.0.9-8.fc10.i386.rpm
gcc-4.3.2-7.i386.rpm
glibc-2.9-2.i686.rpm
glibc-devel-2.9-2.i386.rpm
glibc-headers-2.9-2.i386.rpm
kernel-devel-2.6.27.5-117.fc10.i686.rpm
kernel-headers-2.6.27.5-117.fc10.i386.rpm
libgomp-4.3.2-7.i386.rpm

You can’t run some of them through the RPM utility, so you should probably run them all at the command line. The command line syntax and sequence for these packages is:

rpm -ivh kernel-headers-2.6.27.5-117.fc10.i386.rpm
rpm -ivh kernel-devel-2.6.27.5-117.fc10.i686.rpm
rpm -ivh binutils-2.18.50.0.9-8.fc10.i386.rpm
rpm -ivh libgomp-4.3.2-7.i386.rpm
rpm -ivh glibc-headers-2.9-2.i386.rpm
rpm -ivh glibc-devel-2.9-2.i386.rpm
rpm -ivh glibc-2.9-2.i686.rpm
rpm -ivh gcc-4.3.2-7.i386.rpm

As the root user, you can now install the VMWare Toolkit. While running the installation, you’ll receive a prompt to confirm the /usr/src/linux/include. Don’t accept the default path because it won’t work unless you created a symbolic link. The kernel header files require you to enter the following path:

/usr/src/kernels/2.6.27.5-117.fc10.i686/include

Everything should work finxs e when you compile the modules. Hope this helps a couple folks.

Written by maclochlainn

April 11th, 2009 at 6:01 pm

Posted in Linux, Mac, Red Hat, VMWare