MacLochlainns Weblog

Michael McLaughlin's Technical Blog

Site Admin

Archive for the ‘MongoDB’ Category

MongoDB on Ubuntu

without comments

This post shows how to install, configure, and use MongoDB with JavaScript programs. You need to complete each section in the order provided (based on Cherry Server post).

Step #1: MongoDB Installation

Install the prerequisite packages with the following command:

sudo apt install -y software-properties-common gnupg apt-transport-https ca-certificates

Import the public key for MongoDB on your system using the curl command:

curl -fsSL https://pgp.mongodb.com/server-7.0.asc |  sudo gpg -o /usr/share/keyrings/mongodb-server-7.0.gpg --dearmor

Add MongoDB 7.0 APT repository to the /etc/apt/sources.list.d directory:

echo "deb [ arch=amd64,arm64 signed-by=/usr/share/keyrings/mongodb-server-7.0.gpg ] https://repo.mongodb.org/apt/ubuntu jammy/mongodb-org/7.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-7.0.list

Reload the local package index, which refreshes the local repositories and makes Ubuntu aware of the newly added MongoDB repository:

sudo apt update

Install the mongodb-org meta-package:

sudo apt install -y mongodb-org

Verify the installed version of MongoDB with this command:

mongod --version

It should display:

db version v7.0.6
Build Info: {
    "version": "7.0.6",
    "gitVersion": "66cdc1f28172cb33ff68263050d73d4ade73b9a4",
    "openSSLVersion": "OpenSSL 3.0.2 15 Mar 2022",
    "modules": [],
    "allocator": "tcmalloc",
    "environment": {
        "distmod": "ubuntu2204",
        "distarch": "x86_64",
        "target_arch": "x86_64"
    }
}

Step #2: Start MongoDB Service & Shell

You can verify that the installed mongodb is disabled after initial installation with this command:

sudo systemctl status mongod

It should display:

○ mongod.service - MongoDB Database Server
     Loaded: loaded (/lib/systemd/system/mongod.service; disabled; vendor preset: enabled)
     Active: inactive (dead)
       Docs: https://docs.mongodb.org/manual

Exit the output display from the systemctl utility by typing the escape key, a colon (:) and a q in sequence.

You can start the MongoDB service with this command:

sudo systemctl start mongod

Then, check the MongoDB service:

sudo systemctl status mongod

It displays:

● mongod.service - MongoDB Database Server
     Loaded: loaded (/lib/systemd/system/mongod.service; disabled; vendor preset: enabled)
     Active: active (running) since Thu 2024-03-07 16:38:17 MST; 2s ago
       Docs: https://docs.mongodb.org/manual
   Main PID: 33795 (mongod)
     Memory: 79.2M
        CPU: 706ms
     CGroup: /system.slice/mongod.service
             └─33795 /usr/bin/mongod --config /etc/mongod.conf
Mar 07 16:38:17 student-virtual-machine systemd[1]: Started MongoDB Database Server.
Mar 07 16:38:17 student-virtual-machine mongod[33795]: {"t":{"$date":"2024-03-07T23:38:17.642Z"},"s">

You can confirm that the database is up and running by checking if the server is listening on its default port, which is port 27017. Run the ss command to check the port number.

sudo ss -pnltu | grep 27017

It will display:

tcp   LISTEN 0      4096       127.0.0.1:27017      0.0.0.0:*    users:(("mongod",pid=33795,fd=14))

You can enable the mongodb service at startup with the following command:

sudo systemctl enable mongod

It raised the following error:

Created symlink /etc/systemd/system/multi-user.target.wants/mongod.service → /lib/systemd/system/mongod.service.

Now, start the MongoDB Shell (mongosh) by typing either the explicit or implicit MongoDB Shell command. The explicit one uses the port and database path, which are unnecessary when you’ve successfully started the mongosh service. (Please note that at the time of writing this blog post there is erroneous, or obsolete, content on the MongoDB Documentation Enable Access Control web page.

Explicit connection:

mongosh  --port 27017 --db /var/lib/mongodb --help

This version of the command will display most of the options available in MongoDB but it will suppress warning messages.

$ mongosh [options] [db address] [file names (ending in .js or .mongodb)]
 
  Options:
 
    -h, --help                                 Show this usage information
    -f, --file [arg]                           Load the specified mongosh script
        --host [arg]                           Server to connect to
        --port [arg]                           Port to connect to
        --build-info                           Show build information
        --version                              Show version information
        --quiet                                Silence output from the shell during the connection process
        --shell                                Run the shell after executing files
        --nodb                                 Don't connect to mongod on startup - no 'db address' [arg] expected
        --norc                                 Will not run the '.mongoshrc.js' file on start up
        --eval [arg]                           Evaluate javascript
        --json[=canonical|relaxed]             Print result of --eval as Extended JSON, including errors
        --retryWrites[=true|false]             Automatically retry write operations upon transient network errors (Default: true)
 
  Authentication Options:
 
    -u, --username [arg]                       Username for authentication
    -p, --password [arg]                       Password for authentication
        --authenticationDatabase [arg]         User source (defaults to dbname)
        --authenticationMechanism [arg]        Authentication mechanism
        --awsIamSessionToken [arg]             AWS IAM Temporary Session Token ID
        --gssapiServiceName [arg]              Service name to use when authenticating using GSSAPI/Kerberos
        --sspiHostnameCanonicalization [arg]   Specify the SSPI hostname canonicalization (none or forward, available on Windows)
        --sspiRealmOverride [arg]              Specify the SSPI server realm (available on Windows)
 
  TLS Options:
 
        --tls                                  Use TLS for all connections
        --tlsCertificateKeyFile [arg]          PEM certificate/key file for TLS
        --tlsCertificateKeyFilePassword [arg]  Password for key in PEM file for TLS
        --tlsCAFile [arg]                      Certificate Authority file for TLS
        --tlsAllowInvalidHostnames             Allow connections to servers with non-matching hostnames
        --tlsAllowInvalidCertificates          Allow connections to servers with invalid certificates
        --tlsCertificateSelector [arg]         TLS Certificate in system store (Windows and macOS only)
        --tlsCRLFile [arg]                     Specifies the .pem file that contains the Certificate Revocation List
        --tlsDisabledProtocols [arg]           Comma separated list of TLS protocols to disable [TLS1_0,TLS1_1,TLS1_2]
        --tlsUseSystemCA                       Load the operating system trusted certificate list
        --tlsFIPSMode                          Enable the system TLS library's FIPS mode
 
  API version options:
 
        --apiVersion [arg]                     Specifies the API version to connect with
        --apiStrict                            Use strict API version mode
        --apiDeprecationErrors                 Fail deprecated commands for the specified API version
 
  FLE Options:
 
        --awsAccessKeyId [arg]                 AWS Access Key for FLE Amazon KMS
        --awsSecretAccessKey [arg]             AWS Secret Key for FLE Amazon KMS
        --awsSessionToken [arg]                Optional AWS Session Token ID
        --keyVaultNamespace [arg]              database.collection to store encrypted FLE parameters
        --kmsURL [arg]                         Test parameter to override the URL of the KMS endpoint
 
  DB Address Examples:
 
        foo                                    Foo database on local machine
        192.168.0.5/foo                        Foo database on 192.168.0.5 machine
        192.168.0.5:9999/foo                   Foo database on 192.168.0.5 machine on port 9999
        mongodb://192.168.0.5:9999/foo         Connection string URI can also be used
 
  File Names:
 
        A list of files to run. Files must end in .js and will exit after unless --shell is specified.
 
  Examples:
 
        Start mongosh using 'ships' database on specified connection string:
        $ mongosh mongodb://192.168.0.5:9999/ships
 
  For more information on usage: https://docs.mongodb.com/mongodb-shell.

Implicit connection:

mongosh

You should see the following message with any warning messages:

Current Mongosh Log ID:	65ea502a97f4c1e2b7e12af4
Connecting to:		mongodb://127.0.0.1:27017/?directConnection=true&serverSelectionTimeoutMS=2000&appName=mongosh+2.1.5
Using MongoDB:		7.0.6
Using Mongosh:		2.1.5
 
For mongosh info see: https://docs.mongodb.com/mongodb-shell/
 
To help improve our products, anonymous usage data is collected and sent to MongoDB periodically (https://www.mongodb.com/legal/privacy-policy).
You can opt-out by running the disableTelemetry() command.
 
------
   The server generated these startup warnings when booting
   2024-03-07T16:38:17.818-07:00: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine. See http://dochub.mongodb.org/core/prodnotes-filesystem
   2024-03-07T16:38:18.350-07:00: Access control is not enabled for the database. Read and write access to data and configuration is unrestricted
   2024-03-07T16:38:18.350-07:00: vm.max_map_count is too low
------

You can run opt out of the data collection by running the disableTelemetry() command from the Linux command line. Use the following command (a broader explanation is in the MongoDB Telemetry documentation):

mongosh --nodb --eval "disableTelemetry()"

It should return:

Current Mongosh Log ID:	65eab2df3e663bde3711fa2f
Using Mongosh:		2.1.5
 
For mongosh info see: https://docs.mongodb.com/mongodb-shell/
 
Telemetry is now disabled.

You still have three warning messages to deal with at this point. You should fix the vm.max_map_count warning first. This is a Linux kernel issue. You can determine the current value of the vm.max_map_count value with this command:

cat /proc/sys/vm/max_map_count

It should return the system default value:

65530

You can change it at runtime with this command:

sudo sysctl -w vm.max_map_count=262144

However, you must restart the mongod service to see the change in the mongosh shell. There won’t be a warning message for the kernel parameter value being too low until you reboot your operating system. You can restart your mongod service with this command:

sudo service mongod restart

You can make a change to the /etc/sysctl.conf file to ensure the parameter is set to the correct value each time the system reboots. Simply add the following line as the root user or by using the sudo prefacing a text editor or your choice (like vim or nano) to your /etc/sysctl.conf file:

# Adding vm.max_map_count to sysctl.conf defaults.
vm.max_map_count=262144

At this point, you’ve eliminated two of the warning messages. The next step shows you how to enable Access Control. If you want to check the general server status, run the following command from the Linux Command-Line Interface (CLI):

mongosh --eval "db.serverStatus()" > server_status.log

You can inspect the log file, which should be slightly less than 2,000 lines of output with MongoDB a 7.0.6 installation. Using the command from the Linux CLI is generally the easiest way to inspect the output from the db.serverStatus() function, which is just too long to scroll from the console output.

Step #3: MongoDB Enabling Access Control

Connect to the mongosh …

Step #4: MongoDB Installing Node.js and React.js

Install Node.js with the following command:

sudo apt install -y nodejs

You can check the Node.js version with this command:

node -v

v12.22.9

Install the Node.js package manager npm with the following command:

sudo apt install -y npm

You can check the Node.js version with this command:

npm -v

8.5.1

As always, I hope this helps those looking for concise and complete free answer.

Written by maclochlainn

March 7th, 2024 at 11:10 pm

MongoDB on macOS

without comments

It’s important to document things before I ask my students to do them. This walks through a native installation of MongoDB on the macOS. At least, it does when you’ve already installed Xcode. If you installed Xcode sometime ago, you should update it otherwise you may get an installation error like the following:

==> Installing mongodb-community from mongodb/brew
Error: Your Xcode (11.1) is too outdated.
Please update to Xcode 13.1 (or delete it).
Xcode can be updated from the App Store.

The core MongoDB installation document is in the MongoDB 5.0 installation manual. The following is a step-by-step set of instructions for those new to installing something like MongoDB without using the macOS standard package management software.

  1. The first step requires you install Home Brew, as qualified by the Home Brew Installation page. Before you can install Home Brew, you need to install Xcode on your macOS. After installing Xcode, you need to launch a Terminal session on your macOS and run the following command from the macOS command-line interface (CLI):

    /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

    After the installation, which may take several minutes, you should see this final message on the console:

    ==> Next steps:
    - Run brew help to get started
    - Further documentation:
        https://docs.brew.sh
  2. Now that Home Brew is installed, the next step requires you to update the Homebrew formulat for MongoDB. You use the following command to do that for MongoDB 5:

    brew tap mongodb/brew

    You should see something like the following on your console when you update the Homebrew formula:

    Running `brew update --preinstall`...
    ==> Homebrew is run entirely by unpaid volunteers. Please consider donating:
      https://github.com/Homebrew/brew#donations
    ==> Auto-updated Homebrew!
    Updated 1 tap (homebrew/core).
    ==> Updated Formulae
    Updated 5 formulae.
     
    ==> Tapping mongodb/brew
    Cloning into '/usr/local/Homebrew/Library/Taps/mongodb/homebrew-brew'...
    remote: Enumerating objects: 825, done.
    remote: Counting objects: 100% (322/322), done.
    remote: Compressing objects: 100% (236/236), done.
    remote: Total 825 (delta 165), reused 140 (delta 83), pack-reused 503
    Receiving objects: 100% (825/825), 180.91 KiB | 0 bytes/s, done.
    Resolving deltas: 100% (403/403), done.
    Tapped 14 formulae (30 files, 246.8KB).
  3. Assuming your Xcode is curent, you now install MongoDB with the necessary supporting packages:

    brew install mongodb-community@5.0

    You should see something like the following on your console after you install MongoDB 5:

    ==> Summary
      /usr/local/Cellar/mongodb-community/5.0.5: 11 files, 181.5MB, built in 6 seconds
    ==> Running `brew cleanup mongodb-community`...
    Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
    Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
    ==> Caveats
    ==> mongodb-community
    To start mongodb/brew/mongodb-community now and restart at login:
      brew services start mongodb/brew/mongodb-community
    Or, if you don't want/need a background service you can just run:
      mongod --config /usr/local/etc/mongod.conf
  4. MongoDB really requires a background service, so I suggest you use the following to create the service:

    brew services start mongodb/brew/mongodb-community

    You should see something like the following on your console after you install MongoDB 5:

    ==> Successfully started `mongodb-community` (label: homebrew.mxcl.mongodb-community)

    The installation gave you three key binaries:

    • The MongoDB Server Daemon – mongod
    • The MongoDB sharded cluster query router – mongos
    • The MongoDB Shell – mongosh

    The installation also creates configuration, data, and log files based on physical CPU-Chip hardware:

    • Intel Processor
      • Configuration File: /usr/local/etc/mongodb.conf
      • Log Directory: /usr/local/var/log/mongodb
      • Data Directory: /usr/local/var/mongodb
    • Apple M1 Processor
      • Configuration File: /opt/homebrew/etc/mongodb.conf
      • Log Directory: /opt/homebrew/var/log/mongodb
      • Data Directory: /opt/homebrew/var/mongodb
    • You can now start the MongoDB Shell with the following command in a Terminal window:

      mongosh

      You should see the following when you connect to the MongoDB Shell

      Current Mongosh Log ID:	61e4e2c98f023c2b5fc5a905
      Connecting to:		mongodb://127.0.0.1:27017/?directConnection=true&serverSelectionTimeoutMS=2000
      Using MongoDB:		5.0.5
      Using Mongosh:		1.1.8
       
      For mongosh info see: https://docs.mongodb.com/mongodb-shell/
       
       
      To help improve our products, anonymous usage data is collected and sent to MongoDB periodically (https://www.mongodb.com/legal/privacy-policy).
      You can opt-out by running the disableTelemetry() command.
       
      ------
         The server generated these startup warnings when booting:
         2022-01-16T20:26:18.320-07:00: Access control is not enabled for the database. Read and write access to data and configuration is unrestricted
      ------
       
      test>

As always, I hope the instructions are helpful and clear. If you find an issue let em know about it.

Written by maclochlainn

January 16th, 2022 at 8:45 pm

Fun JavaScript Q?

without comments

I got sent a fun JavaScript question from a student. It was fun for two reasons. They were clearly engaged and trying to figure stuff out on their own and they were just beyond basic syntax issues.

The student wanted to know why this code worked:

// Populate an object with one property.
var notempty = {test:"me"}
 
// Test for and enumerate object properties.
for (var property in notempty) {
  if (!notempty.hasOwnProperty(property)) {
    print("No properties")
  }
  else {
    print("[" + property + "]")
  }
}
print("done")

and returned:

[test]
done

but the behavior changed when the initialized object didn’t have any properties. It failed to print the No properties message:

// Populate an object with one property.
var empty = {}
 
// Test for and enumerate object properties.
for (var property in empty) {
  if (!empty.hasOwnProperty(property)) {
    print("No properties")
  }
  else {
    print("[" + property + "]")
  }
} print("done")

it printed:

done

The answer was easy, there were no properties to enumerate, which meant the loop never executes. However, that’s only part of the answer. The complete answer is that the empty object and the default Object.prototype have no properties to enumerate. You should perform a different test before the loop, like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
// Populate an object without properties.
var empty = {}
 
// Test for and enumerate object properties.
if (Object.keys(empty).length !== 0) {
  for (var property in empty) {
    if (empty.hasOwnProperty(property)) {
      print("[" + property + "]")
    }
  }
} else {
  print("No properties")
}
 
// Print complete message.
print("done")

If you run this from the mongo shell, like this

mongo --nodb --norc < object_properties.js

It prints:

MongoDB shell version v4.0.19
No properties
done
bye

If you run this inside the mongo shell with the load function, like

load("object_properties.js")

It prints:

No properties
done
true

However, you can generate a parsing error when you rewrite line 11 in the if-else statement, like so

if (Object.keys(empty).length !== 0) {
  ...
}
else {
  ...
}

The reason for the error is that JavaScript appends a semicolon (;) at the end of each line. Therefore, the if-else-statement appears as an if-only-statement. which raises the syntax error.

It prints:

MongoDB shell version v4.0.19
2021-08-02T13:38:34.672-0600 E QUERY    [js] SyntaxError: expected expression, got keyword 'else' @(shell):1:0
done
bye

It appears the parsing error is a warning message because the code completes. Personally, I would prefer that they not raise a warning message because it actually implies a style preference.

There’s a next level problem you may run into while trying to implement reflection, or the discovery of existing properties and their value types, of JavaScript objects. Here’s that code, which seems to be missing from many tutorials and articles:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
// Create an object with functions.
// Populate an object without properties.
var notEmpty = { first_name: "Clark"
               , last_name: "Kent"
               , getName: function() { return this.first_name + " " + this.last_name }
               }
 
// Test for and enumerate object properties.
if (Object.keys(notEmpty).length !== 0) {
  for (var property in notEmpty) {
    if (notEmpty.hasOwnProperty(property)) {
      print("[" + property + "] value is a [" + typeof(notEmpty[property]) + "]")
    }
  }
} else {
  print("No properties")
}
 
print("done")

This code on line 12 lets you view a property’s value type:

1
typeof(notEmpty[property])

Only the object[property] subscript notation method works when the property’s value is a variable. The other dot notation syntax to access a property’s value in JavaScript, object.property, doesn’t work without a literal value. Actually, it would work if you embed it inside an eval() function call, but the eval() function is bad juju (black magic) in JavaScript coding.

It prints the following:

MongoDB shell version v4.0.19
[first_name] value is a [string]
[last_name] value is a [string]
[getName] value is a [function]
done
bye

While this shows the basic functionality, it would be more effective to create a function and extend it while adding some output formatting. The next example wouldn’t be found in a JSON (JavaScript Object Notation) collection element but would be found in a NodeJS application managing JSON structures. You would define the constructor to match the data properties of the JSON structure.

The nice thing about a JavaScript function is that “It can specify a list of parameters that will act as variables initialized by the invocation arguments. The body of the function includes variable definitions and statement. (JavaScript: The Good Parts, 2008)” Newer approaches allow you to pass an object as a parameter with named notation inside the object.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
// Create an object with functions.
function Person(first_name, last_name) {
             this.first_name = first_name
           , this.last_name  = last_name
           , this.setFirstName =
               function(first_name) {
                 this.first_name = first_name }
           , this.getName =
               function() {
                 return this.first_name + " " + this.last_name }
         }
 
// Create an instance of a variable and print results.
var myPerson = new Person("Clark","Kent")
print("Initial Name: " + myPerson.getName())
 
// Call the added function and print the new value.
myPerson.setFirstName("Lois")
print("Reset Name:   " + myPerson.getName())
print()
 
// Test for and enumerate object properties.
print("Variable Property Names & Values")
print("====================================")
if (Object.keys(myPerson).length !== 0) {
  for (var property in myPerson) {
    if (myPerson.hasOwnProperty(property)) {
      print("[" + property + "] value is a [" + typeof(myPerson[property]) + "]")
    }
  }
} else {
  print("No properties")
}
print("====================================")
print("done")

It prints the following:

MongoDB shell version v4.0.19
Initial Name: Clark Kent
Reset Name:   Lois Kent
 
Variable Property Names & Values
====================================
[first_name] value is a [string]
[last_name] value is a [string]
[setFirstName] value is a [function]
[getName] value is a [function]
====================================
done
bye

While the prior example shows object construction, this one introduces one variable inheriting the Person function’s properties and another inheriting the same properties before adding a new one to only the variable. The inherited properties, which are functions, are methods of the variable by delegation.

Below is the modified code. It changes the variable from myPerson to myPerson1 and lines 22 through 28 add the new myPerson2 variable and setLastName function. Lines 47 to 58 enumerate across the new second variable, myPerson2 to show you that it has a new setLastName function.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
// Create an object with functions.
function Person(first_name, last_name) {
             this.first_name = first_name
           , this.last_name  = last_name
           , this.setFirstName =
               function(first_name) {
                 this.first_name = first_name }
           , this.getName =
               function() {
                 return this.first_name + " " + this.last_name }
         }
 
// Create an instance of a variable and print results.
var myPerson1 = new Person("Clark","Kent")
print("Initial Name: " + myPerson1.getName())
 
// Call the added function and print the new value.
myPerson1.setFirstName("Lois")
print("Reset Name:   " + myPerson1.getName())
print()
 
// Create an instance of a variable and print results.
var myPerson2 = new Person("Bruce","Wayne")
print("Initial Name: " + myPerson2.getName())
 
// Add a function to the local variable.
function setLastName(last_name) { this.last_name = last_name }
myPerson2.setLastName = setLastName
print()
 
// Test for and enumerate object properties.
print("Variable Property Names & Values")
print("====================================")
if (Object.keys(myPerson1).length !== 0) {
  for (var property in myPerson1) {
    if (myPerson1.hasOwnProperty(property)) {
      print("[" + property + "] value is a [" + typeof(myPerson1[property]) + "]")
    }
  }
} else {
  print("No properties")
}
print("====================================")
print()
 
// Test for and enumerate object properties.
print("Variable Property Names & Values")
print("====================================")
if (Object.keys(myPerson2).length !== 0) {
  for (var property in myPerson2) {
    if (myPerson2.hasOwnProperty(property)) {
      print("[" + property + "] value is a [" + typeof(myPerson2[property]) + "]")
    }
  }
} else {
  print("No properties")
}
print("====================================")
print("done")

The program shows you the following output:

MongoDB shell version v4.0.19
Initial Name: Clark Kent
Reset Name:   Lois Kent
 
Initial Name: Bruce Wayne
 
Variable Property Names & Values
====================================
[first_name] value is a [string]
[last_name] value is a [string]
[setFirstName] value is a [function]
[getName] value is a [function]
====================================
 
Variable Property Names & Values
====================================
[first_name] value is a [string]
[last_name] value is a [string]
[setFirstName] value is a [function]
[getName] value is a [function]
[setLastName] value is a [function]
====================================
done
bye

As always, I hope this helps those working through a similar issue.

Written by maclochlainn

August 2nd, 2021 at 5:21 pm

Mongo List Databases

without comments

After you install MongoDB on your Linux system, you can connect without a user name or password and discover version and many other details about the installation, as shown in this prior post.

You can use the db.adminCommand() to understand the existing databases, as qualified in the online documentation. The basic syntax to list all databases is:

db.adminCommand( {listDatabases:1} )

In a generic Linux installation you should see something like the following:

{
        "databases" : [
                {
                        "name" : "admin",
                        "sizeOnDisk" : 32768,
                        "empty" : false
                },
                {
                        "name" : "config",
                        "sizeOnDisk" : 73728,
                        "empty" : false
                },
                {
                        "name" : "local",
                        "sizeOnDisk" : 90112,
                        "empty" : false
                },
                {
                        "name" : "test",
                        "sizeOnDisk" : 49152,
                        "empty" : false
                }
        ],
        "totalSize" : 245760,
        "ok" : 1
}

You can filter for a test database and find only your authorization to use the test database with the following command:

db.adminCommand( {listDatabases:1, filter: {"name": /test/}, authorizedDatabases: true} )

It returns:

{
        "databases" : [
                {
                        "name" : "test",
                        "sizeOnDisk" : 49152,
                        "empty" : false
                }
        ],
        "totalSize" : 49152,
        "ok" : 1
}

If you just want the names of your authorized databases, use this syntax:

db.adminCommand( {listDatabases:1, nameOnly: true, authorizedDatabases: true} )

It should return:

{
        "databases" : [
                {
                        "name" : "admin"
                },
                {
                        "name" : "config"
                },
                {
                        "name" : "local"
                },
                {
                        "name" : "test"
                }
        ],
        "ok" : 1
}

As always, I hope this helps those trying to sort through the official documentation and learn how to do things.

Written by maclochlainn

June 16th, 2021 at 11:09 pm

Posted in Linux,MongoDB

Tagged with ,

MongoDB Duplicate Key

without comments

How do you avoid encountering a “duplicate key on update” in MongoDB? Well, that’s an interesting question if you’re coming at it from a relational perspective. The answer is understanding that while there may be a natural key, the update needs to use the unique identifier (or, _id). The unique identifier is essentially a primary key based on a system generated surrogate key, assigned by the insertOne() or insertMany() methods. As you’ll notice in the example, it’s not just a numeric value.

The following builds you a test case and is designed to run from the interactive mongo shell. It assumes you understand enough JavaScript to read the code dealing with the JSON (JavaScript Object Notation) structure.

  1. Insert three rows in a people collection with the insertMany() method:

    > db.people.insertMany([
    ...  {"name" : "joe", "age" : 65}
    ... ,{"name" : "joe", "age" : 20}
    ... ,{"name" : "joe", "age" : 49}])

    it returns the following unique identifiers:

    {
            "acknowledged" : true,
            "insertedIds" : [
                    ObjectId("60a4a67f8d03d82365e994ab"),
                    ObjectId("60a4a67f8d03d82365e994ac"),
                    ObjectId("60a4a67f8d03d82365e994ad")
            ]
    }
  2. Assign the return value from a findOne() method against the people collection to a joe variable:

    > joe = db.people.findOne({"name" : "joe", "age" : 20})

    Typically, you can print local variables in the mongo shell with the print command, like:

    print(joe)

    it returns the following:

    [object BSON]

    The object is a Binary JSON object, which requires you use the following command:

    printjson(joe)

    It prints the following JSON element:

    { "_id" : ObjectId("60a4a67f8d03d82365e994ac"), "name" : "joe", "age" : 20 }
  3. Increment the age value from 20 to 21:

    > joe.age++

    You can show the change in the age member’s value by printing the JSON element with the printjson() function:

    { "_id" : ObjectId("60a4a67f8d03d82365e994ac"), "name" : "joe", "age" : 21 }
  4. If you attempt to replace the JSON structure in the MongoDB collection with anything other than the following, you’ll raise a duplicate key error. You must reference the _id value to update the correct element of the collection. The following syntax works:

    db.people.replaceOne({"_id" : joe._id}, joe)

    it should return the following:

    { "acknowledged" : true, "matchedCount" : 1, "modifiedCount" : 1 }

    You can verify that the change only affected the originally queried row with the following command:

    db.people.find()

    it should return the following:

    { "_id" : ObjectId("60a4a67f8d03d82365e994ab"), "name" : "joe", "age" : 65 }
    { "_id" : ObjectId("60a4a67f8d03d82365e994ac"), "name" : "joe", "age" : 21 }
    { "_id" : ObjectId("60a4a67f8d03d82365e994ad"), "name" : "joe", "age" : 49 }

I thought this example may help others because the O’Reily’s MongoDB: The Definitive Guide example (on pages 35-37) uses a manually typed string, which doesn’t lend itself to a programmatic solutions. It also failed to clarify that value returned from the findOne() method and assigned to the local joe variable would be a BSON (Binary JSON) object; and it didn’t cover any means to easily convert the BSON to a JSON text. The reality is there’s no necessity to cast the BSON into a JSON object, also left out of the text. All that said, it’s still an excellent, if not essential, book for your technical library if you’re working with the MongoDB database.

Written by maclochlainn

May 22nd, 2021 at 11:09 pm

MongoDB Two Step

without comments

Sometimes a bit of humor helps with a new topic. Creating a database in MongoDB is a two-step process, like the Texas Two-Step (a nick name for a country/western two-step danced in common time). A Texas Two Step is a one-two, one-two shuffle which is like the two-step process for how you create a MongoDB database.

While databases in MongoDB are a multiuser sandbox like a relational database, you can’t simply create them and grant them privileges. You must first USE the database and then put a collection (a.k.a., equivalent to a table in MySQL) in it. This blog post shows you how to create a MongoDB play database with an actor collection. It shows you the commands to create the a database with one collection in it and how to verify its existence.

You connect through the mongo shell, like:

mongo

The first-step requires you to connect to the play database even though it doesn’t exist. You do that by typing:

> use play

You can verify you’re in the play database by typing the db command, which the mongo shell treats as an expression. It returns play, as the name of the current database.

However, when you call the shell show dbs helper:

> show dbs

It displays:

admin   0.000GB
config  0.000GB
local   0.000GB

You also should note that the play database still doesn’t exist when you run the JavaScript equivalent to the shell show dbs helper:

> db.getMongo().getDBs()

It returns the following list of databases, which excludes the as yet not created play database:

{
  "databases" : [
    {
      "name" : "admin",
      "sizeOnDisk" : 32768,
      "empty" : false
    },
    {
      "name" : "config",
      "sizeOnDisk" : 61440,
      "empty" : false
    },
    {
      "name" : "local",
      "sizeOnDisk" : 81920,
      "empty" : false
    }
  ],
  "totalSize" : 176128,
  "ok" : 1
}

You create an actor collection (a.k.a. the equivalent of a relational table) with the following syntax, where db maps to the play database that you’re using. Note that you must create or insert one document to begin a document collection. While you can use the insert method, you should use either the newer insertOne() or insertMany() methods.

> db.actors.insertOne({"actor" : {"first_name" : "Chris", "last_name" : "Pine"}, "age" : 40 })

A new call to the shell show dbs helper:

> show dbs

displays your new play database:

admin   0.000GB
config  0.000GB
local   0.000GB
play    0.000GB

You can add two more documents (a.k.a. for rows in a relational database) with the insertMany() method:

> db.actors.insertMany([{"actor" : {"first_name" : "Chris", "last_name" : "Evans"}, "age" : 39 }
                       ,{"actor" : {"first_name" : "Chris", "last_name" : "Pratt"}, "age" : 41 }])

A quick word to the JavaScript novices out there. Don’t forget the square brackets ([{},{},...]) in the insertMany() method call or you’ll get an error like this:

2021-04-12T16:20:13.237-0600 E QUERY    [js] TypeError: documents.map is not a function :
DBCollection.prototype.insertMany@src/mongo/shell/crud_api.js:295:1
@(shell):1:1

You call the shell show collections helper or db.getMongo().getCollectionNames() JavaScript function to display the collections in the play database. The show collections displays a list of collections, and the db.getMongo().getCollectionNames() displays an JavaScript array of collections.

If you’re like me, Mongo’s db convention is a bit risky that I could do something in the wrong database. So, I put the following function into my .mongorc.js (a.k.a., MongoDB Resource file):

prompt = function() { var dbName = db; return dbName + "> " }

It ensures you will see the current database name to the left of the prompt (“>”), like:

play>

You can query the documents from the actors collection with the following:

play> db.actors.find().pretty()

It returns:

{
  "_id" : ObjectId("6074c692813c5a85db9cc9df"),
  "actor" : {
    "first_name" : "Chris",
    "last_name" : "Pine"
  },
  "age" : 40
}
{
  "_id" : ObjectId("6074c7ea813c5a85db9cc9e0"),
  "actor" : {
    "first_name" : "Chris",
    "last_name" : "Evans"
  },
  "age" : 39
}
{
  "_id" : ObjectId("6074c7ea813c5a85db9cc9e1"),
  "actor" : {
    "first_name" : "Chris",
    "last_name" : "Pratt"
  },
  "age" : 41
}

As always, I hope this helps those looking for a solution.

Written by maclochlainn

April 12th, 2021 at 5:57 pm

MongoDB Script Test

without comments

There are many ways to test and edit files. A lot of developers only use their favorite Integrated Developer Environment (IDE) but I find testing script files within the scope of a pipelined set of scripts much faster.

The ability to edit a JavaScript file from within the mongo Shell would be nice but unfortunately, it doesn’t exist. You are able to edit a complex variable with a mechanism quite like the Oracle Database. Rather than leave you hanging on how to edit a complex variable in the mongo shell, here are the steps before launching into how to test your JavaScript files:

  1. You can enter this manually during any connection to the mongo shell or put it in your .mongorc.js configuration file, like this in Fedora with the fully qualified filename:

    EDITOR="/usr/bin/vim"
  2. Let’s say you have a stooges array variable that contains “Moe”, “Curly”, and “Larry” and you want to add “Shemp” to the variable. You can edit the stooges array variable with vi and add “Shemp” to it by typing edit and the stooges variable name.

    edit stooges
  3. Then, you can test the stooges array variable’s new values:

    stooges
  4. It returns the following:

    [ "Moe", "Curly", "Shemp", "Larry" ]

Unfortunately, these changes to the demo array variable will be lost after you break the connection. While it does afford a quick test case, you should make the changes in the external JavaScript file. Then, the change is there the the next time you access the resource file.

Here’s my quick edit and test script technique for MongoDB from your present working directory:

  1. Assume you create a compliment.js test file, like:

    /* Declare an array variable and random index value. */
    var compliment = ["Bashful","Doc","Dopey","Grumpy","Happy","Sleepy","Sneezy"]
    var index = Math.floor(Math.random()*7)
     
    /* Print a welcome message. */
    print("Hello, " + compliment[index] + ".")
  2. You can edit the compliment.js file with vi and test the script interactively from the present working directory.

    • You can edit the file with the following syntax:

      vi compliment.js
    • then, you can test the task.sql file:

      mongo --nodb --norc < compliment.js
    • The --nodb option instructs MongoDB to not connect to a database and the --norc option instructs MongoDB to not load your .mongorc.js file. Effectively, disabling the database connection and loading of the MongoDB resource file (.mongorc.js) lets you test your code in MongoDB’s Javascript shell unencumbered by any overhead from the MongoDB server.

      The foregoing script returns the following:

      MongoDB shell version v4.0.19
      Hello, Sneezy.
      bye
    • If you have the desired outcome, you’re done. However, if you need further change you repeat the process.

As always, I hope this helps those looking for a quick way to accomplish a task.

Written by maclochlainn

April 11th, 2021 at 1:45 pm

Node.js Fedora Install

with 2 comments

I want to add the MEAN (MongoDB, Express.js, Angular.js, and Node.js) stack to my backend server development course. This post documents the installation and configuration of components on Fedora 30.

The first step requires installing the Node package. The Node package also contains the Node package manager (npm). You install the Node packages as the root user or as a sudoer user with the following command.

yum install -y npm

It should produce the following installation log:

Last metadata expiration check: 1:10:42 ago on Wed 08 Jul 2020 06:57:52 PM MDT.
Dependencies resolved.
================================================================================================================================
 Package                         Architecture          Version                                     Repository              Size
================================================================================================================================
Installing:
 npm                             x86_64                1:6.13.4-1.10.19.0.1.fc30                   updates                3.8 M
Installing dependencies:
 nodejs                          x86_64                1:10.19.0-1.fc30                            updates                 88 k
 nodejs-libs                     x86_64                1:10.19.0-1.fc30                            updates                9.1 M
Installing weak dependencies:
 nodejs-full-i18n                x86_64                1:10.19.0-1.fc30                            updates                7.3 M
 
Transaction Summary
================================================================================================================================
Install  4 Packages
 
Total download size: 20 M
Installed size: 91 M
Downloading Packages:
(1/4): nodejs-10.19.0-1.fc30.x86_64.rpm                                                         173 kB/s |  88 kB     00:00    
(2/4): nodejs-full-i18n-10.19.0-1.fc30.x86_64.rpm                                               2.8 MB/s | 7.3 MB     00:02    
(3/4): nodejs-libs-10.19.0-1.fc30.x86_64.rpm                                                    2.7 MB/s | 9.1 MB     00:03    
(4/4): npm-6.13.4-1.10.19.0.1.fc30.x86_64.rpm                                                   1.3 MB/s | 3.8 MB     00:02    
--------------------------------------------------------------------------------------------------------------------------------
Total                                                                                           4.9 MB/s |  20 MB     00:04     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Running scriptlet: npm-1:6.13.4-1.10.19.0.1.fc30.x86_64                                                                   1/1 
  Preparing        :                                                                                                        1/1 
  Installing       : nodejs-libs-1:10.19.0-1.fc30.x86_64                                                                    1/4 
  Installing       : nodejs-full-i18n-1:10.19.0-1.fc30.x86_64                                                               2/4 
  Installing       : npm-1:6.13.4-1.10.19.0.1.fc30.x86_64                                                                   3/4 
  Installing       : nodejs-1:10.19.0-1.fc30.x86_64                                                                         4/4 
  Running scriptlet: nodejs-1:10.19.0-1.fc30.x86_64                                                                         4/4 
  Verifying        : nodejs-1:10.19.0-1.fc30.x86_64                                                                         1/4 
  Verifying        : nodejs-full-i18n-1:10.19.0-1.fc30.x86_64                                                               2/4 
  Verifying        : nodejs-libs-1:10.19.0-1.fc30.x86_64                                                                    3/4 
  Verifying        : npm-1:6.13.4-1.10.19.0.1.fc30.x86_64                                                                   4/4 
 
Installed:
  nodejs-1:10.19.0-1.fc30.x86_64           nodejs-full-i18n-1:10.19.0-1.fc30.x86_64     nodejs-libs-1:10.19.0-1.fc30.x86_64    
  npm-1:6.13.4-1.10.19.0.1.fc30.x86_64    
 
Complete!

After installing the Node package, you should use the Node package manager (npm) to install the Node Monitor nodemon. nodemon is a popular utility that automatically lets you restart Node programs when you make changes to the source code.

While npm is installed as part of the Node package, you must use npm to install the Node Monitor. The following command installs the nodemon globally on your Fedora system. The -g flag lets you install it globally, which is important when you manage package.json files.

npm install -g nodemon

You install nodemon globally but most of your web app or project files will be installed locally. Node is a different paradigm than building an Apache or IIS web application because Node provides a framework for you to build a web server.

Here’s a quick Hello World! example that I borrowed a JavaScript helloworld.js file from an excellent Web Development with Node & Express: Leveraging the JavaScript Stack by Ethan Brown. For those who haven’t worked with JavaScript in years, semicolons are optional now.

/* Construct a web server. */
const http = require('http')
const port = process.env.PORT || 3000
 
const server = http.createServer((req, res) => {
  res.writeHead(200, { 'Content-Type': 'text/plain' })
  res.end('Hello world!')
})
 
server.listen(port, () => console.log(`server started on port ${port}); ` +
                                      'press Ctrl-C to terminate...'))

I put this in /var/www/html/node directory, which is owned by the superuser, root. You need to start the server before accessing it from a browser. You can start the program with the following syntax as a privileged user:

node /var/www/html/node/helloworld.js

Then, you can use the localhost to access it with the following URL:

http://localhost:3000

It will display the following:

Next, you need to use the Node Package Manager (npm) to install the Express.js packages. You do that with the following syntax:

npm install -g express express-generator

It should produce a console out put like the following:

npm WARN deprecated mkdirp@0.5.1: Legacy versions of mkdirp are no longer supported. Please update to mkdirp 1.x. (Note that the API surface has changed to use Promises in 1.x.)
/usr/local/bin/express -> /usr/local/lib/node_modules/express-generator/bin/express-cli.js
+ express@4.17.1
+ express-generator@4.16.1
added 60 packages from 42 contributors in 4.798s

After you install all the packages, you can inspect them with the following command. The packages are found in the /usr/local/lib/node_modules/express directory. The listing is generated from the package.json file on Fedora and Ubuntu Linux.

npm list -g

It should display something like this:

/usr/local/lib
├─┬ express@4.17.1
│ ├─┬ accepts@1.3.7
│ │ ├─┬ mime-types@2.1.27
│ │ │ └── mime-db@1.44.0
│ │ └── negotiator@0.6.2
│ ├── array-flatten@1.1.1
│ ├─┬ body-parser@1.19.0
│ │ ├── bytes@3.1.0
│ │ ├── content-type@1.0.4 deduped
│ │ ├── debug@2.6.9 deduped
│ │ ├── depd@1.1.2 deduped
│ │ ├─┬ http-errors@1.7.2
│ │ │ ├── depd@1.1.2 deduped
│ │ │ ├── inherits@2.0.3
│ │ │ ├── setprototypeof@1.1.1 deduped
│ │ │ ├── statuses@1.5.0 deduped
│ │ │ └── toidentifier@1.0.0
│ │ ├─┬ iconv-lite@0.4.24
│ │ │ └── safer-buffer@2.1.2
│ │ ├── on-finished@2.3.0 deduped
│ │ ├── qs@6.7.0 deduped
│ │ ├─┬ raw-body@2.4.0
│ │ │ ├── bytes@3.1.0 deduped
│ │ │ ├── http-errors@1.7.2 deduped
│ │ │ ├── iconv-lite@0.4.24 deduped
│ │ │ └── unpipe@1.0.0 deduped
│ │ └── type-is@1.6.18 deduped
│ ├─┬ content-disposition@0.5.3
│ │ └── safe-buffer@5.1.2 deduped
│ ├── content-type@1.0.4
│ ├── cookie@0.4.0
│ ├── cookie-signature@1.0.6
│ ├─┬ debug@2.6.9
│ │ └── ms@2.0.0
│ ├── depd@1.1.2
│ ├── encodeurl@1.0.2
│ ├── escape-html@1.0.3
│ ├── etag@1.8.1
│ ├─┬ finalhandler@1.1.2
│ │ ├── debug@2.6.9 deduped
│ │ ├── encodeurl@1.0.2 deduped
│ │ ├── escape-html@1.0.3 deduped
│ │ ├── on-finished@2.3.0 deduped
│ │ ├── parseurl@1.3.3 deduped
│ │ ├── statuses@1.5.0 deduped
│ │ └── unpipe@1.0.0
│ ├── fresh@0.5.2
│ ├── merge-descriptors@1.0.1
│ ├── methods@1.1.2
│ ├─┬ on-finished@2.3.0
│ │ └── ee-first@1.1.1
│ ├── parseurl@1.3.3
│ ├── path-to-regexp@0.1.7
│ ├─┬ proxy-addr@2.0.6
│ │ ├── forwarded@0.1.2
│ │ └── ipaddr.js@1.9.1
│ ├── qs@6.7.0
│ ├── range-parser@1.2.1
│ ├── safe-buffer@5.1.2
│ ├─┬ send@0.17.1
│ │ ├── debug@2.6.9 deduped
│ │ ├── depd@1.1.2 deduped
│ │ ├── destroy@1.0.4
│ │ ├── encodeurl@1.0.2 deduped
│ │ ├── escape-html@1.0.3 deduped
│ │ ├── etag@1.8.1 deduped
│ │ ├── fresh@0.5.2 deduped
│ │ ├── http-errors@1.7.2 deduped
│ │ ├── mime@1.6.0
│ │ ├── ms@2.1.1
│ │ ├── on-finished@2.3.0 deduped
│ │ ├── range-parser@1.2.1 deduped
│ │ └── statuses@1.5.0 deduped
│ ├─┬ serve-static@1.14.1
│ │ ├── encodeurl@1.0.2 deduped
│ │ ├── escape-html@1.0.3 deduped
│ │ ├── parseurl@1.3.3 deduped
│ │ └── send@0.17.1 deduped
│ ├── setprototypeof@1.1.1
│ ├── statuses@1.5.0
│ ├─┬ type-is@1.6.18
│ │ ├── media-typer@0.3.0
│ │ └── mime-types@2.1.27 deduped
│ ├── utils-merge@1.0.1
│ └── vary@1.1.2
└─┬ express-generator@4.16.1
  ├── commander@2.15.1
  ├── ejs@2.6.1
  ├─┬ minimatch@3.0.4
  │ └─┬ brace-expansion@1.1.11
  │   ├── balanced-match@1.0.0
  │   └── concat-map@0.0.1
  ├─┬ mkdirp@0.5.1
  │ └── minimist@0.0.8
  └── sorted-object@2.0.1

You can also create a secure node site (HTTPS) with the following additional steps. They include creating a self-signed secure public and private key. This creates the public key:

openssl genrsa -out key.pem

The openssl command will generate a private key key.pem file. It generates something like the following text message to console:

Generating RSA private key, 2048 bit long modulus (2 primes)
...........+++++
.............................................+++++
e is 65537 (0x010001)

Next, you need to generate a self-signed certificate. You do this in two steps.

  1. Create a Distinguished Name (DN) file. The csr.pem file is the DN file. You need it to create a self-signed certificate:

    openssl req -new -key key.pem -out csr.pem

    It will prompt you for values, like the following:

    You are about to be asked to enter information that will be incorporated
    into your certificate request.
    What you are about to enter is what is called a Distinguished Name or a DN.
    There are quite a few fields but you can leave some blank
    For some fields there will be a default value,
    If you enter '.', the field will be left blank.
    -----
    Country Name (2 letter code) [XX]:
    State or Province Name (full name) []:MiddleEarth
    Locality Name (eg, city) [Default City]:Rivendell
    Organization Name (eg, company) [Default Company Ltd]:Fellowship    
    Organizational Unit Name (eg, section) []:Self
    Common Name (eg, your name or your server's hostname) []:Localhost
    Email Address []:bilbo@loth.org
     
    Please enter the following 'extra' attributes
    to be sent with your certificate request
    A challenge password []:friend
    An optional company name []:Bilbo
  2. Use the DN file to create your secure certificate. The following openssl command creates the certificate file by using your private key key.pem and DN csr.pem files.

    openssl x509 -req -days 9999 -in csr.pem -signkey key.pem -out cert.pem

    It should generate a Secure certificate cert.pem file and return something like the following to the console.

    Signature ok
    subject=C = XX, ST = MiddleEarth, L = Rivendell, O = Fellowship, OU = Self, CN = Localhost, emailAddress = bilbo@loth.org
    Getting Private key

You can put these private key (key.pem) and certificate (cert.pem) files in an ssl subdirectory of the directory where you put the JavaScript program. The following creates a secure server page with the following code.

/* Construct a secure web server. */ 
const https = require('https') 
const fs = require('fs') 
const port = process.env.PORT || 3000 
 
const options = { 
 key: fs.readFileSync('ssl/key.pem'), 
 cert: fs.readFileSync('ssl/cert.pem') 
} 
 
const server = https.createServer((options, res) => { 
 res.writeHead(200, { 'Content-Type': 'text/plain' }) 
 res.end('Hello world!') 
}) 
 
server.listen(port, () => console.log(`server started on port ${port}); ` + 
                                     'press Ctrl-C to terminate...'))

If you try launch your browser using the localhost instead of a DNS or file resolved network name on the designated port, it will raise the following security error:

This site can't be reached
https's server IP address could not be found.
DNS_PROBE_FINISHED_NXDOMAIN

An alternate approach to writing a secure server includes using Express.js library. The syntax changes somewhat and you include two new libraries, as shown below:

/* Construct a secure web server. */ 
const https = require('https') 
const express = require('express') 
const fs = require('fs') 
const app = express() 
const port = process.env.PORT || 3001 
 
const options = { 
 key:  fs.readFileSync('ssl/key.pem'), 
 cert: fs.readFileSync('ssl/cert.pem') 
} 
 
https.createServer(options, app).listen(port, () => { 
 console.log(`Express started in ${app.get('env')} mode ` + 
             `on port + ${port}.`) 
})

This will fail with the following error message if you’re running it with a global installation unless you set the $NODE_PATH environment variable correctly. Without setting the variable you may get the following error message:

internal/modules/cjs/loader.js:638 
   throw err; 
   ^ 
 
Error: Cannot find module 'express' 
   at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15) 
   at Function.Module._load (internal/modules/cjs/loader.js:562:25) 
   at Module.require (internal/modules/cjs/loader.js:692:17) 
   at require (internal/modules/cjs/helpers.js:25:18) 
   at Object.<anonymous> (/var/www/html/node/helloworldsecure.js:3:17) 
   at Module._compile (internal/modules/cjs/loader.js:778:30) 
   at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10) 
   at Module.load (internal/modules/cjs/loader.js:653:32) 
   at tryModuleLoad (internal/modules/cjs/loader.js:593:12) 
   at Function.Module._load (internal/modules/cjs/loader.js:585:3)

Sometimes they’ll advise you to do an individual user installation of Express.js to get past this error but that’s not necessary. You just need to set the $NODE_PATH environment variable as follows:

export NODE_PATH=/usr/local/lib/node_modules

This will enable the JavaScript to work without error and without a specific user installation. Assuming you name either of these programs as helloworldsecure.js, you run them with the following command:

node helloworldsecure.js

You can terminate the program with a Ctrl+c or if use the kill -15 pid command if you started it as a background process. You can find the process ID (pid) with the jobs command.

As always, I hope this is helpful to those starting out with this cool technology stack.

Written by maclochlainn

July 8th, 2020 at 9:10 pm

MongoDB on Fedora 30

without comments

I found that MongoDB instructions changed significantly from a year ago. More or less, there was no pre-configured yum repository. Fortunately, the following web page was very helpful though not specific about Fedora.

Installing MongoDB 4.0 on Fedora 30 requires updating the yum repository. You need to create the mongodb.repo file as the root user in the /etc/yum.repos.d directory. The mongodb.repo file should contain the following:

[Mongodb]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/amazon/2013.03/mongodb-org/4.0/x86_64/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-4.0.asc

After you create the appropriate mongodb.repo file, you can run the following command to install the MongoDB database.

def install mongodb-org

It will produce the following log:

MongoDB Repository                            31 kB/s |  21 kB     00:00    
Dependencies resolved.
=============================================================================
 Package                 Arch        Version              Repository    Size
=============================================================================
Installing:
 mongodb-org             x86_64      4.0.12-1.amzn1       Mongodb      5.8 k
Installing dependencies:
 mongodb-org-mongos      x86_64      4.0.12-1.amzn1       Mongodb       11 M
 mongodb-org-server      x86_64      4.0.12-1.amzn1       Mongodb       20 M
 mongodb-org-shell       x86_64      4.0.12-1.amzn1       Mongodb       13 M
 mongodb-org-tools       x86_64      4.0.12-1.amzn1       Mongodb       29 M
 
Transaction Summary
=============================================================================
Install  5 Packages
 
Total download size: 73 M
Installed size: 213 M
Is this ok [y/N]: y
Downloading Packages:
(1/5): mongodb-org-4.0.12-1.amzn1.x86_64.rpm  18 kB/s | 5.8 kB     00:00    
(2/5): mongodb-org-shell-4.0.12-1.amzn1.x86_ 4.6 MB/s |  13 MB     00:02    
(3/5): mongodb-org-mongos-4.0.12-1.amzn1.x86 1.8 MB/s |  11 MB     00:06    
(4/5): mongodb-org-tools-4.0.12-1.amzn1.x86_ 6.6 MB/s |  29 MB     00:04    
(5/5): mongodb-org-server-4.0.12-1.amzn1.x86 2.5 MB/s |  20 MB     00:08    
-----------------------------------------------------------------------------
Total                                        9.0 MB/s |  73 MB     00:08     
warning: /var/cache/dnf/Mongodb-f722cd88d61a4e38/packages/mongodb-org-4.0.12-1.amzn1.x86_64.rpm: Header V3 RSA/SHA1 Signature, key ID e52529d4: NOKEY
MongoDB Repository                           3.0 kB/s | 1.7 kB     00:00    
Importing GPG key 0xE52529D4:
 Userid     : "MongoDB 4.0 Release Signing Key <packaging@mongodb.com>"
 Fingerprint: 9DA3 1620 334B D75D 9DCB 49F3 6881 8C72 E525 29D4
 From       : https://www.mongodb.org/static/pgp/server-4.0.asc
Is this ok [y/N]: y
Key imported successfully
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                     1/1 
  Installing       : mongodb-org-tools-4.0.12-1.amzn1.x86_64             1/5 
  Installing       : mongodb-org-shell-4.0.12-1.amzn1.x86_64             2/5 
  Running scriptlet: mongodb-org-server-4.0.12-1.amzn1.x86_64            3/5 
  Installing       : mongodb-org-server-4.0.12-1.amzn1.x86_64            3/5 
  Running scriptlet: mongodb-org-server-4.0.12-1.amzn1.x86_64            3/5 
  Installing       : mongodb-org-mongos-4.0.12-1.amzn1.x86_64            4/5 
  Installing       : mongodb-org-4.0.12-1.amzn1.x86_64                   5/5 
  Running scriptlet: mongodb-org-4.0.12-1.amzn1.x86_64                   5/5 
  Verifying        : mongodb-org-4.0.12-1.amzn1.x86_64                   1/5 
  Verifying        : mongodb-org-mongos-4.0.12-1.amzn1.x86_64            2/5 
  Verifying        : mongodb-org-server-4.0.12-1.amzn1.x86_64            3/5 
  Verifying        : mongodb-org-shell-4.0.12-1.amzn1.x86_64             4/5 
  Verifying        : mongodb-org-tools-4.0.12-1.amzn1.x86_64             5/5 
 
Installed:
  mongodb-org-4.0.12-1.amzn1.x86_64                                          
  mongodb-org-mongos-4.0.12-1.amzn1.x86_64                                   
  mongodb-org-server-4.0.12-1.amzn1.x86_64                                   
  mongodb-org-shell-4.0.12-1.amzn1.x86_64                                    
  mongodb-org-tools-4.0.12-1.amzn1.x86_64                                    
 
Complete!

You create a MongoDB service with the following syntax as a privileged user in the sudoer list:

sudo systemctl enable mongodb.service

You can then start the mongod service with the following command as a privileged user in the sudoer list:

sudo systemctl start mongod.service

You confirm that it started with the following command as the same privileged user:

sudo service mongod status

It should produce a log file like this:

● mongod.service - SYSV: Mongo is a scalable, document-oriented database.
   Loaded: loaded (/etc/rc.d/init.d/mongod; generated)
   Active: active (running) since Fri 2019-08-16 14:57:22 MDT; 2min 57s ago
     Docs: man:systemd-sysv-generator(8)
  Process: 128115 ExecStart=/etc/rc.d/init.d/mongod start (code=exited, stat>
    Tasks: 27 (limit: 4661)
   Memory: 47.0M
   CGroup: /system.slice/mongod.service
           └─128131 /usr/bin/mongod -f /etc/mongod.conf
 
Aug 16 14:57:21 localhost.localdomain systemd[1]: Starting SYSV: Mongo is a >
Aug 16 14:57:21 localhost.localdomain runuser[128127]: pam_unix(runuser:sess>
Aug 16 14:57:22 localhost.localdomain runuser[128127]: pam_unix(runuser:sess>
Aug 16 14:57:22 localhost.localdomain mongod[128115]: [30B blob data]
Aug 16 14:57:22 localhost.localdomain systemd[1]: Started SYSV: Mongo is a s>
lines 1-15/15 (END)

You close the service log with a “q“. You can determine your version with the following command:

mongod --version

It should show you something like this:

db version v4.0.12
git version: 5776e3cbf9e7afe86e6b29e22520ffb6766e95d4
OpenSSL version: OpenSSL 1.0.0-fips 29 Mar 2010
allocator: tcmalloc
modules: none
build environment:
    distmod: amazon
    distarch: x86_64
    target_arch: x86_64

You can connect to the MongoDB shell with the following command:

mongo

Inside the MongoDB shell, you can run standard MongoDB commands, like:

> use mydb;
switched to db mydb
> db.version()
4.0.12
> db.stats()
{
        "db" : "mydb",
        "collections" : 0,
        "views" : 0,
        "objects" : 0,
        "avgObjSize" : 0,
        "dataSize" : 0,
        "storageSize" : 0,
        "numExtents" : 0,
        "indexes" : 0,
        "indexSize" : 0,
        "fileSize" : 0,
        "fsUsedSize" : 0,
        "fsTotalSize" : 0,
        "ok" : 1
}
> quit()

As always, I hope this helps those looking for the missing steps.

Written by maclochlainn

August 16th, 2019 at 3:11 pm

MongoDB Update Statement

without comments

While discussing the pros and cons of MongoDB, my students wanted to know how to update a specific element in a collection. Collections are like tables in relational databases.

You create the users collection by inserting rows like this:

db.users.insert(
[
  { contact_account: "CA_20170321_0001"
  , first_name: "Jonathan"
  , middle_name: "Eoin"
  , last_name: "MacGregor"
  , addresses:
    {
      street_address: ["1111 Broadway","Suite 101"]
    , city: "Oakland"
    , state: "CA"
    , zip: "94607" 
    }
  }
, { contact_account: "CA_20170328_0001"
  , first_name: "Remington"
  , middle_name: "Alain" 
  , last_name: "Dennison"
  , addresses:
    {
      street_address: ["2222 Broadway","Suite 121"]
    , city: "Oakland"
    , state: "CA"
    , zip: "94607" 
    }
  }
])

You can query the results with the db.users.find({}) command, or you can query the formatted results with the following command:

db.users.find({}).pretty()

You can provide a simple update of middle_name element of a given collection element with the findAndModify() function. The following queries the users collection to find the JSON middle_name element where the contact_account value is equal to the “CA_20170330_0001” string.

db.users.findAndModify(
  { query: { contact_account: "CA_20170328_0001" }
  , update: { $set: { middle_name: "Alan" }}
  , upsert: false })

After changing the middle_name value from “Alain” to “Alan”, you can query the single element of the collection with the following:

db.users.find({ contact_account: "CA_20170328_0001" }).pretty()

It should return the following:

{
        "_id" : ObjectId("5bd7f69ba135dda917665de7"),
        "contact_account" : "CA_20170328_0001",
        "first_name" : "Remington",
        "middle_name" : "Alan",
        "last_name" : "Dennison",
        "addresses" : {
                "street_address" : [
                        "2222 Broadway",
                        "Suite 121"
                ],
                "city" : "Oakland",
                "state" : "CA",
                "zip" : "94607"
        }
}

You can replace the addresses string element value a collection of elements with the following findAndModify() function:

db.users.findAndModify(
  { query: { contact_account: "CA_20170328_0001" }
  , update:
    { $set:
      { addresses:
        [
          {
            active_status: true
          , start_date : new Date("2018-10-30")
          , street_address: ["2222 Broadway","Suite 121"]
          , city: "Oakland"
          , state: "CA"
          , zip: "94607" 
          }
        , {
            active_status: false
          , start_date: new Date("2017-10-01")
          , end_date : new Date("2018-10-29")
          , street_address: ["2222 Broadway","Suite 121"]
          , city: "Oakland"
          , state: "CA"
          , zip: "94607" 
          }
        ]
      }
    }
  , upsert: false })

You can re-query the modified result set with find() function with the same query syntax as used previously. This looks for a specific member element in the collection by matching the contact_account name’s value pair. It is the same as the one used earlier in this blog post.

db.users.find({ contact_account: "CA_20170328_0001" }).pretty()

It should return the following:

{
        "_id" : ObjectId("5bd7f69ba135dda917665de7"),
        "contact_account" : "CA_20170328_0001",
        "first_name" : "Remington",
        "middle_name" : "Alan",
        "last_name" : "Dennison",
        "addresses" : [
                {
                        "active_status" : true,
                        "start_date" : ISODate("2018-10-30T00:00:00Z"),
                        "street_address" : [
                                "2222 Broadway",
                                "Suite 121"
                        ],
                        "city" : "Oakland",
                        "state" : "CA",
                        "zip" : "94607"
                },
                {
                        "active_status" : false,
                        "start_date" : ISODate("2017-10-01T00:00:00Z"),
                        "end_date" : ISODate("2018-10-29T00:00:00Z"),
                        "street_address" : [
                                "2222 Broadway",
                                "Suite 121"
                        ],
                        "city" : "Oakland",
                        "state" : "CA",
                        "zip" : "94607"
                }
        ]
}

As always, I hope this helps someone.

Written by maclochlainn

October 30th, 2018 at 12:22 am

Posted in Linux,MongoDB,Unix

Tagged with