Avoiding problems when upgrading from MongoDB 3.x to 4.x

If you are not familiar with the concept of backward incompatibilities, then you have probably not yet survived a major upgrade! To give you an idea of how important it is to be aware of this when reviewing the change log for MongoDB, this concept is also referred to as code breaks...as in things that can break your code.

MMAPv1 storage engine

MMAPv1 (https://docs.mongodb.com/manual/storage/#mmapv1-storage-engine), the original MongoDB storage engine, has been deprecated in MongoDB 4.0 and removed as of MongoDB 4.2. It has been replaced by WiredTiger, which has been available since MongoDB version 3.0. WiredTiger has been the default since MongoDB version 3.2, so there is a good chance that this backward-incompatible change does not affect your applications nor installation.

If your installation was using the MMAPv1 storage engine before the upgrade, then you immediately notice more efficient memory and disk-space allocation. Simply put, MMAPv1 grabbed as much free memory as it could, and would allocate additional disk space with an insatiable appetite. This made a MongoDB 3 installation using MMAPv1 a bad neighbor on a server that is also doing other things!

Another difference that DevOps engineers appreciate is that embedded documents no longer continue to grow in size after being created. This was an unwanted side effect produced by the MMAPv1 storage engine, which could have potentially affected write performance and lead to data fragmentation.

If, as a result of an upgrade from MongoDB 3 to 4, you are in the unfortunate position of having to update a database that is stored using the MMAPv1 storage engine, then you must manually back up the data on each server prior to the MongoDB 4.x upgrade. After the upgrade has completed, you then need to perform a manual restore.

Replica set protocol version

Communications between members of a replica set are governed by an internal protocol simply referred to as pv0 or pv1.  pv0 was the original protocol. Prior to MongoDB 3.2, the only version available was pv0. MongoDB 4.x dropped support for pv0 and only supports pv1. Accordingly, before you perform an upgrade to MongoDB 4, you must reconfigure all replica sets to pv1 (https://docs.mongodb.com/manual/reference/replica-set-protocol-versions/#modify-replica-set-protocol-version).

Fortunately, this process is quite easy, and can be accomplished by this simple procedure. These steps must then be repeated for each replica set: verify oplog entry replication and upgrade to pv1. Let's go into more detail regarding these two steps.

Verifying that at least one oplog entry has replicated

These steps must be performed on each secondary in the replica set:

1. Use the mongo shell to connect to each secondary in the replica set:

mongo --host <address of secondary>

1. Once connected to the secondary, run the rs.status(); command and check the values of the optimes::appliedOpTime::t key.
2. Repeat this for each secondary and confirm that the t value is greater than -1. This tells us that at least one oplog entry has replicated from the primary to all secondaries.

Upgrading the primary to protocol version 1

You can now upgrade the replica set protocol version to pv1:

1. Use the mongo (https://docs.mongodb.com/manual/mongo/#the-mongo-shell) shell to connect to the primary in the replica set:

 mongo --host <address of primary>

1. Once connected to the primary, run these commands to update the protocol version for the replica set:

cfg = rs.conf();
 cfg.protocolVersion=1;
 rs.reconfig(cfg);

Feature compatibility

A number of the new features available in MongoDB 4.x only work if you update the setFeatureCompatibilityVersion parameter (https://docs.mongodb.com/master/reference/command/setFeatureCompatibilityVersion/#setfeaturecompatibilityversion) in the admin database. The new features affected include the following:

• SCRAM-SHA-256
• New type conversion operators and enhancements
• Multidocument transactions
• $dateToString option changes
• New change stream methods
• Change stream resume token data type changes

To view the current featureCompatibility setting, go through the following steps:

1. Use the mongo shell to connect to your database as a user who has the rights to modify the admin database:

mongo --username <name of user> --password

1. Use this command to view the current setting:

db.adminCommand({getParameter:1, featureCompatibilityVersion:1})

To perform the update, go through the following steps:

1. Use the mongo shell to connect to your database as a user who has the rights to modify the admin database:

mongo --username <name of user> --password

1. You can then update this parameter as follows, substituting 4.0, 4.2, 4.4, and so on in place of <VERSION>:

db.adminCommand( { setFeatureCompatibilityVersion: "<VERSION>" } )

User authentication

When establishing security for database users, you have a choice of several different approaches. One of the most popular approaches is challenge-response. Simply put: the database challenges the user to prove their identity. The response (in most cases), is a username and password combination. In MongoDB 3, this popular approach was implemented by default using MONGODB-CR (MongoDB Challenge Response). As of MongoDB 4, this mechanism is no longer available. This means that when you upgrade from MongoDB 3 to MongoDB 4, you must implement at least its replacement, Salted Challenge Response Authentication Method (SCRAM). 

If your user credentials are in MONGODB-CR format, then you must use the following command to upgrade to SCRAM format:

db.adminCommand({authSchemaUpgrade: 1});

It is critical that you perform this upgrade while still running MongoDB 3. The reason for this is that the authSchemaUpgrade parameter has been removed in MongoDB 4! Another side effect of upgrading to SCRAM is that your application driver might also be affected. Have a look at the SCRAM Driver Support table in the documentation at https://docs.mongodb.com/manual/core/security-scram/#driver-support to be sure. The minimum programming language driver for Python that supports SCRAM authentication, for example, is version 2.8. 

Removed and deprecated items

Any command or executable binary that is removed can potentially cause problems if you are relying on these as part of your application or automated maintenance procedures. Any application that relies upon an item that is been deprecated should be examined and scheduled to be rewritten in a timely manner.

Removed items

The following table shows the removed items:

Significant removed items

The following table shows the removed items:

These commands were deprecated in MongoDB 4.0 and removed as of MongoDB 4.2.

Deprecated SSL configuration options

Another compatibility issue comes from the TLS/SSL configuration options. In MongoDB 3 and MongoDB 4.0, in the configuration files for both mongod (MongoDB database daemon) and mongos (which is used to control a sharded cluster), you could add a series of options under the net.ssl (https://docs.mongodb.com/v4.0/reference/configuration-options/#net-ssl-options) key. As of MongoDB 4.2, these options are deprecated in favor of the net.tls options. The net.tls options have enhanced functionality compared with the net.ssl options. These are covered in detail in Chapter 11, Administering MongoDB Security.