Moving a GroupWise Post Office

For most administrators, moving the post office to a new server is the most unsettling part of a move to a new server. While certainly having an MTA or GWIA or WebAccess down can be frustrating, the thought that something might “go wrong” during a post office move causes some sleepless nights.

We won’t say that there are no dangers in the move to a new server. Frankly, any time you do something as major as changing network operating systems there are difficulties. Learning the new operating system is generally the most work for a Linux move. Most NetWare administrators have at least a general knowledge of Windows networking. But getting everything “just right” is what we’re here to help you do!

You should always look at moving all but the very smallest of post offices as a two stage process. The first stage involves preparing the server and making a first pass copy of the post office data. We are often asked why we do a first pass and second pass of the copy of data from the source server to the new server. It’s simple: time. Unless you can afford to have your post office down for the entire time it takes to copy every file in your post office to the new server, you need to do two passes.

To explain this better, we’ll look at how DBCopy works. DBCopy is optimized to only copy files that are new or have changed since the last run of the utility. When you run DBCopy with only the parameters required to do the job, and no additional switches, DBCopy looks at each file that it is asked to copy and checks to see if there is already a file by that name in the target directory. If the file does not exist in the target directory, DBCopy copies the file. If the filename already exists in the target directory, DBCopy checks the timestamp and only copies the file if the source file is newer.

To understand this even better, we will briefly go over how GroupWise stores data. As you know, there are three data directories for a basic GroupWise post office (we aren’t counting document management directories here just yet). There is the ofuser directory, the ofmsg directory, and the offiles directory structure. When GroupWise delivers a piece of mail, a pointer is put into the ofuser directory that contains the header information for the message (sender, recipient, subject, message size, etc.). In theory then, the actual message is placed in the ofmsg database. This is true, of all message 4KB and smaller. In practice of course, with the increased use of HTML messages, these small messages are becoming fewer and fewer. Any message or attachment that is larger than 4KB is package up in a BLOB file (binary large object) and placed in the offiles directory structure. And there it sits, totally unchanged, until it is finally deleted and purged by the sender and all recipients and it can finally be removed from the system.

Thus, if you were to look at your GroupWise post office data, you would find that the vast majority (oftentimes over 90%) of the data in your post office is static information in the offiles directories. That information only needs to be copied once to your new server. And if that takes 10 hours, 20 hours, even 40 hours, you can sit back and let it happen a few days or weeks before your final move. Then on the actual “migration day” only files that have been created or modified since the first pass will need to be copied. Novell has actually also added an “incremental” switch that allows you to enter a specific date for the copy, thus even avoiding much of the checking against timestamps and file size. So, for example, if you were to make your first copy on December 30, 2011, and on January 7, 2011 you are doing the actual migration, you could put the “-i 01-06-2011” switch to prevent DBCopy from trying to copy files older than your last copy. And notice that we say “older than your last copy” here. You could technically run DBCopy multiple times prior to your final migration. For example, run DBCopy on December 30, 2011, then again on January 6, 2012, and on the 7th when it is time for the actual migration, only one day’s worth of data would need to be copied again.

  • Stage One – this stage can be done as many times prior to your actual migration day as you choose.
    • Create a mount point on Linux for the NetWare server, or map a drive from Windows to the NetWare server.
    • Perform our first pass DBCopy of our post office.
    • Configure the POA on (we won’t load it yet, but it just saves us some time on that stressful migration day!).
  • Stage Two – Migration Day
    • Unload the POA for the post office
    • Rename the post office directory to avoid any accidental access while we are relocating the post office.
    • Perform our final DBCopy pass.
    • Load ConsoleOne and edit the post office location, agent address and link information if necessary.
    • Verify that the HTTP monitor for the POA is properly enabled.
    • Load the POA for the relocated post office.
    • Log into the post office and verify that all prior data is present.
    • Send a message to a user on the same post office, and verify that it is received.
    • Verify that the POA is communicating with its domain’s MTA and force the new MTP settings through the HTTP Monitor if necessary.
    • Send a message to a user on another post office if applicable and verify that it was received
    • Send a message to an external recipient through your GWIA and verify that it was received.
    • Send a message from an external sender to your user through the GWIA and verify that it was received.

Migrating our Post Office – Stage One

Because our post office data store is large, and because we cannot afford to have our e-mail system down for two or three days while we migrate the data, we will make an initial copy of the data while the post office is up and running. Other than some disk I/O increase, our users will not even know we are there!

Stage One can be started quite a long time before the actual migration, but there are a couple of things to keep in mind. The longer in advance you do your first DBCopy, the more “junk” you will have in your offiles directories to try to clean up later. As we noted above when we discussed how DBCopy works, the utility only copies new or newer files to the target location. However, DBCopy does no “reconciliation” of files on the servers. In other words, if you do a first pass copy of your post office on September 1, 2009, and then don’t do your actual migration until September 30, 2009, anything in the offiles structure that has been deleted from the live post office in the meantime will still exist on the Linux server. It would be unwise, therefore, to do a first pass copy before instructing users to clean our their mailboxes and purge their trash! In that case, you would have cleaner databases, but you would still have old files in the offiles directories. And we all know (at least those of us who have been around for any time at all know!) how difficult it is to get rid of old orphaned files in the offiles directories. Thus, it is best to try to have your users clean up before you do your first pass with DBCopy.

Once you have your server prepared and ready for your migration, and you are within your reasonable “migration window” for starting your first pass DBCopy run, it’s time to get down to business.

Attach to Your NetWare Server

Linux – Create a Mount Point for the NetWare Server

In order to move your post office from NetWare, you need to attach to the source server from your new Linux server. To mount the NCP volumes into your Linux file system, perform the following steps:

  1. On the Linux server, open up a terminal window.
  2. “su root” to become root for this session.
  3. Create a directory structure for your mount point. For our purposes, we have created /mnt/gw1 as the mount point for our NetWare or OES2 server that is named gw1. It is suggested that you replace gw1 with the actual name of your source server for ease of recognition later.
  4. We have chosen to connect to our NetWare server via ncpfs. To mount the NetWare server via ncpfs, issue the following command

 

ncpmount -S server -A 123.123.123.123 -U userid -P password /mnt/gw1

replacing your server name for “server” and your server’s IP address for 123.123.123.123. For example

ncpmount -S gw1 -A 192.168.100.200 -U danita.cnc -P password /mnt/gw1

 

NOTE: You might wonder about the “user.context” vs. “.user.context” in this example. While “.user.context” works on some versions of Linux, it fails on others. In my testing, “user.context” has always worked on all versions. You might keep this in mind if you experience any problems authenticating with ncpmount.

 

This will actually mount all volumes from the NetWare server to this location. So, if we cd to /mnt/gw1 will will see a directory called SYS, one called GWDATA, one called GWDOCS and any other volumes that are on that server.

Windows – Map a Drive to Your NetWare Server

It may be that your login script will automatically map a drive for you from your new Windows server to your NetWare server. Otherwise, map a drive by finding the volume in Windows Explorer, right-clicking the volume, and choosing Map Network Drive.

Performing Our First Pass DBCopy of the Post Office.

Decide where you wish to have your post office located on your server (local partition, SAN, etc.) and make sure that your location is ready to receive your data. It is not necessary to have actually created the post office directory on this partition, as the DBCopy utility will create it for you.

Linux

In our example, our NetWare volume is mounted at /mnt/gw1/GWDATA/pos/caledoni. Our new location for our data will be in /grpwise/pos/caledoni.

The absolutely most important thing to remember when performing your DBCopy from NetWare to Linux is the -m switch. This is the “migrate” switch, and it changes the case for all of the data to lowercase. We can’t stress enough how critical this is. We have been involved in migrations where this switch was forgotten. Especially if you plan on doing more than one “first pass” you must be extremely careful. There is nothing more frustrating than having a DBCopy run for 20 hours the first time, all files dutifully converted to lowercase, and then come back for another pass, forgetting the -m switch, and having duplicates of all file names in two different cases! In fact, some of our customers have been so wary of this, that they simply run the Change Case utility that we discussed back in our preparation chapter prior to beginning their migration first pass just to lessen the damage that would be incurred by forgetting the -m switch!

If you have an older version of GroupWise, and your ngwguard.db file on the NetWare server is not all lowercase, it is advisable to change that prior to starting the DBCopy. You must take down your POA in order to change the case for this file of course. While newer versions of DBCopy should do okay with an uppercase ngwguard.db file, we have seen issues with the -m switch failing, and it’s just better to be safe than sorry!

So, getting down to business, here’s how we will copy our data.

 

You will find that DBCopy on the Linux server does not run well from any directory other than /opt/novell/groupwise/agents/bin so we will cd to that location to run the utility. Run DBCopy to copy your files from the source server to the Linux location like this:

./dbcopy -m -p <source> <destination>

so for our situation it would be

./dbcopy -m -p /mnt/gw1/GWDATA/pos/caledoni /grpwise/pos/caledoni

This will copy all needed data from the current post office directory into the directory structure that we have indicated for the destination. In our case, we have chosen to location our post office in /grpwise/pos/caledoni. Our post office is named “Caledonia”. While in theory we should be able to have a post office directory of more than 8 characters on Linux, we do not do this for compatibility and consistency sake.

The -m switch copies the files over in all lower case. The -p switch tells DBCopy that it is migrating a post office. If you are extremely curious and just love watching screens fly by to prove that something is happening (don’t be shy – Danita is just like that herself!), you can also include the -w switch. This writes all of the data to the “window” so that you can see each file as it is copied. If you DO enable this switch though, do not expect to be able to really tell where you are in the copy. DBCopy launches multiple threads (by default 5) to do the copying, and does not seem to always follow any particular order in the directory structure.

Windows

In our example, our current Windows volume is mapped to f: at f:\pos\cal2. Our new location for our data will be in d:\pos\cal2.

So, getting down to business, here’s how we will copy our data.

From a command prompt on your Windows workstation, change to the location where you can copied the DBCopy directory. Run the command as follows:

dbcopy.exe /p <source> <destination>

so for our situation it would be

dbcopy.exe /p f:\pos\cal2 e:\pos\cal2

This will copy all needed data from the current post office directory into the directory structure that we have indicated for the destination.

The /p switch tells DBCopy that it is copying a post office. If you are extremely curious and just love watching screens fly by to prove that something is happening (don’t be shy – Danita is just like that herself!), you can also include the /w switch. This writes all of the data to the “window” so that you can see each file as it is copied. If you DO enable this switch though, do not expect to be able to really tell where you are in the copy. DBCopy launches multiple threads (by default 5) to do the copying, and does not seem to always follow any particular order in the directory structure.

 

Go get a cup of coffee. Or dinner. Or watch a movie. Or just go home for the weekend! This can take a long time, and there is almost no way to calculate it for you. The time it takes you to do current backups of your full post office is a start at an estimation. The faster the connection between your new Linux server and the source server the better. And of course, if you are migrating from an ncp server, ncp isn’t necessarily the fastest of protocols out there, but it requires no special configuration on your NetWare server, and is Novell’s recommended way to copy the data from NetWare to Linux. While on some versions of NetWare you could serve up the post office via CIFS, it probably isn’t worth the effort to try to configure this solely for the purpose of this migration. And under no circumstances should you attempt to do any type of NFS mount to your source server. For the vast majority of sites, you will find it easier and safer to just follow the instructions outlined here for using ncpfs for NetWare or cifs for Windows and having the data copy over via DBCopy that way.

Install and Configure the Agent Software

After the first run of DBCopy, it is advisable to go ahead and configure your POA so that it will be ready to load on migration day.

The configuration process will create a new startup file for the POA, and configure the agent to launch upon startup.

Linux

In a terminal window, go back to the location where you extracted your GroupWise distribution above in Preparing Your Server. Run the install script as root. For example:

 

gwlinux:/sdd/gw12soft # ./install

 

  • Choose Install Products.
  • Choose GroupWise Agents.
  • Choose Configure Agents.
  • At the License Agreement screen, accept the license.
  1. The next screen will look very familiar to GroupWise administrators (see Figure 8-1). This is where we list the agents that will run on this server to create startup files and the startup script for Linux.

 

    1. Figurel-d7.tiffGroupWise Agents Configuration

 

 

  1. Enter the information for your post office, using the local Linux path (see Figure 8-2)

 

    1. Figurel-p1.tiffPost Office Information

 

  • After your post office information is entered, click next to move on to the startup screen.
  • You will be asked if you wish for your agent to load on startup. This box is already checked, but really just looks like a red box. You will notice that if you click on the box, it becomes white (unchecked) and clicking again will recheck the box. It is advisable to have your agents load on startup.
  • Click Exit to leave this setup, and then click the Exit icon (at the bottom right of the screen) to leave the GroupWise installation program.

 

When you perform these tasks, a startup file is created for the post office’s POA, and if you chose to launch on startup, symbolic links were created in the proper run level directories to start the agent automatically. Looking at the /opt/novell/groupwise/agents directory you will see directories named bin, lib and share. Under bin you will see the agent executables, as well as default startup templates. Under the share directory are a couple of directories, and the caledoni.poa startup file that was created by our installation. Let’s run this agent now and see what it looks like.

Windows

To install the software for your agent, you can use your GroupWise CD, Software Distribution Directory, or simply download new software from the Novell downloads site (http://download.novell.com).

In the root of the location of your GroupWise software, you will see the setup.exe program that will be used for this installation. Our installation here is using GroupWise 8.0. Your agent installation may vary slightly. When you run this program, you see the Window in Figure 8-3.

    1. install1.tiffThe main installation screen

 

  • Click on Install GroupWise System.
  • Click Yes on the next screen to accept the license agreement.
    1. Click Next on the next screen to do a standard install. You will see the window in Figure 8-4.
    2. Install GroupWise AgentsFigureN2.tiff

 

 

 

  • “Install Individual Components”.
  • Choose GroupWise Agents as an option to install.
  • At the next screen you will be prompted if you wish to install on NetWare or Windows. Of course we will choose Windows here.
  • The next screen will give you the option of where to install your files on your Windows server, and options for the installation.
    • Install and configure SNMP for GroupWise Agents: If SNMP is available and installed on your Windows server, you can configure your agents here for SNMP.
    • Install as Windows services: Most Windows sites will configure GroupWise agents as services. Otherwise the server would need to remain logged in in order for the agents to run.
    1. After you choose the options on this screen you will click next and be presented with the window in Figure 8-5.
      1. configure1.tiffConfiguring the GroupWise Agents

 

While we are looking at this screen, we should explain its usage so that it’s more easily understood the next time you encounter it. There are a few misconceptions about what is done here, and what you need to put in this location, so we will try to clear those up.

First, this screen is a template for creating new agent startup files (for example the caledoni.poa file), and for configuring your grpwise.ncf file. Let’s take the example of our groupwise system. Our domain is called CNC. If we want to create a new startup file for the Caledonia POA, and place it in our grpwise.ncf (provided that we checked the option above to “Launch the GroupWise agents on system startup”), we would click the Add button as shown in Figure 8-5 and enter the information for our post office.

  1. Adding a post ofice for configurationFigureN7.tiff

By entering this information, we are instructing the installation routine to create us a startup file called caledoni.poa with a /home switch of “d:\pos\caledoni and create Windows services or icons for the post office agent.

The installation does not validate the “name” you put here and indeed, you can name things here anything you like. For example, rather than naming my startup file “caledoni.poa” as shown above, I could just as easily type “cal” in the name field, and the name of the file would instead be cal.poa. If you put in a name that has more than eight characters (for example Caledonia), the file name will be truncated to eight characters.

  1. Next you will be presented with a summary screen showing your choices for the install, and the installation will proceed.
  2. When the installation has completed, choose to launch your agent now.

It is best to try to avoid restarting this server between now and migration day. It would not be a critical problem, except of course that we have configured the server at this point to automatically load the POA on startup. This should not pose any great problems if it were to load unattended, but it would be best to not have to worry about the POA running when it comes time to do our final copy.

As mentioned earlier, you can do a “first run” copy any time you choose between now and the final migration day. It’s usually not that important to do though, unless you want to incrementally reduce the amount of time the copy takes on the final day.

Migration Day

Today’s the day, and we will complete our migration of the post office to our new server. Users have been alerted that the e-mail system will be unavailable for the duration, and we are ready to proceed.Here are the tasks that we will complete in order to accomplish our migration:

  • Unload the source server POA for the post office
  • Rename the post office directory to avoid any accidental access while we are relocating the post office.
  • Perform our final DBCopy pass.
  • Load ConsoleOne and edit the post office location, agent address and link information if necessary.
  • Verify that the HTTP monitor for the POA is properly enabled.
  • Load the POA for the relocated post office.
  • Change the ngwnameserver entries in our local DNS if applicable
  • Log into the post office and verify that all prior data is present.
  • Send a message to a user on the same post office, and verify that it is received.
  • Verify that the POA is communicating with its domain’s MTA and force the new MTP settings through the HTTP Monitor if necessary.
  • Send a message to a user on another post office if applicable and verify that it was received
  • Send a message to an external recipient through your GWIA and verify that it was received.
  • Send a message from an external sender to your user through the GWIA and verify that it was received.

 

It seems like a lot of work, but some of it can be done while the post office is doing its final copy.

 

Unload the POA

On your NetWare server, press F7 at the agent console to unload the agent

 

Rename the post office directory

Just to be safe, we always rename the post office directory on the source server to ensure that no one (or no utility) can access the post office while we are making our final copy.

Perform our final DBCopy pass.

Our final DBCopy pass will be very similar to the first. The big differences here are that we will use the “interval” switch to limit what work DBCopy must do, and we will alter the source directory to reflect that we have renamed our post office directory.

Linux

You will find that DBCopy does not run well on the Linux server from any directory other than /opt/novell/groupwise/agents/bin so we will cd to that location on the destination server to run the utility. Run DBCopy to copy your files from the source server to the destination Linux server location like this:

./dbcopy -m -p -i mm-dd-yyyy <source> <destination>

so let’s assume that we did our last DBCopy to the Linux server on January 6, 2012. Our command would be

./dbcopy -m -p -i 01-05-2012 /mnt/gw1/GWDATA/pos/caledoni.old /grpwise/pos/caledoni

We always make the -i switch the day before our last copy just to be safe!

 

StoreLowerCase on Linux

If all goes well with your DBCopy, a gwcheck with a “storelowercase” option will be issued by the utility at the very end. This tells the ngwguard.db that all files in the GroupWise system are now lowercase. The only time you should ever have to run a gwcheck with a “storelowercase” option yourself if if you do not use DBCopy with an -m and -p switch. We bring this up, because there are still some TIDs floating about that recommend that you DO run a “storelowercase”. And while there are instances where you indeed might need to do this manually, it CANNOT BE STRESSED ENOUGH that you should NEVER run a gwcheck with a “storelowercase” while the POA is loaded. Unfortunately, ConsoleOne will actually allow you to request such a thing through Mailbox/Library Maintenance, which of course will have the POA run the gwcheck itself. This can cause no end of suffering. Please do not do this.

Windows

 

dbcopy.exe /p /i mm-dd-yyyy <source> <destination>

so let’s assume that we did our last DBCopy to the new Windows server on January 6, 2012. Our command would be

dbcopy.exe /p /i 01-05-2012 f:\pos\caledoni.old d:\pos\caledoni

We always make the /i switch the day before our last copy just to be safe!

 

Again, this will take some time. It’s difficult to say how long exactly. However, in a recent migration we did, the initial DBCopy of all of the post office files took about 20 hours, and the final DBCopy took two hours.

While this is processing we can continue to do a bit of work that needs completing.

Additional Migration Tasks to Do While the Final Copy Continues

There are a few steps that need to be accomplished before we are finished with our migration.

Edit the Details of the Post Office in ConsoleOne

Load ConsoleOne and if possible connect to the domain that owns this post office. While the changes could be done from any domain location, being connected to the domain that owns the post office aids in getting the changes propagated more efficiently.

  • Edit the properties of the post office and change the UNC path of the post office to point to the new location.
  • Edit the properties of the POA object, and change the following:
    • On the Identification screen, change the Platform to Linux or Windows
    • On the Agent Settings screen (click the triangle on the GroupWise tab to change to this screen), notice the HTTP Monitor settings. If you have never enabled the HTTP monitors for your agents, you will need to decide on a good userid and password for the HTTP monitors. Please note that this is neither an eDirectory user nor a GroupWise user. This is an entirely made up user and password solely for the use of the HTTP monitors. If all administrators in your organization will have rights to use the HTTP monitors, then it is a good idea to have the same userid and password for all agents. If you need to limit rights to some agents to various groups, set up a userid and password for each group of agents that will be monitored. Enter the userid and password that you have decided on here in this screen.
    • On the Network Address screen, change the IP address of your POA to reflect the IP address of the new server. Make sure that a valid port number is listed in the HTTP Monitor field, and note what that port is.
    • On the Log Settings tab, remove any custom log locations and leave the logging to the default location. Best to make sure the log is also set to verbose!
    • Click OK to save all of the changes.

 

NGWNameserver

If your site is using an ngwnameserver value in DNS to direct users to the POA, you should change this now to point to the new IP address for your POA. This will ensure that users have no disruption of service after the POA comes back up. The DNS server will simply redirect the users to the proper IP address after the migration.

 

And now we wait. There really isn’t much left to do until the copy is complete!

 

Now you can continue on with the next chapter to upgrade your post office.