Migration Tool - Release Candidate 1

Introduction

This technical note contains functional and technical information for the migration tool that is used to migrate EPiServer CMS sites from EPiServer CMS 4.62 to EPiServer CMS 5 SP2.

The EPiServer CMS 4.62 database is migrated in place and receives all the necessary configuration on the target CMS 5 site. After migration, the CMS 4.62 database has been converted to the CMS 5 schema, the CMS 5 site has configuration settings that map to the CMS 4.62 configuration, and its connection string points to the converted CMS 4.62 database.

Table of Contents

• Preparing for Migration
• User Handling
• Register Custom Event Handlers
• Language ID Conversion
• Execution Order
  » Migrate Object Store Data in the Database
  » Migrate Other Data in the Database
  » Migrate Enterprise Sites
  » Migrate Unified File Systems
  » Change LongString Properties to XhtmlString
  » Migrate Links
  » Migrate Page Types
• What Happens After Migration?
• Troubleshooting

Preparing for Migration

There are certain steps that must be carried out before you can migrate your site to EPiServer CMS 5 SP2. These are described below.

• Microsoft SQL Server Database Only.
  The Migration tool works only when EPiServer CMS is configured with a Microsoft SQL 2005 database only.  You need to update to Microsoft SQL 2005 Server first, before using the migration tool. 
   
• Back up the CMS 4.62 database.
  It is recommended that you run the migration on a backup of the database. Check that there is no other traffic towards the site/database during migration. NOTE - The migration process is very database intensive. Increase the performance of the migration process by setting the "Recovery model" for the database to SIMPLE during migration.
  
  
• If necessary, upgrade the EPiServer CMS site to be migrated to 4.62 using EPiServer CMS Manager.
  EPiServer CMS 4.62 is the only supported version for migrating to EPiServer CMS 5.
   
• Install an unmodified EPiServer CMS 5 SP2 site on same machine as the 4.62 site.
  Use EPiServer CMS Manager to install the EPIServer CMS 5 SP2 site.
   
• Install the migration tool on the same machine as the sites.
   
• Ensure that the EPiServer 4.62 assemblies are already installed on the EPiServer CMS 5 SP2 site machine - in the GAC (Global Assembly Cache).  Copy the assemblies to the CMS 5 site machine if they do not exist there. The EPiServer 4.62 assemblies and their versions are listed at the bottom this tech note.
   
• Remove any custom tables added in the database with constraints towards any EPiServer CMS tables.
  For example, if there is a custom table that has a foreign key that is a column in an EPiServer CMS table. The reason for this is that during migration the existing EPiServer CMS tables will be dropped, which requires that no other tables have constraints towards the tables.
   
• Remove or convert any Form properties to XForm properties.
  Check whether any Form properties need to be managed by starting the migration tool, select the different web.config files and click Next. The migration tool checks for any Form properties and displays a message indicating which properties need to be managed before migration. 
   
• Exclude any file systems that you don't want to migrate.
  The migration tool will migrate all native file systems to versioning file systems. If there are any file systems (or part of a file system) that you don't want to migrate, make sure the configuration excludes that before migration. See Migrate Unified File Systems
   
• Remove old page versions.
  One part of the migration is to migrate links from classic format to a permanent link format. During this process each version of each page will be parsed and resaved. If your site has a lot of old versions for pages, we recommend that you remove some of the old versions before migration. 
   
• Seperate page folders.
  If you have all your uploaded content in one folder it is recommended that before you begin to migrate that you separate the page folders from the upload folder.
  
  If you want your page files from 4.62 to be in a separate folder you could take all folders named as a number (those are your page files) in your upload folder and move them to another folder and name that folder "Page Files". Then you add another handler in web.config so it looks something like the configuration below (change the physical path to a valid one):
  <handler pagedirectory="False" virtualname="upload" virtualshare="True" type="EPiServer.FileSystem.Handler.NativeFileSystem,EPiServer">
      <customsettings physicalpath="c:\Upload\Public\upload" />
     </handler>
     <handler pagedirectory="True" virtualname="Pages" virtualshare="True" type="EPiServer.FileSystem.Handler.NativeFileSystem,EPiServer">
      <customsettings physicalpath="c:\Upload\Public\pages" />
 </handler>
  You will also have to add mappings for the folders:
  <mappings>
     <map fromPath="/upload" toPath="/pages" type="EPiServer.FileSystem.Mapping.PageDirectoryMapping,EPiServer" />
 </mappings>
  Documentation explaining how to use PageDirectoryMapping to separate page folders from the upload folder is available in the Unified File System technical note - see the section entitled Move Folders Around / Mappings.

Recommendation:
To minimize the risk of encountering problems whilist migratiing EPiServer CMS 4.62 to EPiServer CMS 5 SP2, it is our recommendation that the migration tool and the SQL Server exist on the same machine whilst migration is being performed. If this is not a possibility and you encounter problems during migration see the troubleshooting section or even the migration tool FAQ.

User Handling

In EPiServer CMS 4, users were identified using a SID (int based). In the ASP.NET membership model used in EPiServer CMS 5, a user is identified by his/her username. During migration all references to users/roles (e.g. regarding access rights) are migrated from a SID reference to a username reference, meaning all access rights will be kept by migration. However, the actual users and roles will not be migrated.

If you were using using internal EPiServer CMS users, i.e. users entirely stored in the EPiServer CMS database, in EPiServer CMS 4, you must configure your EPiServer CMS 5 to use SqlMembershipProvider and then recreate the users/roles after migration. By recreating them with same username as they had in EPiServer CMS 4, all access rights will then be preserved. If you are using Windows or AD users, you are only required to configure your CMS 5 system to use WindowsMembershipProvider or ActiveDirectoryMembershipProvider.

Register Custom Event Handlers

The migration tool exposes a lot of extension points, which makes it possible to register external event handlers that will be called during migration. To register an external event handler, add an entry under the extensions configuration element in the App.config file for the migration tool.

Example:

<?xml version="1.0"?>
<configuration>
  ...  
<migration>
  <extensions>
    <add name="MyCustomHandler" type="CustomMigration.CustomMigrater, CustomMigration" />
  </extensions>
</migration>
...
</configuration>

The classes specified as extension modules must implement the EPiServer.Migration.IMigrationExtension interface, which is specified in the EPiServer.MigrationEngine.dll assembly.

public interface IMigrationExtension
{
    void Initialize(MigrationContext context);
}

At startup the migration tool calls the Initialize method for all extension modules. Each extension module will then register event handlers for those extension points in the MigrationContext.Events class that the module wants to act upon.

Language ID Conversion

In EPiServer CMS 4 it was possible to define language codes for languages. In EPiServer CMS 5 each language code must be parseable as a .NET CultureInfo object. During migration, the migration tool parses each language and if a language code is not parseable as a CultureInfo object, a dialog will be displayed, where language mappings can be performed. (Each language code marked with '*' must be changed). An example is the language code for Danish, which in some previous versions was stored as 'DK', but in CMS 5 should be 'DA'.  

Execution Order

The migration tool uses a workflow, based on Windows Workflow Foundation, as the execution engine. The order in which different parts of the migration takes place is explained below. Some of the steps are explained in more detail below.

1. Migrate object store data in the database.
2. Migrate other data in the database.
3. Update database schema to CMS 5 SP2.
4. Update the web.config file on the target site:
   • according to the setting in the 4.62 site.
   • with any enterprise settings.
5. Register the ASP.NET sql files in database.
6. Register workflow (WF) scripts in database.
7. Migrate unified file systems.
8. Change LongString properties to XhtmlString.
9. Migrate links.
10. Migrate page types.
11. Update EPiServer.IndexingService with new file systems that should be indexed.

If the migration is canceled (by clicking Cancel), the migration will stop as soon as it reaches a checkpoint. The next time migration is started, you will be asked if you want to start from where the previous migration was canceled. It will then start executing the same step from the beginning of that step, with the exception of file and link migration, which will continue from where it was at cancellation.

Migrate Object Store Data in the Database

The migration tool handles all EPiServer CMS-specific objects persisted in the database, except objects with schema name WsrpPortalInfo, WsrpPortalPageSession. These objects are not supported by EPiServer CMS 5.

Migration Of Custom ObjectStore Items

Any custom objects stored in EPiServer ObjectStore will not be automatically migrated with the migration tool. However, the migration tool provides an extension point where it is possible to plug in and add your own module that will handle migration of your custom object types.

This example provides a short explanation of how the plug-in should be done.

1. Create a new class library project and add references to :
   • EPiServer.BaseLibrary
   • EPiServer.Implementation
   • EPiServer.Migration.ObjectStore462
   • EPiServer.MigrationEngine
2. Create a new class and add references to:
   using EPiServer.BaseLibrary;
   using EPiServer.Implementation;
   using EPiServer.Implementation.Serialization;
   using EPiServer.Migration;
   using EPiServer.Migration.ObjectStoreCommon;
3. Implement IMigrationExtension interface.
4. Compile your dll and copy it into the bin folder of EpiServerMigration tool.
5. Modify EPiServerMigrationTool.exe.config by adding a key to handle your custom type.

  <migration>
    <extensions>
   <add name="Test Extension" type="ObjectStoreCustomHandler.MigrationExtension, ObjectStoreCustomsample2" />
    </extensions>
  </migration>  

Here is a short example of custom object migration:

using System;
using System.Collections.Generic;
using System.Reflection;
using System.Text;
using EPiServer.BaseLibrary;
using EPiServer.Implementation;
using EPiServer.Implementation.Serialization;
using EPiServer.Migration;
using EPiServer.Migration.ObjectStoreCommon;

namespace ObjectStoreCustomHandler
{
    public class MigrationExtension : IMigrationExtension
    {
        #region IMigrationExtension Members

        public void Initialize(MigrationContext context)
        {
            //
            //subscribe to the event that triggers when unknown object type is triggered
            //
            context.Events.CustomObjectDetected += new CustomObjectStoreObjectEventHandler(Events_CustomObjectDetected);  
        }
        #endregion

        void Events_CustomObjectDetected(ISession session, ItemInfo item, MigrationEventArgs e)
        {
            //
            //check if the item migrated is your custom type
            //
            if (String.Equals("ObjectStoreCustomType",item.SchemaName,StringComparison.OrdinalIgnoreCase))
            {
                CustomObjectSample.ObjectStoreCustomType myCustomType = new CustomObjectSample.ObjectStoreCustomType(String.Empty);
                //
                // register your custom type for serialization
                //
                ObjectStoreHandler.RegisterSchemaAndType("ObjectStoreCustomType", myCustomType.GetType());
                //
                //load the fields in the proxy object
                //
                ItemFields fields = item.GetItemFields("CustomObjectSample.ObjectStoreCustomType,CustomObjectSample");
                //
                //copy (non public) fields to your custom object
                //
                ObjectStoreHandler.CopyFields(fields.Fields, myCustomType, new string[] { "_name", "_id" }, e.Context);
                //
                //persist into EPiServer CMS 5 database
                //
                session.Save(myCustomType);
            }
        }
    }
}

Migrate Other Data in the Database

The migration tool converts the database "in place", meaning that it converts the data and schema of the source database. We therefore recommend that you back up the database before migration.

Migration Of Custom Property

The migration tool manages conversion of custom properties, if there are such properties in the site to be migrated. The tool creates a skeleton for the property and changes the name of the property to AutoGenerated_XXXX. (The XXXX postfix points to the name of custom property before migration.) After migration, the system engineer should change it to the appropriate custom property. The migration tool creates the property on-the-fly, because the real custom property (assembly) does not exist under the new site yet.

Change the customer property back to the original type by following the steps below:

1. Refactor the code of the custom property to CMS 5 API.
2. Copy the assembly or module under bin directory of the new site.
3. Go to Admin mode and select Edit Custom Properties.
4. Change AutoGenerated_XXXX to XXXX name and choose the appropriate Class name and Assembly Name for the property.

After editing all custom properties, you can delete the AutoGenerateCustomProperty.dll from the bin directory under the new site.

Migrate Enterprise Sites

The migration tool adds some entries to web.config for the enterprise settings, but additional manual steps must be done after migration. 

Enter applicationId, siteUrl, utilUri, uiUrl and pageStartId with appropriate information for each site. Here is a code example for these attributes. Tip! If you browse the site, you will receive error code from IIS which points to applicationId, you can use it for applicationId.

<site applicationId="/LM/W3SVC/1/ROOT" description="Example Site 1">
  ....
<siteSettings   
    pageStartId="0"
    siteUrl="http://enterprise1/"
    uiUrl="http://enterprise1/yourUI/"
    utilUrl="http://enterprise1/Util/"
  ...="" />
 </site> <site applicationId="" description="Example Site 2">
  ....
<siteSettings   
    pageStartId="0"
    siteUrl="http://enterprise2/"
    uiUrl="http://enterprise2/yourUI/"
    utilUrl="http://enterprise2/Util/"
  ...="" />
 </site>


Migrate Unified File Systems

In EPiServer CMS 4.62 the file system was based on an EPiServer CMS-specific technology called Unified File System (UFS). In EPiServer CMS 5 the file systems are based on the ASP.NET technology Virtual Path Providers (VPP). The following conversions are carried out by the migration tool:

• EPiServer.FileSystem.Handler.NativeFileSystem -> EPiServer.Web.Hosting.VirtualPathVersioningProvider
• EPiServer.FileSystem.Handler.VersioningFileSystem -> EPiServer.Web.Hosting.VirtualPathVersioningProvider

If the EPiServer CMS 4.62 site uses an IIS virtual directory that maps a relative folder outside the application, that folder must be copied locally relative to the site, e.g. if there is a virtual directory 'files' in IIS under the application that is configured to point to a physical directory outside the application root. The reason for this is that the migration tool does not have any knowledge of IIS settings and hence it cannot locate the physical folder if it is a virtual directory. 

The reason NativeFileSystem is converted to VirtualPathVersioningProvider is that the versioning provider supports permanent links, which EPiServer.Web.Hosting.VirtualPathNativeProvider does not. VirtualPathNativeProvider does not support metadata either. 

Permanent links were introduced with EPiServer CMS 5. The links are stored in the database as a guid-based link based on a unique identifier - can be a versioning file ID or a page ID. This assures that files can be moved around in the file system without affecting the links, i.e. the links will not be broken. If the conversion from NativeFileSystem to VirtualPathVersioningProvider is not wanted, you can comment out that handler in the CMS 4.62 web.config file before migration. Then after migration you can add VPP configuration that uses EPiServer.Web.Hosting.VirtualPathNativeProvider for that virtual path.

ASP.NET 2.0 applications restart when, for example, new folders are created under the physical root of the application. Because of this EPiServer CMS 5 sites doe not allow VPPs to be configured with a physical path under the application's physical root. When the migration tool migrates a UFS with a physical root under the application root, it will relocate it to the physical root for VPPs that was given when the CMS 5 SP2 site was installed. For UFS handlers that were configured with a physical path outside the application, the migration tool creates the new physical structure in the same location as the UFS system, but in a subfolder Converted (or Converted_1 if Converted already exists). The migration process will not alter the existing filesystem but create new parallell structures. This implies that there must be at least the same amount of disc space free that is used by the UFS filesystems. When the conversion is done, all old files and folders can be removed manually.

The file system migration of NativeFileSystem to VirtualPathVersioningProvider is performed by the migration tool deploying a temporary file ConvertNativeFileSystem.aspx to the CMS 5 site. The migration tool then makes a request for the page, which will then do the work. The reason this is done on the site and not in the migration tool is that the migration of file system requires a web context to execute.

If the migration of file system fails for some reason, for example if the site is not executable or IIS is not startable, the migration process will stop. After making sure the site is up and running, it is possible to continue the migration process by starting the migration tool again and selecting to continue the existing migration process. 

It is possible to configure the file migration to skip certain subfolders. To do this, the file excludedFolders.config should be deployed to the target site root before migration. The following is an example of configuration that skips migration of subfolder "subfolder1" in a file system with root "upload" and a subfolder "subfolder2" that is further down in the file system hierarchy.

<folders>
<folder>upload/subfolder1</folder>
<folder>upload/files/subfolder2</folder>
</folders>

Migrate Custom Unified FileSystems

The migration tool will take care of conversion for the unified file systems EPiServer.FileSystem.Handler.NativeFileSystem and EPiServer.FileSystem.Handler.VersioningFileSystem. They will be converted to VirtualPath provider of type EPiServer.Web.Hosting.VirtualPathVersioningProvider. Other unified file systems will not be converted to a corresponding VirtualPath provider. It is though possible to convert custom unified file systems by setting up an event handler for the event MigrateCustomFileSystem on class MigrationEvents. The event handler will then be called for each custom file system that is registered on the 4.62 site. Download a code sample for a module that converts a custom file system to a corresponding virtual path provider.

Change LongString Properties to XhtmlString

Permanent link support in CMS 5 requires that the property is of type XhtmlString. The migration offers the opportunity to convert LongString properties to XhtmlString properties. Before the migration starts, the migration tool displays a list of all LongString properties so that the user can select which properties to convert. The conversion tries to save each LongString value as an XhtmlString and if every value is successfully saved, the conversion is made on the page type. If however a value on a page is not possible to parse as XhtmlString, the conversion will not be performed.

Migrate Links

Links in CMS 5 are stored in the database as a guid-based link based on a unique identifier - can be a versioning file ID or a page ID. This allows, for example, files to be moved without breaking any existing links. The migration tool tries to rewrite all links in content as permanent links. This step is one of the last steps in the migration and is achieved by the migration tool deploying a temporary file ConvertLinks.aspx to the CMS 5 site. The migration tool then makes a request for the page, which will then do the work. The reason this is done on the site and not in the migration tool is that link migration requires a web context to execute.

If the CMS 4.62 site to migrate is a site configured as a virtual directory, the physical path of the virtual directory should be relocated to the targeting CMS 5 site before migration. The reason for this is that in CMS 4.62 running in a virtual directory, a link to a local resource is stored as "/virdir/Documents/image.gif". So when the migration of links work is performed, the CMS 5 site must run under the same virtual directory, otherwise the VPP configured for virtualpath '~/Documents' will not handle the request. This will result in the link not being rewritten to a permanent link.

If the migration of links fails for some reason, for example if the site is not executable or IIS is not startable, the migration process will stop. After making sure the site is up and running, it is possible to continue the migration process by starting the migration tool again and selecting to continue the existing migration process. 

Migrate Page Types

Migration deploys temporary .aspx files on the target site according to the page types on the source site. The deployed .aspx site will only list all properties on the page and have some navigation. So after migration it is possible to browse the site.

What Happens After Migration?

After data has been migrated, the code needs to be migrated. This means that code for page templates, properties, plug-ins, etc. must be converted to EPiServer CMS 5 API. The section Code Conversion in the tech note Migrating from CMS 4.62 to CMS 5 R1 describes  some of the changes in the API that need to be considered during the code migration.

Troubleshooting

There is a reported error in .NET framework that can occur when a virtual path contains some characters (e.g. ':'), this is reported in KB932522. This can cause link migration to fail for some pages. Any errors during the link migration will be logged to a file MigrateLinksErrors.txt that will be created in the site root of the target CMS 5 site. Then after the migration those pages can be migrated manually by open them in edit mode and republish them. 

The migration requires that the database collation is the same for the database and each text-based column in tables, which is the case when site is installed with EPiServer CMS Manager. The reason for this is that during migration new tables will be created with the same collation as the database. The creation of stored procedures will fail if it tries to do a join on two text-based columns with different collation. If the collation differs, the database should be changed before migration so collation is the same.

The link migration step reads pages including properties from the database and stores the updated values. For large sites this may cause an error in the form of an SQL timeout. The reason for this is that the database read and write operations work on the same tables and if there are a lot of entries in the SQL server, some read and write operations can block each other. In that case migration will be stopped. Since the migration process stores the state of the migration, the migration tool can be restarted and will continue migration where it was canceled.

"The partner transaction manager has disabled its support for remote/network transactions"

If you are receiving the exception: "[Error] System.Transactions.TransactionException: The partner transaction manager has disabled its support for remote/network transactions," see the migration tool FAQ - there are some steps described there that may be of assistance.

EPiServer 4.62 Assemblies with version numbers

ElektroPost.Licensing	version 1.0.0.10
ElektroPost.Win32	version 1.0.0.9
EPiServer.BaseLibrary	version 5.0.2000.15
EPiServer.Enterprise	version 4.62.0.533
EPiServer.Implementation	version 5.0.2000.15
EPiServer.Log.Analyzers	version 4.62.0.533
EPiServer.Log.Core	version 4.62.0.533
EPiServer.Scheduler	version 4.62.0.533
EPiServer.Workflow	version 4.62.0.533
EPiServer.XForms	version 4.62.0.533
log4net	version 1.2.0.30174