Views: 2245
Number of votes: 7
Average rating:

Simple Content Synchronization

I recently received a question from a partner trying to implement a requirement for having no live edits on the production server.  I have received this question a few other times so I thought I would see if I could come up with a solution that would work.  This example is a really simple solution and should probably be extended for production use. For example a service bus should be used for the change set and maybe azure logic apps or scheduled job to read the queue and process it.

The first thing I did was create an initialization module for listening to the publish events.  There is some code to check if these events should run.  This appsetting should only be set on the staging server that is to be used for content changes.

using EPiServer.Core;
using EPiServer.Core.Transfer;
using EPiServer.Enterprise;
using EPiServer.Framework;
using EPiServer.Framework.Initialization;
using EPiServer.Logging;
using EPiServer.ServiceLocation;
using Newtonsoft.Json.Linq;
using System;
using System.Collections.Generic;
using System.Configuration;
using System.IO;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Threading;
using System.Threading.Tasks;

namespace EPiServer.Reference.Commerce.Site.Infrastructure.Initialization
{
    [InitializableModule]
    [ModuleDependency(typeof(Web.InitializationModule))]
    public class PublishSync : IInitializableModule
    {
        private ServiceAccessor<IDataExporter> _dataExporterAccessor;
        private IContentLoader _contentLoader;
        private const string SettingPageFiles = "ExportPageFiles";
        private const string SettingRecursively = "ExportRecursively";
        private const string SettingPageLink = "ExporterPageLink";
        private const string SettingIncludeContentTypeDependencies = "ExportIncludeContentTypeDependencies";
        private string _username;
        private string _password;
        private string _stagingUrl;
        private ILogger _logger = LogManager.GetLogger(typeof(PublishSync));

        public void Initialize(InitializationEngine context)
        {
            if (!bool.TryParse(ConfigurationManager.AppSettings["contentStaging:Enabled"], out var stagingEnabled))
            {
                return;
            }

            _stagingUrl = ConfigurationManager.AppSettings["contentStaging:ProductionUrlBase"];
            _username = ConfigurationManager.AppSettings["contentStaging:Username"];
            _password = ConfigurationManager.AppSettings["contentStaging:Password"];
            _dataExporterAccessor = context.Locate.Advanced.GetInstance<ServiceAccessor<IDataExporter>>();
            _contentLoader = context.Locate.Advanced.GetInstance<IContentLoader>();

            var events = context.Locate.Advanced.GetInstance<IContentEvents>();
            events.PublishedContent += Events_PublishedContent;
        }

        public void Uninitialize(InitializationEngine context)
        {
            var events = context.Locate.Advanced.GetInstance<IContentEvents>();
            events.PublishedContent -= Events_PublishedContent;
        }

        private void Events_PublishedContent(object sender, ContentEventArgs e)
        {
            if (e.Content is PageData || e.Content is BlockData || e.Content is MediaData)
            {
                Task.Run(() => ExportItem(e.Content, _dataExporterAccessor(), _contentLoader));
            }
        }

        private async Task ExportItem(IContent content, IDataExporter exporter, IContentLoader contentLoader)
        {
            var exportedFileLocation = Path.GetTempFileName();
            var stream = new FileStream(exportedFileLocation, FileMode.Create, FileAccess.ReadWrite, FileShare.None);
            var settings = new Dictionary<string, object>();
            settings[SettingPageLink] = content.ContentLink;
            settings[SettingRecursively] = false;
            settings[SettingPageFiles] = true;
            settings[SettingIncludeContentTypeDependencies] = true;

            var sourceRoots = new List<ExportSource>();
            sourceRoots.Add(new ExportSource(content.ContentLink, ExportSource.NonRecursive));

            var options = ExportOptions.DefaultOptions;
            options.ExcludeFiles = false;
            options.IncludeReferencedContentTypes = true;

            IContent parent;
            contentLoader.TryGet(content.ParentLink, out parent);

            var state = new ExportState
            {
                Stream = stream,
                Exporter = exporter,
                FileLocation = exportedFileLocation,
                Options = options,
                SourceRoots = sourceRoots,
                Settings = settings,
                Parent = parent?.ContentGuid ?? Guid.Empty
            };

            if (state.Parent == Guid.Empty)
            {
                return;
            }

            try
            {
                exporter.Export(state.Stream, state.SourceRoots, state.Options);
                exporter.Dispose();
                await SendContent(state.FileLocation, state.Parent);
            }
            catch (Exception ex)
            {
                exporter.Abort();
                exporter.Status.Log.Error("Can't export package because: {0}", ex, ex.Message);
            }
        }


        private async Task SendContent(string file, Guid parentId)
        {
            var token = await GetToken();
            if (string.IsNullOrEmpty(token))
            {
                return;
            }
            using (var client = new HttpClient())
            {
                client.BaseAddress = new Uri(_stagingUrl);
                client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token);
                var content = new MultipartFormDataContent();
                var filestream = new FileStream(file, FileMode.Open);
                content.Add(new StreamContent(filestream), "file", "Import.episerverdata");
                var response = await client.PostAsync($"/episerverapi/import/cms/content/{parentId}", content);
                if (response.StatusCode != HttpStatusCode.OK)
                {
                    _logger.Error(response.Content.ReadAsStringAsync().Result);
                }
            }
        }

        private async Task<string> GetToken()
        {
            using (var client = new HttpClient())
            {
                client.BaseAddress = new Uri(_stagingUrl);
                var fields = new Dictionary<string, string>
                {
                    { "grant_type", "password" },
                    { "username", _username },
                    { "password", _password }
                };
                var response = await client.PostAsync("/episerverapi/token", new FormUrlEncodedContent(fields));
                if (response.StatusCode == HttpStatusCode.OK)
                {
                    var content = response.Content.ReadAsStringAsync().Result;
                    var token = JObject.Parse(content).GetValue("access_token");
                    return token.ToString();
                }
            }
            return null;
        }

        private class ExportState
        {
            public string FileLocation { get; set; }
            public IDataExporter Exporter { get; set; }
            public ExportOptions Options { get; set; }
            public Stream Stream { get; set; }
            public IList<ExportSource> SourceRoots { get; set; }
            public Dictionary<string, object> Settings { get; set; }
            public Guid Parent { get; set; }
        }
    }
}

Next I added an endpoint to be able to update the content with the changeset.  This makes use of some service api classes so a reference to the package will be needed to create the endpoint.  

using EPiServer.Core;
using EPiServer.Enterprise;
using EPiServer.ServiceApi.Configuration;
using EPiServer.ServiceApi.Extensions;
using EPiServer.ServiceApi.Util;
using EPiServer.ServiceLocation;
using EPiServer.Web.Internal;
using System;
using System.IO;
using System.Net;
using System.Net.Http;
using System.Threading.Tasks;
using System.Web.Http;
using System.Web.Http.Description;

namespace EPiServer.Reference.Commerce.Site.Features.Rest
{
    public class ImportController : ApiController
    {
        private static readonly ApiCallLogger _logger = new ApiCallLogger(typeof(ImportController));
        private readonly PermanentLinkMapper _permanentLinkMapper;
        private readonly  ServiceAccessor<IDataImporter> _dataImporterAccessor;

        private const string InvalidMediaTypeMessage = "Wrong Media Type";

        public ImportController(PermanentLinkMapper permanentLinkMapper, ServiceAccessor<IDataImporter> dataImporterAccessor)
        {
            _permanentLinkMapper = permanentLinkMapper;
            _dataImporterAccessor = dataImporterAccessor;
        }

        [Route("episerverapi/import/cms/content/{id:guid}", Name = "mh_UpdateContent")]
        [HttpPost]
        [ResponseType(typeof(Guid))]
        [AuthorizePermission(Permissions.GroupName, Permissions.Write)]
        public virtual async Task<IHttpActionResult> PostCmsImport(Guid id)
        {
            if (!Request.Content.IsMimeMultipartContent())
            {
                _logger.Error(InvalidMediaTypeMessage, Request.CreateResponseException(InvalidMediaTypeMessage, HttpStatusCode.UnsupportedMediaType));
                throw Request.CreateResponseException(InvalidMediaTypeMessage, HttpStatusCode.UnsupportedMediaType);
            }
            
            var file = await Request.GetUploadedFile(UploadPaths.IntegrationDataPath);
            var destinationRoot = _permanentLinkMapper.Find(id);
            if (destinationRoot == null)
            {
                return Ok(id);
            }

            var importerOptions = ImportOptions.DefaultOptions;
            importerOptions.KeepIdentity = true;
            importerOptions.ValidateDestination = true;
            importerOptions.EnsureContentNameUniqueness = true;
            importerOptions.IsTest = false;
            var importer = _dataImporterAccessor();
            Security.PrincipalInfo.RecreatePrincipalForThreading();
            var state = new ImporterState
            {
                Destination = destinationRoot.ContentReference,
                Importer = importer,
                Options = importerOptions,
                Stream = new FileStream(file.LocalFileName, FileMode.Open)
            };

            var message = await ImportFileThread(state);
            if (string.IsNullOrEmpty(message))
            {
                return Ok(id);
            }
            else
            {
                throw Request.CreateResponseException(message, HttpStatusCode.InternalServerError);
            }
        }

        private Task<string> ImportFileThread(ImporterState state)
        {
            return Task.Run(() =>
            {
                try
                {
                    state.Importer.Import(state.Stream, state.Destination, state.Options);
                    return "";
                }
                catch (Exception ex)
                {
                    _logger.Error("Can't import data because, ", ex);
                    return ex.StackTrace;
                }
            });
        }
    }

    public class ImporterState
    {
        public ContentReference Destination { get; set; }
        public IDataImporter Importer { get; set; }
        public Stream Stream { get; set; }
        public ImportOptions Options { get; set; }
    }
}
Aug 09, 2018

( 8/9/2018 1:00:20 PM)

Nice - mirroing version 3.0 :)

Paul Gruffydd
( By Paul Gruffydd, 8/9/2018 3:07:44 PM)

Much as I always disliked mirroring, I like the idea here, particularly if you go with your suggestion of queueing the content and transferring via a separate process.

I was thinking of similar a while back but hit issues where publish events weren't called for scheduled publishes and expiry. Thankfully it looks like that's been resolved now and the PublishedContent event is called in most of the situations where I'd expect it to be. Beyond the publish event it would be good to tie in to the MovedContent event too otherwise, when pages get deleted in thet CMS, the production site wouldn't reflect that. Maybe also the IContentSecurityRepository.ContentSecuritySaved event if ACLs are important.

Good stuff Mark - I suspect I'll be referencing this post in the future.

valdis
( By valdis, 8/20/2018 9:52:30 AM)

this is nais approch. but as advocate of dependency injection - I would recommend to extract all appSettings access code from the job and ask for the `IContentSyncOptions` or similar interface to be injected. and then I could configure container to instruct how to create implementation of `IContentSyncOptions` and from where to get config values.

these are just my humble 2 cents :) no offense

Please login to comment.