Migration strategies

Migrations are often complicated and not very sexy, all the work you put into it, configuration, scripts and code will be more or less useless after the migration has been done. However, choosing the right strategy is essential for a good CRM implementation. I will go through some of the strategies that are common and discuss some of the advantages and disadvantages.

Greenfield
Greenfield migrations are not really migrations, but rather the lack of migration. This means that the system will be set up without any migrated data, no accounts, no contacts etc. The idea is then that all the users will create data in the new system as the feel the need for it.

This is often used for start-ups when there is no or very little data. It can also be used when the data quality is very bad and trying to filter out the "good" data is just to complicated. Normal problems can be large numbers of duplicates, errors in the data itself, like "." and "," being inserted instead of real data just to make the forms savable.

Advantages of Greenfield migrations is that the system will be up and running in no time. Migrations might otherwise take considerable amounts of time. This also mean that the cost of Greenfield migrations are very low. Do note however, that there is need for some general system configuration, like creating users, setting up queues.

The disadvantage is that the users do not feel that the new system is helping them out and they have to reinsert a lot of data which can create considerable amounts of annoyance with the new system. This can in the long term increase the bad-will towards the system, lowering usage which can jeopardize the entire CRM investment.

Standard import
The standard import functionality of Dynamics CRM is reasonably powerful and can usually be used for importing simpler data. It is based on data being stored in csv-files or excel as xml-files. The first of these being a bit problematic as it is rather old and dependant on the regional setting. For instance, the value separator in Sweden is semicolon ";" but in USA comma ",". It is also common that the csv-files contain data that risk messing up the file syntax, breaking the imports. In general the excel-as-xml format is to be preferred as it also contains field names and proper matching.

This functionality has greatly been enhanced in CRM 2011 compared to CRM 4. For instance, it now support uploading of zip-files containing xml-data files with relations between the files. It can even handle multiple relations that otherwise would take three import runs, all in one go. For instance if you have data with accounts containing primary contact and all contacts also contain parent customer.

The limitations of standard import is that it cannot really handle large amounts of data as the maximum upload size is 8 MB (can be changed in an onpremise installation). Complicated data structures are also hard to handle and it is rather time consuming to run it as there is no scripting capabilities. The lack of scripting also makes the ordering of the different imports in a zip more or less impossible. There is also no support for logic like "If the contacts email adress exists, update the data of that contact, otherwise create a new contact" which is often very useful when importing data from multiple sources. It is also hard to do testruns of the migration without a lot of manual steps involved. There is some error handling but it is very rudimentary and cannot handle more complex logic.

The advantages of standard import is mainly that it requires no extra software and it is relativly easy as the options are limited.

Third party products
One of the more powerful options when doing migrations, is using third party products, like Scribe, Import Manager. These include lots of options that the standard import doesn't like:

• ODBC/OLE DB connections to a data source
• Logic to handle update or create
• Scripting - custom logic to handle some of the data conversions
• Run as service - ability to be run in the background as a service
• Reuse for integrations - these tools can also be used for integrations why the effort invested can at least partly be reused.
• Support for custom addons
• More powerful - has support for multi threading and large data sources
• Easier to do test migrations as all steps can be set up in a run-script, less manual labour.
• Can handle more complex data
• Better error handling

Not all products support all of these features.

The main drawbacks are the licensing fees required. Do note that some of these companies have special migration licenses which are not the same as the full license used for integrations. These products are also a bit more complicated due to the larger sets of options available. Hence it also takes some time to learn the product why my recommendation is to choose one and stick with it.

Custom migration program
The most advanced for of migrations need to be done by using custom code. There are really very few limitations to what can be done when writing custom programs that migrated data, it is more up to the skills of the developer and the time available.

Some of the advantages:

• Limitations only in skills and available time and money
• Easy to do test runs
• Complicated error handling logic can be created

The main disadvantages are:

• Demands developers
• Time consuming - from around 100 hours to several thousand hours

General recommendations
Migrations are often complex and it is not until you have worked with the data a bit that you start to get a feel for how problematic it is going to be. Hence I usually never give fixed prices on migrations as it would either be inflated or put me at risk. It is also often the case that customers do not realize the complexity of migrations, especially smaller customers it is therefor essential to involve them a lot in the work.

De-scoping the migration is also very important. Usually not all data is required, especially if the old system can be maintained with read-only access. Try to de-scope in width (which entities are really required) and in depth (how old data is required).

Data quality can also be a problem so try to evaluate this early on. Examples are names being stored both as "First name Last name" and "Last name, First name" or fill-in data like ".", "," or "-" that has been entered into required fields to make the forms savable.

Large migrations might required delta migrations, when migrations are run in two steps, first the major migration, then after it has been completed, a smaller to migrate the data that was changed during the main migration. This will put additional demands on migration scripts and/or code so try to avoid it if possible.

Sometimes the new system required data that doesn't exist and hence has to be created. I usually refer to this as migration of non-existent data. It is easy to forget if your perspective is to map the data from the old system without looking at which data is required in the new.

Migrations between two systems that with very different data models is theoretically complicated. There might, for instance exist data in several places that need to fit into one. Data might need to be restructured in complex ways. In depth knowledge of source and target data model is essential and the proper skills required to understand complex data modelling is also a strong recommendation.

I hope I have shed some light on this subject which can be discussed at length.

Gustaf Westerlund
CEO, Chief Architect and co-Founder at CRM-konsulterna AB
www.crmkonsulterna.se

Unsupported customizations

Unsupported customizations are certain customizations that fall outside of what is supported by Microsoft when developing and customizing for Dynamics CRM. Several aspects of this subject can be discussed in depth and I will try to touch base on some of them.

When creating Microsoft Dynamics CRM, Microsoft realized something very important, the system will need to be customized by partners to fully fit the needs of the customers. However, customizations need to be upgradable and support update rollups and hotfixes. Previously, all upgrades would require rather large efforts in re-writing lot of the code that was custom built for the customer. However, the definition of supported and unsupported customizations and the promise from Microsoft that all supported customizations will be upgradable and updatable at least to the next version, is something that is very good and will ensure a better return of investment of the custom development made for customer and also improve partner-customer relations since paying for upgrades of already made customizations seldom improves the partner-customer relationship.

We have worked with many systems where other consultants and developers have created solutions and here are some of the more common unsupported customizations, why they will cause problems and alterntive solutions:

HTML DOM addressing and modifications
Addressing HTML DOM components directly using document.getElementById and replacing parts of the normal HTML DOM with custom components. For instance, addressing an isv-button in CRM 4.0 and disabling or removing it with javascript. Another example is replacing lookups with dropdows, for instance letting the user select contact for an opportunity.

This is very common i CRM 4.0 but is also quite common in CRM 2011. The problem with this unsupported customization is that when upgrading the system the names of the elements in the HTML DOM will probably change. This is obvious in the case of isv button addressing in CRM 4.0 since the ribbon HTML DOM element names are different. The code will hence break after an upgrade.

Developers doing these kinds of unsupported customizations are usually web developers that are used to being able to modify webpages with javascripts anyway they like and they are not familiar with the SDK and do not forsee the consequences of these unsupported customizations.

Alternative solutions are to strictly use the object model described in the SDK and only use the documented methods. Note that the SDK in CRM 2011 has been improved a lot in this perspective. It is also possible to enable and disable buttons in the ribbon with ribbon customizations. It is also possible to use small iframes with aspx-pages behind (or virtual pages created with javascript) that show custom logic and work with the CRM form using cross-frame scripting. Do note that the hosting application and the cross-framed page need to be in the same security zone in IE for it to allow this. Preferably place them on the same server.

Database modifications
Modifications to the CRM database are also some of the more common unsupported customizations that I have experienced. It can be anything from custom views to new stored procedures or even modifications to the existing procedures. Do note that creating indexes in the database is supported but not upgradable.

Developers making these kinds of customizations usually do not realize the effects that they have and might be used to and very familiar with writing T-SQL. Modifications to the existing tables and views are very hazardous since CRM is a metadata driven application meaning that the database contains metainformation on how the database looks. If the metainformation gets out of sync with the real database, you will have a very nasty problem on your hands.

Additions to the database, without any modifications, will usually work with updates to the system. However, when upgrading or redeploying the system these usually cause quite a lot of problems. It is also a general recommendation to try to keep all business logic in the same layer of the application and to not mix them between the database and higher layers (like plugins). This can sometimes not be avoided however.

So, if you do need to create some custom views or similar, create a database next to the CRM database with hosts these and use cross database addressing to work with the data.

Modifications to CRM files or additions to the CRM installation directory
As the aspx-files of CRM can be found in the CRMweb directory (or inetpub for port 80 installation) it can be tempting to modify these. In CRM 4.0, when there was no support for javascript includes, modification to the file global.js was also quite common. I have also seen systems where entire IIS virtual directories have been set up in the assembly directory of CRM. This is also unsupported.

The reason these customizations should be avoided is that any update rollup that is installed might overwrite these files and you customization might be lost. With upgrades the customization will most certainly be lost or stop working.

There are no good alternatives to these kind of modifications. Try working with Iframed custom pages using cross-frame scripting if possible. Modifications to the global.js are very hazardous since it is not very uncommon for it to be updated in normal update rollups, making any customizations to it rather hard to maintain. This is not necessary in CRM 2011 as the web resource-functionality allows for better reuse of javascript code.

Unsupported customizations are necessary anyway
If the requirement still demands unsupported customizations, and there is no good way around it, I recommned having a discussion with the customer on the effects of creating unsupported customizations in regards to upgrades and increased maintainance costs. If the still feel it is required, I recommend you getting their formal signature on a document describing it as an unsupported customzation and that the customer understands that the function might not be upgradable and that upgradecosts will be fully charged to the customer. It might feel a bit formal, but trust me, when the customer wants to upgrade, you will be happy you have the document.

Another important aspect that you need to remember when creating unsupported customizations, is that you need to take this fact into account. For instance, if addressing an HTML DOM element, make sure to handle the fact that it might not exist, and have some resonable fallback from this.

It is also essential to document each unsupported customization with extra care. I recommend a special document for this. It should include some of the following topics:

• Description of requirement
• Technical desciption of the solution
• Why it has to be made with unsupported customizations - This is important as newer versions of CRM might enable you to rewrite this in a supported manner.
• Possible effects on an installation of an update rollup -
• Possible effects after an upgrade
• Checklist to verify the functionality.

Gustaf Westerlund
CEO, Chief Architect and co-Founder at CRM-konsulterna AB
www.crmkonsulterna.se

Managed or unmanaged solutions - and how to remove attributes from managed solutions

My colleague Rickard and I are working with one of our customers who have gotten managed to get themselves into a rather big mess with huge numbers of attributes on the account entity that should not really be there. If any of you ever wondered what the limitation for number of attributes in Dynamics CRM is, it is around 700 and it is not recommended to use since the system becomes very hard to work with. Proper data modelling with experienced architects is usually a good idea that pays off tenfold later, so that you do not get your self into this situation.

However, the production system at hand has several hundred attributes that were erroneous in a managed solution, and something had to be done.

The first thing you might think of trying is just to remove the attributes from the development environment, packaging a new version of the managed solution and overwriting the production environments managed solution with this new reduced solution. This will, however, not work, as solutions cannot remove attributes (they contain the state of the CRM-system, not the transactions).

After a few more ideas had been tried, we finally registered a support case with Microsoft where a very helpful technician assisted us with a solution to this, based on this very good blog posting by Gonzalo Ruiz. The latest news on this is that it seems to be working, although neither Rickard nor myself is doing the actual work.

This has also caused us to have a very in-depth discussion on if managed solutions are suitable for normal project production systems or not. I had a discussion with former Microsoft employee Peter Björkmarker who is a very skilled architect, and we came to the conclusion that managed solutions are really best suited for ISV-packages, and not so much for normal customer projects where unmanaged solution are easier to work with. This is, however, something I would be more than happy to discuss, so if you have any opinion on the subject, please leave a comment!

Gustaf Westerlund
CEO, Chief Architect and co-Founder at CRM-konsulterna AB
www.crmkonsulterna.se

Getting the CRM url for Silverlight based on the browser

When creating Silverlight applications the normal method of retrieving the url for the CRM server that will be used to connect to the REST endpoint is usually based on the CRM url that is stored in the CRM config database and retrieved based on Xrm.Page.context.getServerUrl().

There can often be problems with using this address, for instance in development environments or similar, as the address returned might not be the address used in Internet Explorer. When this happens, Internet Explorer will block the connection to the REST endpoint since it does not connect to the same URL as the webpage that hosts the Silverlight application.

During the latest course I held for some CRM consultants at Evry, we discussed this and one of the attendees, Klas Adolfsson at Evry, was very quick and wrote an alternative to the getServerUrl()-method that bases the url on the hosting browser url instead of the url context variable from CRM.

HtmlPage.Document.DocumentUri.Scheme + "://" + HtmlPage.Document.DocumentUri.Host + "/" + App.Current.Host.InitParams["orgname"];

Edit: There are some relevant comments to this posting below, please read these if this posting is of interest to you.

Gustaf Westerlund
CEO, Chief Architect and co-Founder at CRM-konsulterna AB
www.crmkonsulterna.se

Problems uploading customizations/solutions can be due to large Templates included

Today I am working with a customer who has used other CRM consultants previously. It is always interesting. In this case the QA/testing environment was not up to date and wasn't acutally working at all. All changes had been made directly in production, not really my cup of coffe, as I like things to be a bit more controlled. Just for the record, it is a CRM 4 system, but the I do believe the same issue applies to CRM 2011 aswell.

So, I wanted to get the QA-environement up to date with the production environment and started by exporting all customizations from production and attempted to import them into the QA-environment. (Backing up that enviroments customizatons first of course). Well, that didn't actually work, and I didn't get a very good error message, just some non-descript error. When looking in the event viewer I got a ASP.NET error saying that "Post size exceeded allowed limits." which of course go me thinking about the size of the customization file. Checking it, I found that it was almost 9 MB, which is above the set  8 MB ceiling. After some pondering about which parts of the customizations might be the issue, I found that the Templates part was about 7.9 MB. So, my suggestion is to keep this in mind especially when exporting customizations/solutions from production systems that Templates can be very large and might brake the maximum transfer size of 8 MB. This maximum size can be changed, but that is another issue.


Gustaf Westerlund
CEO, Chief Architect and co-Founder at CRM-konsulterna AB
www.crmkonsulterna.se

Swedish IRS hacked - shows glitch in responsibilies of online services for government agencies

There is currently rather a lot of media in Sweden about a computer system hosted by a larger Software Integrator outsourced by the Swedish IRS. This system was hacked and about 9000 client accounts were compromised. The discussion in the media is if this a government agency really can delegate this kind of responsibility to a third party or if there will always be some fundamental responsiblity for maintaining the security at the government agency? The security experts and even the Swedish Secretary of IT, Anna-Karin Hatt, indicated that this kind of responsibility could not be delegated.

This question is very relevant in the case of CRM systems in general and Microsoft Dynamics CRM in particular. It would then seem more or less impossible from an government agency CIO perspective to use any kind of cloud based service for line of business systems like CRM/xRM as this would imply that it is impossible to fully maintain the control required over the information in the system. This is no problem for us working with Dynamics CRM as it can be deployed on-premise, but bad news for anyone looking to work with SalesForce.com or any other pure cloud based service as they cannot be used in such conditions.

Gustaf Westerlund
CEO, Chief Architect and co-Founder at CRM-konsulterna AB
www.crmkonsulterna.se

Workflow activities in CRM Online!

Custom workflow activities are one of my favorite extension points of CRM. The power workflows gives the customers are great as they can easily change the content of outgoing emails, todos etc. With the ability of custom workflows I can connect the workflow engine to anything in the world, from credit rating checks, integrations to other systems or just adding a bit more complex logic than what the standard activites include.

One of the major drawbacks of the current version of CRM-Online  (as of march 2012) in comparisson to On-premise is from my perspective that custom workflow activies cannot be deployed in CRM-Online. I asked developer evangelist at Microsoft Sweden, André Henriksson about this and his comment was that it was probably due to the fact that the workflow foundation team didn't support sandboxing of custom workflow activities. At SharePoint Exchange Forum I had the oppotunity to discuss this with SharePoint MVP Eric Shupps and he mentioned that in SharePoint there was full support for custom workflows. This made me a bit puzzled since this implied that the SharePoint team had managed to create a workaround for the lack of isolation for custom workflow activities that the CRM team hadn't.

However, in the upcoming R8, release for Dynamics CRM 2011, Microsoft have annonced (http://crmpublish.blob.core.windows.net/docs/ReleasePreviewGuide.pdf) that there will be support for custom workflow activities in CRM Online. This is great news, but as the functionality will be based on sandboxing there might still be limitations to what can be done within the sandbox, for instance on which external connections are allowed. This remains to be seen, but the news of support custom workflow activities in CRM Online is still something I have waited for a long time!

Gustaf Westerlund
CEO, Chief Architect and co-Founder at CRM-konsulterna AB
www.crmkonsulterna.se

Advanced find and Many to Many (N:N) in CRM 2011

I had a discussion today with a potential customer regarding a part of the functionality of advanced find that I thought I might share as it is not so obvious.

In Dynamics CRM 4, it is not possible, with advanced find to use the "related" entities functionality of advanced find to search for objects that are on the other side of the N:N relationship.

However, in CRM 2011 this is possible. For instance, if a new relationship N:N is created between account and contact, it is possible to use advanced find to find all contacts that are connected to a certain company.

It is not, however, possible to show column data from the related entities through an N:N relationship, it is only possible to pick columns through N:1 relationships (in that direction).

Gustaf Westerlund
CEO, Chief Architect and co-Founder at CRM-konsulterna AB
http://www.blogger.com/www.crmkonsulterna.se

Finding the TCP port for Dynamic Excel Export

The dynamic Excel export functionality of Dynamics CRM is a real killer functionality. I often get standing ovations from users, especially managers, when showing dynamic pivot table functions of CRM, partly due to the fact that many users don't actually know what pivot tables are and find them rahter magical.
I was working with a demo environment today and had some issues getting the dynamic excel to load data from CRM properly. I got the error message: "Connection failed". This was a lab environment with a separate Windows 7 virtual machine in an on-premise installation. This is important, because in IFD or CRM-online, dynamic excel data is tunneled through the outlook addon, but not when using the on-premise installation when it communicates directly with the server.

So I started looking for answers on the internet and found the following thread on the CRM forum which helped out a bit: http://social.microsoft.com/Forums/en/crmdeployment/thread/c2d3029c-5487-435b-90e7-ba030aeba8ac

I started by trying to do the obvious thing by opening the TCP port 1433, the SQL communication port in the firewall. This did not help. According to some of the participants of the thread, this KB article by Microsoft might help: http://support.microsoft.com/kb/968872 - it does more or less the same thing, opens 1433 and some additional ports in the firewall all related to SQL. However, it did not help either.

One of the participants of the thread mentioned that he had opened the TCP port 53021, something I found very strange if it was to work, since ports this high are not supposed to be used as incoming ports but are ephemeral ports. I tried opening this port as well, but it didn't work either. The port number seemd sort of random so it might be that my deployment just used some other high port number.

A colleague of mine working at Hermelin IT-Partner, Anders Jildestrand, then joined me in the hunt for the mysterious TCP port. With his assistance and netstat we got the following result:

C:\Users\administrator>netstat
Active Connections
  Proto  Local Address          Foreign Address        State
  TCP    192.168.75.15:3389     192.168.75.131:29837   ESTABLISHED
  TCP    192.168.75.15:52422    192.168.75.13:5555     ESTABLISHED
  TCP    192.168.75.15:52426    WIN-LGEKH2VQ6FI:epmap  TIME_WAIT
  TCP    192.168.75.15:52427    WIN-LGEKH2VQ6FI:49155  TIME_WAIT
  TCP    192.168.75.15:52428    192.168.75.13:63831    SYN_SENT
C:\Users\administrator>

And if you are sharp eyed, you might spot it, as we had the SQL-server on the CRM server, and the TCP-SYN was sent but never answered, it is of course the last row, hence the TCP port 63831.

After opening this port in the firewall, the dynamic Excel worked just like it should.

The question now remains. Why this port?

We also had a discussion on which program, CRM or SQL Server that is actually answering this call, and it seems that it is actually SQL Server since the ODBC connectionstring in the Excel-file is directed to the SQL-server, not the CRM-server. But I think we could probably investigate this further with fiddler. If you have any interesting input, please leave a comment!

Gustaf Westerlund
CEO, Chief Architect and co-Founder at CRM-konsulterna AB
www.crmkonsulterna.se

Running Microsofts Demo VPC on Windows 7

Running Microsofts CRM Demo VPC (Available on Partnersource) on Windows 7 is a bit problematic but can be done. I found a good blog that showed how it can be done using VirtualBox but there were some extra things that needed to be done to make it work.
As many of you probably are aware, from Windows Server 2008, only 64 bit versions of the server operating systems are available which isn't very strange since the 32 bit versions of Windows only support up to 3.5 GB of memory which is really enough for even a small server.

Hence the Virtual Machine that Microsoft have released with a demo environment of CRM 2011 is based on this 64 bit environment. Since Windows Virtual PC, the Virtualization Technology for Windows 7 does not support 64 bit guest operating systems, it has been created on Hyper-V and is meant to be run on this virtualization technology. However, this is not really what you average CRM salesperson is running in his or her Lenovo X11 for many obvious reasons, power management probably being one of the most striking.

So, it is it just not possible to run the demo environment on a Windows 7 machine? Yes, it is. On the blog Leon's CRM Musings there is a very informative posting on how to make the virtual harddrive work in the VirtualBox virtualization technology. I was quite amazed when I saw this working as this is quite similar to installing an operating system on a computer, ripping out the harddrive and screwing it into a completely different computer and hoping it will work. However, the Hyper-V virtualization and VirtualBox virtualization must be very similar from the guest operating system perspective, otherwise the operatingsystem would have given a lot of errors.

Do note the part in the posting that the demo image does not support being run as a SATA drive but must be run as an IDE harddrive.

There are some comments that both SharePoint and CRM are installed on port 80. This is in part true, but they are installed on different websites separated by different host headers.

However, the sharepoint site did not work when I tried running it, hence the document manangement from CRM did not work either. After checking the IIS website bindings and the host-file (C:\system32\drivers\etc\hosts) I noticed that the host headers for the SharePoint site and some other sites were pointed at the IP 192.169.100.1 and a quick check with ipconfig showed that this was not the ip of the current computer. Hence I just changed the mapping to be 127.0.0.1 (always means "this" computer).

I then closed all IE-windows, flushed the DNS (ipconfig /flushdns) just to be sure, then tried the sharepoint site, and it worked. As did the document management in CRM.

I saw some people in Leon's blog commenting on the networking issues of making the exchange server communicate with the CRM-server. This can be done be setting the networkadapter in VirtualBox to "Internal Network" on the Exchange machine and using two adapters on the CRM machine, one with NAT (external) and one internal using the "Internal Network". Both the internal networks adapaters need to be configured manually, but the external one can have a dynamic IP. If anyone doesn't know how, please leave a comment and I will tell you how.

Regarding optimization of speed I strongly recommend upgrading your computers memory to at least 8 GB so that you can give the virtual machine between 4-6 GB. Try to give it even multiples of 512 MB as that will make for better memory management.

Another well known fact regarding VPC:s is disk access. It is highly recommended that you use a separate disc on a high speed controller (i.e. USB 3.0/eSATA/internal SATA).

As for processor power, it is not as essential, eventhough it is always nice. Try to allocate at least 2 cores to it. One for the SQL-server and one for IIS/GUI.

The recommendation that Leon gives to reduce the memory for the SQL server I am not so sure that is a good idea as CRM is very SQL heavy due to its meta data driven nature. However, if you only have 4 GB:s of memory and you have to make this VPC work, then you might have to try it. Kingston memory modules are very cheap now, 2x4 GB for a T410 is less than €100 last time I checked.

I have several computers, my "sales computer" is a Lenovo T410s Corei5 8Gb with 2 SSD HD. I use one of the harddrive for my host operating system and one for the virtual machines. I have removed the DVD-drive since I never use it, and the extra drive is a lot better use of the space.

Last of all, cudos to Leon for a great solution with VirtualBox for the VPC!

Update:
An additional thing to do is to convert the VHD to the Virtualbox native VDI-format for virtual harddrive using the media Media manager in VirtualBox. This is probably a more optimized format for Virtualbox and hence a bit Quicker. (I have not tested it, it is just a thought)
I have also tried to run the VPC on an external USB SSD, both USB 2.0 and USB 3.0 with 2 GB of RAM with 2 Cores on my Core i5 with the converted disk and it works fine. The SSD gives it a lot of extra speed.

Gustaf Westerlund
CEO, Chief Architect and co-Founder at CRM-konsulterna AB
www.crmkonsulterna.se

Cross browser and CRM Mobile

Today Dennis Michalis, General Manager of Dynamics CRM at Microsoft announced that in Q2 2012 a service update of Microsoft Dynamics CRM 2011 will include cross browser functionality to allow users on a multitude of browsers and operting systems to finally be able to use all the great functionality of Dynamics CRM without having to use workarounds like virtualization or terminal services.

Supported Browsers (From Microsoft Roadmap PDF)

He also announced that Microsoft Dynamics CRM Mobile will be released, a Micrsoft based product for bringing CRM functionality to Windows Phones, iPads/iPhones, Androids and Blackberries.

CRM Anywhere on iPad (From Microsoft Roadmap PDF)

There were also some other updates that were announced that you can read about in the Service Update Roadmap Q2 2012.

The cross browser functionality is something many of my customers have been waiting for a long time and I really look forward to seeing it. My colleague Rickard noted that no Unix/Linux based systems were mentioned in the matrix which is notable. According to what they previously have mentioned the mutlibrowser version will be based on HTML5 there might hence be two reasons for it not being included. Either it is not tested or there might be parts of the system that required Silverlight or some other component that only exist on Windows and Mac operating systems.

The other notable thing is that the mobile client will be big drawback for companies like Resco, TenDigits and CWR Mobile who have very competent mobile clients. This is always a risk for companies working with products close to Microsofts since Microsoft always need to expand and grow. So, if you are a company looking to invest in a mobile solution, think twice, it might be a good idea to wait a month or two and have a look at the Microsoft version of the product. It will require an extra cost so there will still be room for the other mobile clients but as others have previously noted, it is always hard to compete with Microsoft. Their mobile products do still have several upper hands on Microsoft Dynamics CRM Mobile in this early version for example,  it does not support offline functionality and I would presume the mobile API is not as extensive as that of Resco or CWR Mobile (I do not have any experience of TenDigits).

Gustaf Westerlund
CEO, Chief Architect and co-Founder at CRM-konsulterna AB
www.crmkonsulterna.se

Unable to upgrade CRM 4.0 database more than once

My colleague Rickard had an interesting problem yesterday when working with a client of ours. They had a prototype upgrade and now wanted the production upgrade. However, they wanted both of these organizations to be available (yes, will required an Enterprise license or separate SPLA licensing if it is a hosting environment). The prototype upgrade they want to be available as "orgname-old" and the normal upgrade just as "orgname".

An interesting issue arose when Rickard tried to import the CRM 4.0 database into the CRM 2011 environment he receive an error message saying that a CRM 4.0 database can only be imported once into a CRM 2011 deployment. This might be noted somewhere but it is not something that I or my colleague Rickard have ever seen when working with CRM and we didn't find to much about it googling the issue either.

The problem seems to be that Microsoft seems to have cut a corner when upgrading a CRM 4.0 database in comparison to normal redeployment as it does not give the organization a new GUID. Hence you cannot import two organizations twice.

The workaround is, however, quite simple, upgrade the first database, disable it and remove it from deployment manager (do not delete it from the SQL-database).

Upgrade the second database into CRM 2011. As there is no organization with this GUID now, this is no problem.

Now, import the first database (redeployment) that you previously disabled and removed from deployment manager. You can now give it a new name if you like. As this is a redeployment, it will give this a organization a new Guid and the two organizations will not conflict.

If you are creative you can probably think of you own solution using similar techniques.

Big thanks to Rickard for input to this posting!

Gustaf Westerlund
CEO, Chief Architect and co-Founder at CRM-konsulterna AB
www.crmkonsulterna.se

Problems with converting email to case

I was recently working with a customer when I noticed an interesting bug in CRM 2011, in the Outlook Client. When trying to convert an email to either a lead, case or opportunity directly from the Outlook ribbon, it does not set the "regarding" lookup field of the email that was tracked. However, if the email is first tracked, then "viewed in CRM" and then from the CRM email form converted to lead, case or opportunity, the regarding field is set correctly.

To make this a bit more clear I will describe this with a few screenshot below:


We'll start off with opening a received email and pressing the Track button in CRM

This will then allow us to press the convert to Opportunity, Case or Lead, directly from Outlook. A very neat, and new function. It came with CRM 2011. In this example, I am selecting Case.

 This will show a smaller dialog allowing you to select case (not shown here) and also prompting you if you want to open the case or not. I selected to open the case.

After opening the case, I checked the closed activities to see if the attached email that was I just converted to a case, but as you can see below there were none. I was a bit perplexed as I was showing this new functionality to the customer as was expecting to see the email here.

I went back to the email from Outlook and opened it in CRM and found that it had not been correctly connected to the case using the lookup "Regarding" field.

So, I tried it the old way. I sent myself another email, pressed the track in CRM button, and then instead of converting it directly from Outlook, i pressed the "View in CRM" button to open it in CRM.

I then converted it to a case from the ribbon menu from the CRM Email from by pressing the "To Case" button.

I was shown a similar small dialog letting me select subject for the case and similar and then the case form was shown. It was identical to when created from Outlook, but when checking the closed activities there was one important difference;

And by opening the CRM email form from the list shows it clearly:

The regarding field has been set properly.

This is a very unfortunate bug in CRM and I do hope Microsoft solve it quickly as the feature that CRM 2011 adds in Outlook by being able to convert emails to cases, opportunities and leads directly from the email form is very good but as all of you know who work close to sales or customer service people, their time is very precious and every click counts and new features that do not deliver as expected are always annoying to the user. So, until the bug has been fixed, make sure the users open the CRM form to convert the emails.

Update: Jukka Niiranen left a very informative comment to this posting below informing us that Microsoft probably will fix this bug in UR6 planned to be released in Jan 2012.
Gustaf Westerlund
CEO, Chief Architect and co-Founder at CRM-konsulterna AB
www.crmkonsulterna.se

Built in imports in CRM

I have been working with the built in import functionality for imports a bit lately and I must say that it has been improved a quite a lot. It as now reached a level where it is quite usable but there are still problematic areas you have to look out for.

I found this blog posting (http://blogs.msdn.com/b/crm/archive/2010/03/15/importing-multiple-files-using-zip-import.aspx) which was quite interesting and enlighted me to the fact that you can import zip:ed files which is really good since there are some size limitation on file uploads to CRM. Since you usually get really good compression rates on csv and xml files this is really good news.

For those of you who don't know these limitations, the maximum upload file size to CRM is 8 MB. But if you zip the files the total size of files that can be contained within the zipfile can be up to 35 MB. Just to give you an example of the compression rates that can be achieved, we tried compressing a 70 MB xml based excel document and the zip-file we got was only 0.5 MB. That is a ratio of 1/140, less that 1% of the original size.

Zipping the import files is also good for uploading multiple files with at once, for instance when the files have dependencies or when there are attachments as this reduces the manual work a lot.

A bit of advice though. I would strongly advice against using the csv-format since it is very volatile and often brakes and gives rise to a lot of strange errors. It is also dependent on the local regional settings, for instance, in Sweden, it is not comma separated, but semi-colon separated, as this is the standard numeral separator. Very confusing.

On top of this you often have problems with codepages, if you work in other languages than english, and you have to re-save the file in UTF-8 to make it import properly. Today we also had some issues with Excel insisting on formating the Swedish personal number (Social Security Numbers) as 7.41231E+12 instead of the correct 7412311234. We had to manually force it with different kinds of formatting to save the number correctly.

It didn't end there, Excel also found it fitting to add a few empty columns to the end of our CSV-file making it impossible to import into CRM.

So, we have been working to iron these things out and we were getting there but somewhere along the line we decided to skip the CSV and to go all in for the excel as xml format instead. The files that are saved are a lot bigger but with the zip feature it is managable.

Despite the fact that the new xml-format support and the zip-support exist I think the next time I will be even more vigourous in my recommendation for an import tool such as QuickBix Integration Suite, Scribe, or Import Manager. The new import features are great but for smaller imports, not for migrations.

Gustaf Westerlund
CEO, Chief Architect and co-Founder at CRM-konsulterna AB
www.crmkonsulterna.se

Nice blog posting on new CRM 2011 certifications

Back from the summer vacation and discussed CRM 2011 certifications on www.crmforum.se where my friend Fredrik Neiderud hinted about a nice blog posting by Richard Knudson regarding the new CRM 2011 certifications and what new in them.

Please read about it here: http://www.dynamicscrmtrickbag.com/2011/07/11/dynamics-crm-2011-certifications/

Gustaf Westerlund
CEO, Chief Architect and co-Founder at CRM-konsulterna AB
www.crmkonsulterna.se

Some interesting questions on xRM functionality

I have received a few questions from one of the readers of this blog and I will answer them bellow:

I am currently working on a project which is planning to use CRM 2011 as a development platform for migrating an existing  vertical specific LOB application.  As the existing application has a large and complicated database, I am trying to better understand how CRM 2011 can use external data, hence the following questions:
-          Can CRM 2011 use external data assuming it is exposed as an OData feed?

Yes and no. CRM 2011 cannot using some simple mode of configuration connect to an OData feed but there are numerous customizations that can be done using techniques such as javascript, mash-ups, SilverLight, XSLT or ASP.NET that can be used to work with the Odata feed. When I work with my clients, I usually try to understand what they are trying to achieve to be able to recommend a technique that fits their requirements.

-          Will CRM allow creation of Metadata for external Odata?

Yes, as mentioned above the different customization points allow for different ways of communicating and working with external data and metadata such as OData. There are no built in methods that automatically will connect OData to CRM (as I know of anyway).

-          Can CRM security model be used for external Odata?

The CRM security model is rather complex as it incorporates several concepts at the same time;

o   Security roles
o   Organizational context with ownership of objects based on this and in conjunction with the roles.
o   Sharing privileges to individual users and/or teams

This results in the fact that it is rather complex to replicate the privileges for a certain object to a related object in another system. It can be done, but requires in-depth knowledge of the CRM security model and the API:s as all data related to this can be accessed from the API.

As an example of the complexity, let’s say that you have users A, B and C. User A belongs to division inner sales, user B to division outer sales and user C to division service. All have the same role that only allows access to the accounts within the same division. The account “Vodaphone” is owned by A and A creates a new team to which he shares the rights to view the account “Vodaphone”. He then adds B to this team while creating it. Some time later he adds C to this team as well and shares read-write access directly to B as well.

This is an example that despite the fact that they all have the same roles would involve a lot of setting access rights in the external system since the adding of team members to teams indirectly sets their rights. It is not impossible to implement but quite complex and does require a considerable amount of effort. For an experienced CRM developer, probably not less than 200 hours.

When discussing issues like this with my clients, I, as I usually do, try to understand their core needs and motives so that I can suggest a good-enough solution that will give them the best bang of the buck.

-          Does CRM allow the use of external Odata in native forms or custom Silverlight UI has to be developed?

No. Fields (previously called attributes) are directly connected to the database and there is no support for virtual attributes that will retrieve data from an external source.

Integrations to external systems, using any data source, for instance OData, can of course be created that will transfer data to CRM. Data could for instance be periodically sent or sent when there is an update if the external system allows that kind of triggering.

It is also possible to jack into the retrieve or retrieve multiple events that occur when CRM tries to retrieve data so that CRM could pull data from the external data source when data is retrieved. I would not recommend this though since these events occur a lot and it would probably affect performance badly. It is also supported to fetch data directly from the database views using SQL and that would bypass this logic making this data incorrect. This would also cause some problems with working with CRM offline.

The solution you are hinting at using Silverlight is of course also an option but would be similar to some mash-up integration that could also be created using ASP.NET or javascript.

It would sometimes also be an option to create specific reports in CRM that connect to external data sources and integrate the CRM data with the external data in this manner. This method is preferable when using data warehousing or when there are heavy performance issues, as these reports can be cached and scheduled.

Without knowing more about the specific requirements, I would hence recommend some kind of periodic or live integration that transfers the required data to CRM in read-only fields as this would be most in line with how CRM works and would enable all built in features that CRM has to work with this data. It would also be best from a performance perspective and could be scaled if needed.

-          Can external Odata be used in Workflow activities?

Yes, custom workflows activities can be created in all versions of Microsoft Dynamics CRM 2011 except CRM Online where the same logic can be created with some work-around effort using plugins instead. These custom workflow activities are created using the .NET framework typically in C# and can hence easily use any available datasource, like Odata.

On a related front, does CRM allow complicated (i.e. highly normalized and multi-level) data hierarchies to be implemented  natively?  If so, is there a performance penalty?  Would we be better off using this data as an external OData feed or bringing it internal to CRM?

No, CRM does not support some of the more complex OO concepts like interfaces and inheritance. As this is something many of us in the CRM community are looking for and it would also be a great addition to the xRM concept so I am certain it is on Microsoft’s list of features they are trying to implement. How far up, I cannot say.

This is usually not a major issue for me when I work with my clients and I have created very complex verticalizations, for instance for the Telco industry which implement a highly normalized and multi-level information model called SID, in CRM. What you need to do is make it a bit more flat and more concrete which you more or less have to do anyway if you want to make the system usable. It is of course important to be well aware of the simplifications you are making but I do not imagine that any usable system possibly could implement the entire SID information model without simplification and still make the system usable and with any reasonable budget. The abstract class “Party” for instance is just too abstract to be of any real use, but it is of course useful to know that contact, account, competitor, lead, system user etc. do share some information. The implementation would however have benefited from one or two inheritances or interfaces.

If the data should be kept in or outside the system is not easy to answer without deeper knowledge of what you are trying to implement. If you keep it mostly outside, you would have to define a very strict line at which information is kept inside or outside of CRM. Do not, however, underestimate the power of the xRM platform, it is often advantageous to implement large or all parts of the LOB application inside CRM/xRM since you get so much functionality for free when doing this.

In general I would need deeper domain knowledge to be able to give you more direct recommendations and I do not think that this is the forum for that kind of discussion as that requires a dialog and would probably best be done in workshops.

I do like the fact that you sent me these questions as I think they are very relevant when looking at the system from an xRM perspective and put the finger on some of the issues that Microsoft have to improve in the system. If there are other readers out there, if you have questions, please send them to me and I will try to answer them on this blog.

Gustaf Westerlund
CEO, Chief Architect and co-Founder at CRM-konsulterna AB
www.crmkonsulterna.se

Today I was deploying a solution to a production environment and ran into a quite difficult problem; the custom report we had made could not be uploaded to CRM and the error description was rather cryptic. This error description was independent on if we tried to import the entire solution or just added the report directly. The error we received was the following:

Unhandled Exception: System.ServiceModel.FaultException`1[[Microsoft.Xrm.Sdk.OrganizationServiceFault, Microsoft.Xrm.Sdk, Version=5.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35]]:

An error occurred while trying to add the report to Microsoft Dynamics CRM. Try adding the report again. If this problem persists, contact your system administrator.Detail:

http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.microsoft.com/xrm/2011/Contracts">
  -2147188072
  http://schemas.datacontract.org/2004/07/System.Collections.Generic" />
  An error occurred while trying to add the report to Microsoft Dynamics CRM. Try adding the report again. If this problem persists, contact your system administrator.
  2011-06-16T07:43:12.1856313Z
 
 


I Binged/Googled the webb some and found that this article probably describes this error as it fits our environmen setup:

http://support.microsoft.com/kb/2491352

The error was due to the fact that reports written for SQL 2008 R2 are not compatible with SQL 2008. This has some very important effects as many customers might be running SQL 2008 for the production environments and many developers work with SQL 2008 R2 when developing. This difference might not be noticeable in general but this fact will cause these reports  to not be importable into the production environment.
So, a general suggestion is to have all environments, development, development test, acceptance test, training and production on exactly the same version of both Windows server and SQL-server to avoid any errors showing up in one of the environments but not the other, something that can be very problematic to solve due to the difficulty of finding the error since it by definition cannot be replicated on some of the environments.
Gustaf Westerlund
CEO, Chief Architect and co-Founder at CRM-konsulterna AB
www.crmkonsulterna.se

Problem with Custom Workflow Activities in CRM 2011

The workflow activities framework for CRM 2011 has been upgraded from CRM 4.0. The dependance properties have been removed and it now support the simplified datatypes used in CRM 2011 like DateTime.

I was developing a custom activity today and I had an output argument that was of the type EntityReference. However, when I tried to run it I got a weird error:

"The argument of type 'Microsoft.Xrm.Sdk.EntityReference' cannot be used. Make sure that it is declared on an activity."

The argument declaration looked like this:

[Output("The Output of special entity")]
[ReferenceTarget("new_specialentity")]
public OutArgument specOut { get; set; }

And I am trying to set it with the following code:

specOut = new OutArgument();
Entity outent = entities.Entities[0];
specOut.Set(executionContext, outent.ToEntityReference());

I had also tried creating the entity reference with a default contstructor and filling id and logicalname, with the constructor setting id and logicalname. I had even tried setting it using the entity directly but they all give the same error. I had not found anything useful on the internet or in the SDK.

After some rethinking I looked closer at the error message and thought that there might be something wrong with the declaration or definition of the OutArgument. And after some tries I found that the error was that I had created a new OutArgument which is wrong, it is precreated and this was the reason for the error. Hence I just removed the line:

specOut = new OutArgument();

and it work like it should have from the beginning.

It is always interesting to work with new technology!

Gustaf Westerlund
CEO, Chief Architect and co-Founder at CRM-konsulterna AB

www.crmkonsulterna.se

Ignoring system checks during installation of CRM

Sometimes when installing, it can be necessary to allow the installation to take place despite the fact that some of the system checks that are conducted during the installation fail. There are some different places mentioning how to do this, it is actually rather simple, you just have to add the DWORD key IgnoreChecks with a value of "1" to the MSCRM key in the registry [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSCRM].


Gustaf Westerlund
CEO, Chief Architect and co-Founder at CRM-konsulterna AB

www.crmkonsulterna.se

Problems when connecting PluginRegistration Tool to CRM

I was going to deploy some plugins to a server today and when I tried to connect to the CRM with the PlugingRegistration.exe I found got the following error:

Unhandled Exception: System.ServiceModel.FaultException: The server was unable to process the request due to an internal error.

So I switched on the tracing in CRM and read the following from the trace log (edited):

System.IO.FileNotFoundException, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089

Could not load file or assembly 'Microsoft.Xrm.Client, Version=5.0.2.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified.

The error seems to originate from CRM not being able to find the Microsoft.Xrm.Client.dll file. After some fiddeling, I fixed it by copying the file from the SDK\bin directory to CRMWeb\bin directory. It is a bit strange that it is required that you do this, and it will probably be fixed in future rollups of CRM 2011.

By the way, I heard some rumors from the CRM Statement of Direction that CRM 6 will be released Q2 2012 and that it will have multibrowser support. The feature we have all been waiting for, and it is probably going to be more or less the same thing as the 2011 version but with multibrowser support. So Software Assurance will probably be a good idea this year for Microsoft Dynamics CRM.

Gustaf Westerlund
CEO, Chief Architect and co-Founder at CRM-konsulterna AB
www.crmkonsulterna.se