Using the FormXML Library

Previously on Dr. Lambda's blog:

With Microsoft Dynamics CRM we can put client-side scripts on forms. Having a JavaScript file, we have to upload the script, go to the form editor, properties, add the dependencies (one at a time), register the call-back, save and publish the form. Finally, we can test our script, at which point we discover that we used the wrong name for the call-back or forgot one of the dependencies, not to mention that the JavaScript itself could have bugs, and we have to go through most of the process again.

module CrmUtils = begin
  val sdk_version : string
  val publish : unit -> unit
  module FormXml = begin
    type form_info
    val init : XRM.systemform -> form_info

    val register_dependency : string -> form_info -> form_info

    type event_type
    val event_type_to_string : event_type -> string
    val register_callback : event_type -> string -> string -> form_info -> form_info

    type validated
    val validate : form_info -> validated

    val commit : validated -> unit
  end
end

Now, the continuation...

Off the Shelf Usage

The library is intended to be usable without being an F# or functional programmer. Let's explore the library through some examples.

Example 1

Say we have a JavaScript file called Test.Feature.js, which we have already uploaded as a webresource. The file contains a function called onload. Now, we want to register a call-back for this funciton, on the main form for the account entity, which is called Account.

xrm.systemformSet.Individuals.Account
|> CrmUtils.FormXml.init
|> CrmUtils.FormXml.register_callback CrmUtils.FormXml.OnLoad "Test.Feature.js" "onload"
|> CrmUtils.FormXml.validate
|> CrmUtils.FormXml.commit
|> CrmUtils.publish

This will register the dependency and the call-back. This was easy, however imagine we need want to use the same feature on a different form, we only need to change the first line and run it again.

Example 2

The advantage of the library is even clearer if the file had had dependencies. For demonstration say it depended on Test.Utilities.js and jquery.js.

xrm.systemformSet.Individuals.Account
|> CrmUtils.FormXml.init
|> CrmUtils.FormXml.register_dependency "jquery.js"
|> CrmUtils.FormXml.register_dependency "Test.Utilities.js"
|> CrmUtils.FormXml.register_callback CrmUtils.FormXml.OnLoad "Test.Feature.js" "onload"
|> CrmUtils.FormXml.validate
|> CrmUtils.FormXml.commit
|> CrmUtils.publish

Going Beyond

While the library is neat off-the-shelf, it also enables us to go even further. But first, let's warm up with some easy, useful, and obvious features to support the library.

Some General CRM Utilities

Currently our CrmUtils is pretty bare-bones so let's extend it with a few extra functions. Here are functions for finding: a solution, the publisher of a solution, and the prefix used by that publisher.

let solution solName =
  xrm.solutionSet
  |> Seq.find (fun s -> s.uniquename = solName)

let solution_publisher solName =
  let sol = solution solName in
  xrm.publisherSet
  |> Seq.find (fun p -> p.Id = sol.publisherid.Id)

let solution_prefix solName =
  solName |> solution_publisher |> fun p -> p.customizationprefix

Form Selection

Selecting forms with the xrm.systemformSet.Individuals.-method has a few disadvantages: Many forms are called Information, only one is accessible with this method, and it is difficult to know which. Therefore, another nice addition would be a complete and convenient way to uniquely select a form. In order to do this we need to know the name and type of the form, and which entity it is on:

let select_form entity t name =
  xrm.systemformSet
  |> Seq.find (fun sf -> sf.objecttypecode = entity && sf.``type`` = t && sf.name = name)

At this point, we can reiterate example 1:

select_form "account" XRM.systemform_type.Main "Account"
|> CrmUtils.FormXml.init
|> CrmUtils.FormXml.register_dependency "jquery.js"
|> CrmUtils.FormXml.register_dependency "Test.Utilities.js"
|> CrmUtils.FormXml.register_callback CrmUtils.FormXml.OnLoad "Test.Feature.js" "onload"
|> CrmUtils.FormXml.validate
|> CrmUtils.FormXml.commit
|> CrmUtils.publish

Restoring

In the last post, we discussed the risks of using the library, and one of the relaxing arguments was that you could always restore a form, using a back. Let us express this in a function taking a form and a filename.

let restore file (form : XRM.systemform) =
  if form.LogicalName <> "systemform" then
    failwith "Not a form";
  let e = Entity () in
  e.Id <- form.Id;
  e.LogicalName <- "systemform";
  e.Attributes.Add("formxml", System.IO.File.ReadAllText(cfg.rootFolder + file));
  xrm.OrganizationService.Update(e);
  CrmUtils.publish ()

Notice that we publish directly, even though it is time consuming. This function is intended for if something goes wrong. If it does, the user would be under a lot of stress, thus restore should be easy to call, and the user shouldn't need to remember anything, like publishing.

Web Resources

Last week I also mentioned that I had functionality for uploading web resources. I did not think they belonged in the FormXml library, however, in this context it fits perfectly.

There are a few caveats. First, web resources need to be encoded in UTF8 base 64. Second, if the web resource is new we want to create it in the appropriate solution, this requires us to pass an extra parameter, SolutionUniqueName. Only, this parameter is only supported on requests, so instead of calling xrm.OrganizationService.Create we have to execute an explicit CreateRequest. Third, the name of the web resource has to be prefixed by the prefix from the chosen solution.

I have chosen to split file-upload into two, as it is useful to be able to upload strings directly from code, as we shall see later.

let upload_text solName name (content : string) =
  let prefix = CrmUtils.solution_prefix solName in
  let already_exists =
    xrm.webresourceSet
    |> Seq.tryFind (fun wr -> wr.name = prefix + "_" + name) in
  let wf = Entity () in
  wf.LogicalName <- "webresource";
  wf.Attributes.Add("name", prefix + "_" + name);
  wf.Attributes.Add("displayname", name);
  wf.Attributes.Add("webresourcetype", OptionSetValue (int XRM.webresource_webresourcetype.``Script (JScript)``));
  wf.Attributes.Add("content", System.Convert.ToBase64String(System.Text.Encoding.UTF8.GetBytes(content)));
  match already_exists with
    | None ->
      let cr = Messages.CreateRequest () in
      cr.Target <- wf;
      cr.Parameters.Add("SolutionUniqueName", solName);
      xrm.OrganizationService.Execute(cr) |> ignore
    | Some wr ->
      wf.Id <- wr.Id;
      xrm.OrganizationService.Update(wf)

let upload_file solName file =
  System.IO.File.ReadAllText(cfg.rootFolder + @"\" + file)
  |> upload_text solName file

Turning it up to eleven

There is still one final function missing in order to solve the original problem. One unifying function that, given a form, will generate the initial JavaScript, upload it to CRM, and finally register the dependency and call-back. But before we can write that we need to define what should be in our JavaScript files.

let javascript t org (fi : CrmUtils.FormXml.form_info) =
  let fname = CrmUtils.FormXml.event_type_to_string t in
  "var " + org + ";\n" +
  "(function (" + org + ") {\n" +
  "  var " + fi.name + ";\n" +
  "  (function (" + fi.name + ") {\n" +
  "    var Form = Xrm.Page;\n" +
  "    function " + fname + "() {\n" +
  "    }\n" +
  "    " + fi.name + "." + fname + " = " + fname + ";\n" +
  "  }) (" + fi.name + " = " + org + "." + fi.name +
  " || (" + org + "." + fi.name + " = {}));\n"+
  "}) (" + org + " || (" + org + " = {}));"

I like to organize my JavaScript into modules, prefixed by the org argument.

This following function does solves the problem, although it also needs the solution name, the call-back type.

let new_script solName t org form =
  let fi = form |> CrmUtils.FormXml.init in
  let filename = org + "." + fi.name + ".js" in
  let prefix = CrmUtils.solution_prefix solName in
  let name = prefix + "_" + filename in
  if System.IO.File.Exists (cfg.rootFolder + @"\" + filename)
  then
    printf "Error: File '%s' already exists\n" filename;
    exit (1)
  let already_in_crm =
    xrm.webresourceSet
    |> Seq.exists (fun wr -> wr.name = name) in
  if already_in_crm
  then
    printf "Error: File '%s' already in crm\n" name;
    exit (1)
  let content = javascript t org fi in
  upload_text solName filename content;
  System.IO.File.WriteAllText(cfg.rootFolder + @"\" + filename, content);
  fi
  |> CrmUtils.FormXml.register_callback t name (org + "." + fi.name + "." + CrmUtils.FormXml.event_type_to_string t)
  |> CrmUtils.FormXml.validate
  |> CrmUtils.FormXml.commit

More Examples

After all that, working with scripts looks like this:

Start by running:

select_form "contact" XRM.systemform_type.Main "Information"
|> new_script "DrLambda" CrmUtils.FormXml.OnLoad "Test"
|> CrmUtils.publish

Then open Test.Information.js and code.

Then you can update the script with:

upload_file "DrLambda" "Test.Information.js"
|> CrmUtils.publish

Then, when you split up your code into multiple files, or start using libraries you just run:

select_form "contact" XRM.systemform_type.Main "Information"
|> CrmUtils.FormXml.init
|> CrmUtils.FormXml.register_dependency "jquery.js"
|> CrmUtils.FormXml.register_dependency "Test.Utilities.js"
|> CrmUtils.FormXml.validate
|> CrmUtils.FormXml.commit
|> CrmUtils.publish

More common operations require less work.

Scripting Form Scripts

Motivation

With Microsoft Dynamics CRM we can put client-side scripts on forms, which can be very useful.

Unfortunately no part of the process is pleasant. First there is the JavaScript, which I personally really don't enjoy. Then, on the CRM side, we have to upload the script, go to the form editor, properties, add the dependencies (one at a time mind you), register the call-back, save and publish the form. Finally we can test our script, at which point we discover that we used the wrong name for the call-back or forgot one of your dependencies, not to mention that the JavaScript itself could have bugs, and we have to go through the entire process again.

It may seem like a small issue, after all, once the script is up and running we can skip most of the steps above. But if you manage a lot of CRM systems you may find yourself often spending half an hour to an hour just getting a new script running for the first time, excluding coding it. This is too wasteful and error-prone for my taste, so I decided to automate a lot of it.

Remember: this is a technical post, a usage/tutorial post will follow next week.

Prerequicits

Client-side scripts are considered part of a forms layout. The form layout in CRM is represented as XML, and through the standard API the only way to affect it is by manipulating this XML. Thus, before we begin, the reader is expected to be familiar with:

• XML (as a reference CRM's FormXML)
• JavaScript (or something that compiles to JavaScript)
• F#, TypeProviders (in particular the XrmDataProvider)
• and of course Microsoft Dynamics CRM.

Utilities

Let us start by defining some convinient functions.

I find the functional string library to be missing a few useful functions, in this case only one:

module StringUtils =
  (** Partition a string into two at the last occurance of c from the left *)
  let partition_right (c : char) (s : string) =
    let i = s.LastIndexOf c in
    (s.Substring(0, i), s.Substring(i+1)) 

The XML library we will be using comes from C#, which means it uses lots of side-effects and is (mostly) untyped, neither of which are good in my oppinion, however I'm too pressed for time to write my own XML library. Having said that, I prefer to have these functions:

#r "System.Xml.dll"
open System.Xml

module XmlUtils =
  (** Selects a single node, or creates it if it doesn't exist *)
  let rec select_node (doc : XmlDocument) path =
    let lookup = doc.SelectSingleNode path in
    if lookup <> null
    then lookup
    else 
      let (path', node) = StringUtils.partition_right '/' path in
      let new_node = doc.CreateElement node in
      (select_node doc path').AppendChild(new_node)

  (** Selects a list of nodes, and creates the path to it if the path doesn't exist *)
  let select_nodes (doc : XmlDocument) path = 
    let (path', node) = StringUtils.partition_right '/' path in
    let parent = select_node doc path' in
    parent.SelectNodes(node) |> Seq.cast

  (** Appends a node to a path, created the path if it doesn't exist *)
  let append_node (doc : XmlDocument) path node =
    (select_node doc path).AppendChild(node) |> ignore

  let to_list (xs : XmlNodeList) =
    let rec loop i acc =
      if i < 0
      then acc
      else loop (i - 1) ((xs.Item(i)) :: acc) in
    loop (xs.Count) []

Finally, because I want to support multiple versions of CRM, I need to detect which one we are using, which also means we need to instantiate the CRM TypeProvider:

module VersionUtils =
  type version = private { v : int [] }
  let of_string (s : string) = { v = Array.map int (s.Split('.')) }
  let major v = v.v.[0]

#r "microsoft.xrm.sdk.dll"
open Microsoft.Xrm.Sdk
#r "FSharp.Data.DynamicsCRMProvider.dll"
open FSharp.Data.TypeProviders
#r "FSharp.Data.DynamicsCRMProvider.Runtime.dll"
open FSharp.Data.TypeProviders.XrmProvider.Runtime.Common

(* CRM TypeProvider *)
type XDP = XrmDataProvider<"https://some_domain/XRMServices/2011/Organization.svc"
                , Username="username"
                , Password="password">
let xrm = XDP.GetDataContext()
type XRM = XDP.XrmService

#r "Microsoft.Crm.Sdk.Proxy.dll"
open Microsoft.Crm.Sdk.Messages
let sdk_version = 
  let crmVersion = 
    let req = new RetrieveVersionRequest() in
    let resp = xrm.OrganizationService.Execute(req) :?> RetrieveVersionResponse in
    resp.Version
    |> VersionUtils.of_string
    |> VersionUtils.major in
  match crmVersion with
    (* https://support.microsoft.com/da-dk/lifecycle?p1=15707 *)
    | 5 -> "2011"
    | 6 -> "2013"
    | 7 -> "2015"
    | 8 -> "2016"
    | _ -> failwith "Unsupported CRM version"

And also we need to publish our changes eventually:

let publish () = 
  let req = Microsoft.Crm.Sdk.Messages.PublishAllXmlRequest () in
  xrm.OrganizationService.Execute(req);

The Library

The library itself consists of five functions for: initialization, registering dependencies, registering call-backs, validating the XML, commiting the XML to CRM.

When I started designing this library I had a goal. I like chain-calling, so we should have something, let's call it a 'needle', that can be 'threaded' through all the calls. This means that the needle has to be the last argument, and every function has to return it, or a variation of it.

Initialization

For ease of use we have a record containing all information that the library needs:

• the name of the form
• the name of the entity
• the xml for the form
• and a few references to the form.

The initialization is just filling those fields:

type form_info = 
  { name : string;
    entity : string;
    doc : XmlDocument;
    form_id : System.Guid;
    original_formxml : string }

let init (form : XRM.systemform) =
  let xml = form.formxml in
  let doc = new XmlDocument() in
  doc.LoadXml xml; 
  { name = form.name;
    entity = form.objecttypecode;
    doc = doc;
    form_id = form.Id;
    original_formxml = form.formxml }

Registering Dependencies

For registering something, the only real consideration is wether it is already registered or not. Therefore, the structure of the next two functions is the same: check if the argument is already registered, if it is then skip, otherwise register it.

let register_dependency file fi = 
  let already_registered = 
    XmlUtils.select_nodes fi.doc "/form/formLibraries/Library"
    |> Seq.exists (fun n -> 
      n.Attributes.GetNamedItem("name").Value = file) in
  if already_registered
  then 
    printfn "%s is already registered on %s (%s)" 
      file fi.name fi.entity; 
    fi
  else
    let libNode = fi.doc.CreateElement("Library") in
    libNode.SetAttribute("libraryUniqueId", 
      System.Guid.NewGuid().ToString("B"));
    libNode.SetAttribute("name", file);
    XmlUtils.append_node fi.doc "/form/formLibraries" libNode;
    fi

Registering Call-backs

The structure here is essentially the same, although the XML is slightly more involved.

type event_type = OnLoad | OnSave
let event_type_to_string = function OnLoad -> "onload" | OnSave -> "onsave"
let register_callback t file func fi = 
  register_dependency file fi |> ignore;
  let already_registered = 
    XmlUtils.select_nodes fi.doc "/form/events/event/Handlers/Handler"
    |> Seq.exists (fun n -> 
      n.Attributes.GetNamedItem("functionName").Value = func &&
      n.Attributes.GetNamedItem("libraryName").Value = file) in
  if already_registered
  then 
    printfn "%s (%s) is already registered on %s (%s)" 
      func file fi.name fi.entity; 
    fi
  else
    let handlerNode = fi.doc.CreateElement "Handler" in
    handlerNode.SetAttribute("handlerUniqueId", 
      System.Guid.NewGuid().ToString("B"));
    handlerNode.SetAttribute("functionName", func);
    handlerNode.SetAttribute("libraryName", file);
    handlerNode.SetAttribute("enabled", "true");
    handlerNode.SetAttribute("passExecutionContext", "false");
    handlerNode.SetAttribute("parameters", "");
    let handlersNode = fi.doc.CreateElement "Handlers" in
    let eventNode = fi.doc.CreateElement "event" in
    eventNode.SetAttribute("name", event_type_to_string t);
    eventNode.SetAttribute("application", "false");
    eventNode.SetAttribute("active", "false");
    handlersNode.AppendChild(handlerNode) |> ignore;
    eventNode.AppendChild(handlersNode) |> ignore;
    XmlUtils.append_node fi.doc "/form/events" eventNode;
    fi

Validating the XML

This is the most critical part of the library. We also have to spent the most time thinking about it. First consideration is that we don't want to commit something without having validated it. Second when we have validated something we don't want it to change without having to validate it again. We will deal with these in turn.

In order to ensure that something is validated before it is committed we take advantage of the types. We introduce a new type so that commit takes something that you can only get through validate. We still need all the information from form_info, and the way we prevent users from making a validated value themselves is by making it private.

type validated = private { fi : form_info }

Making sure that something doesn't change after validation is a bit more subtle. Because the XML library has side effects someone could accidentally dublicate a file_info, validate one of them, make changes in the other one, thereby changing the xml of the validated one -- because the doc fields point to the same object -- but because the first is already validated we can commit it. It would look something like this:

let fi = CrmUtils.FormXml.init someform in
let v = CrmUtils.FormXml.validate fi in
(* change fi.doc *)
CrmUtils.FormXml.commit v

The solution I went with is to clone all the mutable data, in this case only doc.

open System.Xml.Schema;
let validate fi = 
  let validationEventHandler sender (e : ValidationEventArgs) =
    match e.Severity with
      | XmlSeverityType.Error -> 
        printf "Validation error: %s\n" e.Message; exit (1)
      | XmlSeverityType.Warning -> 
        printf "Validation warning: %s\n" e.Message
      | _ -> failwith "Impossible" in
  (* "Adding a schema to the XmlSchemaSet with the same 
      target namespace and schema location URL as a schema 
      already contained within the XmlSchemaSet will return 
      the original schema object."
      - https://msdn.microsoft.com/en-us/library/1hh8b082(v=vs.110).aspx *)
  fi.doc.Schemas.Add(null, cfg.rootFolder + @"\SDK\" + sdk_version + @"\Schemas\FormXML.xsd") |> ignore;
  fi.doc.Validate(new ValidationEventHandler(validationEventHandler));
  { fi = { fi with doc = fi.doc.Clone() :?> XmlDocument } }

Note: You need the CRM SDKs in ".\SDK\201X"

Commiting the XML to CRM

This part is the dangoures one. We already validated our XML, but what if there is something we overlooked? A universal advice applies here too: "Always backup your data".

Another thing to note is that CRM is a little picky about the XML you submit to it, so we need to suppress the <?xml version="1.0" encoding="UTF-8"?>.

open System.IO
let commit v =
  (* "If the directory already exists, this method does not 
      create a new directory"
      - https://msdn.microsoft.com/en-us/library/54a0at6s(v=vs.110).aspx *)
  Directory.CreateDirectory(cfg.rootFolder + @"\backup") |> ignore;
  File.WriteAllText(cfg.rootFolder + @"\backup\" + 
    System.DateTime.Now.ToString("yyyyMMddHHmmss") + "-" + 
    v.fi.entity + "." + v.fi.name + ".xml", 
    v.fi.original_formxml);
  let ws = new XmlWriterSettings() in
  ws.OmitXmlDeclaration <- true;
  let sw = new StringWriter() in
  let writer = XmlWriter.Create(sw, ws) in
  v.fi.doc.Save(writer);
  let entity = new Entity() in
  entity.Id <- v.fi.form_id;
  entity.LogicalName <- "systemform";
  entity.Attributes.Add("formxml", sw.ToString());
  xrm.OrganizationService.Update(entity)

This concludes the core library. Next week we will look at how to use this library out of the box, and how to build some neat functions on top of it.

Quality Control

As I am devoded to high quality software we should take a step back and examine how solid this library is. Even though this works and could save us a lot of time we shouldn't ignore potential risks such as overwriting files, destroying forms, or in the worst case locking up the entire CRM system.

Is it supported to change the XML of a form?

We already know that FormXML is documented by Microsoft. Further from Microsofts website (Customize entity forms) we have:

Editing the form definitions from an exported managed solution and then re-importing the solution is a supported method to edit entity forms. When manually editing forms we strongly recommend you use an XML editor that allows for schema validation.

Regarding the validation part, as we have seen, our library actually requires the XML to be validated before it can be committed.

If something goes wrong, then what?

As just mentioned, changing the XML is supported and we require the XML to be validated, making it very unlikely for something to go wrong. Regarding the script files, we always check if files exists before writing to them. Finally each time we commit XML to CRM we make a backup of what the XML was before the commit, thus, even if we could destroy XML we can easily restore it.

Acknoledgements

Thanks to Ramón Soto Mathiesen, Jacob Blom Andersen, and Martin Kasban Tange for feedback and discussion during the development of this library.