Scripting Form Scripts

Motivation

With Microsoft Dynamics CRM we can put client-side scripts on forms, which can be very useful.

Unfortunately no part of the process is pleasant. First there is the JavaScript, which I personally really don't enjoy. Then, on the CRM side, we have to upload the script, go to the form editor, properties, add the dependencies (one at a time mind you), register the call-back, save and publish the form. Finally we can test our script, at which point we discover that we used the wrong name for the call-back or forgot one of your dependencies, not to mention that the JavaScript itself could have bugs, and we have to go through the entire process again.

It may seem like a small issue, after all, once the script is up and running we can skip most of the steps above. But if you manage a lot of CRM systems you may find yourself often spending half an hour to an hour just getting a new script running for the first time, excluding coding it. This is too wasteful and error-prone for my taste, so I decided to automate a lot of it.

Remember: this is a technical post, a usage/tutorial post will follow next week.

Prerequicits

Client-side scripts are considered part of a forms layout. The form layout in CRM is represented as XML, and through the standard API the only way to affect it is by manipulating this XML. Thus, before we begin, the reader is expected to be familiar with:

• XML (as a reference CRM's FormXML)
• JavaScript (or something that compiles to JavaScript)
• F#, TypeProviders (in particular the XrmDataProvider)
• and of course Microsoft Dynamics CRM.

Utilities

Let us start by defining some convinient functions.

I find the functional string library to be missing a few useful functions, in this case only one:

module StringUtils =
  (** Partition a string into two at the last occurance of c from the left *)
  let partition_right (c : char) (s : string) =
    let i = s.LastIndexOf c in
    (s.Substring(0, i), s.Substring(i+1)) 

The XML library we will be using comes from C#, which means it uses lots of side-effects and is (mostly) untyped, neither of which are good in my oppinion, however I'm too pressed for time to write my own XML library. Having said that, I prefer to have these functions:

#r "System.Xml.dll"
open System.Xml

module XmlUtils =
  (** Selects a single node, or creates it if it doesn't exist *)
  let rec select_node (doc : XmlDocument) path =
    let lookup = doc.SelectSingleNode path in
    if lookup <> null
    then lookup
    else 
      let (path', node) = StringUtils.partition_right '/' path in
      let new_node = doc.CreateElement node in
      (select_node doc path').AppendChild(new_node)

  (** Selects a list of nodes, and creates the path to it if the path doesn't exist *)
  let select_nodes (doc : XmlDocument) path = 
    let (path', node) = StringUtils.partition_right '/' path in
    let parent = select_node doc path' in
    parent.SelectNodes(node) |> Seq.cast

  (** Appends a node to a path, created the path if it doesn't exist *)
  let append_node (doc : XmlDocument) path node =
    (select_node doc path).AppendChild(node) |> ignore

  let to_list (xs : XmlNodeList) =
    let rec loop i acc =
      if i < 0
      then acc
      else loop (i - 1) ((xs.Item(i)) :: acc) in
    loop (xs.Count) []

Finally, because I want to support multiple versions of CRM, I need to detect which one we are using, which also means we need to instantiate the CRM TypeProvider:

module VersionUtils =
  type version = private { v : int [] }
  let of_string (s : string) = { v = Array.map int (s.Split('.')) }
  let major v = v.v.[0]

#r "microsoft.xrm.sdk.dll"
open Microsoft.Xrm.Sdk
#r "FSharp.Data.DynamicsCRMProvider.dll"
open FSharp.Data.TypeProviders
#r "FSharp.Data.DynamicsCRMProvider.Runtime.dll"
open FSharp.Data.TypeProviders.XrmProvider.Runtime.Common

(* CRM TypeProvider *)
type XDP = XrmDataProvider<"https://some_domain/XRMServices/2011/Organization.svc"
                , Username="username"
                , Password="password">
let xrm = XDP.GetDataContext()
type XRM = XDP.XrmService

#r "Microsoft.Crm.Sdk.Proxy.dll"
open Microsoft.Crm.Sdk.Messages
let sdk_version = 
  let crmVersion = 
    let req = new RetrieveVersionRequest() in
    let resp = xrm.OrganizationService.Execute(req) :?> RetrieveVersionResponse in
    resp.Version
    |> VersionUtils.of_string
    |> VersionUtils.major in
  match crmVersion with
    (* https://support.microsoft.com/da-dk/lifecycle?p1=15707 *)
    | 5 -> "2011"
    | 6 -> "2013"
    | 7 -> "2015"
    | 8 -> "2016"
    | _ -> failwith "Unsupported CRM version"

And also we need to publish our changes eventually:

let publish () = 
  let req = Microsoft.Crm.Sdk.Messages.PublishAllXmlRequest () in
  xrm.OrganizationService.Execute(req);

The Library

The library itself consists of five functions for: initialization, registering dependencies, registering call-backs, validating the XML, commiting the XML to CRM.

When I started designing this library I had a goal. I like chain-calling, so we should have something, let's call it a 'needle', that can be 'threaded' through all the calls. This means that the needle has to be the last argument, and every function has to return it, or a variation of it.

Initialization

For ease of use we have a record containing all information that the library needs:

• the name of the form
• the name of the entity
• the xml for the form
• and a few references to the form.

The initialization is just filling those fields:

type form_info = 
  { name : string;
    entity : string;
    doc : XmlDocument;
    form_id : System.Guid;
    original_formxml : string }

let init (form : XRM.systemform) =
  let xml = form.formxml in
  let doc = new XmlDocument() in
  doc.LoadXml xml; 
  { name = form.name;
    entity = form.objecttypecode;
    doc = doc;
    form_id = form.Id;
    original_formxml = form.formxml }

Registering Dependencies

For registering something, the only real consideration is wether it is already registered or not. Therefore, the structure of the next two functions is the same: check if the argument is already registered, if it is then skip, otherwise register it.

let register_dependency file fi = 
  let already_registered = 
    XmlUtils.select_nodes fi.doc "/form/formLibraries/Library"
    |> Seq.exists (fun n -> 
      n.Attributes.GetNamedItem("name").Value = file) in
  if already_registered
  then 
    printfn "%s is already registered on %s (%s)" 
      file fi.name fi.entity; 
    fi
  else
    let libNode = fi.doc.CreateElement("Library") in
    libNode.SetAttribute("libraryUniqueId", 
      System.Guid.NewGuid().ToString("B"));
    libNode.SetAttribute("name", file);
    XmlUtils.append_node fi.doc "/form/formLibraries" libNode;
    fi

Registering Call-backs

The structure here is essentially the same, although the XML is slightly more involved.

type event_type = OnLoad | OnSave
let event_type_to_string = function OnLoad -> "onload" | OnSave -> "onsave"
let register_callback t file func fi = 
  register_dependency file fi |> ignore;
  let already_registered = 
    XmlUtils.select_nodes fi.doc "/form/events/event/Handlers/Handler"
    |> Seq.exists (fun n -> 
      n.Attributes.GetNamedItem("functionName").Value = func &&
      n.Attributes.GetNamedItem("libraryName").Value = file) in
  if already_registered
  then 
    printfn "%s (%s) is already registered on %s (%s)" 
      func file fi.name fi.entity; 
    fi
  else
    let handlerNode = fi.doc.CreateElement "Handler" in
    handlerNode.SetAttribute("handlerUniqueId", 
      System.Guid.NewGuid().ToString("B"));
    handlerNode.SetAttribute("functionName", func);
    handlerNode.SetAttribute("libraryName", file);
    handlerNode.SetAttribute("enabled", "true");
    handlerNode.SetAttribute("passExecutionContext", "false");
    handlerNode.SetAttribute("parameters", "");
    let handlersNode = fi.doc.CreateElement "Handlers" in
    let eventNode = fi.doc.CreateElement "event" in
    eventNode.SetAttribute("name", event_type_to_string t);
    eventNode.SetAttribute("application", "false");
    eventNode.SetAttribute("active", "false");
    handlersNode.AppendChild(handlerNode) |> ignore;
    eventNode.AppendChild(handlersNode) |> ignore;
    XmlUtils.append_node fi.doc "/form/events" eventNode;
    fi

Validating the XML

This is the most critical part of the library. We also have to spent the most time thinking about it. First consideration is that we don't want to commit something without having validated it. Second when we have validated something we don't want it to change without having to validate it again. We will deal with these in turn.

In order to ensure that something is validated before it is committed we take advantage of the types. We introduce a new type so that commit takes something that you can only get through validate. We still need all the information from form_info, and the way we prevent users from making a validated value themselves is by making it private.

type validated = private { fi : form_info }

Making sure that something doesn't change after validation is a bit more subtle. Because the XML library has side effects someone could accidentally dublicate a file_info, validate one of them, make changes in the other one, thereby changing the xml of the validated one -- because the doc fields point to the same object -- but because the first is already validated we can commit it. It would look something like this:

let fi = CrmUtils.FormXml.init someform in
let v = CrmUtils.FormXml.validate fi in
(* change fi.doc *)
CrmUtils.FormXml.commit v

The solution I went with is to clone all the mutable data, in this case only doc.

open System.Xml.Schema;
let validate fi = 
  let validationEventHandler sender (e : ValidationEventArgs) =
    match e.Severity with
      | XmlSeverityType.Error -> 
        printf "Validation error: %s\n" e.Message; exit (1)
      | XmlSeverityType.Warning -> 
        printf "Validation warning: %s\n" e.Message
      | _ -> failwith "Impossible" in
  (* "Adding a schema to the XmlSchemaSet with the same 
      target namespace and schema location URL as a schema 
      already contained within the XmlSchemaSet will return 
      the original schema object."
      - https://msdn.microsoft.com/en-us/library/1hh8b082(v=vs.110).aspx *)
  fi.doc.Schemas.Add(null, cfg.rootFolder + @"\SDK\" + sdk_version + @"\Schemas\FormXML.xsd") |> ignore;
  fi.doc.Validate(new ValidationEventHandler(validationEventHandler));
  { fi = { fi with doc = fi.doc.Clone() :?> XmlDocument } }

Note: You need the CRM SDKs in ".\SDK\201X"

Commiting the XML to CRM

This part is the dangoures one. We already validated our XML, but what if there is something we overlooked? A universal advice applies here too: "Always backup your data".

Another thing to note is that CRM is a little picky about the XML you submit to it, so we need to suppress the <?xml version="1.0" encoding="UTF-8"?>.

open System.IO
let commit v =
  (* "If the directory already exists, this method does not 
      create a new directory"
      - https://msdn.microsoft.com/en-us/library/54a0at6s(v=vs.110).aspx *)
  Directory.CreateDirectory(cfg.rootFolder + @"\backup") |> ignore;
  File.WriteAllText(cfg.rootFolder + @"\backup\" + 
    System.DateTime.Now.ToString("yyyyMMddHHmmss") + "-" + 
    v.fi.entity + "." + v.fi.name + ".xml", 
    v.fi.original_formxml);
  let ws = new XmlWriterSettings() in
  ws.OmitXmlDeclaration <- true;
  let sw = new StringWriter() in
  let writer = XmlWriter.Create(sw, ws) in
  v.fi.doc.Save(writer);
  let entity = new Entity() in
  entity.Id <- v.fi.form_id;
  entity.LogicalName <- "systemform";
  entity.Attributes.Add("formxml", sw.ToString());
  xrm.OrganizationService.Update(entity)

This concludes the core library. Next week we will look at how to use this library out of the box, and how to build some neat functions on top of it.

Quality Control

As I am devoded to high quality software we should take a step back and examine how solid this library is. Even though this works and could save us a lot of time we shouldn't ignore potential risks such as overwriting files, destroying forms, or in the worst case locking up the entire CRM system.

Is it supported to change the XML of a form?

We already know that FormXML is documented by Microsoft. Further from Microsofts website (Customize entity forms) we have:

Editing the form definitions from an exported managed solution and then re-importing the solution is a supported method to edit entity forms. When manually editing forms we strongly recommend you use an XML editor that allows for schema validation.

Regarding the validation part, as we have seen, our library actually requires the XML to be validated before it can be committed.

If something goes wrong, then what?

As just mentioned, changing the XML is supported and we require the XML to be validated, making it very unlikely for something to go wrong. Regarding the script files, we always check if files exists before writing to them. Finally each time we commit XML to CRM we make a backup of what the XML was before the commit, thus, even if we could destroy XML we can easily restore it.

Acknoledgements

Thanks to Ramón Soto Mathiesen, Jacob Blom Andersen, and Martin Kasban Tange for feedback and discussion during the development of this library.

The First Post

Motivation for this blog

I recently graduated my masters degree in computer science, specializing in Dependent Types (functional programming) and teaching. As such, I have strong interests in high quality software, and knowledge sharing. These are the focuses of this blog. My hope is that this blog will be mutually beneficial for readers and me through code sharing and community interaction. For clarity, I have chosen to focus on two topic. In particular I have chosen Customer Relationship Management (abbr: CRM) systems and Functional Programming (abbr: FP) because I think they are interesting, relevant, and current topics.

CRM

The CRM system I consider is Microsoft Dynamics CRM, mostly online, however there might be the occasional post about on-premise. One of my goals is to be up-to-date so I mostly post about 2015 (until CRM 2016 goes live), but when I build software it should support all the Microsoft supported versions.

Functional Programming

I come from a background in OCaml programming, but am in the process of switching to F#. It works better with CRM, thus making for a more consistent blog. Since most (functional) OCaml code is also valid F# code the switch is pretty seamless, however I still have some style-artifacts.

Format

As this is a technical blog, I try to include code as much as possible. I agree with Martin Fowlers statement that comments are often used to cover up bad code, therefore I stride for my code to be self-documenting. Apart from that, the format depends wither the post is about CRM or FP.

CRM

For CRM, the code will probably be more complex and focusing on software architecture. In technical CRM posts there are always two perspectives: how to build it, and how to use it, I intend to split these into separate posts, hoping that each will be more clear. When I do less technical posts, I intend to include as many screenshots as possible.

Functional Programming

For FP I usually try to make my demonstrations small'n'cute, presenting concepts in a clear, down-to-earth manner. As I am a teacher by heart, I may occasionally post tutorials or exercises, intended to teach useful techniques in a safe and guided fashion.

Some code

As mentioned, I try to post code as much as possible, so here is this posts share.

I was recently challenged by a colleague to make a (Pure) list library where appending was O(1) without using the lazy-keyword. As suggested by the formulation he expected me to implement lazyness through explicit thunks.

type 'a mylist_lazy =
  | Nil
  | Cons of 'a * (unit -> 'a mylist_lazy)

I leave the rest of this library as an exercise for the reader.

However, there is another way to implement such lists, which is quite neat. The key idea is to have a pointer to the end of the list, but because the solution has to be pure, we cannot use references. What we can use however is functions. The intuition is that we postpone putting in the list terminator (nil). Equationally we see:

1 :: 2 :: 3 :: nil = (fun tl -> 1 :: 2 :: 3 :: tl) nil

So, instead of returning lists we return a function taking the tail of the list and returning the final list. Append now just becomes the function composition of two mylists.

type 'a mylist = 'a list -> 'a list
let nil = fun tl -> tl
let cons x xs' = fun tl -> x :: xs' tl
let append xs ys = fun tl -> xs (ys tl)
let is_empty xs = (* exercise *)
let head xs = (* exercise *)
let tail xs = (* exercise *)
let map f xs = (* exercise *)
let to_list xs = (* exercise *)