How to use results of FSharp.Compiler.Services

Question

I'm trying to build a system that is similar to FsBolero (TryWebassembly), Fable Repl and many more that uses Fsharp.Compiler.Services.

So I expect it is feasible to achieve my goals but I encountered a problem that I hope is only a result of my lack of experience with that realm of software development

I'm implementing a service that gives user the power to write custom algorithms (DSL) in the context of the domain system.

The code to compile come as a plain raw string that is fully correct F# code.

Sample DSL algorithm looks like:

let code = """
                module M
                open Lifespace
                open Lifespace.LocationPricing

                let alg (pricing:LocationPricing) =
                    let x=pricing.LocationComparisions.CityLevel.Transportation
                    (8.*x.PublicTransportationStation.Data+ x.RailwayStation.Data+ 5.*x.MunicipalBikeStation.Data) / 14.
            """

that code compiles correctly via CompileToDynamicAssembly. I also provided proper reference to my domain *.dll via -r Fsc parameter.

And here comes my problems as next I have the generated dynamic assembly and want to invoke that algorithm. I do it with reflection (is there any other way?) with f.Invoke(null, [|arg|]) when arg is of type LocationPricing and comes from main/hosting project reference.

The Invoke doesn't work because I have error:

Cannot cast LocationPricing to LocationPricing

I had the same problem when tried to use F# interactive services, the error was similar:

Cannot cast [A]LocationPricing to [B]LocationPricing

I'm aware I have two same dlls in the context and F# does have extern alias syntax to solve it.

But other mentioned public systems somehow deals with that or I'm doing it wrongly.

I will look at code of Bolero and FableRepl but it will definately take some time to understand the pitfalls.

Update: Full code (Azure Function)

namespace AzureFunctionFSharp

open System.IO
open System.Text

open Microsoft.Azure.WebJobs
open Microsoft.Azure.WebJobs.Extensions.Http
open Microsoft.AspNetCore.Http
open Microsoft.AspNetCore.Mvc
open Microsoft.Extensions.Logging

open FSharp.Compiler.SourceCodeServices

open Lifespace.LocationPricing

module UserCodeEval =

    type CalculationResult = {
        Value:float
    }
    type Error = {
        Message:string
    }

    [<FunctionName("UserCodeEvalSampleLocation")>]
    let Run([<HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)>] req: HttpRequest, log: ILogger , [<Blob("ranks/short-ranks.json", FileAccess.Read)>]  myBlob:Stream)=

        log.LogInformation("F# HTTP trigger function processed a request.")



        // confirm valid domain dll location
        // for a in System.AppDomain.CurrentDomain.GetAssemblies() do
        //    if a.FullName.Contains("wrometr.lam.to.ranks") then log.LogInformation(a.Location)

        // let code = req.Query.["code"].ToString()
        // replaced just to show how the user algorithm can looks like
        let code = 
            """
                module M

                open Lifespace
                open Lifespace.LocationPricing
                open Math.MyStatistics
                open MathNet.Numerics.Statistics

                let alg (pricing:LocationPricing) =
                    let x= pricing.LocationComparisions.CityLevel.Transportation
                    (8.*x.PublicTransportationStation.Data+ x.RailwayStation.Data+ 5.*x.MunicipalBikeStation.Data) / 14.
            """

        use reader = new StreamReader(myBlob, Encoding.UTF8)
        let content = reader.ReadToEnd()
        let encode x = LocationPricingStore.DecodeArrayUnpack x 
        let pricings = encode content

        let checker = FSharpChecker.Create()
        let fn = Path.GetTempFileName()
        let fn2 = Path.ChangeExtension(fn, ".fsx")
        let fn3 = Path.ChangeExtension(fn, ".dll")

        File.WriteAllText(fn2, code)

        let errors, exitCode, dynAssembly = 
            checker.CompileToDynamicAssembly(
                [| 
                "-o"; fn3;
                "-a"; fn2
                "-r";@"C:\Users\longer\azure.functions.compiler\bin\Debug\netstandard2.0\bin\MathNet.Numerics.dll"
                "-r";@"C:\Users\longer\azure.functions.compiler\bin\Debug\netstandard2.0\bin\Thoth.Json.Net.dll"
                // below is crucial and obtained with AppDomain resolution on top, comes as a project reference 
                "-r";@"C:\Users\longer\azure.functions.compiler\bin\Debug\netstandard2.0\bin\wrometr.lam.to.ranks.dll"  
                |], execute=None)
             |> Async.RunSynchronously

        let assembly = dynAssembly.Value

        // get one item to test the user algorithm works in the funtion context        
        let arg = pricings.[0].Data.[0]

        let result = 
            match assembly.GetTypes() |> Array.tryFind (fun t -> t.Name = "M") with
            | Some moduleType -> 
                moduleType.GetMethods()
                |> Array.tryFind (fun f -> f.Name = "alg") 
                |> 
                    function 
                    | Some f -> f.Invoke(null, [|arg|]) |> unbox<float>
                    | None -> failwith "Function `f` not found"
            | None -> failwith "Module `M` not found"

        // end of azure function, not important in the problem context      
        let res = req.HttpContext.Response
        match String.length code with
            | 0 -> 
                res.StatusCode <- 400
                ObjectResult({ Message = "No Good, Please provide valid encoded user code"})
            | _ ->
                res.StatusCode <-200
                ObjectResult({ Value = result})

**Update: changing data flow ** To move forward I resigned to use domain types in both places. Instead I do all logic in domain assembly and only pass primitives (strings) to reflected invocation. I'm also suprised a lot that caching still works everytime I do compilation on each Azure Function call. I will experiment as well with FSI, in theory it should be faster than reflection but with additional burden to pass parameters to evaluations

Answer 1

In your example, the code that runs inside your dynamically compiled assembly and the code calling it need to share a type LocationPricing. The error you are seeing typically means that you somehow ended up with different assembly loaded in the process that is calling the dynamically compiled code and the code actually running the computation.

It is hard to say exactly why this happened, but you should be able to check whether this is indeed the case by looking at assemblies loaded in the current App Domain. Say that your shared assembly is MyAssembly. You can run:

for a in System.AppDomain.CurrentDomain.GetAssemblies() do
  if a.FullName.Contains("MyAssembly") then printfn "%s" a.Location

If you were using F# Interactive Services, then a trick to fix this is to start an FSI session and then send an interaction to the service that loads the assembly from the right place. Something along those lines:

let myAsm = System.AppDomain.CurrentDomain.GetAssemblies() |> Seq.find (fun asm ->
  asm.FullName.Contains("MyAssembly"))

fsi.EvalInteraction(sprintf "#r @\"%s\"" myAsm.Location)