Syntax proposal: async/await

[jaredly]

Except using lwt instead of promises? orr we could not tie it to anything in particular and have it construct calls to async_map fn (val) so you could do let async_map = Lwt.map at the top of your file, or let async_map fn val = Js.Promise.then_ (fun v => Js.Promise.resolve (fn v)) val.
Or maybe it should transform to bind instead of map?

[jordwalke]

Sounds good! Perhaps defining something like

module Async = Lwt

At the top of the file could be how you select the async "provider". I really like Lwt's syntactic ppx plugin, and we could provide first class syntax for it.

[jaredly]

Hmmm actually I'm thinking that all I really need is ppx_let https://github.com/janestreet/ppx_let with the proper setup.
Unfortunately it looks like reason doesn't support the 4.03 let%ext syntax :(

[cullophid]

does this not require a decision on what the standard for async should be ?
I was under the impression that that was still not decided...

[jaredly]

Actually, I've got a ppx together that isn't tied to any specific async implementation, as long as it adheres to a general signature.

[brentvatne]

@jaredly - have you thought more about this since July? I'm pretty interested in async/await syntax in reason -- the current async tools are a detractor for people coming from the JS side if you're used to having async/await. It would certainly make it an easier sell to re-write some of Expo's existing JS with Reason.

[jordwalke]

@brentvatne
This has been at the top of our minds this week. There's a couple of approaches. Currently, I'm trying to weight the pros/cons between supporting syntactic extension for promise libraries like Lwt, vs. providing syntactic sugar for callbacks.

Syntactic sugar for callbacks would not full featured, and has some pitfalls - however, it is much lighter weight by default, and is simply sugar for supplying a final callback argument: "Lightest Weight Threads" has a nice ring to it: Any function that accepts a callback of some regular form would be compatible.

You could imagine the let! keyword being used to indicate an async computation.

let asyncIncrement = (i, andThen) => andThen(i + 1);

let myFunc = (initialVal) => {
  let! foo = asyncIncrement(10);
  let z = nonAsync(20, foo);
  lat q = asyncIncrement(z);
  q;
};
let! finalValue = myFunc(200);
print_int(finalValue);

More examples, including how let! / and could enable "parellel" tasks (concurrent, actually):

https://gist.github.com/jordwalke/1cd1f18ef060e2a5226f54f31417f6f2

We could instead build syntactic support for integrating with fully featured promise libraries such as Lwt by using ppx_let. I wonder if the two approaches are not incompatible. I wish there was a way for one to progressively enhance to the other.

[kay-is]

I liked the Backcalls of LiveScript.

[brentvatne]

@jordwalke - the "Lightest Weight Threads" api looks elegant and easy to use. Coming from the JS side, my main concern is that when integrating Reason code with existing JS libs we'd want to be able to use this syntax with functions that return Promises, ideally without having to manually wrap them with another function for interop.

Suppose asyncIncrement returns Js.Promise, then let! foo = asyncIncrement(5) would have a similar desugaring but rather than passing in a callback to asyncIncrement it would pass it in to Js.Promise.then_ . Haven't thought about how to handle .catch though.

[jordwalke]

@brentvatne Yeah, we'd want to create a way to interop with full featured promise libraries (including JS promises).

You could specify a promise provider perhaps:

let myFunction = () => {
   open Promise;
   let! thisIsTheResultOfJSPromise = asyncCall();
   thisIsTheResultOfJSPromise + 10;
};

Or for short:

let myFunction = () => {
   Promise.let thisIsTheResultOfJSPromise = asyncCall();
   thisIsTheResultOfJSPromise + 10;
};

By default, you might consider that there's this hypothetical open Callback by default which uses callbacks for the default behavior. (Or maybe the build system per project could decide the default).

Module.let has the visual affordance of "Library Defined Let".

[hcarty]

I really like the idea of Module.let as it makes things very explicit and, when useful, makes it cleaner and more readable when multiple modules are to mixed - M1.let ...; M2.let ...; ....

A ! suffix on keywords is already used to signify that something is being overridden. It would be nice to make that use consistent, either by choosing something else as the let suffix or changing open! and friends to use a different syntax. That said, it may be nice to save a sugared let! until the language can support it more cleanly with modular implicits or something similar.

[hcarty]

For any syntax it would be nice to support the same for other keywords like switch, try, and if. Lwt's ppx supports this and I think ppx_let does as well.

Lwt.switch promise {
| A => ...
| B => ...
| exception Not_found => ...
}

[cullophid]

why not just use the same api as in js ?

let async f = () => {
let f = await somePromise();
someFunct(f);
}

This would require a standardized promise implementation.

[andreypopp]

I propose adding two syntax constructs which could be described as syntax sugar
for ppx_let and therefore can used with any monadic structures (promises,
option, result, ...).

1. let-do-bindings: let N = do E;
2. do-expressions: do E;

Syntax

Let-do-bindings

Reason:

let N = do E;
M

OCaml:

let%bind N = E in M

OCaml (after preprocessing with ppx_let):

Let_syntax.bind ~f:(fun N -> M) E

Let-do-bindings are used when you need to exract a value form a monad and define
some new computations with it. For promises: "wait for a promise to be resolved,
get its value and return a new promise based on it".

Do-expressions

Reason:

do E;
M

OCaml:

let%bind () = E in M

OCaml (after preprocessing with ppx_let):

Let_syntax.bind ~f:(fun () -> M) E

Do-expressions are used when you need to perform a series of side-effectful
computations for which result values are not defined (they are represented as
unit () and so you care only about the side-effects). For promises: "perform a
series of async actions on a filesystem: copy, move files".

Using syntax with concrete monads

ppx_let desugars let%bind into Let_syntax.bind calls. That means
Let_syntax module should be in scope when using the proposed syntax
constructs.

I like how we can use local module open to bring the needed Let_syntax into
scope:

module Promise = {
  module Let_syntax = {
    let bind = (~f, promise) =>
      Js.Promise.then_(f, promise);
  };
};

let getJSON = (url) => Promise.({
  do AsyncLogService.debug("getting data...");
  let resp = do fetch(url);
  let data = do resp.json();
  Yojson.Safe.parse(data);
});

Parallel bindings

let_ppx supports "parallel"-bindings:

let%bind x = getJSON("http://example.com/x.json")
     and y = getJSON("http://example.com/y.json")
in ...

This maps on let-do-bindings:

let x = do getJSON("http://example.com/x.json")
and y = do getJSON("http://example.com/y.json");
...

The downside (?) is that do keyword is required for each of the bindings in a
group. At the same time I don't think that allowing to omit it for and N
bindings is a good idea — it will bring less clarity.

Note that ppx_let implements parallel bindings in terms of Let_syntax.both
function. You can read more in its README.

Adding an implicit return value for the last do-expression

Currently in a chain of do-expression, the last value must be some monadic
value. For example, with promises:

let copydir = (from, to) => Promise.({
  do Fs.copy(from, to);
  Js.Promise.resolve(());
})

We can add such return value implicitly in terms of the current Let_syntax in
scope.

ppx_let doesn't have this feature so I'd describe it mapping from Reason to
preprocessed OCaml code directly.

Note the refined definition of Promise.Let_syntax which includes return
function to wrap any value into a promise.

Reason:

module Promise = {
  module Let_syntax = {
    let bind = ... as defined before ...

    let return = (v) =>
      Js.Promise.resolve(v);
  };
};

let copydir = (from, to) => Promise.({
  do Fs.copy(from, to);
})

OCaml:

let copydir from to = Promise.(
  Let_syntax.bind
    ~f:(fun () -> Let_syntax.return ())
    (Fs.copy fromb to)
)

Async-specific syntax vs. Monadic syntax

Simply, I believe async-specific keywords don't bring much value given that
do-keyword already fits well with async computations (warning: I'm not native
speaker, I might be totally wrong about that).

The advantage is that we can use the proposed syntax with other monads than
async computations:

• option - computations which may result in values which are absent
• result - computations which may result in errors
• ... other custom monads

The async computations use case already requires us to provide different
"bindings" to the syntax. We want to define async computations at least with 3
different implementations:

• Js.Promise
• lwt lib
• async lib

ppx_let approach solves that elegantly with Let_syntax-convention.

[jordwalke]

Good discussion. Here's some thoughts/questions:

• When would the value automatically be wrapped in return and when would it not? How would you opt out of that behavior?
• The word do does imply side effect imho. It might not be the right word for things like async/option monads.
• It's nice how do is one keyword whose position alternates its meaning between bind and imperative bind.

[andreypopp]

When would the value automatically be wrapped in return and when would it not? How would you opt out of that behavior?

The only case, I can think of, which is useful to introduce an implicit return is when do E; is the last expression:

{
  do E;
}

In that case instead of parsing it as:

let%bind () = E in ()

we parse it as:

let%bind () = E in [%return ()]

which is then transformed into:

Let_syntax.bind ~f(fun () -> Let_syntax.return ()) E

To opt-out — simply add a desired expression after the do E;:

{
  do E;
  return(result);
}

Now the question is how to refer to the current return, the obvious answer is via Let_syntax.return which is in scope already (for ppx_let). That might be noisy so we potentially could extend the Let_syntax convention to have a module Let_syntax.Open_in_body which we will be opened in body of the monadic expression:

let N = do E;
M

will be transformed into:

Let_syntax.bind
  ~f(fun N -> let open Let_syntax.Open_in_body in M)
  E

Similar thing for do-expressions.

The complete example would look like:

module Promise = {
  module Let_syntax = {
    let bind = (~f, promise) =>
      Js.Promise.then_(f, promise);

    module Open_in_body = {
      let return = (v) =>
        Js.Promise.resolve(v)
    }
  };
};

let getJSON = (url) => Promise.({
  let resp = do fetch(url);
  let data = do resp.json();
  return(Yojson.Safe.parse(data));
});

The downside is that return will appear magically after the let-do or do syntax. It won't be possible to write:

let simple = (v) => Promise.({
  return(v);
})

Maybe instead we should ask to put return into Promise directly?

(btw. In my prev comment I forgot to wrap the last expression of getJSON into Promise so that example fixes it)

The word do does imply side effect imho. It might not be the right word for things like async/option monads.

I think do is generic enough:

• option — do means "do try to unwrap"
• async — do means "do wait for resolution"

Now you can say that such "genericness" is not a good thing. But now I think we should've been arguing whether we need a generic monadic syntax or async-centered syntax.

Anyway the separate keyword do (or another) on RHS looks to me like it's a better solution than a monadiclet keyword (let! or lat or let%bind) on LHS:

• monadic let keyword on LHS maybe seen as it introduces bindings in some different way (related to scope), while having a monadic keyword on RHS hints that it is about binding to a value, not the nature of binding a name.

• I don't like punctuation in let! or let%bind. lat is ok but it suites async use case only, I think.

I think it would be easy to explain do for async computations for people with JS background: "it's like await but called do". Then later we can introduce the generic nature of do for other monads (option, result, ...).

The syntax with local module open already looks very similar to async/await functions in JS:

let getJSON = (url) => Promise.({
  let resp = do fetch(url);
  let data = do resp.json();
  return(Yojson.Safe.parse(data));
});

If we can somehow make local module open syntax "less noisy", then it gets even better:

let getJSON = (url) => Promise.{
  let resp = do fetch(url);
  let data = do resp.json();
  return(Yojson.Safe.parse(data));
};