<p>Effect systems allowing locally introducing and eliminating effects have started to appear in (more) practical systems recently, but they've always had a reputation as being a major challenge to implement, especially to support functions that are generic passing through effects or support layered effects, and to be very intrusive to the rest of the language implementation.</p>
<p>There are a few obvious high-level approaches (add a hidden parameter for every effect, global variables pointing to handlers, universal specialisation, ...) and I know there have been more sophisticated proposals too, at least theoretically. If I'm building an interpreter or compiler for a language with this sort of effect system in it today, what is the state of the art <em>in practice</em> for going about it? That is, which approach(es) have been found effective in the real implementations that have employed them, beyond toy proofs of concept?</p>
<p>I'm not so much looking at absolute maximum performance here, which I guess is always going to be heavy specialisation in a very optimising compiler, but which <em>high-level</em> approaches to implementation have been found effective and unobstructive. If any have been ruled out and replaced by something better, that's useful information too.</p>

<implementation><effect-systems>
What's the state of the art for implementing effect handler systems?
CC BY-SA 4.0
2024-02-20T20:49:41.870Z
For what I know the monad/kaloski arrows the only sensible manner to represent IO in a pure language. With the Haskell's predecessor to monads the dialog system being awful and the generalized exceptions (being exceptions an antipattern in itself, generalized exceptions cannot be a good thing)?, I surely missed some, but I cannot think how could be better than monads, my only idea is to make the language to have the a convenient syntax  for monads.
Delfin
CC BY-SA 4.0
2024-02-20T21:27:40.437Z
@Delfin A few things: (1) I think you mean "Kleisli arrows". (2) A convenient syntax for monads has existed for a long time in the form of Haskell's `do` notation. (3) There are definitely reasonable alternatives to monads for handling IO in a pure language, going back a number of years. These include uniqueness types (see Mercury and Clean) and algebraic effects (see Eff and Koka).
David Young
CC BY-SA 4.0
2024-02-20T21:53:40.250Z
@DavidYoung (1) Do notation is 100% an anti-pattern and should never be used, an actual convenient syntax for monad may be developed. (2) 'Algebraic Effects' are 100% not an reasonable way to handle handling IO, with this being no more than an generalization of an anti pattern called exceptions, thus they should never be used like Exceptions should never be used. (3) Not very fan of uniqueness types, I think that Monads being the perfect abstraction for IO given their equivalence with the monoid of all instructions. (4) Do you know of another challengers to the monad?
Delfin
CC BY-SA 4.0
2024-02-20T22:15:45.493Z
I'd just note here that this question *never mentions IO* and is asking about general effect handler systems. You can special-case a single IO effect however you want, but that's not what is being asked here.
Michael Homer
CC BY-SA 4.0
2024-02-20T22:17:00.287Z
@Delfin As someone who has used Haskell on a regular basis for many years, I strongly disagree with (1). Further, I think the idea that `do` is an anti-pattern is *very* unpopular in the Haskell community. It is widely used by me and most others. However, it is of course an opinion someone can hold. There could be a better way to do it as well. But I think `do` is fine, personally. Regarding (2), I am not an expert on algebraic effects, but I believe they are better seen as a generalization of monads rather than of exceptions. For (4), I would suggest you take another look at algebraic effects
David Young
CC BY-SA 4.0
2024-02-20T22:21:08.940Z
@MichaelHomer I don't understand the difference
Delfin
CC BY-SA 4.0
2024-02-20T22:25:57.090Z
@DavidYoung Link to the paper on Algebraic Effects, please.
Delfin
CC BY-SA 4.0
2024-02-20T22:31:53.633Z
@Delfin Here's some examples of what you can do with algebraic effects: https://effekt-lang.org/examples. I don't think any of them involve the kind of stuff Haskell's IO monad does
user
CC BY-SA 4.0
2024-02-20T22:32:40.100Z
@Delfin There have been many papers on algebraic effects, including by extremely accomplished computer scientists such as Gordon Plotkin and Andrej Bauer. An early paper by Plotkin & Power from ~25 years ago is [this one](https://homepages.inf.ed.ac.uk/gdp/publications/Op_Sem_Comp_Lam.pdf). [Here's](https://arxiv.org/abs/1807.05923) a more recent one by Bauer. Also, [here's](https://www.youtube.com/watch?v=atYp386EGo8) a couple lectures on them by Bauer. Probably best to start with the talk. He also goes into the history at the start.
David Young
CC BY-SA 4.0
2024-02-20T22:38:17.097Z
Conventional examples of effects you could use include failure, mutable state, value propagation, exceptions, and, yes, IO. Effect handlers define how to respond to those effects locally, and could vary at different points in the program. A fairly accessible non-IO example is given in [this article about writing "abilities" for Unison](https://www.unison-lang.org/docs/fundamentals/abilities/writing-abilities/), which uses the example of a locally-created mutable key-value store and does not include IO anywhere.
Michael Homer
CC BY-SA 4.0
2024-02-20T22:40:54.707Z
@MichaelHomer also generators, `async`, default values, and a whole host of other things
Seggan
CC BY-SA 4.0
2024-02-20T23:01:04.117Z
@Seggan I am sure that generators and async are isomorphic and they are unacceptable because: https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/
Delfin
CC BY-SA 4.0
2024-02-20T23:57:55.037Z
**Comments have been [moved to chat](https://chat.stackexchange.com/rooms/151581/discussion-on-question-by-michael-homer-whats-the-state-of-the-art-for-implemen); please do not continue the discussion here.** Before posting a comment below this one, please review the [purposes of comments](/help/privileges/comment). Comments that do not request clarification or suggest improvements usually belong as an [answer](/help/how-to-answer), on [meta], or in [chat]. Comments continuing discussion may be removed.
lyxal
CC BY-SA 4.0
2024-02-21T16:35:21.080Z
Too short for an answer but you might want to take a look at [this](https://effekt-lang.org/docs/implementation/lift-inference)
Seggan
<p>Leo White is developing an <a href="https://www.janestreet.com/tech-talks/effective-programming/" rel="nofollow noreferrer">effect system for OCaml 5.0</a>, that might be state-of-the-art in terms of efficiency.</p>

CC BY-SA 4.0
2024-02-28T07:19:03.453Z
2024-02-28T07:19:03.453Z
Andrej Bauer
CC BY-SA 4.0
2024-03-05T11:20:01.730Z
To clarify, Leo is working on a new effect typing system but OCaml 5.0 is already released and has effect handlers but where the typing is analogous to unchecked exceptions in Java.
Max New
# What's the state of the art for implementing effect handler systems?
Posted by: username_0 on 2024-02-20
Effect systems allowing locally introducing and eliminating effects have started to appear in (more) practical systems recently, but they've always had a reputation as being a major challenge to implement, especially to support functions that are generic passing through effects or support layered effects, and to be very intrusive to the rest of the language implementation.

There are a few obvious high-level approaches (add a hidden parameter for every effect, global variables pointing to handlers, universal specialisation, ...) and I know there have been more sophisticated proposals too, at least theoretically. If I'm building an interpreter or compiler for a language with this sort of effect system in it today, what is the state of the art *in practice* for going about it? That is, which approach(es) have been found effective in the real implementations that have employed them, beyond toy proofs of concept?

I'm not so much looking at absolute maximum performance here, which I guess is always going to be heavy specialisation in a very optimising compiler, but which *high-level* approaches to implementation have been found effective and unobstructive. If any have been ruled out and replaced by something better, that's useful information too.
**Tags:** implementation effect-systems
> **Comments:**
> **username_1**: For what I know the monad/kaloski arrows the only sensible manner to represent IO in a pure language. With the Haskell's predecessor to monads the dialog system being awful and the generalized exceptions (being exceptions an antipattern in itself, generalized exceptions cannot be a good thing)?, I surely missed some, but I cannot think how could be better than monads, my only idea is to make the language to have the a convenient syntax  for monads.  
> **username_2**: @Delfin A few things: (1) I think you mean "Kleisli arrows". (2) A convenient syntax for monads has existed for a long time in the form of Haskell's `do` notation. (3) There are definitely reasonable alternatives to monads for handling IO in a pure language, going back a number of years. These include uniqueness types (see Mercury and Clean) and algebraic effects (see Eff and Koka).  
> **username_1**: @DavidYoung (1) Do notation is 100% an anti-pattern and should never be used, an actual convenient syntax for monad may be developed. (2) 'Algebraic Effects' are 100% not an reasonable way to handle handling IO, with this being no more than an generalization of an anti pattern called exceptions, thus they should never be used like Exceptions should never be used. (3) Not very fan of uniqueness types, I think that Monads being the perfect abstraction for IO given their equivalence with the monoid of all instructions. (4) Do you know of another challengers to the monad?  
> **username_0**: I'd just note here that this question *never mentions IO* and is asking about general effect handler systems. You can special-case a single IO effect however you want, but that's not what is being asked here.  
> **username_2**: @Delfin As someone who has used Haskell on a regular basis for many years, I strongly disagree with (1). Further, I think the idea that `do` is an anti-pattern is *very* unpopular in the Haskell community. It is widely used by me and most others. However, it is of course an opinion someone can hold. There could be a better way to do it as well. But I think `do` is fine, personally. Regarding (2), I am not an expert on algebraic effects, but I believe they are better seen as a generalization of monads rather than of exceptions. For (4), I would suggest you take another look at algebraic effects  
> **username_1**: @MichaelHomer I don't understand the difference  
> **username_1**: @DavidYoung Link to the paper on Algebraic Effects, please.  
> **username_3**: @Delfin Here's some examples of what you can do with algebraic effects: https://effekt-lang.org/examples. I don't think any of them involve the kind of stuff Haskell's IO monad does  
> **username_2**: @Delfin There have been many papers on algebraic effects, including by extremely accomplished computer scientists such as Gordon Plotkin and Andrej Bauer. An early paper by Plotkin & Power from ~25 years ago is [this one](https://homepages.inf.ed.ac.uk/gdp/publications/Op_Sem_Comp_Lam.pdf). [Here's](https://arxiv.org/abs/1807.05923) a more recent one by Bauer. Also, [here's](https://www.youtube.com/watch?v=atYp386EGo8) a couple lectures on them by Bauer. Probably best to start with the talk. He also goes into the history at the start.  
> **username_0**: Conventional examples of effects you could use include failure, mutable state, value propagation, exceptions, and, yes, IO. Effect handlers define how to respond to those effects locally, and could vary at different points in the program. A fairly accessible non-IO example is given in [this article about writing "abilities" for Unison](https://www.unison-lang.org/docs/fundamentals/abilities/writing-abilities/), which uses the example of a locally-created mutable key-value store and does not include IO anywhere.  
> **username_4**: @MichaelHomer also generators, `async`, default values, and a whole host of other things  
> **username_1**: @Seggan I am sure that generators and async are isomorphic and they are unacceptable because: https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/  
> **username_5**: **Comments have been [moved to chat](https://chat.stackexchange.com/rooms/151581/discussion-on-question-by-michael-homer-whats-the-state-of-the-art-for-implemen); please do not continue the discussion here.** Before posting a comment below this one, please review the [purposes of comments](/help/privileges/comment). Comments that do not request clarification or suggest improvements usually belong as an [answer](/help/how-to-answer), on [meta], or in [chat]. Comments continuing discussion may be removed.  
> **username_4**: Too short for an answer but you might want to take a look at [this](https://effekt-lang.org/docs/implementation/lift-inference)  
---
## 1 answers
### Answer by username_6 on 2024-02-28. Score: 1
Leo White is developing an [effect system for OCaml 5.0](https://www.janestreet.com/tech-talks/effective-programming/), that might be state-of-the-art in terms of efficiency.
> **Comments:**
> **username_7** (2024-03-05): To clarify, Leo is working on a new effect typing system but OCaml 5.0 is already released and has effect handlers but where the typing is analogous to unchecked exceptions in Java.  
---
