Congratulations!

[Valid RSS] This is a valid RSS feed.

Recommendations

This feed is valid, but interoperability with the widest range of feed readers could be improved by implementing the following recommendations.

Source: http://www.joachim-breitner.de/blog/feeds/categories/1-English.rss

  1. <?xml version="1.0" encoding="UTF-8"?>
  2. <rss xmlns:bib="http://joachim-breitner.de/2004/Website/Bibliography"
  3.     xmlns:content="http://purl.org/rss/1.0/modules/content/"
  4.     xmlns:atom="http://www.w3.org/2005/Atom"
  5.     version="2.0">
  6.   <channel>
  7.      <title>nomeata’s mind shares</title>
  8.      <link>https://www.joachim-breitner.de/blog</link>
  9.      <atom:link rel="self" type="application/rss+xml"
  10.                 href="https://www.joachim-breitner.de/blog_feed.rss"/>
  11.      <description>Joachim Breitners Denkblogade</description>
  12.      <image>
  13.         <url>https://joachim-breitner.de/avatars/avatar_128.png</url>
  14.         <title>nomeata’s mind shares</title>
  15.         <link>https://www.joachim-breitner.de/blog</link>
  16.         <width>128</width>
  17.         <height>128</height>
  18.      </image>
  19.      <item>
  20.         <title>Extrinsic termination proofs for well-founded recursion in Lean</title>
  21.         <link>https://www.joachim-breitner.de/blog/816-Extrinsic_termination_proofs_for_well-founded_recursion_in_Lean</link>
  22.         <guid>https://www.joachim-breitner.de/blog/816-Extrinsic_termination_proofs_for_well-founded_recursion_in_Lean</guid>
  23.         <comments>https://www.joachim-breitner.de/blog/816-Extrinsic_termination_proofs_for_well-founded_recursion_in_Lean#comments</comments>
  24.         <author>mail@joachim-breitner.de (Joachim Breitner)</author>
  25.         <description>&lt;p&gt;A few months ago &lt;a href="/blog/813-Blogging_on_Lean"&gt;I explained&lt;/a&gt; that one reason why this blog has become more quiet is that all my work on Lean is covered elsewhere.&lt;/p&gt;
  26. &lt;p&gt;This post is an exception, because it is an observation that is (arguably) interesting, but does not lead anywhere, so where else to put it than my own blog…&lt;/p&gt;
  27. &lt;p&gt;Want to share your thoughts about this? Please &lt;a href="https://leanprover.zulipchat.com/#narrow/channel/270676-lean4/topic/Extrinsic.20termination.20proofs.20for.20well-founded.20recursion"&gt;join the discussion on the Lean community zulip&lt;/a&gt;!&lt;/p&gt;
  28. &lt;h3 id="background"&gt;Background&lt;/h3&gt;
  29. &lt;p&gt;When defining a function recursively in Lean that has &lt;em&gt;nested recursion&lt;/em&gt;, e.g. a recusive call that is in the argument to a higher-order function like &lt;code&gt;List.map&lt;/code&gt;, then extra attention used to be necessary so that Lean can see that &lt;code&gt;xs.map&lt;/code&gt; applies its argument only elements of the list &lt;code&gt;xs&lt;/code&gt;. The usual idiom is to write &lt;code&gt;xs.attach.map&lt;/code&gt; instead, where &lt;a href="https://leanprover-community.github.io/mathlib4_docs/Init/Data/List/Attach.html#List.attach"&gt;&lt;code&gt;List.attach&lt;/code&gt;&lt;/a&gt; attaches to the list elements a proof that they are in that list. You can read more about this my &lt;a href="https://lean-lang.org/blog/2024-1-11-recursive-definitions-in-lean/"&gt;Lean blog post on recursive definitions&lt;/a&gt; and our &lt;a href="https://lean-lang.org/doc/reference/latest/Recursive-Definitions/Well-Founded-Recursion/#well-founded-recursion"&gt;new shiny reference manual&lt;/a&gt;, look for Example “Nested Recursion in Higher-order Functions”.&lt;/p&gt;
  30. &lt;p&gt;To make this step less tedious I taught Lean to automatically rewrite &lt;code&gt;xs.map&lt;/code&gt; to &lt;code&gt;xs.attach.map&lt;/code&gt; (where suitable) within the construction of well-founded recursion, so that nested recursion just works (&lt;a href="https://github.com/leanprover/lean4/issues/5471"&gt;issue #5471&lt;/a&gt;). We already do such a rewriting to change &lt;code&gt;if c then … else …&lt;/code&gt; to the dependent &lt;code&gt;if h : c then … else …&lt;/code&gt;, but the attach-introduction is much more ambitious (the rewrites are not definitionally equal, there are higher-order arguments etc.) Rewriting the terms in a way that we can still prove the connection later when creating the equational lemmas is hairy at best. Also, we want the whole machinery to be extensible by the user, setting up their own higher order functions to add more facts to the context of the termination proof.&lt;/p&gt;
  31. &lt;p&gt;I implemented it like this (&lt;a href="https://github.com/leanprover/lean4/pull/6744"&gt;PR #6744&lt;/a&gt;) and it ships with 4.18.0, but in the course of this work I thought about a quite different and maybe better™ way to do this, and well-founded recursion in general:&lt;/p&gt;
  32. &lt;h3 id="a-simpler-fix"&gt;A simpler &lt;code&gt;fix&lt;/code&gt;&lt;/h3&gt;
  33. &lt;p&gt;Recall that to use &lt;a href="https://leanprover-community.github.io/mathlib4_docs/Init/WF.html#WellFounded.fix"&gt;WellFounded.fix&lt;/a&gt;&lt;/p&gt;
  34. &lt;pre class="lean"&gt;&lt;code&gt;WellFounded.fix : (hwf : WellFounded r) (F : (x : α) → ((y : α) → r y x → C y) → C x) (x : α) : C x&lt;/code&gt;&lt;/pre&gt;
  35. &lt;p&gt;we have to rewrite the functorial of the recursive function, which naturally has type&lt;/p&gt;
  36. &lt;pre class="lean"&gt;&lt;code&gt;F : ((y : α) →  C y) → ((x : α) → C x)&lt;/code&gt;&lt;/pre&gt;
  37. &lt;p&gt;to the one above, where all recursive calls take the termination proof &lt;code&gt;r y x&lt;/code&gt;. This is a fairly hairy operation, mangling the type of matcher’s motives and whatnot.&lt;/p&gt;
  38. &lt;p&gt;Things are simpler for recursive definitions using &lt;a href="https://lean-lang.org/doc/reference/latest/Recursive-Definitions/Partial-Fixpoint-Recursion/#partial-fixpoint"&gt;the new &lt;code&gt;partial_fixpoint&lt;/code&gt; machinery&lt;/a&gt;, where we use &lt;a href="https://leanprover-community.github.io/mathlib4_docs/Init/Internal/Order/Basic.html#Lean.Order.fix"&gt;Lean.Order.fix&lt;/a&gt;&lt;/p&gt;
  39. &lt;pre class="lean"&gt;&lt;code&gt;Lean.Order.fix : [CCPO α] (F : β → β) (hmono : monotone F) : β&lt;/code&gt;&lt;/pre&gt;
  40. &lt;p&gt;so the functorial’s type is unmodified (here &lt;code&gt;β&lt;/code&gt; will be &lt;code&gt;((x : α) → C x)&lt;/code&gt;), and everything else is in the propositional side-condition &lt;code&gt;montone F&lt;/code&gt;. For this predicate we have a syntax-guided compositional tactic, and it’s easily extensible, e.g. by&lt;/p&gt;
  41. &lt;pre class="lean"&gt;&lt;code&gt;theorem monotone_mapM (f : γ → α → m β) (xs : List α) (hmono : monotone f) :
  42.    monotone (fun x =&gt; xs.mapM (f x)) &lt;/code&gt;&lt;/pre&gt;
  43. &lt;p&gt;Once given, we don’t care about the content of that proof. In particular proving the unfolding theorem only deals with the unmodified &lt;code&gt;F&lt;/code&gt; that closely matches the function definition as written by the user. Much simpler!&lt;/p&gt;
  44. &lt;h3 id="isabelle-has-it-easier"&gt;Isabelle has it easier&lt;/h3&gt;
  45. &lt;p&gt;&lt;a href="https://isabelle.in.tum.de/"&gt;Isabelle&lt;/a&gt; also supports well-founded recursion, and has great support for nested recursion. And it’s much simpler!&lt;/p&gt;
  46. &lt;p&gt;There, all you have to do to make nested recursion work is to define a congruence lemma of the form, for &lt;code&gt;List.map&lt;/code&gt; something like our &lt;a href="https://leanprover-community.github.io/mathlib4_docs/Init/Data/List/Lemmas.html#List.map_congr_left"&gt;List.map_congr_left&lt;/a&gt;&lt;/p&gt;
  47. &lt;pre class="lean"&gt;&lt;code&gt;List.map_congr_left : (h : ∀ a ∈ l, f a = g a) :
  48.    List.map f l = List.map g l&lt;/code&gt;&lt;/pre&gt;
  49. &lt;p&gt;This is because in Isabelle, too, the termination proofs is a side-condition that essentially states “the functorial &lt;code&gt;F&lt;/code&gt; calls its argument &lt;code&gt;f&lt;/code&gt; only on smaller arguments”.&lt;/p&gt;
  50. &lt;h3 id="can-we-have-it-easy-too"&gt;Can we have it easy, too?&lt;/h3&gt;
  51. &lt;p&gt;I had wished we could do the same in Lean for a while, but that form of congruence lemma just isn’t strong enough for us.&lt;/p&gt;
  52. &lt;p&gt;But maybe there is a way to do it, using an existential to give a witness that &lt;code&gt;F&lt;/code&gt; can alternatively implemented using the more restrictive argument. The following &lt;code&gt;callsOn P F&lt;/code&gt; predicate can express that &lt;code&gt;F&lt;/code&gt; calls its higher-order argument only on arguments that satisfy the predicate &lt;code&gt;P&lt;/code&gt;:&lt;/p&gt;
  53. &lt;pre class="lean"&gt;&lt;code&gt;section setup
  54.  
  55. variable {α : Sort u}
  56. variable {β : α → Sort v}
  57. variable {γ : Sort w}
  58.  
  59. def callsOn (P : α → Prop) (F : (∀ y, β y) → γ) :=
  60.  ∃ (F': (∀ y, P y → β y) → γ), ∀ f, F' (fun y _ =&gt; f y) = F f
  61.  
  62. variable (R : α → α → Prop)
  63. variable (F : (∀ y, β y) → (∀ x, β x))
  64.  
  65. local infix:50 " ≺ " =&gt; R
  66.  
  67. def recursesVia : Prop := ∀ x, callsOn (· ≺ x) (fun f =&gt; F f x)
  68.  
  69. noncomputable def fix (wf : WellFounded R) (h : recursesVia R F) : (∀ x, β x) :=
  70.  wf.fix (fun x =&gt; (h x).choose)
  71.  
  72. def fix_eq (wf : WellFounded R) h x :
  73.    fix R F wf h x = F (fix R F wf h) x := by
  74.  unfold fix
  75.  rw [wf.fix_eq]
  76.  apply (h x).choose_spec&lt;/code&gt;&lt;/pre&gt;
  77. &lt;p&gt;This allows nice compositional lemmas to discharge &lt;code&gt;callsOn&lt;/code&gt; predicates:&lt;/p&gt;
  78. &lt;pre class="lean"&gt;&lt;code&gt;theorem callsOn_base (y : α) (hy : P y) :
  79.    callsOn P (fun (f : ∀ x, β x) =&gt; f y) := by
  80.  exists fun f =&gt; f y hy
  81.  intros; rfl
  82.  
  83. @[simp]
  84. theorem callsOn_const (x : γ) :
  85.    callsOn P (fun (_ : ∀ x, β x) =&gt; x) :=
  86.  ⟨fun _ =&gt; x, fun _ =&gt; rfl⟩
  87.  
  88. theorem callsOn_app
  89.    {γ₁ : Sort uu} {γ₂ : Sort ww}
  90.    (F₁ :  (∀ y, β y) → γ₂ → γ₁) -- can this also support dependent types?
  91.    (F₂ :  (∀ y, β y) → γ₂)
  92.    (h₁ : callsOn P F₁)
  93.    (h₂ : callsOn P F₂) :
  94.    callsOn P (fun f =&gt; F₁ f (F₂ f)) := by
  95.  obtain ⟨F₁', h₁⟩ := h₁
  96.  obtain ⟨F₂', h₂⟩ := h₂
  97.  exists (fun f =&gt; F₁' f (F₂' f))
  98.  intros; simp_all
  99.  
  100. theorem callsOn_lam
  101.    {γ₁ : Sort uu}
  102.    (F : γ₁ → (∀ y, β y) → γ) -- can this also support dependent types?
  103.    (h : ∀ x, callsOn P (F x)) :
  104.    callsOn P (fun f x =&gt; F x f) := by
  105.  exists (fun f x =&gt; (h x).choose f)
  106.  intro f
  107.  ext x
  108.  apply (h x).choose_spec
  109.  
  110. theorem callsOn_app2
  111.    {γ₁ : Sort uu} {γ₂ : Sort ww}
  112.    (g : γ₁ → γ₂ → γ)
  113.    (F₁ :  (∀ y, β y) → γ₁) -- can this also support dependent types?
  114.    (F₂ :  (∀ y, β y) → γ₂)
  115.    (h₁ : callsOn P F₁)
  116.    (h₂ : callsOn P F₂) :
  117.    callsOn P (fun f =&gt; g (F₁ f) (F₂ f)) := by
  118.  apply_rules [callsOn_app, callsOn_const]&lt;/code&gt;&lt;/pre&gt;
  119. &lt;p&gt;With this setup, we can have the following, possibly user-defined, lemma expressing that &lt;code&gt;List.map&lt;/code&gt; calls its arguments only on elements of the list:&lt;/p&gt;
  120. &lt;pre class="lean"&gt;&lt;code&gt;theorem callsOn_map (δ : Type uu) (γ : Type ww)
  121.    (P : α → Prop) (F : (∀ y, β y) → δ → γ) (xs : List δ)
  122.    (h : ∀ x, x ∈ xs → callsOn P (fun f =&gt; F f x)) :
  123.    callsOn P (fun f =&gt; xs.map (fun x =&gt; F f x)) := by
  124.  suffices callsOn P (fun f =&gt; xs.attach.map (fun ⟨x, h⟩ =&gt; F f x)) by
  125.    simpa
  126.  apply callsOn_app
  127.  · apply callsOn_app
  128.    · apply callsOn_const
  129.    · apply callsOn_lam
  130.      intro ⟨x', hx'⟩
  131.      dsimp
  132.      exact (h x' hx')
  133.  · apply callsOn_const
  134.  
  135. end setup&lt;/code&gt;&lt;/pre&gt;
  136. &lt;p&gt;So here is the (manual) construction of a nested &lt;code&gt;map&lt;/code&gt; for trees:&lt;/p&gt;
  137. &lt;pre class="lean"&gt;&lt;code&gt;section examples
  138.  
  139. structure Tree (α : Type u) where
  140.  val : α
  141.  cs : List (Tree α)
  142.  
  143. -- essentially
  144. -- def Tree.map (f : α → β) : Tree α → Tree β :=
  145. --   fun t =&gt; ⟨f t.val, t.cs.map Tree.map⟩)
  146. noncomputable def Tree.map (f : α → β) : Tree α → Tree β :=
  147.  fix (sizeOf · &amp;lt; sizeOf ·) (fun map t =&gt; ⟨f t.val, t.cs.map map⟩)
  148.    (InvImage.wf (sizeOf ·) WellFoundedRelation.wf) &amp;lt;| by
  149.  intro ⟨v, cs⟩
  150.  dsimp only
  151.  apply callsOn_app2
  152.  · apply callsOn_const
  153.  · apply callsOn_map
  154.    intro t' ht'
  155.    apply callsOn_base
  156.    -- ht' : t' ∈ cs -- !
  157.    -- ⊢ sizeOf t' &amp;lt; sizeOf { val := v, cs := cs }
  158.    decreasing_trivial
  159.  
  160. end examples&lt;/code&gt;&lt;/pre&gt;
  161. &lt;p&gt;This makes me happy!&lt;/p&gt;
  162. &lt;p&gt;All details of the construction are now contained in a proof that can proceed by a syntax-driven tactic and that’s easily and (likely robustly) extensible by the user. It also means that we can share a lot of code paths (e.g. everything related to equational theorems) between well-founded recursion and &lt;code&gt;partial_fixpoint&lt;/code&gt;.&lt;/p&gt;
  163. &lt;p&gt;I wonder if this construction is really as powerful as our current one, or if there are certain (likely dependently typed) functions where this doesn’t fit, but the &lt;code&gt;β&lt;/code&gt; above is dependent, so it looks good.&lt;/p&gt;
  164. &lt;p&gt;With this construction, functions defined by well-founded recursion will reduce even worse in the kernel, I assume. &lt;a href="https://github.com/leanprover/lean4/issues/5192"&gt;This may be a good thing&lt;/a&gt;.&lt;/p&gt;
  165. &lt;h3 id="the-cake-is-a-lie"&gt;The cake is a lie&lt;/h3&gt;
  166. &lt;p&gt;What unfortunately kills this idea, though, is the generation of the &lt;a href="https://lean-lang.org/blog/2024-5-17-functional-induction/"&gt;functional induction principles&lt;/a&gt;, which I believe is not (easily) possible with this construction: The functional induction principle is proved by massaging &lt;code&gt;F&lt;/code&gt; to return a proof, but since the extra assumptions (e.g. for &lt;code&gt;ite&lt;/code&gt; or &lt;code&gt;List.map&lt;/code&gt;) only exist in the termination proof, they are not available in &lt;code&gt;F&lt;/code&gt;.&lt;/p&gt;
  167. &lt;p&gt;Oh wey, how anticlimactic.&lt;/p&gt;
  168. &lt;h3 id="ps-path-dependencies"&gt;PS: Path dependencies&lt;/h3&gt;
  169. &lt;p&gt;Curiously, if we didn’t have functional induction at this point yet, then very likely I’d change Lean to use this construction, and then we’d either not get functional induction, or it would be implemented very differently, maybe a more syntactic approach that would re-prove termination. I guess that’s called path dependence.&lt;/p&gt;</description>
  170.         <pubDate>Mon, 10 Mar 2025 18:47:59 +0100</pubDate>
  171.      </item>
  172.      <item>
  173.         <title>Coding on my eInk Tablet</title>
  174.         <link>https://www.joachim-breitner.de/blog/815-Coding_on_my_eInk_Tablet</link>
  175.         <guid>https://www.joachim-breitner.de/blog/815-Coding_on_my_eInk_Tablet</guid>
  176.         <comments>https://www.joachim-breitner.de/blog/815-Coding_on_my_eInk_Tablet#comments</comments>
  177.         <author>mail@joachim-breitner.de (Joachim Breitner)</author>
  178.         <description>&lt;p&gt;For many years I wished I had a setup that would allow me to work (that is, code) productively outside in the bright sun. It’s winter right now, but when its summer again it’s always a bit. this weekend I got closer to that goal.&lt;/p&gt;
  179. &lt;p&gt;TL;DR: Using &lt;a href="https://github.com/coder/code-server"&gt;&lt;code&gt;code-server&lt;/code&gt;&lt;/a&gt; on a beefy machine seems to be quite neat.&lt;/p&gt;
  180. &lt;figure&gt;
  181. &lt;img src="//www.joachim-breitner.de/various/code-server-tablet.jpg" alt="Passively lit coding"/&gt;
  182. &lt;figcaption aria-hidden="true"&gt;Passively lit coding&lt;/figcaption&gt;
  183. &lt;/figure&gt;
  184. &lt;h3 id="personal-history"&gt;Personal history&lt;/h3&gt;
  185. &lt;p&gt;Looking back at my own old blog entries I find one from 10 years ago describing how I bought a &lt;a href="https://www.joachim-breitner.de/blog/660-Using_my_Kobo_eBook_reader_as_an_external_eInk_monitor"&gt;Kobo eBook reader&lt;/a&gt; with the intent of using it as an external monitor for my laptop. It seems that I got a proof-of-concept setup working, using VNC, but it was tedious to set up, and I never actually used that. I subsequently noticed that the eBook reader is rather useful to read eBooks, and it has been in heavy use for that every since.&lt;/p&gt;
  186. &lt;p&gt;Four years ago I gave this old idea another shot and bought an &lt;a href="https://onyxboox.com/boox_maxlumi"&gt;Onyx BOOX Max Lumi&lt;/a&gt;. This is an A4-sized tablet running Android and had the very promising feature of an HDMI &lt;em&gt;input&lt;/em&gt;. So hopefully I’d attach it to my laptop and it just works™. Turns out that this never worked as well as I hoped: Even if I set the resolution to exactly the tablet’s screen’s resolution I got blurry output, and it also drained the battery a lot, so I gave up on this. I subsequently noticed that the tablet is rather useful to take notes, and it has been in sporadic use for that.&lt;/p&gt;
  187. &lt;p&gt;Going off on this tangent: I later learned that the HDMI input of this device appears to the system like a camera input, and I don’t have to use Boox’s “monitor” app but could other apps like &lt;a href="https://f-droid.org/de/packages/troop.com.freedcam/"&gt;FreeDCam&lt;/a&gt; as well. This somehow managed to fix the resolution issues, but the setup still wasn’t as convenient to be used regularly.&lt;/p&gt;
  188. &lt;p&gt;I also played around with pure terminal approaches, e.g. SSH’ing into a system, but since my usual workflow was never purely text-based (I was at least used to using a window manager instead of a terminal multiplexer like &lt;a href="https://www.gnu.org/software/screen/"&gt;&lt;code&gt;screen&lt;/code&gt;&lt;/a&gt; or &lt;a href="https://github.com/tmux/tmux/wiki"&gt;&lt;code&gt;tmux&lt;/code&gt;&lt;/a&gt;) that never led anywhere either.&lt;/p&gt;
  189. &lt;h3 id="vscode-working-remotely"&gt;VSCode, working remotely&lt;/h3&gt;
  190. &lt;p&gt;Since these attempts I have started a &lt;a href="https://www.joachim-breitner.de/blog/809-Joining_the_Lean_FRO"&gt;new job working on the Lean theorem prover&lt;/a&gt;, and working on or with Lean basically means using VSCode. (There is a &lt;a href="https://github.com/Julian/lean.nvim/"&gt;very good neovim plugin as well&lt;/a&gt;, but I’m using VSCode nevertheless, if only to make sure I am dogfooding our default user experience).&lt;/p&gt;
  191. &lt;p&gt;My colleagues have said good things about using VSCode with the remote SSH extension to work on a beefy machine, so I gave this a try now as well, and while it’s not a complete game changer for me, it does make certain tasks (rebuilding everything after a switching branches, running the test suite) very convenient. And it’s a bit spooky to run these work loads without the laptop’s fan spinning up.&lt;/p&gt;
  192. &lt;p&gt;In this setup, the workspace is remote, but VSCode still runs locally. But it made me wonder about my old goal of being able to work reasonably efficient on my eInk tablet. Can I replicate this setup there?&lt;/p&gt;
  193. &lt;p&gt;VSCode itself doesn’t run on Android directly. There are project that run a Linux chroot or in termux on the Android system, and then you can VNC to connect to it (e.g. on &lt;a href="https://andronix-app.gitbook.io/andronix-app/software/ides/vs-code"&gt;Andronix&lt;/a&gt;)… but that did not seem promising. It seemed fiddly, and I probably should take it easy on the tablet’s system.&lt;/p&gt;
  194. &lt;h3 id="code-server-running-remotely"&gt;code-server, running remotely&lt;/h3&gt;
  195. &lt;p&gt;A more promising option is &lt;a href="https://github.com/coder/code-server"&gt;&lt;code&gt;code-server&lt;/code&gt;&lt;/a&gt;. This is a fork of VSCode (actually of VSCodium) that runs completely on the remote machine, and the client machine just needs a browser. I set that up this weekend and found that I was able to do a little bit of work reasonably.&lt;/p&gt;
  196. &lt;h4 id="access"&gt;Access&lt;/h4&gt;
  197. &lt;p&gt;With &lt;code&gt;code-server&lt;/code&gt; one has to decide &lt;a href="https://coder.com/docs/code-server/guide"&gt;how to expose it safely enough&lt;/a&gt;. I decided against the tunnel-over-SSH option, as I expected that to be somewhat tedious to set up (both initially and for each session) on the android system, and I liked the idea of being able to use any device to work in my environment.&lt;/p&gt;
  198. &lt;p&gt;I also decided against the more involved “reverse proxy behind proper hostname with SSL” setups, because they involve a few extra steps, and some of them I cannot do as I do not have root access on the shared beefy machine I wanted to use.&lt;/p&gt;
  199. &lt;p&gt;That left me with the option of using a &lt;code&gt;code-server’&lt;/code&gt;s built-in support for self-signed certificates and a password:&lt;/p&gt;
  200. &lt;pre&gt;&lt;code&gt;$ cat .config/code-server/config.yaml
  201. bind-addr: 1.2.3.4:8080
  202. auth: password
  203. password: xxxxxxxxxxxxxxxxxxxxxxxx
  204. cert: true&lt;/code&gt;&lt;/pre&gt;
  205. &lt;p&gt;With trust-on-first-use this seems reasonably secure.&lt;/p&gt;
  206. &lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt; I noticed that the browsers would forget that I trust this self-signed cert after restarting the browser, and also that I cannot “install” the page (as a &lt;a href="https://de.wikipedia.org/wiki/Progressive_Web_App"&gt;Progressive Web App&lt;/a&gt;) unless it has a valid certificate. But since I don’t have superuser access to that machine, I can’t just follow the official recommendation of using a reverse proxy on port 80 or 431 with automatic certificates. Instead, I pointed a hostname that I control to that machine, obtained a certificate manually on my laptop (using &lt;code&gt;acme.sh&lt;/code&gt;) and copied the files over, so the configuration now reads as follows:&lt;/p&gt;
  207. &lt;pre&gt;&lt;code&gt;bind-addr: 1.2.3.4:3933
  208. auth: password
  209. password: xxxxxxxxxxxxxxxxxxxxxxxx
  210. cert: .acme.sh/foobar.nomeata.de_ecc/foobar.nomeata.de.cer
  211. cert-key: .acme.sh/foobar.nomeata.de_ecc/foobar.nomeata.de.key&lt;/code&gt;&lt;/pre&gt;
  212. &lt;p&gt;(This is getting very specific to my particular needs and constraints, so I’ll spare you the details.)&lt;/p&gt;
  213. &lt;h4 id="service"&gt;Service&lt;/h4&gt;
  214. &lt;p&gt;To keep &lt;code&gt;code-server&lt;/code&gt; running I created a systemd service that’s managed by my user’s systemd instance:&lt;/p&gt;
  215. &lt;pre&gt;&lt;code&gt;~ $ cat ~/.config/systemd/user/code-server.service
  216. [Unit]
  217. Description=code-server
  218. After=network-online.target
  219.  
  220. [Service]
  221. Environment=PATH=/home/joachim/.nix-profile/bin:/nix/var/nix/profiles/default/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
  222. ExecStart=/nix/var/nix/profiles/default/bin/nix run nixpkgs#code-server
  223.  
  224. [Install]
  225. WantedBy=default.target&lt;/code&gt;&lt;/pre&gt;
  226. &lt;p&gt;(I am using &lt;code&gt;nix&lt;/code&gt; as a package manager on a Debian system there, hence the additional &lt;code&gt;PATH&lt;/code&gt; and complex &lt;code&gt;ExecStart&lt;/code&gt;. If you have a more conventional setup then you do not have to worry about &lt;code&gt;Environment&lt;/code&gt; and can likely use &lt;code&gt;ExecStart=code-server&lt;/code&gt;.&lt;/p&gt;
  227. &lt;p&gt;For this to survive me logging out I had to ask the system administrator to run &lt;code&gt;loginctl enable-linger joachim&lt;/code&gt;, so that &lt;a href="https://unix.stackexchange.com/q/396522/20526"&gt;systemd allows my jobs to linger&lt;/a&gt;.&lt;/p&gt;
  228. &lt;h4 id="git-credentials"&gt;Git credentials&lt;/h4&gt;
  229. &lt;p&gt;The next issue to be solved was how to access the git repositories. The work is all on public repositories, but I still need a way to push my work. With the classic VSCode-SSH-remote setup from my laptop, this is no problem: My local SSH key is forwarded using the SSH agent, so I can seamlessly use that on the other side. But with &lt;code&gt;code-server&lt;/code&gt; there is no SSH key involved.&lt;/p&gt;
  230. &lt;p&gt;I could create a new SSH key and store it on the server. That did not seem appealing, though, because SSH keys on Github always have full access. It wouldn’t be horrible, but I still wondered if I can do better.&lt;/p&gt;
  231. &lt;p&gt;I thought of creating &lt;a href="https://github.blog/security/application-security/introducing-fine-grained-personal-access-tokens-for-github/"&gt;fine-grained personal access tokens&lt;/a&gt; that only me to push code to specific repositories, and nothing else, and just store them permanently on the remote server. Still a neat and convenient option, but creating PATs for our org requires approval and I didn’t want to bother anyone on the weekend.&lt;/p&gt;
  232. &lt;p&gt;So I am experimenting with Github’s &lt;a href="https://github.com/git-ecosystem/git-credential-manager"&gt;git-credential-manager&lt;/a&gt; now. I have configured it to use git’s credential cache with an elevated timeout, so that once I log in, I don’t have to again for one workday.&lt;/p&gt;
  233. &lt;pre&gt;&lt;code&gt;$ nix-env -iA nixpkgs.git-credential-manager
  234. $ git-credential-manager configure
  235. $ git config --global credential.credentialStore cache
  236. $ git config --global credential.cacheOptions "--timeout 36000"&lt;/code&gt;&lt;/pre&gt;
  237. &lt;p&gt;To login, I have to &lt;code&gt;https://github.com/login/device&lt;/code&gt; on an authenticated device (e.g. my phone) and enter a 8-character code. Not too shabby in terms of security. I only wish that webpage would not require me to press Tab after each character…&lt;/p&gt;
  238. &lt;p&gt;This still grants rather broad permissions to the &lt;code&gt;code-server&lt;/code&gt;, but at least only temporarily&lt;/p&gt;
  239. &lt;h4 id="android-setup"&gt;Android setup&lt;/h4&gt;
  240. &lt;p&gt;On the client side I could now open &lt;code&gt;https://host.example.com:8080&lt;/code&gt; in Firefox on my eInk Android tablet, click through the warning about self-signed certificates, log in with the fixed password mentioned above, and start working!&lt;/p&gt;
  241. &lt;p&gt;I switched to a theme that supposedly is eInk-optimized (&lt;a href="https://open-vsx.org/extension/Mufanza/e-ink-theme"&gt;eInk by Mufanza&lt;/a&gt;). It’s not perfect (e.g. git diffs are unhelpful because it is not possible to distinguish deleted from added lines), but it’s a start. There are more &lt;a href="https://marketplace.visualstudio.com/items?itemName=eddjrn.e-ink"&gt;eInk themes&lt;/a&gt; on the official Visual Studio Marketplace, but because &lt;code&gt;code-server&lt;/code&gt; is a fork it cannot use that marketplace, and for example this theme &lt;a href="https://gitlab.com/eddjrn/vs-code-e-ink-theme/-/issues/3"&gt;isn’t on Open-VSX&lt;/a&gt;.&lt;/p&gt;
  242. &lt;p&gt;For some reason the F11 key doesn’t work, but going fullscreen is crucial, because screen estate is scarce in this setup. I can go fullscreen using VSCode’s command palette (Ctrl-P) and invoking the command there, but Firefox often jumps out of the fullscreen mode, which is annoying. I still have to pay attention to when that’s happening; maybe its the Esc key, which I am of course using a lot due to me using vim bindings.&lt;/p&gt;
  243. &lt;p&gt;A more annoying problem was that on my Boox tablet, sometimes the on-screen keyboard would pop up, which is seriously annoying! It took me a while to track this down: The Boox has two virtual keyboards installed: The usual Google ASOP keyboard, and the Onyx Keyboard. The former is clever enough to stay hidden when there is a physical keyboard attached, but the latter isn’t. Moreover, pressing Shift-Ctrl on the physical keyboard rotates through the virtual keyboards. Now, VSCode has many keyboard shortcuts that require Shift-Ctrl (especially on an eInk device, where you really want to avoid using the mouse). And the limited settings exposed by the Boox Android system do not allow you configure that or disable the Onyx keyboard! To solve this, I had to install the KISS Launcher, which would allow me to see more Android settings, and in particular allow me to &lt;a href="https://www.reddit.com/r/Onyx_Boox/comments/13rhcg4/comment/k95ngmj/"&gt;disable the Onyx keyboard&lt;/a&gt;. So this is fixed.&lt;/p&gt;
  244. &lt;p&gt;&lt;del&gt;I was hoping to improve the experience even more by opening the web page as a Progressive Web App (PWA), &lt;a href="https://coder.com/docs/code-server/FAQ#how-do-i-make-my-keyboard-shortcuts-work"&gt;as described in the &lt;code&gt;code-server&lt;/code&gt; FAQ&lt;/a&gt;. Unfortunately, that did not work. Firefox on Android did not recognize the site as a PWA (even though it recognizes a &lt;a href="https://whatpwacando.today/"&gt;PWA test page&lt;/a&gt;). And I couldn’t use Chrome either because (unlike Firefox) it would not consider a site with a self-signed certificate as a secure context, and then &lt;code&gt;code-server&lt;/code&gt; does not &lt;a href="https://coder.com/docs/code-server/FAQ#why-do-web-views-not-work"&gt;work fully&lt;/a&gt;. Maybe this is just some bug that gets fixed in later versions.&lt;/del&gt;&lt;/p&gt;
  245. &lt;p&gt;Now that I use a proper certificate, I can use it as a Progressive Web App, and with Firefox on Android this starts the app in full-screen mode (no system bars, no location bar). The F11 key still does’t work, and using the command palette to enter fullscreen does nothing visible, but then Esc leaves that fullscreen mode and I suddenly have the system bars again. But maybe if I just don’t do that I get the full screen experience. We’ll see.&lt;/p&gt;
  246. &lt;p&gt;I did not work enough with this yet to assess how much the smaller screen estate, the lack of colors and the slower refresh rate will bother me. I probably need to hide Lean’s InfoView more often, and maybe use the &lt;a href="https://marketplace.visualstudio.com/items?itemName=usernamehw.errorlens"&gt;Error Lens extension&lt;/a&gt;, to avoid having to split my screen vertically.&lt;/p&gt;
  247. &lt;p&gt;I also cannot easily work on a park bench this way, with a tablet and a separate external keyboard. I’d need at least a table, or some additional piece of hardware that turns tablet + keyboard into some laptop-like structure that I can put on my, well, lap. There are &lt;a href="https://onyxboox.com/accessory#g-onyx-boox-cases"&gt;cases for Onyx products&lt;/a&gt; that include a keyboard, and maybe they work on the lap, but they don’t have the Trackpoint that I have on my &lt;a href="https://support.lenovo.com/de/en/accessories/acc500164-thinkpad-trackpoint-keyboard-ii-overview-and-service-parts"&gt;ThinkPad TrackPoint Keyboard II&lt;/a&gt;, and how can you live without that?&lt;/p&gt;
  248. &lt;h4 id="conclusion"&gt;Conclusion&lt;/h4&gt;
  249. &lt;p&gt;After this initial setup chances are good that entering and using this environment is convenient enough for me to actually use it; we will see when it gets warmer.&lt;/p&gt;
  250. &lt;p&gt;A few bits could be better. In particular logging in and authenticating GitHub access could be both more convenient and more safe – I could imagine that when I open the page I confirm that on my phone (maybe with a fingerprint), and that temporarily grants access to the &lt;code&gt;code-server&lt;/code&gt; and to specific GitHub repositories only. Is that easily possible?&lt;/p&gt;</description>
  251.         <pubDate>Sun, 02 Feb 2025 16:07:35 +0100</pubDate>
  252.      </item>
  253.      <item>
  254.         <title>Do surprises get larger?</title>
  255.         <link>https://www.joachim-breitner.de/blog/814-Do_surprises_get_larger_</link>
  256.         <guid>https://www.joachim-breitner.de/blog/814-Do_surprises_get_larger_</guid>
  257.         <comments>https://www.joachim-breitner.de/blog/814-Do_surprises_get_larger_#comments</comments>
  258.         <author>mail@joachim-breitner.de (Joachim Breitner)</author>
  259.         <description>&lt;h3 id="the-setup"&gt;The setup&lt;/h3&gt;
  260. &lt;p&gt;Imagine you are living on a riverbank. Every now and then, the river swells and you have high water. The first few times this may come as a surprise, but soon you learn that such floods are a recurring occurrence at that river, and you make suitable preparation. Let’s say you feel well-prepared against any flood that is no higher than the highest one observed so far. The more floods you have seen, the higher that mark is, and the better prepared you are. But of course, eventually a higher flood will occur that surprises you.&lt;/p&gt;
  261. &lt;p&gt;Of course such new record floods are happening rarer and rarer as you have seen more of them. I was wondering though: By how much do the new records exceed the previous high mark? Does this excess decrease or increase over time?&lt;/p&gt;
  262. &lt;p&gt;A priori both could be. When the high mark is already rather high, maybe new record floods will just barley pass that mark? Or maybe, simply because new records are so rare events, when they do occur, they can be surprisingly bad?&lt;/p&gt;
  263. &lt;p&gt;This post is a leisurely mathematical investigating of this question, which of course isn’t restricted to high waters; it could be anything that produces a measurement repeatedly and (mostly) independently – weather events, sport results, dice rolls.&lt;/p&gt;
  264. &lt;p&gt;The answer of course depends on the &lt;em&gt;distribution&lt;/em&gt; of results: How likely is each possible results.&lt;/p&gt;
  265. &lt;h3 id="dice-are-simple"&gt;Dice are simple&lt;/h3&gt;
  266. &lt;p&gt;With dice rolls the answer is rather simple. Let our measurement be how often you can roll a die until it shows a 6. This simple game we can repeat many times, and keep track of our record. Let’s say the record happens to be 7 rolls. If in the next run we roll the die 7 times, and it still does not show a 6, then we know that we have broken the record, and every further roll increases by how much we beat the old record.&lt;/p&gt;
  267. &lt;p&gt;But note that how often we will now roll the die is completely independent of what happened before!&lt;/p&gt;
  268. &lt;p&gt;So for this game the answer is: The excess with which the record is broken is always the same.&lt;/p&gt;
  269. &lt;p&gt;Mathematically speaking this is because the distribution of “rolls until the die shows a 6” is &lt;a href="https://en.wikipedia.org/wiki/Memorylessness"&gt;&lt;em&gt;memoryless&lt;/em&gt;&lt;/a&gt;. Such distributions are rather special, its essentially just the example we gave (a &lt;a href="https://en.wikipedia.org/wiki/Geometric_distribution"&gt;geometric distribution&lt;/a&gt;), or its continuous analogue (the &lt;a href="https://en.wikipedia.org/wiki/Exponential_distribution"&gt;exponential distributions&lt;/a&gt;, for example the time until a radioactive particle decays).&lt;/p&gt;
  270. &lt;h3 id="mathematical-formulation"&gt;Mathematical formulation&lt;/h3&gt;
  271. &lt;p&gt;With this out of the way, let us look at some other distributions, and for that, introduce some mathematical notations. Let &lt;span class="math inline"&gt;&lt;em&gt;X&lt;/em&gt;&lt;/span&gt; be a random variable with probability density function &lt;span class="math inline"&gt;&lt;em&gt;φ&lt;/em&gt;(&lt;em&gt;x&lt;/em&gt;)&lt;/span&gt; and cumulative distribution function &lt;span class="math inline"&gt;&lt;em&gt;Φ&lt;/em&gt;(&lt;em&gt;x&lt;/em&gt;)&lt;/span&gt;, and &lt;span class="math inline"&gt;&lt;em&gt;a&lt;/em&gt;&lt;/span&gt; be the previous record. We are interested in the behavior of&lt;/p&gt;
  272. &lt;p&gt;&lt;span class="math display"&gt;&lt;em&gt;Y&lt;/em&gt;(&lt;em&gt;a&lt;/em&gt;) = &lt;em&gt;X&lt;/em&gt; − &lt;em&gt;a&lt;/em&gt; ∣ &lt;em&gt;X&lt;/em&gt; &gt; &lt;em&gt;x&lt;/em&gt;&lt;/span&gt;&lt;/p&gt;
  273. &lt;p&gt;i.e. by how much &lt;span class="math inline"&gt;&lt;em&gt;X&lt;/em&gt;&lt;/span&gt; exceeds &lt;span class="math inline"&gt;&lt;em&gt;a&lt;/em&gt;&lt;/span&gt; under the condition that it did exceed &lt;span class="math inline"&gt;&lt;em&gt;a&lt;/em&gt;&lt;/span&gt;. How does &lt;span class="math inline"&gt;&lt;em&gt;Y&lt;/em&gt;&lt;/span&gt; change as &lt;span class="math inline"&gt;&lt;em&gt;a&lt;/em&gt;&lt;/span&gt; increases? In particular, how does the expected value of the excess &lt;span class="math inline"&gt;&lt;em&gt;e&lt;/em&gt;(&lt;em&gt;a&lt;/em&gt;) = &lt;em&gt;E&lt;/em&gt;(&lt;em&gt;Y&lt;/em&gt;(&lt;em&gt;a&lt;/em&gt;))&lt;/span&gt; change?&lt;/p&gt;
  274. &lt;h3 id="uniform-distribution"&gt;Uniform distribution&lt;/h3&gt;
  275. &lt;p&gt;If &lt;span class="math inline"&gt;&lt;em&gt;X&lt;/em&gt;&lt;/span&gt; is uniformly distributed between, say, 0 and 1, then a new record will appear uniformly distributed between &lt;span class="math inline"&gt;&lt;em&gt;a&lt;/em&gt;&lt;/span&gt; and &lt;span class="math inline"&gt;1&lt;/span&gt;, and as that range gets smaller, the excess must get smaller as well. More precisely,&lt;/p&gt;
  276. &lt;p&gt;&lt;span class="math display"&gt;&lt;em&gt;e&lt;/em&gt;(&lt;em&gt;a&lt;/em&gt;) = &lt;em&gt;E&lt;/em&gt;(&lt;em&gt;X&lt;/em&gt; − &lt;em&gt;a&lt;/em&gt; ∣ &lt;em&gt;X&lt;/em&gt; &gt; &lt;em&gt;a&lt;/em&gt;) = &lt;em&gt;E&lt;/em&gt;(&lt;em&gt;X&lt;/em&gt; ∣ &lt;em&gt;X&lt;/em&gt; &gt; &lt;em&gt;a&lt;/em&gt;) − &lt;em&gt;a&lt;/em&gt; = (1 − &lt;em&gt;a&lt;/em&gt;)/2&lt;/span&gt;&lt;/p&gt;
  277. &lt;p&gt;This not very interesting linear line is plotted in blue in this diagram:&lt;/p&gt;
  278. &lt;figure&gt;
  279. &lt;img src="//www.joachim-breitner.de/various/surprises-plot1.svg" alt="The expected record surpass for the uniform distribution"/&gt;
  280. &lt;figcaption aria-hidden="true"&gt;The expected record surpass for the uniform distribution&lt;/figcaption&gt;
  281. &lt;/figure&gt;
  282. &lt;p&gt;The orange line with the logarithmic scale on the right tries to convey how unlikely it is to surpass the record value &lt;span class="math inline"&gt;&lt;em&gt;a&lt;/em&gt;&lt;/span&gt;: it shows how many attempts we expect before the record is broken. This can be calculated by &lt;span class="math inline"&gt;&lt;em&gt;n&lt;/em&gt;(&lt;em&gt;a&lt;/em&gt;) = 1/(1 − &lt;em&gt;Φ&lt;/em&gt;(&lt;em&gt;a&lt;/em&gt;))&lt;/span&gt;.&lt;/p&gt;
  283. &lt;h3 id="normal-distribution"&gt;Normal distribution&lt;/h3&gt;
  284. &lt;p&gt;For the normal distribution (with median &lt;span class="math inline"&gt;0&lt;/span&gt; and standard derivation &lt;span class="math inline"&gt;1&lt;/span&gt;, to keep things simple), we can look up the expected value of the &lt;a href="https://en.wikipedia.org/wiki/Truncated_normal_distribution#One_sided_truncation_(of_lower_tail)%5B6%5D"&gt;one-sided truncated normal distribution&lt;/a&gt; and obtain&lt;/p&gt;
  285. &lt;p&gt;&lt;span class="math display"&gt;&lt;em&gt;e&lt;/em&gt;(&lt;em&gt;a&lt;/em&gt;) = &lt;em&gt;E&lt;/em&gt;(&lt;em&gt;X&lt;/em&gt; ∣ &lt;em&gt;X&lt;/em&gt; &gt; &lt;em&gt;a&lt;/em&gt;) − &lt;em&gt;a&lt;/em&gt; = &lt;em&gt;φ&lt;/em&gt;(&lt;em&gt;a&lt;/em&gt;)/(1 − &lt;em&gt;Φ&lt;/em&gt;(&lt;em&gt;a&lt;/em&gt;)) − &lt;em&gt;a&lt;/em&gt;&lt;/span&gt;&lt;/p&gt;
  286. &lt;p&gt;Now is this growing or shrinking? We can plot this an have a quick look:&lt;/p&gt;
  287. &lt;figure&gt;
  288. &lt;img src="//www.joachim-breitner.de/various/surprises-plot2.svg" alt="The expected record surpass for the normal distribution"/&gt;
  289. &lt;figcaption aria-hidden="true"&gt;The expected record surpass for the normal distribution&lt;/figcaption&gt;
  290. &lt;/figure&gt;
  291. &lt;p&gt;Indeed it is, too, a decreasing function!&lt;/p&gt;
  292. &lt;p&gt;(As a sanity check we can see that &lt;span class="math inline"&gt;&lt;em&gt;e&lt;/em&gt;(0) = √(2/&lt;em&gt;π&lt;/em&gt;)&lt;/span&gt;, which is the expected value of the &lt;a href="https://en.wikipedia.org/wiki/Half-normal_distribution"&gt;half-normal distribution&lt;/a&gt;, as it should.)&lt;/p&gt;
  293. &lt;h3 id="could-it-be-any-different"&gt;Could it be any different?&lt;/h3&gt;
  294. &lt;p&gt;This settles my question: It seems that each new surprisingly high water will tend to be less surprising than the previously – assuming high waters were uniformly or normally distributed, which is unlikely to be helpful.&lt;/p&gt;
  295. &lt;p&gt;This does raise the question, though, if there are probability distributions for which &lt;span class="math inline"&gt;&lt;em&gt;e&lt;/em&gt;(&lt;em&gt;a&lt;/em&gt;)&lt;/span&gt; is be increasing?&lt;/p&gt;
  296. &lt;p&gt;I can try to construct one, and because it’s a bit easier, I’ll consider a discrete distribution on the positive natural numbers, and consider at &lt;span class="math inline"&gt;&lt;em&gt;g&lt;/em&gt;(0) = &lt;em&gt;E&lt;/em&gt;(&lt;em&gt;X&lt;/em&gt;)&lt;/span&gt; and &lt;span class="math inline"&gt;&lt;em&gt;g&lt;/em&gt;(1) = &lt;em&gt;E&lt;/em&gt;(&lt;em&gt;X&lt;/em&gt; − 1 ∣ &lt;em&gt;X&lt;/em&gt; &gt; 1)&lt;/span&gt;. What does it take for &lt;span class="math inline"&gt;&lt;em&gt;g&lt;/em&gt;(1) &gt; &lt;em&gt;g&lt;/em&gt;(0)&lt;/span&gt;? Using &lt;span class="math inline"&gt;&lt;em&gt;E&lt;/em&gt;(&lt;em&gt;X&lt;/em&gt;) = &lt;em&gt;p&lt;/em&gt; + (1 − &lt;em&gt;p&lt;/em&gt;)&lt;em&gt;E&lt;/em&gt;(&lt;em&gt;X&lt;/em&gt; ∣ &lt;em&gt;X&lt;/em&gt; &gt; 1)&lt;/span&gt; for &lt;span class="math inline"&gt;&lt;em&gt;p&lt;/em&gt; = &lt;em&gt;P&lt;/em&gt;(&lt;em&gt;X&lt;/em&gt; = 1)&lt;/span&gt; we find that in order to have &lt;span class="math inline"&gt;&lt;em&gt;g&lt;/em&gt;(1) &gt; &lt;em&gt;g&lt;/em&gt;(0)&lt;/span&gt;, we need &lt;span class="math inline"&gt;&lt;em&gt;E&lt;/em&gt;(&lt;em&gt;X&lt;/em&gt;) &gt; 1/&lt;em&gt;p&lt;/em&gt;&lt;/span&gt;.&lt;/p&gt;
  297. &lt;p&gt;This is plausible because we get equality when &lt;span class="math inline"&gt;&lt;em&gt;E&lt;/em&gt;(&lt;em&gt;X&lt;/em&gt;) = 1/&lt;em&gt;p&lt;/em&gt;&lt;/span&gt;, as it precisely the case for the geometric distribution. And it is also plausible that it helps if &lt;span class="math inline"&gt;&lt;em&gt;p&lt;/em&gt;&lt;/span&gt; is large (so that the next first record is likely just 1) and if, nevertheless, &lt;span class="math inline"&gt;&lt;em&gt;E&lt;/em&gt;(&lt;em&gt;X&lt;/em&gt;)&lt;/span&gt; is large (so that if we do get an outcome other than 1, it’s much larger).&lt;/p&gt;
  298. &lt;p&gt;Starting with the geometric distribution, where &lt;span class="math inline"&gt;&lt;em&gt;P&lt;/em&gt;(&lt;em&gt;X&lt;/em&gt; &gt; &lt;em&gt;n&lt;/em&gt; ∣ &lt;em&gt;X&lt;/em&gt; ≥ &lt;em&gt;n&lt;/em&gt;) = &lt;em&gt;p&lt;/em&gt;&lt;sub&gt;&lt;em&gt;n&lt;/em&gt;&lt;/sub&gt; = &lt;em&gt;p&lt;/em&gt;&lt;/span&gt; (the probability of again not rolling a six) is constant, it seems that these &lt;span class="math inline"&gt;&lt;em&gt;p&lt;/em&gt;&lt;sub&gt;&lt;em&gt;n&lt;/em&gt;&lt;/sub&gt;&lt;/span&gt; is increasing, we get the desired behavior. So let &lt;span class="math inline"&gt;&lt;em&gt;p&lt;/em&gt;&lt;sub&gt;1&lt;/sub&gt; &amp;lt; &lt;em&gt;p&lt;/em&gt;&lt;sub&gt;2&lt;/sub&gt; &amp;lt; &lt;em&gt;p&lt;/em&gt;&lt;sub&gt;&lt;em&gt;n&lt;/em&gt;&lt;/sub&gt; &amp;lt; …&lt;/span&gt; be an increasing sequence of probabilities, and define &lt;span class="math inline"&gt;&lt;em&gt;X&lt;/em&gt;&lt;/span&gt; so that &lt;span class="math inline"&gt;&lt;em&gt;P&lt;/em&gt;(&lt;em&gt;X&lt;/em&gt; = &lt;em&gt;n&lt;/em&gt;) = &lt;em&gt;p&lt;/em&gt;&lt;sub&gt;1&lt;/sub&gt; ⋅ ⋯ ⋅ &lt;em&gt;p&lt;/em&gt;&lt;sub&gt;&lt;em&gt;n&lt;/em&gt; − 1&lt;/sub&gt; ⋅ (1 − &lt;em&gt;p&lt;/em&gt;&lt;sub&gt;&lt;em&gt;n&lt;/em&gt;&lt;/sub&gt;)&lt;/span&gt; (imagine the die wears off and the more often you roll it, the less likely it shows a 6). Then for this variation of the game, every new record tends to exceed the previous more than previous records. As the &lt;span class="math inline"&gt;&lt;em&gt;p&lt;/em&gt;&lt;/span&gt; increase, we get a flatter long end in the probability distribution.&lt;/p&gt;
  299. &lt;h3 id="gamma-distribution"&gt;Gamma distribution&lt;/h3&gt;
  300. &lt;p&gt;To get a nice plot, I’ll take the intuition from this and turn to continuous distributions. The Wikipedia page for the exponential distribution says it is a special case of the &lt;a href="https://en.m.wikipedia.org/wiki/Gamma_distribution"&gt;gamma distribution&lt;/a&gt;, which has an additional shape parameter &lt;span class="math inline"&gt;&lt;em&gt;α&lt;/em&gt;&lt;/span&gt;, and it seems that it could influence the shape of the distribution to be and make the probability distribution have a longer end. Let’s play around with &lt;span class="math inline"&gt;&lt;em&gt;β&lt;/em&gt; = 2&lt;/span&gt; and &lt;span class="math inline"&gt;&lt;em&gt;α&lt;/em&gt; = 0.5&lt;/span&gt;, &lt;span class="math inline"&gt;1&lt;/span&gt; and &lt;span class="math inline"&gt;1.5&lt;/span&gt;:&lt;/p&gt;
  301. &lt;figure&gt;
  302. &lt;img src="//www.joachim-breitner.de/various/surprises-plot3.svg" alt="The expected record surpass for the gamma distribution"/&gt;
  303. &lt;figcaption aria-hidden="true"&gt;The expected record surpass for the gamma distribution&lt;/figcaption&gt;
  304. &lt;/figure&gt;
  305. &lt;ul&gt;
  306. &lt;li&gt;&lt;p&gt;For &lt;span class="math inline"&gt;&lt;em&gt;α&lt;/em&gt; = 1&lt;/span&gt; (dotted) this should just be the exponential distribution, and we see that &lt;span class="math inline"&gt;&lt;em&gt;e&lt;/em&gt;(&lt;em&gt;a&lt;/em&gt;)&lt;/span&gt; is flat, as predicted earlier.&lt;/p&gt;&lt;/li&gt;
  307. &lt;li&gt;&lt;p&gt;For larger &lt;span class="math inline"&gt;&lt;em&gt;α&lt;/em&gt;&lt;/span&gt; (dashed) the graph does not look much different from the one for the normal distribution – not a surprise, as for &lt;span class="math inline"&gt;&lt;em&gt;α&lt;/em&gt; → ∞&lt;/span&gt;, the gamma distribution turns into the normal distribution.&lt;/p&gt;&lt;/li&gt;
  308. &lt;li&gt;&lt;p&gt;For smaller &lt;span class="math inline"&gt;&lt;em&gt;α&lt;/em&gt;&lt;/span&gt; (solid) we get the desired effect: &lt;span class="math inline"&gt;&lt;em&gt;e&lt;/em&gt;(&lt;em&gt;a&lt;/em&gt;)&lt;/span&gt; is increasing. This means that new records tend to break records more impressively.&lt;/p&gt;&lt;/li&gt;
  309. &lt;/ul&gt;
  310. &lt;p&gt;The orange line shows that this comes at a cost: for a given old record &lt;span class="math inline"&gt;&lt;em&gt;a&lt;/em&gt;&lt;/span&gt;, new records are harder to come by with smaller &lt;span class="math inline"&gt;&lt;em&gt;α&lt;/em&gt;&lt;/span&gt;.&lt;/p&gt;
  311. &lt;h3 id="conclusion"&gt;Conclusion&lt;/h3&gt;
  312. &lt;p&gt;As usual, it all depends on the distribution. Otherwise, not much, it’s late.&lt;/p&gt;</description>
  313.         <pubDate>Sun, 30 Jun 2024 15:28:31 +0200</pubDate>
  314.      </item>
  315.      <item>
  316.         <title>Blogging on Lean</title>
  317.         <link>https://www.joachim-breitner.de/blog/813-Blogging_on_Lean</link>
  318.         <guid>https://www.joachim-breitner.de/blog/813-Blogging_on_Lean</guid>
  319.         <comments>https://www.joachim-breitner.de/blog/813-Blogging_on_Lean#comments</comments>
  320.         <author>mail@joachim-breitner.de (Joachim Breitner)</author>
  321.         <description>&lt;p&gt;This blog has become a bit quiet &lt;a href="https://www.joachim-breitner.de/blog/809-Joining_the_Lean_FRO"&gt;since I joined the Lean FRO&lt;/a&gt;. One reasons is of course that I can now improve things about Lean, rather than blog about what I think should be done (which, by contraposition, means I shouldn’t blog about what can be improved…). A better reason is that some of the things I’d otherwise write here are now published on the &lt;a href="https://lean-lang.org/blog"&gt;official Lean blog&lt;/a&gt;, in particular two lengthy technical posts explaining aspects of Lean that I worked on:&lt;/p&gt;
  322. &lt;ul&gt;
  323. &lt;li&gt;&lt;a href="https://lean-lang.org/blog/2024-1-11-recursive-definitions-in-lean"&gt;Recursive definitions in Lean&lt;/a&gt;&lt;/li&gt;
  324. &lt;li&gt;&lt;a href="https://lean-lang.org/blog/2024-5-17-functional-induction"&gt;Functional Induction theorems&lt;/a&gt;&lt;/li&gt;
  325. &lt;/ul&gt;
  326. &lt;p&gt;It would not be useful to re-publish them here because the technology &lt;a href="https://github.com/leanprover/verso"&gt;&lt;code&gt;verso&lt;/code&gt;&lt;/a&gt; behind the Lean blog, created by my colleage David Thrane Christansen, enables such fancy features like type-checked code snippets, including output and lots of information on hover. So I’ll be content with just cross-linking my posts from here.&lt;/p&gt;</description>
  327.         <pubDate>Fri, 31 May 2024 13:47:06 +0100</pubDate>
  328.      </item>
  329.      <item>
  330.         <title>Convenient sandboxed development environment</title>
  331.         <link>https://www.joachim-breitner.de/blog/812-Convenient_sandboxed_development_environment</link>
  332.         <guid>https://www.joachim-breitner.de/blog/812-Convenient_sandboxed_development_environment</guid>
  333.         <comments>https://www.joachim-breitner.de/blog/812-Convenient_sandboxed_development_environment#comments</comments>
  334.         <author>mail@joachim-breitner.de (Joachim Breitner)</author>
  335.         <description>&lt;p&gt;I like using one machine and setup for everything, from serious development work to hobby projects to managing my finances. This is very convenient, as often the lines between these are blurred. But it is also scary if I think of the large number of people who I have to trust to not want to extract all my personal data. Whenever I run a &lt;code&gt;cabal install&lt;/code&gt;, or a fun VSCode extension gets updated, or anything like that, I am running code that could be malicious or buggy.&lt;/p&gt;
  336. &lt;p&gt;In a way it is surprising and reassuring that, as far as I can tell, this commonly does not happen. Most open source developers out there seem to be nice and well-meaning, after all.&lt;/p&gt;
  337. &lt;h3 id="convenient-or-it-wont-happen"&gt;Convenient or it won’t happen&lt;/h3&gt;
  338. &lt;p&gt;Nevertheless I thought I should do something about this. The safest option would probably to use dedicated virtual machines for the development work, with very little interaction with my main system. But knowing me, that did not seem likely to happen, as it sounded like a fair amount of hassle. So I aimed for a viable compromise between security and convenient, and one that does not get too much in the way of my current habits.&lt;/p&gt;
  339. &lt;p&gt;For instance, it seems desirable to have the project files accessible from my unconstrained environment. This way, I could perform certain actions that need access to secret keys or tokens, but are (unlikely) to run code (e.g. &lt;code&gt;git push&lt;/code&gt;, &lt;code&gt;git pull&lt;/code&gt; from private repositories, &lt;code&gt;gh pr create&lt;/code&gt;) from “the outside”, and the actual build environment can do without access to these secrets.&lt;/p&gt;
  340. &lt;p&gt;The user experience I thus want is a quick way to enter a “development environment” where I can do most of the things I need to do while programming (network access, running command line and GUI programs), with access to the current project, but without access to my actual &lt;code&gt;/home&lt;/code&gt; directory.&lt;/p&gt;
  341. &lt;p&gt;I initially followed the blog post &lt;a href="https://msucharski.eu/posts/application-isolation-nixos-containers/"&gt;“Application Isolation using NixOS Containers” by Marcin Sucharski&lt;/a&gt; and got something working that mostly did what I wanted, but then a colleague pointed out that tools like &lt;a href="https://github.com/netblue30/firejail"&gt;&lt;code&gt;firejail&lt;/code&gt;&lt;/a&gt; can achieve roughly the same with a less “global” setup. I tried to use &lt;code&gt;firejail&lt;/code&gt;, but found it to be a bit too inflexible for my particular whims, so I ended up writing a small wrapper around the lower level sandboxing tool &lt;a href="Bubblewrap"&gt;https://github.com/containers/bubblewrap&lt;/a&gt;.&lt;/p&gt;
  342. &lt;h3 id="selective-bubblewrapping"&gt;Selective bubblewrapping&lt;/h3&gt;
  343. &lt;p&gt;This script, called &lt;code&gt;dev&lt;/code&gt; and included below, builds a new filesystem namespace with minimal &lt;code&gt;/proc&lt;/code&gt; and &lt;code&gt;/dev&lt;/code&gt; directories, it’s own &lt;code&gt;/tmp&lt;/code&gt; directories. It then binds-mound some directories to make the host’s NixOS system available inside the container (&lt;code&gt;/bin&lt;/code&gt;, &lt;code&gt;/usr&lt;/code&gt;, the nix store including domain socket, stuff for OpenGL applications). My user’s home directory is taken from &lt;code&gt;~/.dev-home&lt;/code&gt; and some configuration files are bind-mounted for convenient sharing. I intentionally don’t share most of the configuration – for example, a &lt;code&gt;direnv enable&lt;/code&gt; in the dev environment should not affect the main environment. The X11 socket for graphical applications and the corresponding &lt;code&gt;.Xauthority&lt;/code&gt; file is made available. And finally, if I run &lt;code&gt;dev&lt;/code&gt; in a project directory, this project directory is bind mounted writable, and the current working directory is preserved.&lt;/p&gt;
  344. &lt;p&gt;The effect is that I can type &lt;code&gt;dev&lt;/code&gt; on the command line to enter “dev mode” rather conveniently. I can run development tools, including graphical ones like VSCode, and especially the latter with its extensions is part of the sandbox. To do a &lt;code&gt;git push&lt;/code&gt; I either exit the development environment (Ctrl-D) or open a separate terminal. Overall, the inconvenience of switching back and forth seems worth the extra protection.&lt;/p&gt;
  345. &lt;p&gt;Clearly, isn’t going to hold against a determined and maybe targeted attacker (e.g. access to the X11 and the nix daemon socket can probably be used to escape easily). But I hope it will help against a compromised dev dependency that just deletes or exfiltrates data, like keys or passwords, from the usual places in &lt;code&gt;$HOME&lt;/code&gt;.&lt;/p&gt;
  346. &lt;h3 id="rough-corners"&gt;Rough corners&lt;/h3&gt;
  347. &lt;p&gt;There is more polishing that could be done.&lt;/p&gt;
  348. &lt;ul&gt;
  349. &lt;li&gt;&lt;p&gt;In particular, clicking on a link inside VSCode in the container will currently open Firefox inside the container, without access to my settings and cookies etc. Ideally, links would be opened in the Firefox running outside. This is a problem that has a solution in the world of applications that are sandboxed with Flatpak, and involves a bunch of moving parts (a &lt;a href="https://github.com/flatpak/xdg-desktop-portal"&gt;xdg-desktop-portal&lt;/a&gt; user service, a &lt;a href="https://github.com/flatpak/xdg-dbus-proxy"&gt;filtering dbus proxy&lt;/a&gt;, exposing access to that proxy in the container). I experimented with that for a bit longer than I should have, but could not get it to work to satisfaction (even without a container involved, I could not get &lt;code&gt;xdg-desktop-portal&lt;/code&gt; to heed my default browser settings…). For now I will live with manually copying and pasting URLs, we’ll see how long this lasts.&lt;/p&gt;&lt;/li&gt;
  350. &lt;li&gt;&lt;p&gt;With this setup (and unlike the NixOS container setup I tried first), the same applications are installed inside and outside. It might be useful to separate the set of installed programs: There is simply no point in running &lt;code&gt;evolution&lt;/code&gt; or &lt;code&gt;firefox&lt;/code&gt; inside the container, and if I do not even have VSCode or &lt;code&gt;cabal&lt;/code&gt; available outside, so that it’s less likely that I forget to enter &lt;code&gt;dev&lt;/code&gt; before using these tools.&lt;/p&gt;
  351. &lt;p&gt;It shouldn’t be too hard to cargo-cult some of the NixOS Containers infrastructure to be able to have a separate system configuration that I can manage as part of my normal system configuration and make available to &lt;code&gt;bubblewrap&lt;/code&gt; here.&lt;/p&gt;&lt;/li&gt;
  352. &lt;/ul&gt;
  353. &lt;p&gt;So likely I will refine this some more over time. Or get tired of typing &lt;code&gt;dev&lt;/code&gt; and going back to what I did before…&lt;/p&gt;
  354. &lt;h3 id="the-script"&gt;The script&lt;/h3&gt;
  355. &lt;details&gt;
  356. &lt;summary&gt;
  357. The &lt;code&gt;dev&lt;/code&gt; script (at the time of writing)
  358. &lt;/summary&gt;
  359. &lt;div class="sourceCode" id="cb1"&gt;&lt;pre class="sourceCode bash"&gt;&lt;code class="sourceCode bash"&gt;&lt;span id="cb1-1"&gt;&lt;a href="#cb1-1" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="co"&gt;#!/usr/bin/env bash&lt;/span&gt;&lt;/span&gt;
  360. &lt;span id="cb1-2"&gt;&lt;a href="#cb1-2" aria-hidden="true" tabindex="-1"/&gt;&lt;/span&gt;
  361. &lt;span id="cb1-3"&gt;&lt;a href="#cb1-3" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="va"&gt;extra&lt;/span&gt;&lt;span class="op"&gt;=&lt;/span&gt;&lt;span class="va"&gt;()&lt;/span&gt;&lt;/span&gt;
  362. &lt;span id="cb1-4"&gt;&lt;a href="#cb1-4" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="cf"&gt;if&lt;/span&gt; &lt;span class="kw"&gt;[[&lt;/span&gt; &lt;span class="st"&gt;"&lt;/span&gt;&lt;span class="va"&gt;$PWD&lt;/span&gt;&lt;span class="st"&gt;"&lt;/span&gt; &lt;span class="ot"&gt;==&lt;/span&gt; /home/jojo/build/&lt;span class="pp"&gt;*&lt;/span&gt; &lt;span class="kw"&gt;]]&lt;/span&gt; &lt;span class="kw"&gt;||&lt;/span&gt; &lt;span class="kw"&gt;[[&lt;/span&gt; &lt;span class="st"&gt;"&lt;/span&gt;&lt;span class="va"&gt;$PWD&lt;/span&gt;&lt;span class="st"&gt;"&lt;/span&gt; &lt;span class="ot"&gt;==&lt;/span&gt; /home/jojo/projekte/programming/&lt;span class="pp"&gt;*&lt;/span&gt; &lt;span class="kw"&gt;]]&lt;/span&gt;&lt;/span&gt;
  363. &lt;span id="cb1-5"&gt;&lt;a href="#cb1-5" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="cf"&gt;then&lt;/span&gt;&lt;/span&gt;
  364. &lt;span id="cb1-6"&gt;&lt;a href="#cb1-6" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="va"&gt;extra&lt;/span&gt;&lt;span class="op"&gt;+=&lt;/span&gt;&lt;span class="va"&gt;(&lt;/span&gt;--bind &lt;span class="st"&gt;"&lt;/span&gt;&lt;span class="va"&gt;$PWD&lt;/span&gt;&lt;span class="st"&gt;"&lt;/span&gt; &lt;span class="st"&gt;"&lt;/span&gt;&lt;span class="va"&gt;$PWD&lt;/span&gt;&lt;span class="st"&gt;"&lt;/span&gt; --chdir &lt;span class="st"&gt;"&lt;/span&gt;&lt;span class="va"&gt;$PWD&lt;/span&gt;&lt;span class="st"&gt;"&lt;/span&gt;&lt;span class="va"&gt;)&lt;/span&gt;&lt;/span&gt;
  365. &lt;span id="cb1-7"&gt;&lt;a href="#cb1-7" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="cf"&gt;fi&lt;/span&gt;&lt;/span&gt;
  366. &lt;span id="cb1-8"&gt;&lt;a href="#cb1-8" aria-hidden="true" tabindex="-1"/&gt;&lt;/span&gt;
  367. &lt;span id="cb1-9"&gt;&lt;a href="#cb1-9" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="cf"&gt;if&lt;/span&gt; &lt;span class="bu"&gt;[&lt;/span&gt; &lt;span class="ot"&gt;-n&lt;/span&gt; &lt;span class="st"&gt;"&lt;/span&gt;&lt;span class="va"&gt;$1&lt;/span&gt;&lt;span class="st"&gt;"&lt;/span&gt; &lt;span class="bu"&gt;]&lt;/span&gt;&lt;/span&gt;
  368. &lt;span id="cb1-10"&gt;&lt;a href="#cb1-10" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="cf"&gt;then&lt;/span&gt;&lt;/span&gt;
  369. &lt;span id="cb1-11"&gt;&lt;a href="#cb1-11" aria-hidden="true" tabindex="-1"/&gt;    &lt;span class="va"&gt;cmd&lt;/span&gt;&lt;span class="op"&gt;=&lt;/span&gt;&lt;span class="va"&gt;(&lt;/span&gt; &lt;span class="st"&gt;"&lt;/span&gt;&lt;span class="va"&gt;$@&lt;/span&gt;&lt;span class="st"&gt;"&lt;/span&gt; &lt;span class="va"&gt;)&lt;/span&gt;&lt;/span&gt;
  370. &lt;span id="cb1-12"&gt;&lt;a href="#cb1-12" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="cf"&gt;else&lt;/span&gt;&lt;/span&gt;
  371. &lt;span id="cb1-13"&gt;&lt;a href="#cb1-13" aria-hidden="true" tabindex="-1"/&gt;    &lt;span class="va"&gt;cmd&lt;/span&gt;&lt;span class="op"&gt;=&lt;/span&gt;&lt;span class="va"&gt;(&lt;/span&gt; bash &lt;span class="va"&gt;)&lt;/span&gt;&lt;/span&gt;
  372. &lt;span id="cb1-14"&gt;&lt;a href="#cb1-14" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="cf"&gt;fi&lt;/span&gt;&lt;/span&gt;
  373. &lt;span id="cb1-15"&gt;&lt;a href="#cb1-15" aria-hidden="true" tabindex="-1"/&gt;&lt;/span&gt;
  374. &lt;span id="cb1-16"&gt;&lt;a href="#cb1-16" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="co"&gt;# Caveats:&lt;/span&gt;&lt;/span&gt;
  375. &lt;span id="cb1-17"&gt;&lt;a href="#cb1-17" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="co"&gt;# * access to all of `/etc`&lt;/span&gt;&lt;/span&gt;
  376. &lt;span id="cb1-18"&gt;&lt;a href="#cb1-18" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="co"&gt;# * access to `/nix/var/nix/daemon-socket/socket`, and is trusted user (but needed to run nix)&lt;/span&gt;&lt;/span&gt;
  377. &lt;span id="cb1-19"&gt;&lt;a href="#cb1-19" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="co"&gt;# * access to X11&lt;/span&gt;&lt;/span&gt;
  378. &lt;span id="cb1-20"&gt;&lt;a href="#cb1-20" aria-hidden="true" tabindex="-1"/&gt;&lt;/span&gt;
  379. &lt;span id="cb1-21"&gt;&lt;a href="#cb1-21" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="bu"&gt;exec&lt;/span&gt; bwrap &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  380. &lt;span id="cb1-22"&gt;&lt;a href="#cb1-22" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="at"&gt;--unshare-all&lt;/span&gt; &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  381. &lt;span id="cb1-23"&gt;&lt;a href="#cb1-23" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  382. &lt;span id="cb1-24"&gt;&lt;a href="#cb1-24" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="kw"&gt;`&lt;/span&gt;&lt;span class="co"&gt;# blank slate&lt;/span&gt;&lt;span class="kw"&gt;`&lt;/span&gt; &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  383. &lt;span id="cb1-25"&gt;&lt;a href="#cb1-25" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="at"&gt;--share-net&lt;/span&gt; &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  384. &lt;span id="cb1-26"&gt;&lt;a href="#cb1-26" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="at"&gt;--proc&lt;/span&gt; /proc &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  385. &lt;span id="cb1-27"&gt;&lt;a href="#cb1-27" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="at"&gt;--dev&lt;/span&gt; /dev &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  386. &lt;span id="cb1-28"&gt;&lt;a href="#cb1-28" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="at"&gt;--tmpfs&lt;/span&gt; /tmp &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  387. &lt;span id="cb1-29"&gt;&lt;a href="#cb1-29" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="at"&gt;--tmpfs&lt;/span&gt; /run/user/1000 &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  388. &lt;span id="cb1-30"&gt;&lt;a href="#cb1-30" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  389. &lt;span id="cb1-31"&gt;&lt;a href="#cb1-31" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="kw"&gt;`&lt;/span&gt;&lt;span class="co"&gt;# Needed for GLX applications, in paticular alacritty&lt;/span&gt;&lt;span class="kw"&gt;`&lt;/span&gt; &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  390. &lt;span id="cb1-32"&gt;&lt;a href="#cb1-32" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="at"&gt;--dev-bind&lt;/span&gt; /dev/dri /dev/dri &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  391. &lt;span id="cb1-33"&gt;&lt;a href="#cb1-33" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="at"&gt;--ro-bind&lt;/span&gt; /sys/dev/char /sys/dev/char &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  392. &lt;span id="cb1-34"&gt;&lt;a href="#cb1-34" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="at"&gt;--ro-bind&lt;/span&gt; /sys/devices/pci0000:00 /sys/devices/pci0000:00 &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  393. &lt;span id="cb1-35"&gt;&lt;a href="#cb1-35" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="at"&gt;--ro-bind&lt;/span&gt; /run/opengl-driver /run/opengl-driver &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  394. &lt;span id="cb1-36"&gt;&lt;a href="#cb1-36" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  395. &lt;span id="cb1-37"&gt;&lt;a href="#cb1-37" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="at"&gt;--ro-bind&lt;/span&gt; /bin /bin &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  396. &lt;span id="cb1-38"&gt;&lt;a href="#cb1-38" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="at"&gt;--ro-bind&lt;/span&gt; /usr /usr &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  397. &lt;span id="cb1-39"&gt;&lt;a href="#cb1-39" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="at"&gt;--ro-bind&lt;/span&gt; /run/current-system /run/current-system &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  398. &lt;span id="cb1-40"&gt;&lt;a href="#cb1-40" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="at"&gt;--ro-bind&lt;/span&gt; /nix /nix &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  399. &lt;span id="cb1-41"&gt;&lt;a href="#cb1-41" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="at"&gt;--ro-bind&lt;/span&gt; /etc /etc &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  400. &lt;span id="cb1-42"&gt;&lt;a href="#cb1-42" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="at"&gt;--ro-bind&lt;/span&gt; /run/systemd/resolve/stub-resolv.conf /run/systemd/resolve/stub-resolv.conf &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  401. &lt;span id="cb1-43"&gt;&lt;a href="#cb1-43" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  402. &lt;span id="cb1-44"&gt;&lt;a href="#cb1-44" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="at"&gt;--bind&lt;/span&gt; ~/.dev-home /home/jojo &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  403. &lt;span id="cb1-45"&gt;&lt;a href="#cb1-45" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="at"&gt;--ro-bind&lt;/span&gt; ~/.config/alacritty ~/.config/alacritty  &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  404. &lt;span id="cb1-46"&gt;&lt;a href="#cb1-46" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="at"&gt;--ro-bind&lt;/span&gt; ~/.config/nvim ~/.config/nvim  &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  405. &lt;span id="cb1-47"&gt;&lt;a href="#cb1-47" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="at"&gt;--ro-bind&lt;/span&gt; ~/.local/share/nvim ~/.local/share/nvim  &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  406. &lt;span id="cb1-48"&gt;&lt;a href="#cb1-48" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="at"&gt;--ro-bind&lt;/span&gt; ~/.bin ~/.bin &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  407. &lt;span id="cb1-49"&gt;&lt;a href="#cb1-49" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  408. &lt;span id="cb1-50"&gt;&lt;a href="#cb1-50" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="at"&gt;--bind&lt;/span&gt; /tmp/.X11-unix/X0 /tmp/.X11-unix/X0 &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  409. &lt;span id="cb1-51"&gt;&lt;a href="#cb1-51" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="at"&gt;--bind&lt;/span&gt; ~/.Xauthority ~/.Xauthority &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  410. &lt;span id="cb1-52"&gt;&lt;a href="#cb1-52" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="at"&gt;--setenv&lt;/span&gt; DISPLAY :0 &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  411. &lt;span id="cb1-53"&gt;&lt;a href="#cb1-53" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  412. &lt;span id="cb1-54"&gt;&lt;a href="#cb1-54" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="at"&gt;--setenv&lt;/span&gt; container dev &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  413. &lt;span id="cb1-55"&gt;&lt;a href="#cb1-55" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="st"&gt;"&lt;/span&gt;&lt;span class="va"&gt;${extra&lt;/span&gt;&lt;span class="op"&gt;[@]&lt;/span&gt;&lt;span class="va"&gt;}&lt;/span&gt;&lt;span class="st"&gt;"&lt;/span&gt; &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  414. &lt;span id="cb1-56"&gt;&lt;a href="#cb1-56" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="at"&gt;--&lt;/span&gt; &lt;span class="dt"&gt;\&lt;/span&gt;&lt;/span&gt;
  415. &lt;span id="cb1-57"&gt;&lt;a href="#cb1-57" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="st"&gt;"&lt;/span&gt;&lt;span class="va"&gt;${cmd&lt;/span&gt;&lt;span class="op"&gt;[@]&lt;/span&gt;&lt;span class="va"&gt;}&lt;/span&gt;&lt;span class="st"&gt;"&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  416. &lt;/details&gt;</description>
  417.         <pubDate>Mon, 11 Mar 2024 21:39:58 +0100</pubDate>
  418.      </item>
  419.      <item>
  420.         <title>GHC Steering Committee Retrospective</title>
  421.         <link>https://www.joachim-breitner.de/blog/811-GHC_Steering_Committee_Retrospective</link>
  422.         <guid>https://www.joachim-breitner.de/blog/811-GHC_Steering_Committee_Retrospective</guid>
  423.         <comments>https://www.joachim-breitner.de/blog/811-GHC_Steering_Committee_Retrospective#comments</comments>
  424.         <author>mail@joachim-breitner.de (Joachim Breitner)</author>
  425.         <description>&lt;p&gt;After seven years of service as member and secretary on the GHC Steering Committee, I have resigned from that role. So this is a good time to look back and retrace the formation of the GHC proposal process and committee.&lt;/p&gt;
  426. &lt;p&gt;In my memory, I helped define and shape the proposal process, optimizing it for effectiveness and throughput, but memory can be misleading, and judging from the paper trail in my email archives, this was indeed mostly Ben Gamari’s and Richard Eisenberg’s achievement: Already in Summer of 2016, Ben Gamari set up the &lt;a href="https://github.com/ghc-proposals/ghc-proposals"&gt;ghc-proposals Github repository&lt;/a&gt; with a sketch of a process and sent out a &lt;a href="https://mail.haskell.org/pipermail/glasgow-haskell-users/2016-September/026396.html"&gt;call for nominations&lt;/a&gt; on the GHC user’s mailing list, which I replied to. The Simons picked the first set of members, and in the fall of 2016 we discussed the committee’s by-laws and procedures. As so often, Richard was an influential shaping force here.&lt;/p&gt;
  427. &lt;h3 id="three-ingredients"&gt;Three ingredients&lt;/h3&gt;
  428. &lt;p&gt;For example, it was him that suggested that for each proposal we have one committee member be the “Shepherd”, overseeing the discussion. I believe this was one ingredient for the process effectiveness: There is always one person in charge, and thus we avoid the delays incurred when any one of a non-singleton set of volunteers have to do the next step (and everyone hopes someone else does it).&lt;/p&gt;
  429. &lt;p&gt;The next ingredient was that we do not usually require a vote among all members (again, not easy with volunteers with limited bandwidth and occasional phases of absence). Instead, the shepherd makes a recommendation (accept/reject), and if the other committee members do not complain, this silence is taken as consent, and we come to a decision. It seems this idea can also be traced back on Richard, who suggested that “once a decision is requested, the shepherd [generates] consensus. If consensus is elusive, then we vote.”&lt;/p&gt;
  430. &lt;p&gt;At the end of the year we agreed and wrote down these rules, created the mailing list for our internal, but &lt;a href="https://mail.haskell.org/pipermail/ghc-steering-committee/"&gt;publicly archived committee discussions&lt;/a&gt;, and began accepting proposals, starting with &lt;a href="https://github.com/ghc-proposals/ghc-proposals/pull/6"&gt;Adam Gundry’s &lt;code&gt;OverloadedRecordFields&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;
  431. &lt;p&gt;At that point, there was no “secretary” role yet, so how I did become one? It seems that in February 2017 I started to &lt;a href="https://mail.haskell.org/pipermail/ghc-steering-committee/2017-February/000040.html"&gt;clean-up and refine the process documentation&lt;/a&gt;, fixing “bugs in the process” (like requiring authors to set Github labels when they don’t even have permissions to do that). This in particular meant that someone from the committee had to manually handle submissions and so on, and by the aforementioned principle that at every step there ought to be exactly one person in change, the role of a secretary followed naturally. In the &lt;a href="https://mail.haskell.org/pipermail/ghc-steering-committee/2017-February/000044.html"&gt;email in which I described that role&lt;/a&gt; I wrote:&lt;/p&gt;
  432. &lt;blockquote&gt;
  433. &lt;p&gt;Simon already shoved me towards picking up the “secretary” hat, to reduce load on Ben.&lt;/p&gt;
  434. &lt;/blockquote&gt;
  435. &lt;p&gt;So when I &lt;a href="https://github.com/ghc-proposals/ghc-proposals/commit/63d050c380b2aa380581cc9b9aafb2a7c022556a"&gt;merged the updated process documentation&lt;/a&gt;, I already listed myself “secretary”.&lt;/p&gt;
  436. &lt;p&gt;It wasn’t just Simon’s shoving that put my into the role, though. I dug out my original self-nomination email to Ben, and among other things I wrote:&lt;/p&gt;
  437. &lt;blockquote&gt;
  438. &lt;p&gt;I also hope that there is going to be clear responsibilities and a clear workflow among the committee. E.g. someone (possibly rotating), maybe called the secretary, who is in charge of having an initial look at proposals and then assigning it to a member who shepherds the proposal.&lt;/p&gt;
  439. &lt;/blockquote&gt;
  440. &lt;p&gt;So it is hardly a surprise that I became secretary, when it was dear to my heart to have a smooth continuous process here.&lt;/p&gt;
  441. &lt;p&gt;I am rather content with the result: These three ingredients – single secretary, per-proposal shepherds, silence-is-consent – helped the committee to be effective throughout its existence, even as every once in a while individual members dropped out.&lt;/p&gt;
  442. &lt;h3 id="ulterior-motivation"&gt;Ulterior motivation&lt;/h3&gt;
  443. &lt;p&gt;I must admit, however, there was an ulterior motivation behind me grabbing the secretary role: Yes, I &lt;em&gt;did&lt;/em&gt; want the committee to succeed, and I &lt;em&gt;did&lt;/em&gt; want that authors receive timely, good and decisive feedback on their proposals – but I &lt;em&gt;did not&lt;/em&gt; really want to have to do that part.&lt;/p&gt;
  444. &lt;p&gt;I am, in fact, a lousy proposal reviewer. I am too generous when reading proposals, and more likely mentally fill gaps in a specification rather than spotting them. Always optimistically assuming that the authors surely know what they are doing, rather than critically assessing the impact, the implementation cost and the interaction with other language features.&lt;/p&gt;
  445. &lt;p&gt;And, maybe more importantly: why should I know which changes are good and which are not so good in the long run? Clearly, the authors cared enough about a proposal to put it forward, so there is some need… and I do believe that Haskell should stay an evolving and innovating language… but how does this help me decide about this or that particular feature.&lt;/p&gt;
  446. &lt;p&gt;I even, during the formation of the committee, explicitly asked that we write down some guidance on “Vision and Guideline”; do we want to foster change or innovation, or be selective gatekeepers? Should we accept features that are proven to be useful, or should we accept features so that they can prove to be useful? This discussion, however, did not lead to a concrete result, and the assessment of proposals relied on the sum of each member’s personal preference, expertise and gut feeling. I am not saying that this was a mistake: It is hard to come up with a general guideline here, and even harder to find one that does justice to each individual proposal.&lt;/p&gt;
  447. &lt;p&gt;So the secret motivation for me to grab the secretary post was that I could contribute without having to judge proposals. Being secretary allowed me to assign most proposals to others to shepherd, and only once in a while myself took care of a proposal, when it seemed to be very straight-forward. Sneaky, ain’t it?&lt;/p&gt;
  448. &lt;h3 id="years-later"&gt;7 Years later&lt;/h3&gt;
  449. &lt;p&gt;For years to come I happily played secretary: When an author finished their proposal and public discussion ebbed down they would ping me on GitHub, I would &lt;a href="https://mail.haskell.org/pipermail/ghc-steering-committee/2022-July/002837.html"&gt;pick a suitable shepherd&lt;/a&gt; among the committee and ask them to judge the proposal. Eventually, the committee would come to a conclusion, usually by implicit consent, sometimes by voting, and I’d merge the pull request and update the metadata thereon. Every few months I’d &lt;a href="https://mail.haskell.org/pipermail/ghc-steering-committee/2024-January/003683.html"&gt;summarize the current state of affairs&lt;/a&gt; to the committee (what happened since the last update, which proposals are currently on our plate), and once per year gathered the data for &lt;a href="https://youtu.be/LFIL0myeOlo?si=P316QJs0EBSWe4Uy&amp;amp;t=3955"&gt;Simon Peyton Jones’ annually GHC Status Report&lt;/a&gt;. Sometimes some members needed a nudge or two to act. Some would eventually step down, and I’d sent around a call for nominations and when the nominations came in, distributed them off-list among the committee and tallied the votes.&lt;/p&gt;
  450. &lt;p&gt;Initially, that was exciting. For a long while it was a pleasant and rewarding routine. Eventually, it became a mere chore. I noticed that I didn’t quite care so much anymore about some of the discussion, and there was a decent amount of naval-gazing, meta-discussions and some wrangling about claims of authority that was probably useful and necessary, but wasn’t particularly fun.&lt;/p&gt;
  451. &lt;p&gt;I also began to notice weaknesses in the processes that I helped shape: We could really use some more automation for showing proposal statuses, notifying people when they have to act, and nudging them when they don’t. The whole silence-is-assent approach is good for throughput, but not necessary great for quality, and maybe the committee members need to be pushed more firmly to engage with each proposal. Like GHC itself, the committee processes deserve continuous refinement and refactoring, and since I could not muster the motivation to change my now well-trod secretarial ways, it was time for me to step down.&lt;/p&gt;
  452. &lt;p&gt;Luckily, Adam Gundry volunteered to take over, and that makes me feel much less bad for quitting. Thanks for that!&lt;/p&gt;
  453. &lt;p&gt;And although I am for my day job now &lt;a href="https://lean-lang.org/"&gt;enjoying a language&lt;/a&gt; that has many of the things out of the box that for Haskell are still only language extensions or even just future proposals (dependent types, &lt;code&gt;BlockArguments&lt;/code&gt;, &lt;code&gt;do&lt;/code&gt; notation with &lt;code&gt;(← foo)&lt;/code&gt; expressions and 💜 Unicode), I’m still around, hosting the &lt;a href="https://haskell.foundation/podcast/"&gt;Haskell Interlude Podcast&lt;/a&gt;, writing on this blog and hanging out at ZuriHac etc.&lt;/p&gt;</description>
  454.         <pubDate>Thu, 25 Jan 2024 01:21:41 +0100</pubDate>
  455.      </item>
  456.      <item>
  457.         <title>The Haskell Interlude Podcast</title>
  458.         <link>https://www.joachim-breitner.de/blog/810-The_Haskell_Interlude_Podcast</link>
  459.         <guid>https://www.joachim-breitner.de/blog/810-The_Haskell_Interlude_Podcast</guid>
  460.         <comments>https://www.joachim-breitner.de/blog/810-The_Haskell_Interlude_Podcast#comments</comments>
  461.         <author>mail@joachim-breitner.de (Joachim Breitner)</author>
  462.         <description>&lt;p&gt;It was pointed out to me that I have not blogged about this, so better now than never:&lt;/p&gt;
  463. &lt;p&gt;Since 2021 I am – together with four other hosts – producing a regular podcast about Haskell, the &lt;a href="https://haskell.foundation/podcast/"&gt;&lt;strong&gt;Haskell Interlude&lt;/strong&gt;&lt;/a&gt;. Roughly every two weeks two of us interview someone from the Haskell Community, and we chat for approximately an hour about how they came to Haskell, what they are doing with it, why they are doing it and what else is on their mind. Sometimes we talk to very famous people, like Simon Peyton Jones, and sometimes to people who maybe should be famous, but aren’t quite yet.&lt;/p&gt;
  464. &lt;p&gt;For most episodes we also have a transcript, so you can read the interviews instead, if you prefer, and you should find the podcast on most podcast apps as well. I do not know how reliable these statistics are, but supposedly we regularly have around 1300 listeners. We don’t get much feedback, however, so if you like the show, or dislike it, or have feedback, let us know (for example on the &lt;a href="https://discourse.haskell.org/"&gt;Haskell Disourse&lt;/a&gt;, which has a thread for each episode).&lt;/p&gt;
  465. &lt;p&gt;At the time of writing, we released 40 episodes. For the benefit of my (likely hypothetical) fans, or those who want to train an AI voice model for nefarious purposes, here is the list of episodes co-hosted by me:&lt;/p&gt;
  466. &lt;ul&gt;
  467. &lt;li&gt;&lt;a href="https://haskell.foundation/podcast/3"&gt;Gabriella Gonzales&lt;/a&gt;&lt;/li&gt;
  468. &lt;li&gt;&lt;a href="https://haskell.foundation/podcast/4"&gt;Jasper Van der Jeugt&lt;/a&gt;&lt;/li&gt;
  469. &lt;li&gt;&lt;a href="https://haskell.foundation/podcast/5"&gt;Chris Smith&lt;/a&gt;&lt;/li&gt;
  470. &lt;li&gt;&lt;a href="https://haskell.foundation/podcast/9"&gt;Sebastian Graf&lt;/a&gt;&lt;/li&gt;
  471. &lt;li&gt;&lt;a href="https://haskell.foundation/podcast/11"&gt;Simon Peyton Jones&lt;/a&gt;&lt;/li&gt;
  472. &lt;li&gt;&lt;a href="https://haskell.foundation/podcast/14"&gt;Ryan Trinkle&lt;/a&gt;&lt;/li&gt;
  473. &lt;li&gt;&lt;a href="https://haskell.foundation/podcast/15"&gt;Facundo Dominguez&lt;/a&gt;&lt;/li&gt;
  474. &lt;li&gt;&lt;a href="https://haskell.foundation/podcast/19"&gt;Marc Scholten&lt;/a&gt;&lt;/li&gt;
  475. &lt;li&gt;&lt;a href="https://haskell.foundation/podcast/23"&gt;Ben Gamari&lt;/a&gt;&lt;/li&gt;
  476. &lt;li&gt;&lt;a href="https://haskell.foundation/podcast/25"&gt;Andrew Lelechenko (Bodigrim)&lt;/a&gt;&lt;/li&gt;
  477. &lt;li&gt;&lt;a href="https://haskell.foundation/podcast/29"&gt;ZuriHac 2023 special&lt;/a&gt;&lt;/li&gt;
  478. &lt;li&gt;&lt;a href="https://haskell.foundation/podcast/31"&gt;Arnaud Spiwack&lt;/a&gt;&lt;/li&gt;
  479. &lt;li&gt;&lt;a href="https://haskell.foundation/podcast/37"&gt;John MacFarlane&lt;/a&gt;&lt;/li&gt;
  480. &lt;li&gt;&lt;a href="https://haskell.foundation/podcast/41"&gt;Moritz Angermann&lt;/a&gt;&lt;/li&gt;
  481. &lt;li&gt;&lt;a href="https://haskell.foundation/podcast/42"&gt;Jezen Thomas&lt;/a&gt;&lt;/li&gt;
  482. &lt;/ul&gt;
  483. &lt;p&gt;Can’t decide where to start? The one with Ryan Trinkle might be my favorite.&lt;/p&gt;
  484. &lt;p&gt;Thanks to the Haskell Foundation and its sponsors for supporting this podcast (hosting, editing, transscription).&lt;/p&gt;</description>
  485.         <pubDate>Fri, 22 Dec 2023 10:04:42 +0100</pubDate>
  486.      </item>
  487.      <item>
  488.         <title>Joining the Lean FRO</title>
  489.         <link>https://www.joachim-breitner.de/blog/809-Joining_the_Lean_FRO</link>
  490.         <guid>https://www.joachim-breitner.de/blog/809-Joining_the_Lean_FRO</guid>
  491.         <comments>https://www.joachim-breitner.de/blog/809-Joining_the_Lean_FRO#comments</comments>
  492.         <author>mail@joachim-breitner.de (Joachim Breitner)</author>
  493.         <description>&lt;p&gt;Tomorrow is going to be a new first day in a new job for me: I am joining the &lt;a href="https://lean-fro.org/"&gt;Lean FRO&lt;/a&gt;, and I’m excited.&lt;/p&gt;
  494. &lt;h3 id="what-is-lean"&gt;What is Lean?&lt;/h3&gt;
  495. &lt;p&gt;&lt;a href="https://lean-lang.org/about/"&gt;Lean&lt;/a&gt; is the new kid on the block of theorem provers.&lt;/p&gt;
  496. &lt;p&gt;It’s a pure functional programming language (like Haskell, with and on which I have worked a lot), but it’s dependently typed (which Haskell may be evolving to be as well, but rather slowly and carefully). It has a refreshing syntax, built on top of a rather good (I have been told, not an expert here) macro system.&lt;/p&gt;
  497. &lt;p&gt;As a dependently typed programming language, it is also a theorem prover, or proof assistant, and there exists already a lively community of mathematicians who started to formalize mathematics in a coherent library, creatively called &lt;a href="https://github.com/leanprover-community/mathlib4"&gt;mathlib&lt;/a&gt;.&lt;/p&gt;
  498. &lt;h3 id="what-is-a-fro"&gt;What is a FRO?&lt;/h3&gt;
  499. &lt;p&gt;A &lt;a href="https://www.convergentresearch.org/"&gt;Focused Research Organization&lt;/a&gt; has the organizational form of a small start up (small team, little overhead, a few years of runway), but its goals and measure for success are not commercial, as funding is provided by donors (in the case of the Lean FRO, the Simons Foundation International, the Alfred P. Sloan Foundation, and Richard Merkin). This allows us to build something that we believe is a contribution for the greater good, even though it’s not (or not yet) commercially interesting enough and does not fit other forms of funding (such as research grants) well. This is a very comfortable situation to be in.&lt;/p&gt;
  500. &lt;h3 id="why-am-i-excited"&gt;Why am I excited?&lt;/h3&gt;
  501. &lt;p&gt;To me, working on Lean seems to be the perfect mix: I have been working on language implementation for about a decade now, and always with a preference for functional languages. Add to that my interest in theorem proving, where I have used Isabelle and Coq so far, and played with Agda and others. So technically, clearly up my alley.&lt;/p&gt;
  502. &lt;p&gt;Furthermore, the language isn’t too old, and plenty of interesting things are simply still to do, rather than tried before. The ecosystem is still evolving, so there is a good chance to have some impact.&lt;/p&gt;
  503. &lt;p&gt;On the other hand, the language isn’t too young either. It is no longer an open question whether we will have users: we have them already, they hang out on &lt;a href="https://leanprover.zulipchat.com/"&gt;zulip&lt;/a&gt;, so if I improve something, there is likely someone going to be happy about it, which is great. And the community seems to be welcoming and full of nice people.&lt;/p&gt;
  504. &lt;p&gt;Finally, &lt;a href="https://github.com/leanprover-community/mathlib4"&gt;this library of mathematics&lt;/a&gt; that these users are building is itself an amazing artifact: Lots of math in a consistent, machine-readable, maintained, documented, checked form! With a little bit of optimism I can imagine this changing how math research and education will be done in the future. It could be for math what Wikipedia is for encyclopedic knowledge and OpenStreetMap for maps – and the thought of facilitating that excites me.&lt;/p&gt;
  505. &lt;p&gt;With this new job I find that when I am telling friends and colleagues about it, I do not hesitate or hedge when asked why I am doing this. This is a good sign.&lt;/p&gt;
  506. &lt;h3 id="what-will-i-be-doing"&gt;What will I be doing?&lt;/h3&gt;
  507. &lt;p&gt;We’ll see what main tasks I’ll get to tackle initially, but knowing myself, I expect I’ll get broadly involved.&lt;/p&gt;
  508. &lt;p&gt;To get up to speed I started playing around with a few things already, and for example created &lt;a href="https://loogle.lean-lang.org/"&gt;Loogle&lt;/a&gt;, a Mathlib search engine inspired by Haskell’s &lt;a href="https://hoogle.haskell.org/"&gt;Hoogle&lt;/a&gt;, including a Zulip bot integration. This seems to be useful and quite well received, so I’ll continue maintaining that.&lt;/p&gt;
  509. &lt;p&gt;Expect more about this and other contributions here in the future.&lt;/p&gt;</description>
  510.         <pubDate>Wed, 01 Nov 2023 21:47:06 +0100</pubDate>
  511.      </item>
  512.      <item>
  513.         <title>Squash your Github PRs with one click</title>
  514.         <link>https://www.joachim-breitner.de/blog/808-Squash_your_Github_PRs_with_one_click</link>
  515.         <guid>https://www.joachim-breitner.de/blog/808-Squash_your_Github_PRs_with_one_click</guid>
  516.         <comments>https://www.joachim-breitner.de/blog/808-Squash_your_Github_PRs_with_one_click#comments</comments>
  517.         <author>mail@joachim-breitner.de (Joachim Breitner)</author>
  518.         <description>&lt;p&gt;TL;DR: Squash your PRs with one click at &lt;a href="https://squasher.nomeata.de/" class="uri"&gt;https://squasher.nomeata.de/&lt;/a&gt;.&lt;/p&gt;
  519. &lt;p&gt;Very recently I got this response from the project maintainer at a pull request I contributed: “Thanks, approved, please squash so that I can merge.”&lt;/p&gt;
  520. &lt;p&gt;It’s nice that my contribution can go it, but why did the maintainer not just press the “Squash and merge button”, and instead adds the this unnecessary roundtrip to the process? Anyways, maintainers make the rules, so I play by them. But unlike the maintainer, who can squash-and-merge with just one click, squashing the PR’s branch is surprisingly laberous: Github does not allow you to do that via the Web UI (and hence on mobile), and it seems you are expected to go to your computer and juggle with &lt;code&gt;git rebase --interactive&lt;/code&gt;.&lt;/p&gt;
  521. &lt;p&gt;I found this rather annoying, so I created &lt;a href="https://squasher.nomeata.de/"&gt;&lt;strong&gt;Squasher&lt;/strong&gt;&lt;/a&gt;, a simple service that will squash your branch for you. There is no configuration, just paste the PR url. It will use the PR title and body as the commit message (which is obviously the right way™), and create the commit in your name:&lt;/p&gt;
  522. &lt;figure&gt;
  523. &lt;img src="/various/squasher.png" alt="Squasher in action"/&gt;
  524. &lt;figcaption aria-hidden="true"&gt;Squasher in action&lt;/figcaption&gt;
  525. &lt;/figure&gt;
  526. &lt;p&gt;If you find this useful, or found it to be buggy, let me know. The code is at &lt;a href="https://github.com/nomeata/squasher" class="uri"&gt;https://github.com/nomeata/squasher&lt;/a&gt; if you are curious about it.&lt;/p&gt;</description>
  527.         <pubDate>Sun, 29 Oct 2023 22:46:56 +0100</pubDate>
  528.      </item>
  529.      <item>
  530.         <title>Left recursive parser combinators via sharing</title>
  531.         <link>https://www.joachim-breitner.de/blog/807-Left_recursive_parser_combinators_via_sharing</link>
  532.         <guid>https://www.joachim-breitner.de/blog/807-Left_recursive_parser_combinators_via_sharing</guid>
  533.         <comments>https://www.joachim-breitner.de/blog/807-Left_recursive_parser_combinators_via_sharing#comments</comments>
  534.         <author>mail@joachim-breitner.de (Joachim Breitner)</author>
  535.         <description>&lt;p&gt;At this year’s &lt;a href="https://icfp23.sigplan.org/"&gt;ICFP in Seattle&lt;/a&gt; I gave a talk about my &lt;a href="https://hackage.haskell.org/package/rec-def"&gt;rec-def&lt;/a&gt; Haskell library, which I have blogged about before here. While my &lt;a href="https://doi.org/10.1145/3607853"&gt;functional pearl paper&lt;/a&gt; focuses on a concrete use-case and the tricks of the implementation, in my talk I put the emphasis on the high-level idea: it beholds of a declarative lazy functional like Haskell that recursive equations just work whenever they describe a (unique) solution. Like in the paper, I used equations between sets as the running example, and only conjectured that it should also work for other domains, in particular parser combinators.&lt;/p&gt;
  536. &lt;p&gt;Naturally, someone called my bluff and asked if I actually tried it. I had not, but I should have, because it works nicely and is actually more straight-forward than with sets. I wrote up a prototype and showed it off a few days later as a lightning talk at &lt;a href="https://icfp23.sigplan.org/home/haskellsymp-2023"&gt;Haskell Symposium&lt;/a&gt;; here is the write up that goes along with that.&lt;/p&gt;
  537. &lt;h3 id="parser-combinators"&gt;Parser combinators&lt;/h3&gt;
  538. &lt;p&gt;Parser combinators are libraries that provide little functions (combinators) that you compose to define your parser directly in your programming language, as opposed to using external tools that read some grammar description and generate parser code, and are quite popular in Haskell (e.g. &lt;a href="https://hackage.haskell.org/package/parsec"&gt;parsec&lt;/a&gt;, &lt;a href="https://hackage.haskell.org/package/attoparsec"&gt;attoparsec&lt;/a&gt;, &lt;a href="https://hackage.haskell.org/package/megaparsec"&gt;megaparsec&lt;/a&gt;).&lt;/p&gt;
  539. &lt;p&gt;Let us define a little parser that recognizes sequences of &lt;code&gt;a&lt;/code&gt;s:&lt;/p&gt;
  540. &lt;div class="sourceCode" id="cb1"&gt;&lt;pre class="sourceCode haskell"&gt;&lt;code class="sourceCode haskell"&gt;&lt;span id="cb1-1"&gt;&lt;a href="#cb1-1" aria-hidden="true" tabindex="-1"/&gt;ghci&lt;span class="op"&gt;&gt;&lt;/span&gt; &lt;span class="kw"&gt;let&lt;/span&gt; aaa &lt;span class="ot"&gt;=&lt;/span&gt; tok &lt;span class="ch"&gt;'a'&lt;/span&gt; &lt;span class="op"&gt;*&gt;&lt;/span&gt; aaa &lt;span class="op"&gt;&amp;lt;|&gt;&lt;/span&gt; &lt;span class="fu"&gt;pure&lt;/span&gt; ()&lt;/span&gt;
  541. &lt;span id="cb1-2"&gt;&lt;a href="#cb1-2" aria-hidden="true" tabindex="-1"/&gt;ghci&lt;span class="op"&gt;&gt;&lt;/span&gt; parse aaa &lt;span class="st"&gt;"aaaa"&lt;/span&gt;&lt;/span&gt;
  542. &lt;span id="cb1-3"&gt;&lt;a href="#cb1-3" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="dt"&gt;Just&lt;/span&gt; ()&lt;/span&gt;
  543. &lt;span id="cb1-4"&gt;&lt;a href="#cb1-4" aria-hidden="true" tabindex="-1"/&gt;ghci&lt;span class="op"&gt;&gt;&lt;/span&gt; parse aaa &lt;span class="st"&gt;"aabaa"&lt;/span&gt;&lt;/span&gt;
  544. &lt;span id="cb1-5"&gt;&lt;a href="#cb1-5" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="dt"&gt;Nothing&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  545. &lt;h3 id="left-recursion"&gt;Left-recursion&lt;/h3&gt;
  546. &lt;p&gt;This works nicely, but just because we were lucky: We wrote the parser to recurse on the right (of the &lt;code&gt;*&gt;&lt;/code&gt;), and this happens to work. If we put the recursive call first, it doesn’t anymore:&lt;/p&gt;
  547. &lt;div class="sourceCode" id="cb2"&gt;&lt;pre class="sourceCode haskell"&gt;&lt;code class="sourceCode haskell"&gt;&lt;span id="cb2-1"&gt;&lt;a href="#cb2-1" aria-hidden="true" tabindex="-1"/&gt;ghci&lt;span class="op"&gt;&gt;&lt;/span&gt; &lt;span class="kw"&gt;let&lt;/span&gt; aaa &lt;span class="ot"&gt;=&lt;/span&gt; aaa &lt;span class="op"&gt;&amp;lt;*&lt;/span&gt; tok &lt;span class="ch"&gt;'a'&lt;/span&gt; &lt;span class="op"&gt;&amp;lt;|&gt;&lt;/span&gt; &lt;span class="fu"&gt;pure&lt;/span&gt; ()&lt;/span&gt;
  548. &lt;span id="cb2-2"&gt;&lt;a href="#cb2-2" aria-hidden="true" tabindex="-1"/&gt;ghci&lt;span class="op"&gt;&gt;&lt;/span&gt; parse aaa &lt;span class="st"&gt;"aaaa"&lt;/span&gt;&lt;/span&gt;
  549. &lt;span id="cb2-3"&gt;&lt;a href="#cb2-3" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="op"&gt;^&lt;/span&gt;&lt;span class="dt"&gt;CInterrupted&lt;/span&gt;&lt;span class="op"&gt;.&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  550. &lt;p&gt;This is a well-known problem (see for example &lt;a href="https://doi.org/10.1145/3471874.3472984"&gt;Nicolas Wu’s overview paper&lt;/a&gt;), all the common parser combinator libraries cannot handle it and the usual advise is to refactor your grammar to avoid left recursion.&lt;/p&gt;
  551. &lt;p&gt;But there are some libraries that can handle left recursion, at least with a little help from the programmer. I found two variations:&lt;/p&gt;
  552. &lt;ul&gt;
  553. &lt;li&gt;&lt;p&gt;The library provides an explicit fix point combinator, and as long as that is used, left-recursion works. This is for example described by &lt;a href="https://link.springer.com/chapter/10.1007/978-3-540-77442-6_12"&gt;Frost, Hafiz and Callaghan&lt;/a&gt; by, and (of course) &lt;a href="https://okmij.org/ftp/Haskell/LeftRecursion.hs"&gt;Oleg Kiselyov&lt;/a&gt; has an implementation of this too.&lt;/p&gt;&lt;/li&gt;
  554. &lt;li&gt;&lt;p&gt;The library expects explicit labels on recursive productions, so that the library can recognize left-recursion. I found an implementation of this idea in the &lt;a href="https://hackage.haskell.org/package/Agda-2.6.3/docs/Agda-Utils-Parser-MemoisedCPS.html"&gt;&lt;code&gt;Agda.Utils.Parser.MemoisedCPS&lt;/code&gt;&lt;/a&gt; module in the Agda code, the &lt;a href="https://hackage.haskell.org/package/gll-0.4.1.0/docs/GLL-Combinators-Interface.html"&gt;&lt;code&gt;gll&lt;/code&gt; library&lt;/a&gt; seems to follow this style and &lt;a href="https://discourse.haskell.org/t/reusing-haskells-binding-when-defining-context-free-grammars/5960?u=nomeata"&gt;Jaro discusses it as well&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;
  555. &lt;/ul&gt;
  556. &lt;p&gt;I took the module from the Agda source and simplified a bit for the purposes of this demonstration (&lt;a href="https://github.com/nomeata/left-rec-parse/blob/master/Parser.hs"&gt;&lt;code&gt;Parser.hs&lt;/code&gt;&lt;/a&gt;). Indeed, I can make the left-recursive grammar work:&lt;/p&gt;
  557. &lt;div class="sourceCode" id="cb3"&gt;&lt;pre class="sourceCode haskell"&gt;&lt;code class="sourceCode haskell"&gt;&lt;span id="cb3-1"&gt;&lt;a href="#cb3-1" aria-hidden="true" tabindex="-1"/&gt;ghci&lt;span class="op"&gt;&gt;&lt;/span&gt; &lt;span class="kw"&gt;let&lt;/span&gt; aaa &lt;span class="ot"&gt;=&lt;/span&gt; memoise &lt;span class="st"&gt;":-)"&lt;/span&gt; &lt;span class="op"&gt;$&lt;/span&gt; aaa &lt;span class="op"&gt;&amp;lt;*&lt;/span&gt; tok &lt;span class="ch"&gt;'a'&lt;/span&gt; &lt;span class="op"&gt;&amp;lt;|&gt;&lt;/span&gt; &lt;span class="fu"&gt;pure&lt;/span&gt; ()&lt;/span&gt;
  558. &lt;span id="cb3-2"&gt;&lt;a href="#cb3-2" aria-hidden="true" tabindex="-1"/&gt;ghci&lt;span class="op"&gt;&gt;&lt;/span&gt; parse aaa &lt;span class="st"&gt;"aaaa"&lt;/span&gt;&lt;/span&gt;
  559. &lt;span id="cb3-3"&gt;&lt;a href="#cb3-3" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="dt"&gt;Just&lt;/span&gt; ()&lt;/span&gt;
  560. &lt;span id="cb3-4"&gt;&lt;a href="#cb3-4" aria-hidden="true" tabindex="-1"/&gt;ghci&lt;span class="op"&gt;&gt;&lt;/span&gt; parse aaa &lt;span class="st"&gt;"aabaa"&lt;/span&gt;&lt;/span&gt;
  561. &lt;span id="cb3-5"&gt;&lt;a href="#cb3-5" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="dt"&gt;Nothing&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  562. &lt;p&gt;It does not matter what I pass to &lt;code&gt;memoise&lt;/code&gt;, as long as I do not pass the same key when memoising a different parser.&lt;/p&gt;
  563. &lt;p&gt;For reference, an excerpt of the the API of &lt;code&gt;Parser&lt;/code&gt;:&lt;/p&gt;
  564. &lt;div class="sourceCode" id="cb4"&gt;&lt;pre class="sourceCode haskell"&gt;&lt;code class="sourceCode haskell"&gt;&lt;span id="cb4-1"&gt;&lt;a href="#cb4-1" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="kw"&gt;data&lt;/span&gt; &lt;span class="dt"&gt;Parser&lt;/span&gt; k tok a &lt;span class="co"&gt;-- k is type of keys, tok type of tokens (e.g. Char)&lt;/span&gt;&lt;/span&gt;
  565. &lt;span id="cb4-2"&gt;&lt;a href="#cb4-2" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="kw"&gt;instance&lt;/span&gt; &lt;span class="dt"&gt;Functor&lt;/span&gt; (&lt;span class="dt"&gt;Parser&lt;/span&gt; k tok)&lt;/span&gt;
  566. &lt;span id="cb4-3"&gt;&lt;a href="#cb4-3" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="kw"&gt;instance&lt;/span&gt; &lt;span class="dt"&gt;Applicative&lt;/span&gt; (&lt;span class="dt"&gt;Parser&lt;/span&gt; k tok)&lt;/span&gt;
  567. &lt;span id="cb4-4"&gt;&lt;a href="#cb4-4" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="kw"&gt;instance&lt;/span&gt; &lt;span class="dt"&gt;Alternative&lt;/span&gt; (&lt;span class="dt"&gt;Parser&lt;/span&gt; k tok)&lt;/span&gt;
  568. &lt;span id="cb4-5"&gt;&lt;a href="#cb4-5" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="kw"&gt;instance&lt;/span&gt; &lt;span class="dt"&gt;Monad&lt;/span&gt; (&lt;span class="dt"&gt;Parser&lt;/span&gt; k tok)&lt;/span&gt;
  569. &lt;span id="cb4-6"&gt;&lt;a href="#cb4-6" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;parse ::&lt;/span&gt; &lt;span class="dt"&gt;Parser&lt;/span&gt; k tok a &lt;span class="ot"&gt;-&gt;&lt;/span&gt; [tok] &lt;span class="ot"&gt;-&gt;&lt;/span&gt; &lt;span class="dt"&gt;Maybe&lt;/span&gt; a&lt;/span&gt;
  570. &lt;span id="cb4-7"&gt;&lt;a href="#cb4-7" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;sat ::&lt;/span&gt; (tok &lt;span class="ot"&gt;-&gt;&lt;/span&gt; &lt;span class="dt"&gt;Bool&lt;/span&gt;) &lt;span class="ot"&gt;-&gt;&lt;/span&gt; &lt;span class="dt"&gt;Parser&lt;/span&gt; k tok tok&lt;/span&gt;
  571. &lt;span id="cb4-8"&gt;&lt;a href="#cb4-8" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;tok ::&lt;/span&gt; &lt;span class="dt"&gt;Eq&lt;/span&gt; tok &lt;span class="ot"&gt;=&gt;&lt;/span&gt; tok &lt;span class="ot"&gt;-&gt;&lt;/span&gt; &lt;span class="dt"&gt;Parser&lt;/span&gt; k tok tok&lt;/span&gt;
  572. &lt;span id="cb4-9"&gt;&lt;a href="#cb4-9" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;memoise ::&lt;/span&gt; &lt;span class="dt"&gt;Ord&lt;/span&gt; k &lt;span class="ot"&gt;=&gt;&lt;/span&gt; k &lt;span class="ot"&gt;-&gt;&lt;/span&gt; &lt;span class="dt"&gt;Parser&lt;/span&gt; k tok a &lt;span class="ot"&gt;-&gt;&lt;/span&gt; &lt;span class="dt"&gt;Parser&lt;/span&gt; k tok a&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  573. &lt;h3 id="left-recursion-through-sharing"&gt;Left-recursion through sharing&lt;/h3&gt;
  574. &lt;p&gt;To follow the agenda set out in my talk, I now want to wrap that parser in a way that relieves me from having to insert the calls to &lt;code&gt;memoise&lt;/code&gt;. To start, I import that parser qualified, define a newtype around it, and start lifting some of the functions:&lt;/p&gt;
  575. &lt;div class="sourceCode" id="cb5"&gt;&lt;pre class="sourceCode haskell"&gt;&lt;code class="sourceCode haskell"&gt;&lt;span id="cb5-1"&gt;&lt;a href="#cb5-1" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="kw"&gt;import&lt;/span&gt; &lt;span class="kw"&gt;qualified&lt;/span&gt; &lt;span class="dt"&gt;Parser&lt;/span&gt; &lt;span class="kw"&gt;as&lt;/span&gt; &lt;span class="dt"&gt;P&lt;/span&gt;&lt;/span&gt;
  576. &lt;span id="cb5-2"&gt;&lt;a href="#cb5-2" aria-hidden="true" tabindex="-1"/&gt;&lt;/span&gt;
  577. &lt;span id="cb5-3"&gt;&lt;a href="#cb5-3" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="kw"&gt;newtype&lt;/span&gt; &lt;span class="dt"&gt;Parser&lt;/span&gt; tok a &lt;span class="ot"&gt;=&lt;/span&gt; &lt;span class="dt"&gt;MkP&lt;/span&gt; {&lt;span class="ot"&gt; unP ::&lt;/span&gt; &lt;span class="dt"&gt;P.Parser&lt;/span&gt; &lt;span class="dt"&gt;Unique&lt;/span&gt; tok a }&lt;/span&gt;
  578. &lt;span id="cb5-4"&gt;&lt;a href="#cb5-4" aria-hidden="true" tabindex="-1"/&gt;&lt;/span&gt;
  579. &lt;span id="cb5-5"&gt;&lt;a href="#cb5-5" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;parse ::&lt;/span&gt; &lt;span class="dt"&gt;Parser&lt;/span&gt; tok a &lt;span class="ot"&gt;-&gt;&lt;/span&gt; [tok] &lt;span class="ot"&gt;-&gt;&lt;/span&gt; &lt;span class="dt"&gt;Maybe&lt;/span&gt; a&lt;/span&gt;
  580. &lt;span id="cb5-6"&gt;&lt;a href="#cb5-6" aria-hidden="true" tabindex="-1"/&gt;parses (&lt;span class="dt"&gt;MkP&lt;/span&gt; p) &lt;span class="ot"&gt;=&lt;/span&gt; P.parse p&lt;/span&gt;
  581. &lt;span id="cb5-7"&gt;&lt;a href="#cb5-7" aria-hidden="true" tabindex="-1"/&gt;&lt;/span&gt;
  582. &lt;span id="cb5-8"&gt;&lt;a href="#cb5-8" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;sat ::&lt;/span&gt; &lt;span class="dt"&gt;Typeable&lt;/span&gt; tok &lt;span class="ot"&gt;=&gt;&lt;/span&gt; (tok &lt;span class="ot"&gt;-&gt;&lt;/span&gt; &lt;span class="dt"&gt;Bool&lt;/span&gt;) &lt;span class="ot"&gt;-&gt;&lt;/span&gt; &lt;span class="dt"&gt;Parser&lt;/span&gt; tok tok&lt;/span&gt;
  583. &lt;span id="cb5-9"&gt;&lt;a href="#cb5-9" aria-hidden="true" tabindex="-1"/&gt;sat p &lt;span class="ot"&gt;=&lt;/span&gt; &lt;span class="dt"&gt;MkP&lt;/span&gt; (P.sat p)&lt;/span&gt;
  584. &lt;span id="cb5-10"&gt;&lt;a href="#cb5-10" aria-hidden="true" tabindex="-1"/&gt;&lt;/span&gt;
  585. &lt;span id="cb5-11"&gt;&lt;a href="#cb5-11" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;tok ::&lt;/span&gt; &lt;span class="dt"&gt;Eq&lt;/span&gt; tok &lt;span class="ot"&gt;=&gt;&lt;/span&gt; tok &lt;span class="ot"&gt;-&gt;&lt;/span&gt; &lt;span class="dt"&gt;Parser&lt;/span&gt; tok tok&lt;/span&gt;
  586. &lt;span id="cb5-12"&gt;&lt;a href="#cb5-12" aria-hidden="true" tabindex="-1"/&gt;tok t &lt;span class="ot"&gt;=&lt;/span&gt; &lt;span class="dt"&gt;MkP&lt;/span&gt; (P.tok t)&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  587. &lt;p&gt;So far, nothing interesting had to happen, because so far I cannot build recursive parsers. The first interesting combinator that allows me to do that is &lt;code&gt;&amp;lt;*&gt;&lt;/code&gt; from the &lt;code&gt;Applicative&lt;/code&gt; class, so I should use &lt;code&gt;memoise&lt;/code&gt; there. The question is: Where does the unique key come from?&lt;/p&gt;
  588. &lt;h3 id="proprioception"&gt;Proprioception&lt;/h3&gt;
  589. &lt;p&gt;As with the rec-def library, pure code won’t do, and I have to get my hands dirty: I really want a fresh unique label out of thin air. To that end, I define the following combinator, with naming aided by Richard Eisenberg:&lt;/p&gt;
  590. &lt;div class="sourceCode" id="cb6"&gt;&lt;pre class="sourceCode haskell"&gt;&lt;code class="sourceCode haskell"&gt;&lt;span id="cb6-1"&gt;&lt;a href="#cb6-1" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;propriocept ::&lt;/span&gt; (&lt;span class="dt"&gt;Unique&lt;/span&gt; &lt;span class="ot"&gt;-&gt;&lt;/span&gt; a) &lt;span class="ot"&gt;-&gt;&lt;/span&gt; a&lt;/span&gt;
  591. &lt;span id="cb6-2"&gt;&lt;a href="#cb6-2" aria-hidden="true" tabindex="-1"/&gt;propriocept f &lt;span class="ot"&gt;=&lt;/span&gt; unsafePerformIO &lt;span class="op"&gt;$&lt;/span&gt; f &lt;span class="op"&gt;&amp;lt;$&gt;&lt;/span&gt; newUnique&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  592. &lt;p&gt;A thunk defined with &lt;code&gt;propriocept&lt;/code&gt; will know about it’s own identity, and will be able to tell itself apart from other such thunks. This gives us a form of observable sharing, precisely what we need. But before we return to our parser combinators, let us briefly explore this combinator.&lt;/p&gt;
  593. &lt;p&gt;Using &lt;code&gt;propriocept&lt;/code&gt; I can define an operation &lt;code&gt;cons :: [Int] -&gt; [Int]&lt;/code&gt; that records (the hash of) this &lt;code&gt;Unique&lt;/code&gt; in the list:&lt;/p&gt;
  594. &lt;div class="sourceCode" id="cb7"&gt;&lt;pre class="sourceCode haskell"&gt;&lt;code class="sourceCode haskell"&gt;&lt;span id="cb7-1"&gt;&lt;a href="#cb7-1" aria-hidden="true" tabindex="-1"/&gt;ghci&lt;span class="op"&gt;&gt;&lt;/span&gt; &lt;span class="kw"&gt;let&lt;/span&gt; cons xs &lt;span class="ot"&gt;=&lt;/span&gt; propriocept (\x &lt;span class="ot"&gt;-&gt;&lt;/span&gt; hashUnique x &lt;span class="op"&gt;:&lt;/span&gt; xs)&lt;/span&gt;
  595. &lt;span id="cb7-2"&gt;&lt;a href="#cb7-2" aria-hidden="true" tabindex="-1"/&gt;ghci&lt;span class="op"&gt;&gt;&lt;/span&gt; &lt;span class="op"&gt;:&lt;/span&gt;t cons&lt;/span&gt;
  596. &lt;span id="cb7-3"&gt;&lt;a href="#cb7-3" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;cons ::&lt;/span&gt; [&lt;span class="dt"&gt;Int&lt;/span&gt;] &lt;span class="ot"&gt;-&gt;&lt;/span&gt; [&lt;span class="dt"&gt;Int&lt;/span&gt;]&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  597. &lt;p&gt;This lets us see the identity of a list cell, that is, of the concrete object in memory.&lt;/p&gt;
  598. &lt;p&gt;Naturally, if we construct a finite list, each list cell is different:&lt;/p&gt;
  599. &lt;div class="sourceCode" id="cb8"&gt;&lt;pre class="sourceCode haskell"&gt;&lt;code class="sourceCode haskell"&gt;&lt;span id="cb8-1"&gt;&lt;a href="#cb8-1" aria-hidden="true" tabindex="-1"/&gt;ghci&lt;span class="op"&gt;&gt;&lt;/span&gt; cons (cons (cons []))&lt;/span&gt;
  600. &lt;span id="cb8-2"&gt;&lt;a href="#cb8-2" aria-hidden="true" tabindex="-1"/&gt;[&lt;span class="dv"&gt;1&lt;/span&gt;,&lt;span class="dv"&gt;2&lt;/span&gt;,&lt;span class="dv"&gt;3&lt;/span&gt;]&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  601. &lt;p&gt;And if we do that again, we see that fresh list cells are allocated:&lt;/p&gt;
  602. &lt;div class="sourceCode" id="cb9"&gt;&lt;pre class="sourceCode haskell"&gt;&lt;code class="sourceCode haskell"&gt;&lt;span id="cb9-1"&gt;&lt;a href="#cb9-1" aria-hidden="true" tabindex="-1"/&gt;ghci&lt;span class="op"&gt;&gt;&lt;/span&gt; cons (cons (cons []))&lt;/span&gt;
  603. &lt;span id="cb9-2"&gt;&lt;a href="#cb9-2" aria-hidden="true" tabindex="-1"/&gt;[&lt;span class="dv"&gt;4&lt;/span&gt;,&lt;span class="dv"&gt;5&lt;/span&gt;,&lt;span class="dv"&gt;6&lt;/span&gt;]&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  604. &lt;p&gt;We can create an infinite list; if we do it without sharing, every cell is separate:&lt;/p&gt;
  605. &lt;div class="sourceCode" id="cb10"&gt;&lt;pre class="sourceCode haskell"&gt;&lt;code class="sourceCode haskell"&gt;&lt;span id="cb10-1"&gt;&lt;a href="#cb10-1" aria-hidden="true" tabindex="-1"/&gt;ghci&lt;span class="op"&gt;&gt;&lt;/span&gt; &lt;span class="fu"&gt;take&lt;/span&gt; &lt;span class="dv"&gt;20&lt;/span&gt; (acyclic &lt;span class="dv"&gt;0&lt;/span&gt;)&lt;/span&gt;
  606. &lt;span id="cb10-2"&gt;&lt;a href="#cb10-2" aria-hidden="true" tabindex="-1"/&gt;[&lt;span class="dv"&gt;7&lt;/span&gt;,&lt;span class="dv"&gt;8&lt;/span&gt;,&lt;span class="dv"&gt;9&lt;/span&gt;,&lt;span class="dv"&gt;10&lt;/span&gt;,&lt;span class="dv"&gt;11&lt;/span&gt;,&lt;span class="dv"&gt;12&lt;/span&gt;,&lt;span class="dv"&gt;13&lt;/span&gt;,&lt;span class="dv"&gt;14&lt;/span&gt;,&lt;span class="dv"&gt;15&lt;/span&gt;,&lt;span class="dv"&gt;16&lt;/span&gt;,&lt;span class="dv"&gt;17&lt;/span&gt;,&lt;span class="dv"&gt;18&lt;/span&gt;,&lt;span class="dv"&gt;19&lt;/span&gt;,&lt;span class="dv"&gt;20&lt;/span&gt;,&lt;span class="dv"&gt;21&lt;/span&gt;,&lt;span class="dv"&gt;22&lt;/span&gt;,&lt;span class="dv"&gt;23&lt;/span&gt;,&lt;span class="dv"&gt;24&lt;/span&gt;,&lt;span class="dv"&gt;25&lt;/span&gt;,&lt;span class="dv"&gt;26&lt;/span&gt;]&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  607. &lt;p&gt;but if we tie the knot using sharing, all the cells in the list are actually the same:&lt;/p&gt;
  608. &lt;div class="sourceCode" id="cb11"&gt;&lt;pre class="sourceCode haskell"&gt;&lt;code class="sourceCode haskell"&gt;&lt;span id="cb11-1"&gt;&lt;a href="#cb11-1" aria-hidden="true" tabindex="-1"/&gt;ghci&lt;span class="op"&gt;&gt;&lt;/span&gt; &lt;span class="kw"&gt;let&lt;/span&gt; cyclic &lt;span class="ot"&gt;=&lt;/span&gt; cons cyclic&lt;/span&gt;
  609. &lt;span id="cb11-2"&gt;&lt;a href="#cb11-2" aria-hidden="true" tabindex="-1"/&gt;ghci&lt;span class="op"&gt;&gt;&lt;/span&gt; &lt;span class="fu"&gt;take&lt;/span&gt; &lt;span class="dv"&gt;20&lt;/span&gt; cyclic&lt;/span&gt;
  610. &lt;span id="cb11-3"&gt;&lt;a href="#cb11-3" aria-hidden="true" tabindex="-1"/&gt;[&lt;span class="dv"&gt;27&lt;/span&gt;,&lt;span class="dv"&gt;27&lt;/span&gt;,&lt;span class="dv"&gt;27&lt;/span&gt;,&lt;span class="dv"&gt;27&lt;/span&gt;,&lt;span class="dv"&gt;27&lt;/span&gt;,&lt;span class="dv"&gt;27&lt;/span&gt;,&lt;span class="dv"&gt;27&lt;/span&gt;,&lt;span class="dv"&gt;27&lt;/span&gt;,&lt;span class="dv"&gt;27&lt;/span&gt;,&lt;span class="dv"&gt;27&lt;/span&gt;,&lt;span class="dv"&gt;27&lt;/span&gt;,&lt;span class="dv"&gt;27&lt;/span&gt;,&lt;span class="dv"&gt;27&lt;/span&gt;,&lt;span class="dv"&gt;27&lt;/span&gt;,&lt;span class="dv"&gt;27&lt;/span&gt;,&lt;span class="dv"&gt;27&lt;/span&gt;,&lt;span class="dv"&gt;27&lt;/span&gt;,&lt;span class="dv"&gt;27&lt;/span&gt;,&lt;span class="dv"&gt;27&lt;/span&gt;,&lt;span class="dv"&gt;27&lt;/span&gt;]&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  611. &lt;p&gt;We can achieve the same using &lt;a href="https://hackage.haskell.org/package/base/docs/Data-Function.html#v:fix"&gt;&lt;code&gt;fix&lt;/code&gt; from &lt;code&gt;Data.Function&lt;/code&gt;&lt;/a&gt;:&lt;/p&gt;
  612. &lt;div class="sourceCode" id="cb12"&gt;&lt;pre class="sourceCode haskell"&gt;&lt;code class="sourceCode haskell"&gt;&lt;span id="cb12-1"&gt;&lt;a href="#cb12-1" aria-hidden="true" tabindex="-1"/&gt;ghci&lt;span class="op"&gt;&gt;&lt;/span&gt; &lt;span class="kw"&gt;import&lt;/span&gt; &lt;span class="dt"&gt;Data.Function&lt;/span&gt;&lt;/span&gt;
  613. &lt;span id="cb12-2"&gt;&lt;a href="#cb12-2" aria-hidden="true" tabindex="-1"/&gt;ghci&lt;span class="op"&gt;&gt;&lt;/span&gt; &lt;span class="fu"&gt;take&lt;/span&gt; &lt;span class="dv"&gt;20&lt;/span&gt; (fix cons)&lt;/span&gt;
  614. &lt;span id="cb12-3"&gt;&lt;a href="#cb12-3" aria-hidden="true" tabindex="-1"/&gt;[&lt;span class="dv"&gt;28&lt;/span&gt;,&lt;span class="dv"&gt;28&lt;/span&gt;,&lt;span class="dv"&gt;28&lt;/span&gt;,&lt;span class="dv"&gt;28&lt;/span&gt;,&lt;span class="dv"&gt;28&lt;/span&gt;,&lt;span class="dv"&gt;28&lt;/span&gt;,&lt;span class="dv"&gt;28&lt;/span&gt;,&lt;span class="dv"&gt;28&lt;/span&gt;,&lt;span class="dv"&gt;28&lt;/span&gt;,&lt;span class="dv"&gt;28&lt;/span&gt;,&lt;span class="dv"&gt;28&lt;/span&gt;,&lt;span class="dv"&gt;28&lt;/span&gt;,&lt;span class="dv"&gt;28&lt;/span&gt;,&lt;span class="dv"&gt;28&lt;/span&gt;,&lt;span class="dv"&gt;28&lt;/span&gt;,&lt;span class="dv"&gt;28&lt;/span&gt;,&lt;span class="dv"&gt;28&lt;/span&gt;,&lt;span class="dv"&gt;28&lt;/span&gt;,&lt;span class="dv"&gt;28&lt;/span&gt;,&lt;span class="dv"&gt;28&lt;/span&gt;]&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  615. &lt;p&gt;I explore these heap structures more visually in a &lt;a href="https://www.youtube.com/playlist?list=PL4FcLyLhO9jggmkqJyJ2i9pCSiDpwKiVu"&gt;series of screencasts&lt;/a&gt;.&lt;/p&gt;
  616. &lt;p&gt;So with &lt;code&gt;propriocept&lt;/code&gt; we can distinguish different heap objects, and also recognize when we come across the same heap object again.&lt;/p&gt;
  617. &lt;h3 id="left-recursion-through-sharing-cont."&gt;Left-recursion through sharing (cont.)&lt;/h3&gt;
  618. &lt;p&gt;With that we return to our parser. We define a smart constructor for the new &lt;code&gt;Parser&lt;/code&gt; that passes the unique from &lt;code&gt;propriocept&lt;/code&gt; to the underlying parser’s &lt;code&gt;memoise&lt;/code&gt; function:&lt;/p&gt;
  619. &lt;div class="sourceCode" id="cb13"&gt;&lt;pre class="sourceCode haskell"&gt;&lt;code class="sourceCode haskell"&gt;&lt;span id="cb13-1"&gt;&lt;a href="#cb13-1" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;withMemo ::&lt;/span&gt; &lt;span class="dt"&gt;P.Parser&lt;/span&gt; &lt;span class="dt"&gt;Unique&lt;/span&gt; tok a &lt;span class="ot"&gt;-&gt;&lt;/span&gt; &lt;span class="dt"&gt;Parser&lt;/span&gt; tok a&lt;/span&gt;
  620. &lt;span id="cb13-2"&gt;&lt;a href="#cb13-2" aria-hidden="true" tabindex="-1"/&gt;withMemo p &lt;span class="ot"&gt;=&lt;/span&gt; propriocept &lt;span class="op"&gt;$&lt;/span&gt; \u &lt;span class="ot"&gt;-&gt;&lt;/span&gt; &lt;span class="dt"&gt;MkP&lt;/span&gt; &lt;span class="op"&gt;$&lt;/span&gt; P.memoise u p&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  621. &lt;p&gt;If we now use this in the definition of all possibly recursive parsers, then the necessary calls to &lt;code&gt;memoise&lt;/code&gt; will be in place:&lt;/p&gt;
  622. &lt;div class="sourceCode" id="cb14"&gt;&lt;pre class="sourceCode haskell"&gt;&lt;code class="sourceCode haskell"&gt;&lt;span id="cb14-1"&gt;&lt;a href="#cb14-1" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="kw"&gt;instance&lt;/span&gt; &lt;span class="dt"&gt;Functor&lt;/span&gt; (&lt;span class="dt"&gt;Parser&lt;/span&gt; tok) &lt;span class="kw"&gt;where&lt;/span&gt;&lt;/span&gt;
  623. &lt;span id="cb14-2"&gt;&lt;a href="#cb14-2" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="fu"&gt;fmap&lt;/span&gt; f p &lt;span class="ot"&gt;=&lt;/span&gt; withMemo (&lt;span class="fu"&gt;fmap&lt;/span&gt; f (unP p))&lt;/span&gt;
  624. &lt;span id="cb14-3"&gt;&lt;a href="#cb14-3" aria-hidden="true" tabindex="-1"/&gt;&lt;/span&gt;
  625. &lt;span id="cb14-4"&gt;&lt;a href="#cb14-4" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="kw"&gt;instance&lt;/span&gt; &lt;span class="dt"&gt;Applicative&lt;/span&gt; (&lt;span class="dt"&gt;Parser&lt;/span&gt; tok) &lt;span class="kw"&gt;where&lt;/span&gt;&lt;/span&gt;
  626. &lt;span id="cb14-5"&gt;&lt;a href="#cb14-5" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="fu"&gt;pure&lt;/span&gt; x &lt;span class="ot"&gt;=&lt;/span&gt; &lt;span class="dt"&gt;MkP&lt;/span&gt; (&lt;span class="fu"&gt;pure&lt;/span&gt; x)&lt;/span&gt;
  627. &lt;span id="cb14-6"&gt;&lt;a href="#cb14-6" aria-hidden="true" tabindex="-1"/&gt;  p1 &lt;span class="op"&gt;&amp;lt;*&gt;&lt;/span&gt; p2 &lt;span class="ot"&gt;=&lt;/span&gt; withMemo (unP p1 &lt;span class="op"&gt;&amp;lt;*&gt;&lt;/span&gt; unP p2)&lt;/span&gt;
  628. &lt;span id="cb14-7"&gt;&lt;a href="#cb14-7" aria-hidden="true" tabindex="-1"/&gt;&lt;/span&gt;
  629. &lt;span id="cb14-8"&gt;&lt;a href="#cb14-8" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="kw"&gt;instance&lt;/span&gt; &lt;span class="dt"&gt;Alternative&lt;/span&gt; (&lt;span class="dt"&gt;Parser&lt;/span&gt; tok) &lt;span class="kw"&gt;where&lt;/span&gt;&lt;/span&gt;
  630. &lt;span id="cb14-9"&gt;&lt;a href="#cb14-9" aria-hidden="true" tabindex="-1"/&gt;  empty &lt;span class="ot"&gt;=&lt;/span&gt; &lt;span class="dt"&gt;MkP&lt;/span&gt; empty&lt;/span&gt;
  631. &lt;span id="cb14-10"&gt;&lt;a href="#cb14-10" aria-hidden="true" tabindex="-1"/&gt;  p1 &lt;span class="op"&gt;&amp;lt;|&gt;&lt;/span&gt; p2 &lt;span class="ot"&gt;=&lt;/span&gt; withMemo (unP p1 &lt;span class="op"&gt;&amp;lt;|&gt;&lt;/span&gt; unP p2)&lt;/span&gt;
  632. &lt;span id="cb14-11"&gt;&lt;a href="#cb14-11" aria-hidden="true" tabindex="-1"/&gt;&lt;/span&gt;
  633. &lt;span id="cb14-12"&gt;&lt;a href="#cb14-12" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="kw"&gt;instance&lt;/span&gt; &lt;span class="dt"&gt;Monad&lt;/span&gt; (&lt;span class="dt"&gt;Parser&lt;/span&gt; tok) &lt;span class="kw"&gt;where&lt;/span&gt;&lt;/span&gt;
  634. &lt;span id="cb14-13"&gt;&lt;a href="#cb14-13" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="fu"&gt;return&lt;/span&gt; &lt;span class="ot"&gt;=&lt;/span&gt; &lt;span class="fu"&gt;pure&lt;/span&gt;&lt;/span&gt;
  635. &lt;span id="cb14-14"&gt;&lt;a href="#cb14-14" aria-hidden="true" tabindex="-1"/&gt;  p1 &lt;span class="op"&gt;&gt;&gt;=&lt;/span&gt; f &lt;span class="ot"&gt;=&lt;/span&gt; withMemo &lt;span class="op"&gt;$&lt;/span&gt; unP p1 &lt;span class="op"&gt;&gt;&gt;=&lt;/span&gt; unP &lt;span class="op"&gt;.&lt;/span&gt; f&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  636. &lt;p&gt;And indeed, it works (see &lt;a href="https://github.com/nomeata/left-rec-parse/blob/master/RParser.hs"&gt;&lt;code&gt;RParser.hs&lt;/code&gt;&lt;/a&gt; for the full code):&lt;/p&gt;
  637. &lt;div class="sourceCode" id="cb15"&gt;&lt;pre class="sourceCode haskell"&gt;&lt;code class="sourceCode haskell"&gt;&lt;span id="cb15-1"&gt;&lt;a href="#cb15-1" aria-hidden="true" tabindex="-1"/&gt;ghci&lt;span class="op"&gt;&gt;&lt;/span&gt; &lt;span class="kw"&gt;let&lt;/span&gt; aaa &lt;span class="ot"&gt;=&lt;/span&gt; aaa &lt;span class="op"&gt;&amp;lt;*&lt;/span&gt; tok &lt;span class="ch"&gt;'a'&lt;/span&gt; &lt;span class="op"&gt;&amp;lt;|&gt;&lt;/span&gt; &lt;span class="fu"&gt;pure&lt;/span&gt; ()&lt;/span&gt;
  638. &lt;span id="cb15-2"&gt;&lt;a href="#cb15-2" aria-hidden="true" tabindex="-1"/&gt;ghci&lt;span class="op"&gt;&gt;&lt;/span&gt; parse aaa &lt;span class="st"&gt;"aaaa"&lt;/span&gt;&lt;/span&gt;
  639. &lt;span id="cb15-3"&gt;&lt;a href="#cb15-3" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="dt"&gt;Just&lt;/span&gt; ()&lt;/span&gt;
  640. &lt;span id="cb15-4"&gt;&lt;a href="#cb15-4" aria-hidden="true" tabindex="-1"/&gt;ghci&lt;span class="op"&gt;&gt;&lt;/span&gt; parse aaa &lt;span class="st"&gt;"aabaa"&lt;/span&gt;&lt;/span&gt;
  641. &lt;span id="cb15-5"&gt;&lt;a href="#cb15-5" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="dt"&gt;Nothing&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  642. &lt;h3 id="a-larger-example"&gt;A larger example&lt;/h3&gt;
  643. &lt;p&gt;Let us try this on a larger example, and parse (simple) BNF grammars. Here is a data type describing them&lt;/p&gt;
  644. &lt;div class="sourceCode" id="cb16"&gt;&lt;pre class="sourceCode haskell"&gt;&lt;code class="sourceCode haskell"&gt;&lt;span id="cb16-1"&gt;&lt;a href="#cb16-1" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="kw"&gt;type&lt;/span&gt; &lt;span class="dt"&gt;Ident&lt;/span&gt; &lt;span class="ot"&gt;=&lt;/span&gt; &lt;span class="dt"&gt;String&lt;/span&gt;&lt;/span&gt;
  645. &lt;span id="cb16-2"&gt;&lt;a href="#cb16-2" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="kw"&gt;type&lt;/span&gt; &lt;span class="dt"&gt;RuleRhs&lt;/span&gt; &lt;span class="ot"&gt;=&lt;/span&gt; [&lt;span class="dt"&gt;Seq&lt;/span&gt;]&lt;/span&gt;
  646. &lt;span id="cb16-3"&gt;&lt;a href="#cb16-3" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="kw"&gt;type&lt;/span&gt; &lt;span class="dt"&gt;Seq&lt;/span&gt; &lt;span class="ot"&gt;=&lt;/span&gt; [&lt;span class="dt"&gt;Atom&lt;/span&gt;]&lt;/span&gt;
  647. &lt;span id="cb16-4"&gt;&lt;a href="#cb16-4" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="kw"&gt;data&lt;/span&gt; &lt;span class="dt"&gt;Atom&lt;/span&gt; &lt;span class="ot"&gt;=&lt;/span&gt; &lt;span class="dt"&gt;Lit&lt;/span&gt; &lt;span class="dt"&gt;String&lt;/span&gt; &lt;span class="op"&gt;|&lt;/span&gt; &lt;span class="dt"&gt;NonTerm&lt;/span&gt; &lt;span class="dt"&gt;Ident&lt;/span&gt; &lt;span class="kw"&gt;deriving&lt;/span&gt; &lt;span class="dt"&gt;Show&lt;/span&gt;&lt;/span&gt;
  648. &lt;span id="cb16-5"&gt;&lt;a href="#cb16-5" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="kw"&gt;type&lt;/span&gt; &lt;span class="dt"&gt;Rule&lt;/span&gt; &lt;span class="ot"&gt;=&lt;/span&gt; (&lt;span class="dt"&gt;Ident&lt;/span&gt;, &lt;span class="dt"&gt;RuleRhs&lt;/span&gt;)&lt;/span&gt;
  649. &lt;span id="cb16-6"&gt;&lt;a href="#cb16-6" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="kw"&gt;type&lt;/span&gt; &lt;span class="dt"&gt;BNF&lt;/span&gt; &lt;span class="ot"&gt;=&lt;/span&gt; [&lt;span class="dt"&gt;Rule&lt;/span&gt;]&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  650. &lt;p&gt;For the concrete syntax, I’d like to be able to parse something like&lt;/p&gt;
  651. &lt;div class="sourceCode" id="cb17"&gt;&lt;pre class="sourceCode haskell"&gt;&lt;code class="sourceCode haskell"&gt;&lt;span id="cb17-1"&gt;&lt;a href="#cb17-1" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;numExp ::&lt;/span&gt; &lt;span class="dt"&gt;String&lt;/span&gt;&lt;/span&gt;
  652. &lt;span id="cb17-2"&gt;&lt;a href="#cb17-2" aria-hidden="true" tabindex="-1"/&gt;numExp &lt;span class="ot"&gt;=&lt;/span&gt; &lt;span class="fu"&gt;unlines&lt;/span&gt;&lt;/span&gt;
  653. &lt;span id="cb17-3"&gt;&lt;a href="#cb17-3" aria-hidden="true" tabindex="-1"/&gt;    [ &lt;span class="st"&gt;"term   := sum;"&lt;/span&gt;&lt;/span&gt;
  654. &lt;span id="cb17-4"&gt;&lt;a href="#cb17-4" aria-hidden="true" tabindex="-1"/&gt;    , &lt;span class="st"&gt;"pdigit := '1' | '2' | '3' | '4' | '5' | '6' | '7' | '8' | '9';"&lt;/span&gt;&lt;/span&gt;
  655. &lt;span id="cb17-5"&gt;&lt;a href="#cb17-5" aria-hidden="true" tabindex="-1"/&gt;    , &lt;span class="st"&gt;"digit  := '0' | pdigit;"&lt;/span&gt;&lt;/span&gt;
  656. &lt;span id="cb17-6"&gt;&lt;a href="#cb17-6" aria-hidden="true" tabindex="-1"/&gt;    , &lt;span class="st"&gt;"pnum   := pdigit | pnum digit;"&lt;/span&gt;&lt;/span&gt;
  657. &lt;span id="cb17-7"&gt;&lt;a href="#cb17-7" aria-hidden="true" tabindex="-1"/&gt;    , &lt;span class="st"&gt;"num    := '0' | pnum;"&lt;/span&gt;&lt;/span&gt;
  658. &lt;span id="cb17-8"&gt;&lt;a href="#cb17-8" aria-hidden="true" tabindex="-1"/&gt;    , &lt;span class="st"&gt;"prod   := atom | atom '*' prod;"&lt;/span&gt;&lt;/span&gt;
  659. &lt;span id="cb17-9"&gt;&lt;a href="#cb17-9" aria-hidden="true" tabindex="-1"/&gt;    , &lt;span class="st"&gt;"sum    := prod | prod '+' sum;"&lt;/span&gt;&lt;/span&gt;
  660. &lt;span id="cb17-10"&gt;&lt;a href="#cb17-10" aria-hidden="true" tabindex="-1"/&gt;    , &lt;span class="st"&gt;"atom   := num | '(' term ')';"&lt;/span&gt;&lt;/span&gt;
  661. &lt;span id="cb17-11"&gt;&lt;a href="#cb17-11" aria-hidden="true" tabindex="-1"/&gt;    ]&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  662. &lt;p&gt;so here is a possible parser; mostly straight-forward use of parser combinator:&lt;/p&gt;
  663. &lt;div class="sourceCode" id="cb18"&gt;&lt;pre class="sourceCode haskell"&gt;&lt;code class="sourceCode haskell"&gt;&lt;span id="cb18-1"&gt;&lt;a href="#cb18-1" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="kw"&gt;type&lt;/span&gt; &lt;span class="dt"&gt;P&lt;/span&gt; &lt;span class="ot"&gt;=&lt;/span&gt; &lt;span class="dt"&gt;Parser&lt;/span&gt; &lt;span class="dt"&gt;Char&lt;/span&gt;&lt;/span&gt;
  664. &lt;span id="cb18-2"&gt;&lt;a href="#cb18-2" aria-hidden="true" tabindex="-1"/&gt;&lt;/span&gt;
  665. &lt;span id="cb18-3"&gt;&lt;a href="#cb18-3" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;snoc ::&lt;/span&gt; [a] &lt;span class="ot"&gt;-&gt;&lt;/span&gt; a &lt;span class="ot"&gt;-&gt;&lt;/span&gt; [a]&lt;/span&gt;
  666. &lt;span id="cb18-4"&gt;&lt;a href="#cb18-4" aria-hidden="true" tabindex="-1"/&gt;snoc xs x &lt;span class="ot"&gt;=&lt;/span&gt; xs &lt;span class="op"&gt;++&lt;/span&gt; [x]&lt;/span&gt;
  667. &lt;span id="cb18-5"&gt;&lt;a href="#cb18-5" aria-hidden="true" tabindex="-1"/&gt;&lt;/span&gt;
  668. &lt;span id="cb18-6"&gt;&lt;a href="#cb18-6" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;l ::&lt;/span&gt; &lt;span class="dt"&gt;P&lt;/span&gt; a &lt;span class="ot"&gt;-&gt;&lt;/span&gt; &lt;span class="dt"&gt;P&lt;/span&gt; a&lt;/span&gt;
  669. &lt;span id="cb18-7"&gt;&lt;a href="#cb18-7" aria-hidden="true" tabindex="-1"/&gt;l p &lt;span class="ot"&gt;=&lt;/span&gt; p &lt;span class="op"&gt;&amp;lt;|&gt;&lt;/span&gt; l p &lt;span class="op"&gt;&amp;lt;*&lt;/span&gt; sat &lt;span class="fu"&gt;isSpace&lt;/span&gt;&lt;/span&gt;
  670. &lt;span id="cb18-8"&gt;&lt;a href="#cb18-8" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;quote ::&lt;/span&gt; &lt;span class="dt"&gt;P&lt;/span&gt; &lt;span class="dt"&gt;Char&lt;/span&gt;&lt;/span&gt;
  671. &lt;span id="cb18-9"&gt;&lt;a href="#cb18-9" aria-hidden="true" tabindex="-1"/&gt;quote &lt;span class="ot"&gt;=&lt;/span&gt; tok &lt;span class="ch"&gt;'\''&lt;/span&gt;&lt;/span&gt;
  672. &lt;span id="cb18-10"&gt;&lt;a href="#cb18-10" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;quoted ::&lt;/span&gt; &lt;span class="dt"&gt;P&lt;/span&gt; a &lt;span class="ot"&gt;-&gt;&lt;/span&gt; &lt;span class="dt"&gt;P&lt;/span&gt; a&lt;/span&gt;
  673. &lt;span id="cb18-11"&gt;&lt;a href="#cb18-11" aria-hidden="true" tabindex="-1"/&gt;quoted p &lt;span class="ot"&gt;=&lt;/span&gt; quote &lt;span class="op"&gt;*&gt;&lt;/span&gt; p &lt;span class="op"&gt;&amp;lt;*&lt;/span&gt; quote&lt;/span&gt;
  674. &lt;span id="cb18-12"&gt;&lt;a href="#cb18-12" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;str ::&lt;/span&gt; &lt;span class="dt"&gt;P&lt;/span&gt; &lt;span class="dt"&gt;String&lt;/span&gt;&lt;/span&gt;
  675. &lt;span id="cb18-13"&gt;&lt;a href="#cb18-13" aria-hidden="true" tabindex="-1"/&gt;str &lt;span class="ot"&gt;=&lt;/span&gt; some (sat (&lt;span class="fu"&gt;not&lt;/span&gt; &lt;span class="op"&gt;.&lt;/span&gt; (&lt;span class="op"&gt;==&lt;/span&gt; &lt;span class="ch"&gt;'\''&lt;/span&gt;)))&lt;/span&gt;
  676. &lt;span id="cb18-14"&gt;&lt;a href="#cb18-14" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;ident ::&lt;/span&gt; &lt;span class="dt"&gt;P&lt;/span&gt; &lt;span class="dt"&gt;Ident&lt;/span&gt;&lt;/span&gt;
  677. &lt;span id="cb18-15"&gt;&lt;a href="#cb18-15" aria-hidden="true" tabindex="-1"/&gt;ident &lt;span class="ot"&gt;=&lt;/span&gt; some (sat (\c &lt;span class="ot"&gt;-&gt;&lt;/span&gt; &lt;span class="fu"&gt;isAlphaNum&lt;/span&gt; c &lt;span class="op"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="fu"&gt;isAscii&lt;/span&gt; c))&lt;/span&gt;
  678. &lt;span id="cb18-16"&gt;&lt;a href="#cb18-16" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;atom ::&lt;/span&gt; &lt;span class="dt"&gt;P&lt;/span&gt; &lt;span class="dt"&gt;Atom&lt;/span&gt;&lt;/span&gt;
  679. &lt;span id="cb18-17"&gt;&lt;a href="#cb18-17" aria-hidden="true" tabindex="-1"/&gt;atom &lt;span class="ot"&gt;=&lt;/span&gt; &lt;span class="dt"&gt;Lit&lt;/span&gt;     &lt;span class="op"&gt;&amp;lt;$&gt;&lt;/span&gt; l (quoted str)&lt;/span&gt;
  680. &lt;span id="cb18-18"&gt;&lt;a href="#cb18-18" aria-hidden="true" tabindex="-1"/&gt;   &lt;span class="op"&gt;&amp;lt;|&gt;&lt;/span&gt; &lt;span class="dt"&gt;NonTerm&lt;/span&gt; &lt;span class="op"&gt;&amp;lt;$&gt;&lt;/span&gt; l ident&lt;/span&gt;
  681. &lt;span id="cb18-19"&gt;&lt;a href="#cb18-19" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;eps ::&lt;/span&gt; &lt;span class="dt"&gt;P&lt;/span&gt; ()&lt;/span&gt;
  682. &lt;span id="cb18-20"&gt;&lt;a href="#cb18-20" aria-hidden="true" tabindex="-1"/&gt;eps &lt;span class="ot"&gt;=&lt;/span&gt; void &lt;span class="op"&gt;$&lt;/span&gt; l (tok &lt;span class="ch"&gt;'ε'&lt;/span&gt;)&lt;/span&gt;
  683. &lt;span id="cb18-21"&gt;&lt;a href="#cb18-21" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;sep ::&lt;/span&gt; &lt;span class="dt"&gt;P&lt;/span&gt; ()&lt;/span&gt;
  684. &lt;span id="cb18-22"&gt;&lt;a href="#cb18-22" aria-hidden="true" tabindex="-1"/&gt;sep &lt;span class="ot"&gt;=&lt;/span&gt; void &lt;span class="op"&gt;$&lt;/span&gt; some (sat &lt;span class="fu"&gt;isSpace&lt;/span&gt;)&lt;/span&gt;
  685. &lt;span id="cb18-23"&gt;&lt;a href="#cb18-23" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;sq ::&lt;/span&gt; &lt;span class="dt"&gt;P&lt;/span&gt; &lt;span class="dt"&gt;Seq&lt;/span&gt;&lt;/span&gt;
  686. &lt;span id="cb18-24"&gt;&lt;a href="#cb18-24" aria-hidden="true" tabindex="-1"/&gt;sq &lt;span class="ot"&gt;=&lt;/span&gt; []   &lt;span class="op"&gt;&amp;lt;$&lt;/span&gt; eps&lt;/span&gt;
  687. &lt;span id="cb18-25"&gt;&lt;a href="#cb18-25" aria-hidden="true" tabindex="-1"/&gt; &lt;span class="op"&gt;&amp;lt;|&gt;&lt;/span&gt; snoc &lt;span class="op"&gt;&amp;lt;$&gt;&lt;/span&gt; sq &lt;span class="op"&gt;&amp;lt;*&lt;/span&gt; sep &lt;span class="op"&gt;&amp;lt;*&gt;&lt;/span&gt; atom&lt;/span&gt;
  688. &lt;span id="cb18-26"&gt;&lt;a href="#cb18-26" aria-hidden="true" tabindex="-1"/&gt; &lt;span class="op"&gt;&amp;lt;|&gt;&lt;/span&gt; &lt;span class="fu"&gt;pure&lt;/span&gt; &lt;span class="op"&gt;&amp;lt;$&gt;&lt;/span&gt; atom&lt;/span&gt;
  689. &lt;span id="cb18-27"&gt;&lt;a href="#cb18-27" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;ruleRhs ::&lt;/span&gt; &lt;span class="dt"&gt;P&lt;/span&gt; &lt;span class="dt"&gt;RuleRhs&lt;/span&gt;&lt;/span&gt;
  690. &lt;span id="cb18-28"&gt;&lt;a href="#cb18-28" aria-hidden="true" tabindex="-1"/&gt;ruleRhs &lt;span class="ot"&gt;=&lt;/span&gt; &lt;span class="fu"&gt;pure&lt;/span&gt; &lt;span class="op"&gt;&amp;lt;$&gt;&lt;/span&gt; sq&lt;/span&gt;
  691. &lt;span id="cb18-29"&gt;&lt;a href="#cb18-29" aria-hidden="true" tabindex="-1"/&gt;      &lt;span class="op"&gt;&amp;lt;|&gt;&lt;/span&gt; snoc &lt;span class="op"&gt;&amp;lt;$&gt;&lt;/span&gt; ruleRhs &lt;span class="op"&gt;&amp;lt;*&lt;/span&gt; l (tok &lt;span class="ch"&gt;'|'&lt;/span&gt;) &lt;span class="op"&gt;&amp;lt;*&gt;&lt;/span&gt; sq&lt;/span&gt;
  692. &lt;span id="cb18-30"&gt;&lt;a href="#cb18-30" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;rule ::&lt;/span&gt; &lt;span class="dt"&gt;P&lt;/span&gt; &lt;span class="dt"&gt;Rule&lt;/span&gt;&lt;/span&gt;
  693. &lt;span id="cb18-31"&gt;&lt;a href="#cb18-31" aria-hidden="true" tabindex="-1"/&gt;rule &lt;span class="ot"&gt;=&lt;/span&gt; (,) &lt;span class="op"&gt;&amp;lt;$&gt;&lt;/span&gt; l ident &lt;span class="op"&gt;&amp;lt;*&lt;/span&gt; l (tok &lt;span class="ch"&gt;':'&lt;/span&gt; &lt;span class="op"&gt;*&gt;&lt;/span&gt; tok &lt;span class="ch"&gt;'='&lt;/span&gt;) &lt;span class="op"&gt;&amp;lt;*&gt;&lt;/span&gt; ruleRhs &lt;span class="op"&gt;&amp;lt;*&lt;/span&gt; l (tok &lt;span class="ch"&gt;';'&lt;/span&gt;)&lt;/span&gt;
  694. &lt;span id="cb18-32"&gt;&lt;a href="#cb18-32" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;bnf ::&lt;/span&gt; &lt;span class="dt"&gt;P&lt;/span&gt; &lt;span class="dt"&gt;BNF&lt;/span&gt;&lt;/span&gt;
  695. &lt;span id="cb18-33"&gt;&lt;a href="#cb18-33" aria-hidden="true" tabindex="-1"/&gt;bnf &lt;span class="ot"&gt;=&lt;/span&gt; &lt;span class="fu"&gt;pure&lt;/span&gt; &lt;span class="op"&gt;&amp;lt;$&gt;&lt;/span&gt; rule&lt;/span&gt;
  696. &lt;span id="cb18-34"&gt;&lt;a href="#cb18-34" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="op"&gt;&amp;lt;|&gt;&lt;/span&gt; snoc &lt;span class="op"&gt;&amp;lt;$&gt;&lt;/span&gt; bnf &lt;span class="op"&gt;&amp;lt;*&gt;&lt;/span&gt; rule&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  697. &lt;p&gt;I somewhat sillily used &lt;code&gt;snoc&lt;/code&gt; rather than &lt;code&gt;(:)&lt;/code&gt; to build my lists, just so that I can show off all the left-recursion in this grammar.&lt;/p&gt;
  698. &lt;h3 id="sharing-is-tricky"&gt;Sharing is tricky&lt;/h3&gt;
  699. &lt;p&gt;Let’s try it:&lt;/p&gt;
  700. &lt;div class="sourceCode" id="cb19"&gt;&lt;pre class="sourceCode haskell"&gt;&lt;code class="sourceCode haskell"&gt;&lt;span id="cb19-1"&gt;&lt;a href="#cb19-1" aria-hidden="true" tabindex="-1"/&gt;ghci&lt;span class="op"&gt;&gt;&lt;/span&gt; parse bnf numExp&lt;/span&gt;
  701. &lt;span id="cb19-2"&gt;&lt;a href="#cb19-2" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="op"&gt;^&lt;/span&gt;&lt;span class="dt"&gt;CInterrupted&lt;/span&gt;&lt;span class="op"&gt;.&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  702. &lt;p&gt;What a pity, it does not work! What went wrong?&lt;/p&gt;
  703. &lt;p&gt;The underlying library can handle left-recursion if it can recognize it by seeing a &lt;code&gt;memoise&lt;/code&gt; label passed again. This works fine in all the places where we re-use a parser definition (e.g. in &lt;code&gt;bnf&lt;/code&gt;), but it really requires that values are shared!&lt;/p&gt;
  704. &lt;p&gt;If we look carefully at our definition of &lt;code&gt;l&lt;/code&gt; (which parses a lexeme, i.e. something possibly followed by whitespace), then it recurses via a fresh function call, and the program will keep expanding the definition – just like the &lt;code&gt;acyclic&lt;/code&gt; above:&lt;/p&gt;
  705. &lt;div class="sourceCode" id="cb20"&gt;&lt;pre class="sourceCode haskell"&gt;&lt;code class="sourceCode haskell"&gt;&lt;span id="cb20-1"&gt;&lt;a href="#cb20-1" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;l ::&lt;/span&gt; &lt;span class="dt"&gt;P&lt;/span&gt; a &lt;span class="ot"&gt;-&gt;&lt;/span&gt; &lt;span class="dt"&gt;P&lt;/span&gt; a&lt;/span&gt;
  706. &lt;span id="cb20-2"&gt;&lt;a href="#cb20-2" aria-hidden="true" tabindex="-1"/&gt;l p &lt;span class="ot"&gt;=&lt;/span&gt; p &lt;span class="op"&gt;&amp;lt;|&gt;&lt;/span&gt; l p &lt;span class="op"&gt;&amp;lt;*&lt;/span&gt; sat &lt;span class="fu"&gt;isSpace&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  707. &lt;p&gt;The fix (sic!) is to make sure that the recursive call is using the parser we are currently defining, which we can easily do with a local definition:&lt;/p&gt;
  708. &lt;div class="sourceCode" id="cb21"&gt;&lt;pre class="sourceCode haskell"&gt;&lt;code class="sourceCode haskell"&gt;&lt;span id="cb21-1"&gt;&lt;a href="#cb21-1" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;l ::&lt;/span&gt; &lt;span class="dt"&gt;P&lt;/span&gt; a &lt;span class="ot"&gt;-&gt;&lt;/span&gt; &lt;span class="dt"&gt;P&lt;/span&gt; a&lt;/span&gt;
  709. &lt;span id="cb21-2"&gt;&lt;a href="#cb21-2" aria-hidden="true" tabindex="-1"/&gt;l p &lt;span class="ot"&gt;=&lt;/span&gt; p'&lt;/span&gt;
  710. &lt;span id="cb21-3"&gt;&lt;a href="#cb21-3" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="kw"&gt;where&lt;/span&gt; p' &lt;span class="ot"&gt;=&lt;/span&gt; p &lt;span class="op"&gt;&amp;lt;|&gt;&lt;/span&gt; p' &lt;span class="op"&gt;&amp;lt;*&lt;/span&gt; sat &lt;span class="fu"&gt;isSpace&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  711. &lt;p&gt;With this little fix, the parser can parse the example grammar:&lt;/p&gt;
  712. &lt;div class="sourceCode" id="cb22"&gt;&lt;pre class="sourceCode haskell"&gt;&lt;code class="sourceCode haskell"&gt;&lt;span id="cb22-1"&gt;&lt;a href="#cb22-1" aria-hidden="true" tabindex="-1"/&gt;ghci&lt;span class="op"&gt;&gt;&lt;/span&gt; parse bnf numExp&lt;/span&gt;
  713. &lt;span id="cb22-2"&gt;&lt;a href="#cb22-2" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="dt"&gt;Just&lt;/span&gt; [(&lt;span class="st"&gt;"term"&lt;/span&gt;,[[&lt;span class="dt"&gt;NonTerm&lt;/span&gt; &lt;span class="st"&gt;"sum"&lt;/span&gt;]]),(&lt;span class="st"&gt;"pdigit"&lt;/span&gt;,[[&lt;span class="dt"&gt;Lit&lt;/span&gt; &lt;span class="st"&gt;"1"&lt;/span&gt;],…&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  714. &lt;h3 id="going-meta"&gt;Going meta&lt;/h3&gt;
  715. &lt;p&gt;The main demonstration is over, but since we now have already have a parser for grammar descriptions at hand, let’s go a bit further and &lt;em&gt;dynamically&lt;/em&gt; construct a parser from such a description. The parser should only accept strings according to that grammar, and return a parse tree annotated with the non-terminals used:&lt;/p&gt;
  716. &lt;div class="sourceCode" id="cb23"&gt;&lt;pre class="sourceCode haskell"&gt;&lt;code class="sourceCode haskell"&gt;&lt;span id="cb23-1"&gt;&lt;a href="#cb23-1" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;interp ::&lt;/span&gt; &lt;span class="dt"&gt;BNF&lt;/span&gt; &lt;span class="ot"&gt;-&gt;&lt;/span&gt; &lt;span class="dt"&gt;P&lt;/span&gt; &lt;span class="dt"&gt;String&lt;/span&gt;&lt;/span&gt;
  717. &lt;span id="cb23-2"&gt;&lt;a href="#cb23-2" aria-hidden="true" tabindex="-1"/&gt;interp bnf &lt;span class="ot"&gt;=&lt;/span&gt; parsers &lt;span class="op"&gt;M.!&lt;/span&gt; start&lt;/span&gt;
  718. &lt;span id="cb23-3"&gt;&lt;a href="#cb23-3" aria-hidden="true" tabindex="-1"/&gt;  &lt;span class="kw"&gt;where&lt;/span&gt;&lt;/span&gt;
  719. &lt;span id="cb23-4"&gt;&lt;a href="#cb23-4" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;    start ::&lt;/span&gt; &lt;span class="dt"&gt;Ident&lt;/span&gt;&lt;/span&gt;
  720. &lt;span id="cb23-5"&gt;&lt;a href="#cb23-5" aria-hidden="true" tabindex="-1"/&gt;    start &lt;span class="ot"&gt;=&lt;/span&gt; &lt;span class="fu"&gt;fst&lt;/span&gt; (&lt;span class="fu"&gt;head&lt;/span&gt; bnf)&lt;/span&gt;
  721. &lt;span id="cb23-6"&gt;&lt;a href="#cb23-6" aria-hidden="true" tabindex="-1"/&gt;&lt;/span&gt;
  722. &lt;span id="cb23-7"&gt;&lt;a href="#cb23-7" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;    parsers ::&lt;/span&gt; &lt;span class="dt"&gt;M.Map&lt;/span&gt; &lt;span class="dt"&gt;Ident&lt;/span&gt; (&lt;span class="dt"&gt;P&lt;/span&gt; &lt;span class="dt"&gt;String&lt;/span&gt;)&lt;/span&gt;
  723. &lt;span id="cb23-8"&gt;&lt;a href="#cb23-8" aria-hidden="true" tabindex="-1"/&gt;    parsers &lt;span class="ot"&gt;=&lt;/span&gt; M.fromList [ (i, parseRule i rhs) &lt;span class="op"&gt;|&lt;/span&gt; (i, rhs) &lt;span class="ot"&gt;&amp;lt;-&lt;/span&gt; bnf ]&lt;/span&gt;
  724. &lt;span id="cb23-9"&gt;&lt;a href="#cb23-9" aria-hidden="true" tabindex="-1"/&gt;&lt;/span&gt;
  725. &lt;span id="cb23-10"&gt;&lt;a href="#cb23-10" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;    parseRule ::&lt;/span&gt; &lt;span class="dt"&gt;Ident&lt;/span&gt; &lt;span class="ot"&gt;-&gt;&lt;/span&gt; &lt;span class="dt"&gt;RuleRhs&lt;/span&gt; &lt;span class="ot"&gt;-&gt;&lt;/span&gt; &lt;span class="dt"&gt;P&lt;/span&gt; &lt;span class="dt"&gt;String&lt;/span&gt;&lt;/span&gt;
  726. &lt;span id="cb23-11"&gt;&lt;a href="#cb23-11" aria-hidden="true" tabindex="-1"/&gt;    parseRule ident rhs &lt;span class="ot"&gt;=&lt;/span&gt; trace &lt;span class="op"&gt;&amp;lt;$&gt;&lt;/span&gt; asum (&lt;span class="fu"&gt;map&lt;/span&gt; parseSeq rhs)&lt;/span&gt;
  727. &lt;span id="cb23-12"&gt;&lt;a href="#cb23-12" aria-hidden="true" tabindex="-1"/&gt;      &lt;span class="kw"&gt;where&lt;/span&gt; trace s &lt;span class="ot"&gt;=&lt;/span&gt; ident &lt;span class="op"&gt;++&lt;/span&gt; &lt;span class="st"&gt;"("&lt;/span&gt; &lt;span class="op"&gt;++&lt;/span&gt; s &lt;span class="op"&gt;++&lt;/span&gt; &lt;span class="st"&gt;")"&lt;/span&gt;&lt;/span&gt;
  728. &lt;span id="cb23-13"&gt;&lt;a href="#cb23-13" aria-hidden="true" tabindex="-1"/&gt;&lt;/span&gt;
  729. &lt;span id="cb23-14"&gt;&lt;a href="#cb23-14" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;    parseSeq ::&lt;/span&gt; &lt;span class="dt"&gt;Seq&lt;/span&gt; &lt;span class="ot"&gt;-&gt;&lt;/span&gt; &lt;span class="dt"&gt;P&lt;/span&gt; &lt;span class="dt"&gt;String&lt;/span&gt;&lt;/span&gt;
  730. &lt;span id="cb23-15"&gt;&lt;a href="#cb23-15" aria-hidden="true" tabindex="-1"/&gt;    parseSeq &lt;span class="ot"&gt;=&lt;/span&gt; &lt;span class="fu"&gt;fmap&lt;/span&gt; &lt;span class="fu"&gt;concat&lt;/span&gt; &lt;span class="op"&gt;.&lt;/span&gt; &lt;span class="fu"&gt;traverse&lt;/span&gt; parseAtom&lt;/span&gt;
  731. &lt;span id="cb23-16"&gt;&lt;a href="#cb23-16" aria-hidden="true" tabindex="-1"/&gt;&lt;/span&gt;
  732. &lt;span id="cb23-17"&gt;&lt;a href="#cb23-17" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;    parseAtom ::&lt;/span&gt; &lt;span class="dt"&gt;Atom&lt;/span&gt; &lt;span class="ot"&gt;-&gt;&lt;/span&gt; &lt;span class="dt"&gt;P&lt;/span&gt; &lt;span class="dt"&gt;String&lt;/span&gt;&lt;/span&gt;
  733. &lt;span id="cb23-18"&gt;&lt;a href="#cb23-18" aria-hidden="true" tabindex="-1"/&gt;    parseAtom (&lt;span class="dt"&gt;Lit&lt;/span&gt; s) &lt;span class="ot"&gt;=&lt;/span&gt; &lt;span class="fu"&gt;traverse&lt;/span&gt; tok s&lt;/span&gt;
  734. &lt;span id="cb23-19"&gt;&lt;a href="#cb23-19" aria-hidden="true" tabindex="-1"/&gt;    parseAtom (&lt;span class="dt"&gt;NonTerm&lt;/span&gt; i) &lt;span class="ot"&gt;=&lt;/span&gt; parsers &lt;span class="op"&gt;M.!&lt;/span&gt; i&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  735. &lt;p&gt;Let’s see it in action (full code in &lt;a href="https://github.com/nomeata/left-rec-parse/blob/master/BNFEx.hs"&gt;&lt;code&gt;BNFEx.hs&lt;/code&gt;&lt;/a&gt;):&lt;/p&gt;
  736. &lt;div class="sourceCode" id="cb24"&gt;&lt;pre class="sourceCode haskell"&gt;&lt;code class="sourceCode haskell"&gt;&lt;span id="cb24-1"&gt;&lt;a href="#cb24-1" aria-hidden="true" tabindex="-1"/&gt;ghci&lt;span class="op"&gt;&gt;&lt;/span&gt; &lt;span class="dt"&gt;Just&lt;/span&gt; bnfp &lt;span class="ot"&gt;=&lt;/span&gt; parse bnf numExp&lt;/span&gt;
  737. &lt;span id="cb24-2"&gt;&lt;a href="#cb24-2" aria-hidden="true" tabindex="-1"/&gt;ghci&lt;span class="op"&gt;&gt;&lt;/span&gt; &lt;span class="op"&gt;:&lt;/span&gt;t bnfp&lt;/span&gt;
  738. &lt;span id="cb24-3"&gt;&lt;a href="#cb24-3" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="ot"&gt;bnfp ::&lt;/span&gt; &lt;span class="dt"&gt;BNF&lt;/span&gt;&lt;/span&gt;
  739. &lt;span id="cb24-4"&gt;&lt;a href="#cb24-4" aria-hidden="true" tabindex="-1"/&gt;ghci&lt;span class="op"&gt;&gt;&lt;/span&gt; parse (inter&lt;/span&gt;
  740. &lt;span id="cb24-5"&gt;&lt;a href="#cb24-5" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="fu"&gt;interact&lt;/span&gt;  interp&lt;/span&gt;
  741. &lt;span id="cb24-6"&gt;&lt;a href="#cb24-6" aria-hidden="true" tabindex="-1"/&gt;ghci&lt;span class="op"&gt;&gt;&lt;/span&gt; parse (interp bnfp) &lt;span class="st"&gt;"12+3*4"&lt;/span&gt;&lt;/span&gt;
  742. &lt;span id="cb24-7"&gt;&lt;a href="#cb24-7" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="dt"&gt;Just&lt;/span&gt; &lt;span class="st"&gt;"term(sum(prod(atom(num(pnum(pnum(pdigit(1))digit(pdigit(2))))))+sum(prod(atom(num(pnum(pdigit(3))))*prod(atom(num(pnum(pdigit(4)))))))))"&lt;/span&gt;&lt;/span&gt;
  743. &lt;span id="cb24-8"&gt;&lt;a href="#cb24-8" aria-hidden="true" tabindex="-1"/&gt;ghci&lt;span class="op"&gt;&gt;&lt;/span&gt; parse (interp bnfp) &lt;span class="st"&gt;"12+3*4+"&lt;/span&gt;&lt;/span&gt;
  744. &lt;span id="cb24-9"&gt;&lt;a href="#cb24-9" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="dt"&gt;Nothing&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  745. &lt;p&gt;It is worth noting that the &lt;code&gt;numExp&lt;/code&gt; grammar is also left-recursive, so implementing &lt;code&gt;interp&lt;/code&gt; with a conventional parser combinator library would not have worked. But thanks to our &lt;code&gt;propriocept&lt;/code&gt; tick, it does! Again, the sharing is important; in the code above it is the map &lt;code&gt;parsers&lt;/code&gt; that is defined in terms of itself, and will ensure that the left-recursive productions will work.&lt;/p&gt;
  746. &lt;h3 id="closing-thoughts"&gt;Closing thoughts&lt;/h3&gt;
  747. &lt;p&gt;I am using &lt;code&gt;unsafePerformIO&lt;/code&gt;, so I need to justify its use. Clearly, &lt;code&gt;propriocept&lt;/code&gt; is &lt;em&gt;not&lt;/em&gt; a pure function, and it’s type is a lie. In general, using it will break the nice equational properties of Haskell, as we have seen in our experiments with &lt;code&gt;cons&lt;/code&gt;.&lt;/p&gt;
  748. &lt;p&gt;In the case of our parser library, however, we use it in specific ways, namely to feed a fresh name to &lt;code&gt;memoise&lt;/code&gt;. Assuming the underlying parser library’s behavior does not observably depend on where and with which key &lt;code&gt;memoise&lt;/code&gt; is used, this results in a properly pure interface, and all is well again. (NB: I did not investigate if this assumption actually holds for the parser library used here, or if it may for example affect the order of parse trees returned.)&lt;/p&gt;
  749. &lt;p&gt;I also expect that this implementation, which will memoise &lt;em&gt;every&lt;/em&gt; parser involved, will be rather slow. It seems plausible to analyze the graph structure and figure out which &lt;code&gt;memoise&lt;/code&gt; calls are actually needed to break left-recursion (at least if we drop the &lt;code&gt;Monad&lt;/code&gt; instance or always memoise &lt;code&gt;&gt;&gt;=&lt;/code&gt;).&lt;/p&gt;
  750. &lt;p&gt;If you liked this post, you might enjoy reading &lt;a href="https://doi.org/10.1145/3607853"&gt;the paper about rec-def&lt;/a&gt;, watch one of my talks about it (MuniHac, BOBKonf, ICFP23; the presentation evolved over time), or if you just want to see more about how things are laid out on Haskell’s heap, go to &lt;a href="https://www.youtube.com/playlist?list=PL4FcLyLhO9jggmkqJyJ2i9pCSiDpwKiVu"&gt;my screen casts exploring the Haskell heap&lt;/a&gt;.&lt;/p&gt;</description>
  751.         <pubDate>Sun, 10 Sep 2023 17:16:08 -0700</pubDate>
  752.      </item>
  753.      <item>
  754.         <title>Generating bibtex bibliographies from DOIs via DBLP</title>
  755.         <link>https://www.joachim-breitner.de/blog/806-Generating_bibtex_bibliographies_from_DOIs_via_DBLP</link>
  756.         <guid>https://www.joachim-breitner.de/blog/806-Generating_bibtex_bibliographies_from_DOIs_via_DBLP</guid>
  757.         <comments>https://www.joachim-breitner.de/blog/806-Generating_bibtex_bibliographies_from_DOIs_via_DBLP#comments</comments>
  758.         <author>mail@joachim-breitner.de (Joachim Breitner)</author>
  759.         <description>&lt;p&gt;I sometimes &lt;a href="https://www.joachim-breitner.de/publications"&gt;write papers&lt;/a&gt; and part of paper writing is assembling the bibliography. In my case, this is done using BibTeX. So when I need to add another citation, I have to find suitable data in Bibtex format.&lt;/p&gt;
  760. &lt;p&gt;Often I copy snippets from &lt;code&gt;.bib&lt;/code&gt; files from earlier paper.&lt;/p&gt;
  761. &lt;p&gt;Or I search for the paper on &lt;a href="https://dblp.org/"&gt;DBLP&lt;/a&gt;, which in my experience has highest quality BibTeX entries and best coverage of computer science related publications, copy it to my &lt;code&gt;.bib&lt;/code&gt; file, and change the key to whatever I want to refer the paper by.&lt;/p&gt;
  762. &lt;p&gt;But in the days of pervasive use of DOIs (digital object identifiers) for almost all publications, manually keeping the data in bibtex files seems outdated. Instead I’d rather just put the two pieces of data I care about: the &lt;em&gt;key&lt;/em&gt; that I want to use for citation, and the &lt;em&gt;doi&lt;/em&gt;. The rest I do not want to be bothered with.&lt;/p&gt;
  763. &lt;p&gt;So I wrote a small script that takes a &lt;code&gt;.yaml&lt;/code&gt; file like&lt;/p&gt;
  764. &lt;div class="sourceCode" id="cb1"&gt;&lt;pre class="sourceCode yaml"&gt;&lt;code class="sourceCode yaml"&gt;&lt;span id="cb1-1"&gt;&lt;a href="#cb1-1" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="fu"&gt;entries&lt;/span&gt;&lt;span class="kw"&gt;:&lt;/span&gt;&lt;/span&gt;
  765. &lt;span id="cb1-2"&gt;&lt;a href="#cb1-2" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="at"&gt;  &lt;/span&gt;&lt;span class="fu"&gt;unsafePerformIO&lt;/span&gt;&lt;span class="kw"&gt;:&lt;/span&gt;&lt;span class="at"&gt; 10.1007/10722298_3&lt;/span&gt;&lt;/span&gt;
  766. &lt;span id="cb1-3"&gt;&lt;a href="#cb1-3" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="at"&gt;  &lt;/span&gt;&lt;span class="fu"&gt;dejafu&lt;/span&gt;&lt;span class="kw"&gt;:&lt;/span&gt;&lt;span class="at"&gt; 10.1145/2804302.2804306&lt;/span&gt;&lt;/span&gt;
  767. &lt;span id="cb1-4"&gt;&lt;a href="#cb1-4" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="at"&gt;  &lt;/span&gt;&lt;span class="fu"&gt;runST&lt;/span&gt;&lt;span class="kw"&gt;:&lt;/span&gt;&lt;span class="at"&gt; 10.1145/3158152&lt;/span&gt;&lt;/span&gt;
  768. &lt;span id="cb1-5"&gt;&lt;a href="#cb1-5" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="at"&gt;  &lt;/span&gt;&lt;span class="fu"&gt;quickcheck&lt;/span&gt;&lt;span class="kw"&gt;:&lt;/span&gt;&lt;span class="at"&gt; 10.1145/351240.351266&lt;/span&gt;&lt;/span&gt;
  769. &lt;span id="cb1-6"&gt;&lt;a href="#cb1-6" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="at"&gt;  &lt;/span&gt;&lt;span class="fu"&gt;optimiser&lt;/span&gt;&lt;span class="kw"&gt;:&lt;/span&gt;&lt;span class="at"&gt; 10.1016/S0167-6423(97)00029-4&lt;/span&gt;&lt;/span&gt;
  770. &lt;span id="cb1-7"&gt;&lt;a href="#cb1-7" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="at"&gt;  &lt;/span&gt;&lt;span class="fu"&gt;sabry&lt;/span&gt;&lt;span class="kw"&gt;:&lt;/span&gt;&lt;span class="at"&gt; 10.1017/s0956796897002943&lt;/span&gt;&lt;/span&gt;
  771. &lt;span id="cb1-8"&gt;&lt;a href="#cb1-8" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="at"&gt;  &lt;/span&gt;&lt;span class="fu"&gt;concurrent&lt;/span&gt;&lt;span class="kw"&gt;:&lt;/span&gt;&lt;span class="at"&gt; 10.1145/237721.237794&lt;/span&gt;&lt;/span&gt;
  772. &lt;span id="cb1-9"&gt;&lt;a href="#cb1-9" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="at"&gt;  &lt;/span&gt;&lt;span class="fu"&gt;launchbury&lt;/span&gt;&lt;span class="kw"&gt;:&lt;/span&gt;&lt;span class="at"&gt; 10.1145/158511.158618&lt;/span&gt;&lt;/span&gt;
  773. &lt;span id="cb1-10"&gt;&lt;a href="#cb1-10" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="at"&gt;  &lt;/span&gt;&lt;span class="fu"&gt;datafun&lt;/span&gt;&lt;span class="kw"&gt;:&lt;/span&gt;&lt;span class="at"&gt; 10.1145/2951913.2951948&lt;/span&gt;&lt;/span&gt;
  774. &lt;span id="cb1-11"&gt;&lt;a href="#cb1-11" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="at"&gt;  &lt;/span&gt;&lt;span class="fu"&gt;observable-sharing&lt;/span&gt;&lt;span class="kw"&gt;:&lt;/span&gt;&lt;span class="at"&gt; 10.1007/3-540-46674-6_7&lt;/span&gt;&lt;/span&gt;
  775. &lt;span id="cb1-12"&gt;&lt;a href="#cb1-12" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="at"&gt;  &lt;/span&gt;&lt;span class="fu"&gt;kildall-73&lt;/span&gt;&lt;span class="kw"&gt;:&lt;/span&gt;&lt;span class="at"&gt; 10.1145/512927.512945&lt;/span&gt;&lt;/span&gt;
  776. &lt;span id="cb1-13"&gt;&lt;a href="#cb1-13" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="at"&gt;  &lt;/span&gt;&lt;span class="fu"&gt;kam-ullman-76&lt;/span&gt;&lt;span class="kw"&gt;:&lt;/span&gt;&lt;span class="at"&gt; 10.1145/321921.321938&lt;/span&gt;&lt;/span&gt;
  777. &lt;span id="cb1-14"&gt;&lt;a href="#cb1-14" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="at"&gt;  &lt;/span&gt;&lt;span class="fu"&gt;spygame&lt;/span&gt;&lt;span class="kw"&gt;:&lt;/span&gt;&lt;span class="at"&gt; 10.1145/3371101&lt;/span&gt;&lt;/span&gt;
  778. &lt;span id="cb1-15"&gt;&lt;a href="#cb1-15" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="at"&gt;  &lt;/span&gt;&lt;span class="fu"&gt;cocaml&lt;/span&gt;&lt;span class="kw"&gt;:&lt;/span&gt;&lt;span class="at"&gt; 10.3233/FI-2017-1473&lt;/span&gt;&lt;/span&gt;
  779. &lt;span id="cb1-16"&gt;&lt;a href="#cb1-16" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="at"&gt;  &lt;/span&gt;&lt;span class="fu"&gt;secrets&lt;/span&gt;&lt;span class="kw"&gt;:&lt;/span&gt;&lt;span class="at"&gt; 10.1017/S0956796802004331&lt;/span&gt;&lt;/span&gt;
  780. &lt;span id="cb1-17"&gt;&lt;a href="#cb1-17" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="at"&gt;  &lt;/span&gt;&lt;span class="fu"&gt;modular&lt;/span&gt;&lt;span class="kw"&gt;:&lt;/span&gt;&lt;span class="at"&gt; 10.1017/S0956796817000016&lt;/span&gt;&lt;/span&gt;
  781. &lt;span id="cb1-18"&gt;&lt;a href="#cb1-18" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="at"&gt;  &lt;/span&gt;&lt;span class="fu"&gt;longley&lt;/span&gt;&lt;span class="kw"&gt;:&lt;/span&gt;&lt;span class="at"&gt; 10.1145/317636.317775&lt;/span&gt;&lt;/span&gt;
  782. &lt;span id="cb1-19"&gt;&lt;a href="#cb1-19" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="at"&gt;  &lt;/span&gt;&lt;span class="fu"&gt;nievergelt&lt;/span&gt;&lt;span class="kw"&gt;:&lt;/span&gt;&lt;span class="at"&gt; 10.1145/800152.804906&lt;/span&gt;&lt;/span&gt;
  783. &lt;span id="cb1-20"&gt;&lt;a href="#cb1-20" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="at"&gt;  &lt;/span&gt;&lt;span class="fu"&gt;runST2&lt;/span&gt;&lt;span class="kw"&gt;:&lt;/span&gt;&lt;span class="at"&gt; 10.1145/3527326&lt;/span&gt;&lt;/span&gt;
  784. &lt;span id="cb1-21"&gt;&lt;a href="#cb1-21" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="at"&gt;  &lt;/span&gt;&lt;span class="fu"&gt;polakow&lt;/span&gt;&lt;span class="kw"&gt;:&lt;/span&gt;&lt;span class="at"&gt; 10.1145/2804302.2804309&lt;/span&gt;&lt;/span&gt;
  785. &lt;span id="cb1-22"&gt;&lt;a href="#cb1-22" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="at"&gt;  &lt;/span&gt;&lt;span class="fu"&gt;lvars&lt;/span&gt;&lt;span class="kw"&gt;:&lt;/span&gt;&lt;span class="at"&gt; 10.1145/2502323.2502326&lt;/span&gt;&lt;/span&gt;
  786. &lt;span id="cb1-23"&gt;&lt;a href="#cb1-23" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="at"&gt;  &lt;/span&gt;&lt;span class="fu"&gt;typesafe-sharing&lt;/span&gt;&lt;span class="kw"&gt;:&lt;/span&gt;&lt;span class="at"&gt; 10.1145/1596638.1596653&lt;/span&gt;&lt;/span&gt;
  787. &lt;span id="cb1-24"&gt;&lt;a href="#cb1-24" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="at"&gt;  &lt;/span&gt;&lt;span class="fu"&gt;pure-functional&lt;/span&gt;&lt;span class="kw"&gt;:&lt;/span&gt;&lt;span class="at"&gt; 10.1007/978-3-642-14162-1_17&lt;/span&gt;&lt;/span&gt;
  788. &lt;span id="cb1-25"&gt;&lt;a href="#cb1-25" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="at"&gt;  &lt;/span&gt;&lt;span class="fu"&gt;clairvoyant&lt;/span&gt;&lt;span class="kw"&gt;:&lt;/span&gt;&lt;span class="at"&gt; 10.1145/3341718&lt;/span&gt;&lt;/span&gt;
  789. &lt;span id="cb1-26"&gt;&lt;a href="#cb1-26" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="fu"&gt;subs&lt;/span&gt;&lt;span class="kw"&gt;:&lt;/span&gt;&lt;/span&gt;
  790. &lt;span id="cb1-27"&gt;&lt;a href="#cb1-27" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="at"&gt;  &lt;/span&gt;&lt;span class="kw"&gt;-&lt;/span&gt;&lt;span class="at"&gt; &lt;/span&gt;&lt;span class="fu"&gt;replace&lt;/span&gt;&lt;span class="kw"&gt;:&lt;/span&gt;&lt;span class="at"&gt; Peyton Jones&lt;/span&gt;&lt;/span&gt;
  791. &lt;span id="cb1-28"&gt;&lt;a href="#cb1-28" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="at"&gt;    &lt;/span&gt;&lt;span class="fu"&gt;with&lt;/span&gt;&lt;span class="kw"&gt;:&lt;/span&gt;&lt;span class="at"&gt; &lt;/span&gt;&lt;span class="st"&gt;'{Peyton Jones}'&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  792. &lt;p&gt;and turns it into a nice &lt;code&gt;.bibtex&lt;/code&gt; file:&lt;/p&gt;
  793. &lt;pre class="shell"&gt;&lt;code&gt;$ ./doi2bib.py &amp;lt; doibib.yaml &gt; dblp.bib
  794. $ head dblp.bib
  795. @inproceedings{unsafePerformIO,
  796.  author       = {Simon L. {Peyton Jones} and
  797.                  Simon Marlow and
  798.                  Conal Elliott},
  799.  editor       = {Pieter W. M. Koopman and
  800.                  Chris Clack},
  801.  title        = {Stretching the Storage Manager: Weak Pointers and Stable Names in
  802.                  Haskell},
  803.  booktitle    = {Implementation of Functional Languages, 11th International Workshop,
  804.                  IFL'99, Lochem, The Netherlands, September 7-10, 1999, Selected Papers},&lt;/code&gt;&lt;/pre&gt;
  805. &lt;p&gt;The last bit allows me to do some fine-tuning of the file, because unfortunately, not even DBLP BibTeX files are perfect, for example in the presence of two family names.&lt;/p&gt;
  806. &lt;p&gt;Now I have less moving parts to worry about, and a more consistent bibliography.&lt;/p&gt;
  807. &lt;p&gt;The script is rather small, so I’ll just share it here:&lt;/p&gt;
  808. &lt;div class="sourceCode" id="cb3"&gt;&lt;pre class="sourceCode python"&gt;&lt;code class="sourceCode python"&gt;&lt;span id="cb3-1"&gt;&lt;a href="#cb3-1" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="co"&gt;#!/usr/bin/env python3&lt;/span&gt;&lt;/span&gt;
  809. &lt;span id="cb3-2"&gt;&lt;a href="#cb3-2" aria-hidden="true" tabindex="-1"/&gt;&lt;/span&gt;
  810. &lt;span id="cb3-3"&gt;&lt;a href="#cb3-3" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="im"&gt;import&lt;/span&gt; sys&lt;/span&gt;
  811. &lt;span id="cb3-4"&gt;&lt;a href="#cb3-4" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="im"&gt;import&lt;/span&gt; yaml&lt;/span&gt;
  812. &lt;span id="cb3-5"&gt;&lt;a href="#cb3-5" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="im"&gt;import&lt;/span&gt; requests&lt;/span&gt;
  813. &lt;span id="cb3-6"&gt;&lt;a href="#cb3-6" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="im"&gt;import&lt;/span&gt; requests_cache&lt;/span&gt;
  814. &lt;span id="cb3-7"&gt;&lt;a href="#cb3-7" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="im"&gt;import&lt;/span&gt; re&lt;/span&gt;
  815. &lt;span id="cb3-8"&gt;&lt;a href="#cb3-8" aria-hidden="true" tabindex="-1"/&gt;&lt;/span&gt;
  816. &lt;span id="cb3-9"&gt;&lt;a href="#cb3-9" aria-hidden="true" tabindex="-1"/&gt;requests_cache.install_cache(backend&lt;span class="op"&gt;=&lt;/span&gt;&lt;span class="st"&gt;'sqlite'&lt;/span&gt;)&lt;/span&gt;
  817. &lt;span id="cb3-10"&gt;&lt;a href="#cb3-10" aria-hidden="true" tabindex="-1"/&gt;&lt;/span&gt;
  818. &lt;span id="cb3-11"&gt;&lt;a href="#cb3-11" aria-hidden="true" tabindex="-1"/&gt;data &lt;span class="op"&gt;=&lt;/span&gt; yaml.safe_load(sys.stdin)&lt;/span&gt;
  819. &lt;span id="cb3-12"&gt;&lt;a href="#cb3-12" aria-hidden="true" tabindex="-1"/&gt;&lt;/span&gt;
  820. &lt;span id="cb3-13"&gt;&lt;a href="#cb3-13" aria-hidden="true" tabindex="-1"/&gt;&lt;span class="cf"&gt;for&lt;/span&gt; key, doi &lt;span class="kw"&gt;in&lt;/span&gt; data[&lt;span class="st"&gt;'entries'&lt;/span&gt;].items():&lt;/span&gt;
  821. &lt;span id="cb3-14"&gt;&lt;a href="#cb3-14" aria-hidden="true" tabindex="-1"/&gt;    bib &lt;span class="op"&gt;=&lt;/span&gt; requests.get(&lt;span class="ss"&gt;f"https://dblp.org/doi/&lt;/span&gt;&lt;span class="sc"&gt;{&lt;/span&gt;doi&lt;span class="sc"&gt;}&lt;/span&gt;&lt;span class="ss"&gt;.bib"&lt;/span&gt;).text&lt;/span&gt;
  822. &lt;span id="cb3-15"&gt;&lt;a href="#cb3-15" aria-hidden="true" tabindex="-1"/&gt;    bib &lt;span class="op"&gt;=&lt;/span&gt; re.sub(&lt;span class="st"&gt;'{DBLP.*,'&lt;/span&gt;, &lt;span class="st"&gt;'{'&lt;/span&gt; &lt;span class="op"&gt;+&lt;/span&gt; key &lt;span class="op"&gt;+&lt;/span&gt; &lt;span class="st"&gt;','&lt;/span&gt;, bib)&lt;/span&gt;
  823. &lt;span id="cb3-16"&gt;&lt;a href="#cb3-16" aria-hidden="true" tabindex="-1"/&gt;    &lt;span class="cf"&gt;for&lt;/span&gt; subs &lt;span class="kw"&gt;in&lt;/span&gt; data[&lt;span class="st"&gt;'subs'&lt;/span&gt;]:&lt;/span&gt;
  824. &lt;span id="cb3-17"&gt;&lt;a href="#cb3-17" aria-hidden="true" tabindex="-1"/&gt;        bib &lt;span class="op"&gt;=&lt;/span&gt; re.sub(subs[&lt;span class="st"&gt;'replace'&lt;/span&gt;], subs[&lt;span class="st"&gt;'with'&lt;/span&gt;], bib)&lt;/span&gt;
  825. &lt;span id="cb3-18"&gt;&lt;a href="#cb3-18" aria-hidden="true" tabindex="-1"/&gt;    &lt;span class="bu"&gt;print&lt;/span&gt;(bib)&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  826. &lt;p&gt;There are similar projects out there, e.g. &lt;a href="https://github.com/cr-marcstevens/dblpbibtex"&gt;&lt;code&gt;dblpbibtex&lt;/code&gt;&lt;/a&gt; in C++ and &lt;a href="https://github.com/PJK/dblpbib"&gt;&lt;code&gt;dblpbib&lt;/code&gt;&lt;/a&gt; in Ruby. These allow direct use of &lt;code&gt;\cite{DBLP:rec/conf/isit/BreitnerS20}&lt;/code&gt; in Latex, which is also nice, but for now I like to choose more speaking citation keys myself.&lt;/p&gt;</description>
  827.         <pubDate>Wed, 12 Jul 2023 14:32:19 +0200</pubDate>
  828.      </item>
  829.      <item>
  830.         <title>ICFP Pearl preprint on rec-def</title>
  831.         <link>https://www.joachim-breitner.de/blog/805-ICFP_Pearl_preprint_on_rec-def</link>
  832.         <guid>https://www.joachim-breitner.de/blog/805-ICFP_Pearl_preprint_on_rec-def</guid>
  833.         <comments>https://www.joachim-breitner.de/blog/805-ICFP_Pearl_preprint_on_rec-def#comments</comments>
  834.         <author>mail@joachim-breitner.de (Joachim Breitner)</author>
  835.         <description>&lt;p&gt;I submitted a Functional Pearl to this &lt;a href="https://icfp23.sigplan.org/"&gt;year’s ICFP&lt;/a&gt; and it got accepted!&lt;/p&gt;
  836. &lt;p&gt;It is about the idea of using Haskell’s inherent ability to define &lt;em&gt;recursive equations&lt;/em&gt;, and use them for more than just functions and lazy data structures. I blogged about this before (&lt;a href="https://www.joachim-breitner.de/blog/792-More_recursive_definitions"&gt;introducing the idea&lt;/a&gt;, &lt;a href="https://www.joachim-breitner.de/blog/793-rec-def__Behind_the_scenes"&gt;behind the scenes&lt;/a&gt;, applications to &lt;a href="https://www.joachim-breitner.de/blog/794-rec-def__Program_analysis_case_study"&gt;program analysis&lt;/a&gt;, &lt;a href="https://www.joachim-breitner.de/blog/795-rec-def__Dominators_case_study"&gt;graph algorithms&lt;/a&gt; and &lt;a href="https://www.joachim-breitner.de/blog/796-rec-def__Minesweeper_case_study"&gt;minesweeper&lt;/a&gt;), but hopefully the paper brings out the idea even more clearly. The constructive feedback from a few friendly readers (Claudio, Sebastian, and also the anonymous reviewers) certainly improved the paper.&lt;/p&gt;
  837. &lt;blockquote&gt;
  838. &lt;h3 id="abstract"&gt;Abstract&lt;/h3&gt;
  839. &lt;p&gt;Haskell’s laziness allows the programmer to solve some problems naturally and declaratively via recursive equations. Unfortunately, if the input is “too recursive”, these very elegant idioms can fall into the dreaded black hole, and the programmer has to resort to more pedestrian approaches.&lt;/p&gt;
  840. &lt;p&gt;It does not have to be that way: We built variants of common pure data structures (Booleans, sets) where recursive definitions are productive. Internally, the infamous unsafePerformIO is at work, but the user only sees a beautiful and pure API, and their pretty recursive idioms – magically – work again.&lt;/p&gt;
  841. &lt;/blockquote&gt;
  842. &lt;p&gt;If you are curious, please have a look at the &lt;a href="//www.joachim-breitner.de/publications/rec-def-pearl.pdf"&gt;current version of the paper&lt;/a&gt;. Any feedback is welcome; even more if it comes before July 11, because then I can include it in the camera ready version.&lt;/p&gt;
  843. &lt;hr/&gt;
  844. &lt;p&gt;There are still some open questions around this work. What bothers me maybe most is the lack of a denotational semantics that unifies the partial order underlying the Haskell fragment, and the partial order underlying the domain of the embedded equations.&lt;/p&gt;
  845. &lt;p&gt;The crux of the probem is maybe best captured by this question:&lt;/p&gt;
  846. &lt;blockquote&gt;
  847. &lt;p&gt;Imagine an untyped lambda calculus with constructors, lazy evaluation, and an operation &lt;code&gt;rseq&lt;/code&gt; that recursively evaluates constructors, but terminates in the presence of cycles. So for example&lt;/p&gt;
  848. &lt;pre&gt;&lt;code&gt;rseq (let x    = 1 :: x    in x   ) ≡ ()
  849. rseq (let x () = 1 :: x () in x ()) ≡ ⊥&lt;/code&gt;&lt;/pre&gt;
  850. &lt;p&gt;In this language, knot tying is observable. What is the “nicest” denotational semantics.&lt;/p&gt;
  851. &lt;/blockquote&gt;
  852. &lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;: I made some progress via a &lt;a href="https://discourse.haskell.org/t/icfp-pearl-on-rec-def/6626/14?u=nomeata"&gt;discussion on the Haskell Discource&lt;/a&gt; and started &lt;a href="https://www.joachim-breitner.de/publications/rec-def-denotational.pdf"&gt;some rough notes on a denotational semantics&lt;/a&gt;.&lt;/p&gt;</description>
  853.         <pubDate>Thu, 22 Jun 2023 18:21:11 +0200</pubDate>
  854.      </item>
  855.      <item>
  856.         <title>The curious case of the half-half Bitcoin ECDSA nonces</title>
  857.         <link>https://www.joachim-breitner.de/blog/804-The_curious_case_of_the_half-half_Bitcoin_ECDSA_nonces</link>
  858.         <guid>https://www.joachim-breitner.de/blog/804-The_curious_case_of_the_half-half_Bitcoin_ECDSA_nonces</guid>
  859.         <comments>https://www.joachim-breitner.de/blog/804-The_curious_case_of_the_half-half_Bitcoin_ECDSA_nonces#comments</comments>
  860.         <author>mail@joachim-breitner.de (Joachim Breitner)</author>
  861.         <description>&lt;p&gt;This is the week of the Gulaschprogrammiernacht, a yearly Chaos Computer Club even in Karlsruhe, so it was exactly a year ago that I sat in my AirBnB room and went over the slides for my talk &lt;a href="https://media.ccc.de/v/gpn20-66-lattice-attacks-on-ethereum-bitcoin-and-https"&gt;“Lattice Attacks on Ethereum, Bitcoin, and HTTPS”&lt;/a&gt; that I would give there.&lt;/p&gt;
  862. &lt;p&gt;It reports on &lt;a href="https://eprint.iacr.org/2019/023"&gt;research&lt;/a&gt; done with &lt;a href="https://cseweb.ucsd.edu/~nadiah/"&gt;Nadia Heninger&lt;/a&gt; while I was in Phildalephia, and I really liked giving that talk: At some point we look at some rather odd signatures we found on the bitcoin blockchain, and part of the signature (the “nonce”) happens to share some bytes with the secret key. A clear case of some buffer overlap in a memory unsafe language, which I, as a fan of languages like Haskell, are very happy to sneer at!&lt;/p&gt;
  863. &lt;figure&gt;
  864. &lt;img src="/various/biased-nonces-slide-17.svg" alt="A sneery slide"/&gt;
  865. &lt;figcaption aria-hidden="true"&gt;A sneery slide&lt;/figcaption&gt;
  866. &lt;/figure&gt;
  867. &lt;p&gt;But last year, as I was going over &lt;a href="https://www.joachim-breitner.de/publications/BiasedNonces-GPN20-2022-05-20.pdf"&gt;the slides&lt;/a&gt; I looked at the raw data again for some reason, and I found that we overlooked something: Not only was the the upper half ot the nonce equal to the lower half of the secret key, but he lower half of the nonce was also equal to the upper half of the message hash!&lt;/p&gt;
  868. &lt;p&gt;This now looks much less like an accident to me, and more like a (overly) simple form of deterministic nonce creation… so much for my nice anecdote. (I still used the anecdote in my talk, followed up with an “actually”.)&lt;/p&gt;
  869. &lt;p&gt;When I told Nadia about this, she got curious as well, and quickly saw that from a signature with such a nonce, one can rather easily extract the secret key. So together with her student Dylan Rowe, we implemented this analysis and searched the bitcoin blockchain for more instance of such signatures. We did find a few, and were even able to trace them back to a somewhat infamous bitcoin activist going under the pseudonym Amaclin.&lt;/p&gt;
  870. &lt;p&gt;This research and sleuthing turned into another paper, &lt;a href="https://eprint.iacr.org/2023/841"&gt;“The curious case of the half-half Bitcoin ECDSA nonces”&lt;/a&gt;, to be presented at AfricaCrypt 2023. Enjoy!&lt;/p&gt;</description>
  871.         <pubDate>Wed, 07 Jun 2023 08:42:28 +0200</pubDate>
  872.      </item>
  873.      <item>
  874.         <title>Giving back to OPLSS</title>
  875.         <link>https://www.joachim-breitner.de/blog/803-Giving_back_to_OPLSS</link>
  876.         <guid>https://www.joachim-breitner.de/blog/803-Giving_back_to_OPLSS</guid>
  877.         <comments>https://www.joachim-breitner.de/blog/803-Giving_back_to_OPLSS#comments</comments>
  878.         <author>mail@joachim-breitner.de (Joachim Breitner)</author>
  879.         <description>&lt;p&gt;Nine years ago, when I was a PhD student, I attended the &lt;a href="https://www.cs.uoregon.edu/research/summerschool/"&gt;Oregon Programming Language Summer School&lt;/a&gt; in Eugene. I had a great time and learned a lot.&lt;/p&gt;
  880. &lt;figure&gt;
  881. &lt;img src="/various/OPLSS14Photo-small.jpg" alt="The OPLSS’14 full image"/&gt;
  882. &lt;figcaption aria-hidden="true"&gt;The OPLSS’14 &lt;a href="/various/OPLSS14Photo.jpg"&gt;full image&lt;/a&gt;&lt;/figcaption&gt;
  883. &lt;/figure&gt;
  884. &lt;p&gt;Learning some of the things I learned there, and meeting some of the people I met there, also led to me graduating, which led to me becoming &lt;a href="https://www.cis.upenn.edu/~plclub/"&gt;a PostDoc at UPenn&lt;/a&gt;, which led to me later joining DFINITY to implement the &lt;a href="https://github.com/dfinity/motoko/"&gt;Motoko programming language&lt;/a&gt; and help design and specify the public interface of their “Internet Computer”, including the &lt;a href="https://medium.com/dfinity/how-internet-computer-responses-are-certified-as-authentic-2ff1bb1ea659"&gt;response certification&lt;/a&gt; (&lt;a href="https://www.youtube.com/watch?v=mZbFhRIHIiY"&gt;video&lt;/a&gt;).&lt;/p&gt;
  885. &lt;p&gt;So when the &lt;a href="https://icdevs.org/"&gt;ICDevs&lt;/a&gt; non-profit offered a &lt;a href="https://icdevs.org/bounties/2023/01/09/36-Signing-Tree-and-DER-Encoding.html"&gt;development bounty for a Motoko library implementing the merkle trees involved in certification&lt;/a&gt;, this sounded like a fun little coding task, so I completed it; likely with less effort than it would have taken someone who first had to get into these topics.&lt;/p&gt;
  886. &lt;p&gt;The bounty was quite generous, at US$ 10k, and I was too vain to “just” have it donated to some large charity, as I &lt;a href="//www.joachim-breitner.de/blog/798-Pro-charity_consulting"&gt;recently with a few coding and consulting gigs&lt;/a&gt;, and looked for more personal. So, the ICDevs guys and I agreed to donate the money to &lt;a href="https://www.cs.uoregon.edu/research/summerschool/summer23/"&gt;this year’s OPLSS&lt;/a&gt;, where I heard it can cover the cost of about 8 students, and hopefully helps the PL cause.&lt;/p&gt;
  887. &lt;p&gt;(You will not find us listed as sponsors because for some reason, a “donation” instead of “sponsorship” comes with less strings attached to the organizers.)&lt;/p&gt;</description>
  888.         <pubDate>Sun, 04 Jun 2023 14:18:33 +0200</pubDate>
  889.      </item>
  890.      <item>
  891.         <title>More thoughts on a bootstrappable GHC</title>
  892.         <link>https://www.joachim-breitner.de/blog/802-More_thoughts_on_a_bootstrappable_GHC</link>
  893.         <guid>https://www.joachim-breitner.de/blog/802-More_thoughts_on_a_bootstrappable_GHC</guid>
  894.         <comments>https://www.joachim-breitner.de/blog/802-More_thoughts_on_a_bootstrappable_GHC#comments</comments>
  895.         <author>mail@joachim-breitner.de (Joachim Breitner)</author>
  896.         <description>&lt;p&gt;The &lt;a href="https://bootstrappable.org/"&gt;bootstrappable builds project&lt;/a&gt; tries to find ways of building all our software from source, without relying on binary artifacts. A noble goal, and one that is often thwarted by languages with self-hosting compilers, like GHC: In order to build GHC, you need GHC. A &lt;a href="https://github.com/NixOS/nixpkgs/pull/227914"&gt;Pull Request against nixpkgs&lt;/a&gt;, adding first steps of the bootstrapping pipeline, reminded me of the issue with GHC, which I have noted down &lt;a href="https://www.joachim-breitner.de/blog/748-Thoughts_on_bootstrapping_GHC"&gt;some thoughts about&lt;/a&gt; before and I played around a bit more.&lt;/p&gt;
  897. &lt;p&gt;The most promising attempt to bootstrap GHC was done by &lt;a href="https://elephly.net/posts/2017-01-09-bootstrapping-haskell-part-1.html"&gt;rekado in 2017&lt;/a&gt;. He observed that Hugs is maybe the most recently maintained bootstrappable (since written in C) Haskell compiler, but noticed that “it cannot deal with mutually recursive module dependencies, which is a feature that even the earliest versions of GHC rely on. This means that running a variant of GHC inside of Hugs is not going to work without major changes.” He then tries to bootstrap another very old Haskell compiler (nhc) with Hugs, and makes good but incomplete progress.&lt;/p&gt;
  898. &lt;p&gt;This made me wonder: What &lt;em&gt;if&lt;/em&gt; Hugs supported mutually recursive modules? Would that make a big difference? Anthony Clayden &lt;a href="https://www.joachim-breitner.de/blog/748-Thoughts_on_bootstrapping_GHC#comment_3"&gt;keeps advocating Hugs as a viable Haskell implementation&lt;/a&gt;, so maybe if that was the main blocker, then adding support to Hugs for that is probably not too hard (at least in a compile-the-strongly-connected-component-as-one-unit mode) and worthwhile?&lt;/p&gt;
  899. &lt;h3 id="all-of-ghc-in-one-file"&gt;All of GHC in one file?&lt;/h3&gt;
  900. &lt;p&gt;That reminded me of a situation I was in before, where I had to combine multiple Haskell modules into one before: For my talk &lt;a href="https://www.youtube.com/watch?v=2kKvVe673MA"&gt;“Lock-step simulation is child’s play”&lt;/a&gt; I wrote a multi-player game, a simulation environment for it, and a presentation tool around it, all in the &lt;a href="https://code.world"&gt;CodeWorld&lt;/a&gt; programming environment, which supports only a single module. So I hacked the a small tool &lt;a href="https://github.com/nomeata/hs-all-in-one/"&gt;hs-all-in-one&lt;/a&gt; that takes multiple Haskell modules and combines them into one, mangling the names to avoid name clashes.&lt;/p&gt;
  901. &lt;p&gt;This made me wonder: Can I turn &lt;em&gt;all&lt;/em&gt; of GHC into one module, and compile that?&lt;/p&gt;
  902. &lt;p&gt;At this point I have probably left the direct path towards bootstrapping, but I kinda good hooked.&lt;/p&gt;
  903. &lt;ol type="1"&gt;
  904. &lt;li&gt;&lt;p&gt;Using GHC’s &lt;code&gt;hadrian/ghci&lt;/code&gt; tool, I got it to produce the necessary generated files (e.g. from &lt;code&gt;happy&lt;/code&gt; grammars) and spit out the lit of modules that make up GHC, which I could feed to &lt;code&gt;hs-all-in-one&lt;/code&gt;.&lt;/p&gt;&lt;/li&gt;
  905. &lt;li&gt;&lt;p&gt;It uses &lt;a href="https://hackage.haskell.org/package/haskell-src-exts"&gt;&lt;code&gt;haskell-src-exts&lt;/code&gt;&lt;/a&gt; for parsing, and it was almost able to parse all of that. It has a different opinion about how &lt;a href="https://github.com/haskell-suite/haskell-src-exts/issues/468"&gt;&lt;code&gt;MultiWayIf&lt;/code&gt; should be indented&lt;/a&gt;, whether &lt;a href="https://github.com/haskell-suite/haskell-src-exts/issues/469"&gt;&lt;code&gt;EmptyCase&lt;/code&gt; needs &lt;code&gt;{}&lt;/code&gt;&lt;/a&gt; and &lt;a href="https://github.com/haskell-suite/haskell-src-exts/issues/470"&gt;issues&lt;/a&gt; &lt;a href="https://github.com/haskell-suite/haskell-src-exts/issues/471"&gt;pretty-printing&lt;/a&gt; some promoted values, but otherwise the round-tripping worked fine, and I as able to generate a large file (680,000 loc, 41 MB) that passes GHC’s parser.&lt;/p&gt;&lt;/li&gt;
  906. &lt;li&gt;&lt;p&gt;It also uses &lt;a href="https://hackage.haskell.org/package/haskell-names"&gt;&lt;code&gt;haskell-names&lt;/code&gt;&lt;/a&gt; to resolve names.&lt;/p&gt;
  907. &lt;p&gt;This library is less up-to-date with various Haskell features, so I added support for renaming in some pragmas (&lt;code&gt;ANN&lt;/code&gt;, &lt;code&gt;SPECIALIZE&lt;/code&gt;), pattern signatures etc.&lt;/p&gt;
  908. &lt;p&gt;For my previous use-case I could just combine all the imports, knowing that I would not introduce conflicts. For GHC, this is far from true: Some modules import &lt;code&gt;Data.Map.Strict&lt;/code&gt;, others &lt;code&gt;Data.Map.Lazy&lt;/code&gt;, and yet others introduce names that clash with stuff imported from the prelude… so I had to change the tool to fully qualify all imported values. This isn’t so bad, I can do that using &lt;code&gt;haskell-names&lt;/code&gt;, if I somehow know what all the modules in &lt;code&gt;base&lt;/code&gt;, &lt;code&gt;containers&lt;/code&gt;, &lt;code&gt;transformers&lt;/code&gt; and &lt;code&gt;array&lt;/code&gt; export.&lt;/p&gt;
  909. &lt;p&gt;The &lt;code&gt;haskell-names&lt;/code&gt; library itself comes with a hard-coded database of &lt;code&gt;base&lt;/code&gt; exports, but it is incomplete and doesn’t help me with, say, &lt;code&gt;containers&lt;/code&gt;.&lt;/p&gt;
  910. &lt;p&gt;I then wrote a little parser for the &lt;code&gt;.txt&lt;/code&gt; files that &lt;code&gt;haddock&lt;/code&gt; produces for the benefit of &lt;code&gt;hoogle&lt;/code&gt;, and that are conveniently installed along the packages (at least on nix). This would have been great, if these files wouldn’t simply omit all &lt;em&gt;reexported&lt;/em&gt; entities! I added some manual hacks (&lt;code&gt;Data.Map&lt;/code&gt; has the same exports as &lt;code&gt;Data.IntMap&lt;/code&gt;; &lt;code&gt;Prelude&lt;/code&gt; exports all entities as known by &lt;code&gt;haskell-names&lt;/code&gt;, but those that are also exported from &lt;code&gt;Data.List&lt;/code&gt;, use the symbol from there…)&lt;/p&gt;
  911. &lt;p&gt;I played this game of whack-a-mole for a while, solving many of the problems that GHC’s renamer reports, but eventually stopped to write this blog post. I am fairly confident that this could be pulled through, though.&lt;/p&gt;&lt;/li&gt;
  912. &lt;/ol&gt;
  913. &lt;h3 id="back-to-bootstrapping"&gt;Back to bootstrapping&lt;/h3&gt;
  914. &lt;p&gt;So what if we could pull this through? We’d have a very large code file that GHC may be able to compile to produce a &lt;code&gt;ghc&lt;/code&gt; binary without exhausting my RAM. But that doesn’t help with bootstrapping yet.&lt;/p&gt;
  915. &lt;p&gt;If lack of support for recursive modules is all that Hugs is missing, we’d be done indeed. But quite contrary, it is probably the least of our worries, given that contemporary GHC uses many many other features not supported by Hugs.&lt;/p&gt;
  916. &lt;p&gt;Some of them a syntactic and can easily be rewritten to more normal Haskell in a preprocessing step (e.g. &lt;code&gt;MultiWayIf&lt;/code&gt;).&lt;/p&gt;
  917. &lt;p&gt;Others are deep and hard (&lt;code&gt;GADTs&lt;/code&gt;, Pattern synonyms, Type Families), and prohibit attempting to compile a current version of GHC (even if its all one module) with Hugs. So one would certainly have to go back in time and find a version of GHC that is not yet using all these features. For example, the &lt;a href="https://gitlab.haskell.org/ghc/ghc/-/commit/889c084e943779e76d19f2ef5e970ff655f511eb"&gt;first use of GADTs&lt;/a&gt; was introduced by Simon Marlow in 2011, so this suggests going back to at least GHC 7.0.4, maybe earlier.&lt;/p&gt;
  918. &lt;p&gt;Still, being able to mangle the source code before passing it to Hugs is probably a useful thing. This poses the question whether Hugs can compile such a tool; in particular, is it capable of compiling &lt;code&gt;haskell-src-exts&lt;/code&gt;, which I am not too optimistic about either. Did someone check this already?&lt;/p&gt;
  919. &lt;p&gt;So one plan of attack could be&lt;/p&gt;
  920. &lt;ol type="1"&gt;
  921. &lt;li&gt;&lt;p&gt;Identify an old version of GHC that&lt;/p&gt;
  922. &lt;ul&gt;
  923. &lt;li&gt;One &lt;em&gt;can&lt;/em&gt; use to bootstrap subsequent versions until today.&lt;/li&gt;
  924. &lt;li&gt;Is old enough to use as few features not supported by hugs as possible.&lt;/li&gt;
  925. &lt;li&gt;Is still new enough so that one can obtain a compatible toolchain.&lt;/li&gt;
  926. &lt;/ul&gt;&lt;/li&gt;
  927. &lt;li&gt;&lt;p&gt;Wrangle the build system to tell you which files to compile, with which preprocessor flags etc.&lt;/p&gt;&lt;/li&gt;
  928. &lt;li&gt;&lt;p&gt;Boostrap all pre-processing tools used by GHC (&lt;code&gt;cpphs&lt;/code&gt; or use plan cpp, &lt;code&gt;happy&lt;/code&gt;, &lt;code&gt;alex&lt;/code&gt;).&lt;/p&gt;&lt;/li&gt;
  929. &lt;li&gt;&lt;p&gt;For every language feature not supported by Hugs, either&lt;/p&gt;
  930. &lt;ul&gt;
  931. &lt;li&gt;Implement it in Hugs,&lt;/li&gt;
  932. &lt;li&gt;Manually edit the source code to avoid compiling the problematic code, if it is optional (e.g. in an optimization pass)&lt;/li&gt;
  933. &lt;li&gt;Rewrite the problematic code&lt;/li&gt;
  934. &lt;li&gt;Write a pre-processing tool (like the one above) that compiles the feature away&lt;/li&gt;
  935. &lt;/ul&gt;&lt;/li&gt;
  936. &lt;li&gt;&lt;p&gt;Similarly, since Hugs probably ships a &lt;code&gt;base&lt;/code&gt; that is different than what GHC, or the libraries used by GHC expects, either adjust Hugs’ &lt;code&gt;base&lt;/code&gt;, or modify the GHC code that uses it.&lt;/p&gt;&lt;/li&gt;
  937. &lt;/ol&gt;
  938. &lt;p&gt;My actual plan, though, for now is to throw these thoughts out, maybe make some noise on &lt;a href="https://discourse.haskell.org/t/what-s-needed-to-bootstrap-ghc-with-hugs/6205"&gt;Discourse&lt;/a&gt;, &lt;a href="https://mastodon.online/@nomeata/110263917613134533"&gt;Mastodon&lt;/a&gt;, &lt;a href="https://twitter.com/nomeata/status/1651125309700206593"&gt;Twitter&lt;/a&gt; and &lt;a href="https://lobste.rs/s/di37ga"&gt;lobste.rs&lt;/a&gt;, and then let it sit and hope someone else will pick it up.&lt;/p&gt;</description>
  939.         <pubDate>Wed, 26 Apr 2023 07:41:50 +0200</pubDate>
  940.      </item>
  941.   </channel>
  942. </rss>

If you would like to create a banner that links to this page (i.e. this validation result), do the following:

  1. Download the "valid RSS" banner.

  2. Upload the image to your own server. (This step is important. Please do not link directly to the image on this server.)

  3. Add this HTML to your page (change the image src attribute if necessary):

If you would like to create a text link instead, here is the URL you can use:

http://www.feedvalidator.org/check.cgi?url=http%3A//www.joachim-breitner.de/blog/feeds/categories/1-English.rss

Home · About · News · Docs · Terms

 
Copyright © 2002-9 Sam Ruby, Mark Pilgrim, Joseph Walton, and Phil Ringnalda