Hyperlink Your Heart - Programminghttps://blog.hyperlinkyourheart.com/2022-05-26T20:16:00+02:00Until there's nothing left.Nimpressions2022-05-26T20:16:00+02:002022-05-26T20:16:00+02:00Kevin Houlihantag:blog.hyperlinkyourheart.com,2022-05-26:/nimpressions.html<p>First impressions of the Nim programming language</p><p>Python is my go-to language for personal projects, and even client projects when I can get away with it (though usually those are Windows based and within the .Net ecosystem, so I stick with C#). However, it often gives me pause to be using one of the slowest and <a href="https://thenewstack.io/which-programming-languages-use-the-least-electricity/" title="Programming languages energy efficiency">least energy efficient</a> languages available - I might do another post about that, but suffice it to say that it doesn’t align with my values to needlessly waste resources.</p>
<p>The ideal would be a language that’s as easy to write as Python, but as fast and energy efficient as C, or close to it. Well recently I came across a language that claims be both of those things: <a href="https://nim-lang.org/" title="Nim homepage">Nim</a>.</p>
<p>I put together a simple command line application (named <a href="https://github.com/khoulihan/luz" title="Luz on GitHub">Luz</a>) in Nim this week in order to try it out. Appropriately enough given my reason for trying Nim, it just shows the current electricity rate band, and optionally a chart, because where I live there are two peak periods during the day when it is better not to do anything power-intensive. I went on to make a start on a very simple Gemini server called Sparkle, which is still a <span class="caps">WIP</span>. Here are some of my thoughts on the experience as a mediocre developer with some Python and C# experience.</p>
<p><a href="https://github.com/khoulihan/luz" title="Luz on GitHub"><img alt="Luz in action" src="https://blog.hyperlinkyourheart.com/images/nimpressions/luz_screenshot.png" title="Going from bad to worse"></a></p>
<h2>choosenim</h2>
<p>Nim has a <a href="https://github.com/dom96/choosenim" title="choosenim on GitHub">tool for installing its toolchain</a> and and switching between different versions of the compiler, similar to pyenv. Unfortunately it didn’t work for me on Pop! <span class="caps">OS</span> 22.04 due to it having too new a version of libssl. I was able to install the Nim compiler manually easily enough by just downloading the tarball and copying the contents to an appropriate location, and then adding the <code>bin</code> directory to my path. There was an install script in the tarball but it didn’t copy everything for some reason.</p>
<p>Not a great start, and I’m not sure what I’m missing out on by not using choosenim, but I can figure that out later if I continue using the language.</p>
<h2>Typing</h2>
<p>Static typing is something I’m well used to from C# of course, but I don’t engage with Python’s type hinting at all. There is type inference in many situations, and many familiar collection types such as sets, tables, sequences and tuples which are as convenient to instantiate as their Python equivalents, though of course you can’t mix unrelated types within them (aside from tuples)(and why would you do that anyway, you monster). Mostly it is just convenient to know at compile time where there are type mismatches, rather than hearing about them at runtime or just getting weird behaviour.</p>
<p>Nim is only very minimally object-oriented. There is inheritance, but not multiple-inheritance, mixins, or anything resembling the interfaces or traits of other languages. This is probably one of the most concerning aspects of the language for me. It seems like it will inevitably lead to repeated code at some point if procedures can’t accept abstract interfaces as input instead of concrete types.</p>
<p>On the other hand I try to steer away from an object-oriented style in Python unless it really makes sense for the problem I’m working on. In Luz, the classes I created were little more than structs, with no inheritance required, and that’s perfectly sufficient for many problems.</p>
<p>There are also apparently <a href="https://github.com/yglukhov/iface" title="iface library on GitHub">libraries</a> that create a means to specify interfaces using meta-programming, but that’s not something I’ve explored yet.</p>
<div class="highlight"><pre><span></span><code><span class="k">type</span>
<span class="w"> </span><span class="n">Holiday</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">ref</span><span class="w"> </span><span class="k">object</span>
<span class="w"> </span><span class="n">date</span><span class="p">:</span><span class="w"> </span><span class="n">DateTime</span>
<span class="w"> </span><span class="n">localName</span><span class="p">:</span><span class="w"> </span><span class="nb">string</span>
<span class="w"> </span><span class="n">name</span><span class="p">:</span><span class="w"> </span><span class="nb">string</span>
<span class="w"> </span><span class="n">countryCode</span><span class="p">:</span><span class="w"> </span><span class="nb">string</span>
<span class="w"> </span><span class="n">fixed</span><span class="p">:</span><span class="w"> </span><span class="nb">bool</span>
<span class="w"> </span><span class="n">global</span><span class="p">:</span><span class="w"> </span><span class="nb">bool</span>
<span class="w"> </span><span class="n">counties</span><span class="p">:</span><span class="w"> </span><span class="n">Option</span><span class="o">[</span><span class="nb">seq</span><span class="o">[</span><span class="nb">string</span><span class="o">]]</span>
<span class="w"> </span><span class="n">launchYear</span><span class="p">:</span><span class="w"> </span><span class="n">Option</span><span class="o">[</span><span class="nb">int</span><span class="o">]</span>
<span class="kd">var</span><span class="w"> </span><span class="n">holidays</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">initTable</span><span class="o">[</span><span class="nb">int</span><span class="p">,</span><span class="w"> </span><span class="nb">seq</span><span class="o">[</span><span class="n">Holiday</span><span class="o">]]</span><span class="p">()</span>
<span class="k">proc</span><span class="w"> </span><span class="nf">isHoliday</span><span class="o">*</span><span class="p">(</span><span class="n">d</span><span class="p">:</span><span class="w"> </span><span class="n">DateTime</span><span class="p">):</span><span class="w"> </span><span class="nb">bool</span><span class="w"> </span><span class="o">=</span>
<span class="w"> </span><span class="n">result</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kp">false</span>
<span class="w"> </span><span class="c"># This will occur if API key was not provided</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="ow">not</span><span class="w"> </span><span class="n">holidays</span><span class="p">.</span><span class="n">hasKey</span><span class="p">(</span><span class="n">d</span><span class="p">.</span><span class="n">year</span><span class="p">):</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">result</span>
<span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">y</span><span class="p">,</span><span class="w"> </span><span class="n">h</span><span class="w"> </span><span class="ow">in</span><span class="w"> </span><span class="n">holidays</span><span class="o">[</span><span class="n">d</span><span class="p">.</span><span class="n">year</span><span class="o">]</span><span class="p">:</span>
<span class="w"> </span><span class="c"># global indicates that the holiday applies to the whole country</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="n">h</span><span class="p">.</span><span class="n">global</span><span class="p">:</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="n">h</span><span class="p">.</span><span class="n">date</span><span class="p">.</span><span class="n">yearday</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="n">d</span><span class="p">.</span><span class="n">yearday</span><span class="p">:</span>
<span class="w"> </span><span class="n">result</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kp">true</span>
<span class="w"> </span><span class="k">break</span>
</code></pre></div>
<h2>Uniform Function Call Syntax</h2>
<p>This is really neat - any procedure or function can be called as if it is a method of the type of its first parameter.</p>
<div class="highlight"><pre><span></span><code><span class="k">proc</span><span class="w"> </span><span class="nf">sendErrorResponse</span><span class="p">(</span>
<span class="w"> </span><span class="n">requestSocket</span><span class="p">:</span><span class="w"> </span><span class="n">AsyncSocket</span><span class="p">,</span>
<span class="w"> </span><span class="n">code</span><span class="p">:</span><span class="w"> </span><span class="n">StatusCode</span><span class="p">,</span>
<span class="w"> </span><span class="n">meta</span><span class="p">:</span><span class="w"> </span><span class="nb">string</span>
<span class="p">)</span><span class="w"> </span><span class="sx">{.async.}</span><span class="w"> </span><span class="o">=</span>
<span class="w"> </span><span class="n">await</span><span class="w"> </span><span class="n">requestSocket</span><span class="p">.</span><span class="n">send</span><span class="p">(</span><span class="o">&</span><span class="s">"{ord(code)} {meta}</span><span class="se">\r\L</span><span class="s">"</span><span class="p">)</span>
<span class="k">proc</span><span class="w"> </span><span class="nf">processRequest</span><span class="p">(</span><span class="n">requestSocket</span><span class="p">:</span><span class="w"> </span><span class="n">AsyncSocket</span><span class="p">)</span><span class="w"> </span><span class="sx">{.async.}</span><span class="w"> </span><span class="o">=</span>
<span class="w"> </span><span class="p">...</span>
<span class="w"> </span><span class="c"># These calls are equivalent</span>
<span class="w"> </span><span class="n">await</span><span class="w"> </span><span class="n">requestSocket</span><span class="p">.</span><span class="n">sendErrorResponse</span><span class="p">(</span>
<span class="w"> </span><span class="n">StatusCode</span><span class="p">.</span><span class="n">notFound</span><span class="p">,</span>
<span class="w"> </span><span class="s">"Not Found"</span>
<span class="w"> </span><span class="p">)</span>
<span class="w"> </span><span class="n">await</span><span class="w"> </span><span class="n">sendErrorResponse</span><span class="p">(</span>
<span class="w"> </span><span class="n">requestSocket</span><span class="p">,</span>
<span class="w"> </span><span class="n">StatusCode</span><span class="p">.</span><span class="n">notFound</span><span class="p">,</span>
<span class="w"> </span><span class="s">"Not Found"</span>
<span class="w"> </span><span class="p">)</span>
</code></pre></div>
<p>This means that any type can be “extended” in a sense just by writing procedures with that type as the first parameter, no need for sub-classing or a special extension method syntax.</p>
<h2>Blocks</h2>
<p>One neat little feature is that you can open a new code block anywhere, with or without a name, and as well as being visually separated from the code around it it will have its own scope. A <code>break</code> statement will break out of that block, but not the containing one.</p>
<p>I didn’t find much use for this in either of the projects I’ve worked on so far, but it’s definitely something I can see being useful for longer procedures and certain control-flow situations.</p>
<h2>Closures</h2>
<p>Nim supports passing around references to procedures, which allows for a number of neat constructs, including closures. The below procedure creates a closure that animates a spinner when called in a loop while waiting for an <span class="caps">IO</span> operation to conclude. It contains everything it needs, including a constant.</p>
<div class="highlight"><pre><span></span><code><span class="k">proc</span><span class="w"> </span><span class="nf">getDisplayProgressClosure</span><span class="p">():</span><span class="w"> </span><span class="n">proc</span><span class="p">()</span><span class="w"> </span><span class="o">=</span>
<span class="w"> </span><span class="k">const</span><span class="w"> </span><span class="n">phases</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">[</span><span class="s">"🮪"</span><span class="p">,</span><span class="w"> </span><span class="s">"🮫"</span><span class="p">,</span><span class="w"> </span><span class="s">"🮭"</span><span class="p">,</span><span class="w"> </span><span class="s">"🮬"</span><span class="o">]</span>
<span class="w"> </span><span class="kd">var</span><span class="w"> </span><span class="n">lastTime</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">now</span><span class="p">()</span>
<span class="w"> </span><span class="kd">var</span><span class="w"> </span><span class="n">phase</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">0</span>
<span class="w"> </span><span class="kd">var</span><span class="w"> </span><span class="n">initial</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kp">true</span>
<span class="w"> </span><span class="k">proc</span><span class="w"> </span><span class="nf">displayProgress</span><span class="p">()</span><span class="w"> </span><span class="o">=</span>
<span class="w"> </span><span class="k">let</span><span class="w"> </span><span class="n">elapsed</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">now</span><span class="p">()</span><span class="w"> </span><span class="o">-</span><span class="w"> </span><span class="n">lastTime</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="n">elapsed</span><span class="p">.</span><span class="n">inMilliseconds</span><span class="w"> </span><span class="o">></span><span class="w"> </span><span class="mi">100</span><span class="w"> </span><span class="ow">or</span><span class="w"> </span><span class="n">initial</span><span class="p">:</span>
<span class="w"> </span><span class="n">lastTime</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">now</span><span class="p">()</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="ow">not</span><span class="w"> </span><span class="n">initial</span><span class="p">:</span>
<span class="w"> </span><span class="n">erasePrevious</span>
<span class="w"> </span><span class="n">initial</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kp">false</span>
<span class="w"> </span><span class="n">styledEcho</span><span class="p">(</span>
<span class="w"> </span><span class="n">fgGreen</span><span class="p">,</span>
<span class="w"> </span><span class="o">&</span><span class="s">"{phases[phase]}"</span><span class="p">,</span>
<span class="w"> </span><span class="n">fgCyan</span><span class="p">,</span>
<span class="w"> </span><span class="s">" Retrieving holidays..."</span>
<span class="w"> </span><span class="p">)</span>
<span class="w"> </span><span class="n">inc</span><span class="p">(</span><span class="n">phase</span><span class="p">)</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="n">phase</span><span class="w"> </span><span class="o">></span><span class="w"> </span><span class="n">phases</span><span class="p">.</span><span class="n">high</span><span class="p">:</span><span class="w"> </span><span class="n">phase</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">0</span>
<span class="w"> </span><span class="n">result</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">displayProgress</span>
</code></pre></div>
<h2>Templates <span class="amp">&</span> Compile Time Execution</h2>
<p>One of the most exciting features of Nim, for me, is the ability to execute code at compile time, and otherwise manipulate the final state of the code.</p>
<p>For example to embed a file in a binary in C# you have to set a property against the file in the <span class="caps">IDE</span> (or maybe in the project file) to make it an embedded resource, and then do some reflection to pull it back out at runtime. In Nim, you can just call <code>readFile</code> and assign the result to a constant.</p>
<div class="highlight"><pre><span></span><code><span class="k">const</span><span class="w"> </span><span class="n">DEFAULT_BANDS</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">readFile</span><span class="w"> </span><span class="s">"./config/bands.json"</span>
<span class="k">const</span><span class="w"> </span><span class="n">DEFAULT_CONFIG</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">readFile</span><span class="w"> </span><span class="s">"./config/luz.toml"</span>
</code></pre></div>
<p>There is also a compile-time branching statement, <code>when</code>. This is similar to the pre-processor <code>#if</code> in C#, or <code>#ifdef</code> in C, but it fits more naturally with the rest of the code.</p>
<p>Templates allow you to insert specified code in other parts of the codebase, with substitutions, before compilation. One use for this is as an alternative to short procedures, so the code gets inlined, saving a function call.</p>
<p>I feel like I’m only at the start of getting my head around this feature. I thought it might be a good way to output variations of a procedure for operating on different types, but I’m not sure the result is readable or concise enough to be worthwhile:</p>
<div class="highlight"><pre><span></span><code><span class="k">template</span><span class="w"> </span><span class="nf">createGetSetting</span><span class="p">(</span>
<span class="w"> </span><span class="n">valueType</span><span class="p">:</span><span class="w"> </span><span class="n">untyped</span><span class="p">,</span>
<span class="w"> </span><span class="n">argValueTypeGet</span><span class="p">:</span><span class="w"> </span><span class="n">untyped</span><span class="p">,</span>
<span class="w"> </span><span class="n">envValueTypeGet</span><span class="p">:</span><span class="w"> </span><span class="n">untyped</span><span class="p">,</span>
<span class="w"> </span><span class="n">confValueTypeGet</span><span class="p">:</span><span class="w"> </span><span class="n">untyped</span>
<span class="p">)</span><span class="w"> </span><span class="o">=</span>
<span class="w"> </span><span class="k">proc</span><span class="w"> </span><span class="nf">getSetting</span><span class="p">(</span>
<span class="w"> </span><span class="n">args</span><span class="p">:</span><span class="w"> </span><span class="n">Table</span><span class="o">[</span><span class="nb">string</span><span class="p">,</span><span class="w"> </span><span class="n">Value</span><span class="o">]</span><span class="p">,</span>
<span class="w"> </span><span class="n">arg</span><span class="p">:</span><span class="w"> </span><span class="nb">string</span><span class="p">,</span>
<span class="w"> </span><span class="n">conf</span><span class="p">:</span><span class="w"> </span><span class="n">TomlValueRef</span><span class="p">,</span>
<span class="w"> </span><span class="n">confSection</span><span class="p">:</span><span class="w"> </span><span class="nb">string</span><span class="p">,</span>
<span class="w"> </span><span class="n">confKey</span><span class="p">:</span><span class="w"> </span><span class="nb">string</span><span class="p">,</span>
<span class="w"> </span><span class="n">env</span><span class="p">:</span><span class="w"> </span><span class="nb">string</span><span class="p">,</span>
<span class="w"> </span><span class="n">default</span><span class="p">:</span><span class="w"> </span><span class="n">valueType</span>
<span class="w"> </span><span class="p">):</span><span class="w"> </span><span class="p">(</span><span class="n">valueType</span><span class="p">,</span><span class="w"> </span><span class="n">ConfigVariableSource</span><span class="p">)</span><span class="w"> </span><span class="o">=</span>
<span class="w"> </span><span class="n">result</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="n">default</span><span class="p">,</span><span class="w"> </span><span class="n">ConfigVariableSource</span><span class="p">.</span><span class="n">Default</span><span class="p">)</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="n">arg</span><span class="w"> </span><span class="ow">in</span><span class="w"> </span><span class="n">args</span><span class="p">:</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="n">args</span><span class="o">[</span><span class="n">arg</span><span class="o">]</span><span class="p">.</span><span class="n">kind</span><span class="w"> </span><span class="o">!=</span><span class="w"> </span><span class="n">vkNone</span><span class="p">:</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="n">argValueTypeGet</span><span class="p">(</span><span class="n">args</span><span class="o">[</span><span class="n">arg</span><span class="o">]</span><span class="p">),</span>
<span class="w"> </span><span class="n">ConfigVariableSource</span><span class="p">.</span><span class="n">CommandLine</span>
<span class="w"> </span><span class="p">)</span>
<span class="w"> </span><span class="k">let</span><span class="w"> </span><span class="n">envStr</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">getEnv</span><span class="p">(</span><span class="n">env</span><span class="p">,</span><span class="w"> </span><span class="s">""</span><span class="p">)</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="n">envStr</span><span class="w"> </span><span class="o">!=</span><span class="w"> </span><span class="s">""</span><span class="p">:</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="n">envValueTypeGet</span><span class="p">(</span><span class="n">envStr</span><span class="p">),</span>
<span class="w"> </span><span class="n">ConfigVariableSource</span><span class="p">.</span><span class="n">Environment</span>
<span class="w"> </span><span class="p">)</span>
<span class="w"> </span><span class="n">result</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="n">conf</span><span class="o">[</span><span class="n">confSection</span><span class="o">][</span><span class="n">confKey</span><span class="o">]</span><span class="p">.</span><span class="n">confValueTypeGet</span><span class="p">(),</span>
<span class="w"> </span><span class="n">ConfigVariableSource</span><span class="p">.</span><span class="n">ConfigFile</span>
<span class="w"> </span><span class="p">)</span>
<span class="k">proc</span><span class="w"> </span><span class="nf">splitOnComma</span><span class="p">(</span><span class="n">val</span><span class="p">:</span><span class="w"> </span><span class="nb">string</span><span class="p">):</span><span class="w"> </span><span class="nb">seq</span><span class="o">[</span><span class="nb">string</span><span class="o">]</span><span class="w"> </span><span class="o">=</span>
<span class="w"> </span><span class="n">result</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">val</span><span class="p">.</span><span class="n">split</span><span class="p">(</span><span class="sc">','</span><span class="p">)</span>
<span class="k">proc</span><span class="w"> </span><span class="nf">getStringSequence</span><span class="p">(</span><span class="n">value</span><span class="p">:</span><span class="w"> </span><span class="n">TomlValueRef</span><span class="p">):</span><span class="w"> </span><span class="nb">seq</span><span class="o">[</span><span class="nb">string</span><span class="o">]</span><span class="w"> </span><span class="o">=</span>
<span class="w"> </span><span class="k">let</span><span class="w"> </span><span class="n">values</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">value</span><span class="p">.</span><span class="n">getElems</span><span class="p">()</span>
<span class="w"> </span><span class="n">result</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">@[]</span>
<span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">v</span><span class="w"> </span><span class="ow">in</span><span class="w"> </span><span class="n">values</span><span class="p">:</span>
<span class="w"> </span><span class="n">result</span><span class="p">.</span><span class="n">add</span><span class="w"> </span><span class="n">v</span><span class="p">.</span><span class="n">getStr</span><span class="p">()</span>
<span class="k">proc</span><span class="w"> </span><span class="nf">parseIntArg</span><span class="p">(</span><span class="n">val</span><span class="p">:</span><span class="w"> </span><span class="n">Value</span><span class="p">):</span><span class="w"> </span><span class="nb">int</span><span class="w"> </span><span class="o">=</span>
<span class="w"> </span><span class="n">result</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">parseInt</span><span class="p">(</span><span class="o">$</span><span class="n">val</span><span class="p">)</span>
<span class="n">createGetSetting</span><span class="p">(</span><span class="nb">string</span><span class="p">,</span><span class="w"> </span><span class="p">`</span><span class="o">$</span><span class="p">`,</span><span class="w"> </span><span class="p">`</span><span class="o">$</span><span class="p">`,</span><span class="w"> </span><span class="n">getStr</span><span class="p">)</span>
<span class="n">createGetSetting</span><span class="p">(</span><span class="nb">int</span><span class="p">,</span><span class="w"> </span><span class="n">parseIntArg</span><span class="p">,</span><span class="w"> </span><span class="n">parseInt</span><span class="p">,</span><span class="w"> </span><span class="n">getInt</span><span class="p">)</span>
<span class="n">createGetSetting</span><span class="p">(</span><span class="nb">bool</span><span class="p">,</span><span class="w"> </span><span class="n">toBool</span><span class="p">,</span><span class="w"> </span><span class="n">parseBool</span><span class="p">,</span><span class="w"> </span><span class="n">getBool</span><span class="p">)</span>
<span class="n">createGetSetting</span><span class="p">(</span><span class="nb">seq</span><span class="o">[</span><span class="nb">string</span><span class="o">]</span><span class="p">,</span><span class="w"> </span><span class="p">`</span><span class="o">@</span><span class="p">`,</span><span class="w"> </span><span class="n">splitOnComma</span><span class="p">,</span><span class="w"> </span><span class="n">getStringSequence</span><span class="p">)</span>
</code></pre></div>
<p>The result of the above code is four different procedures called <code>getSetting</code> which look for a setting in the command line arguments, an environment variable, or a config file, and return it as the expected type.</p>
<p>Even though the above code is a mess and I’m probably going to rethink it, I will say this - writing the template was surprisingly intuitive.</p>
<p>Nim’s meta-programming features become even more powerful with macros and pragmas, but I haven’t really gotten into them yet so I can’t say much about them.</p>
<h2>Standard Library</h2>
<p>There’s some pretty great stuff in the standard library, including very easy to use asynchronous http and networking libraries, and parsers for a variety of text-based file formats. Everything seems to be appropriately cross-platform as well. I haven’t got much else to say about it!</p>
<h2>Python Modules</h2>
<p>Something I’m always looking out for in a language is the ability to write Python modules in it. There seem to be a <a href="https://github.com/Pebaz/nimporter#nimporter" title="Nimporter on GitHub">couple</a> of Nim <a href="https://github.com/sstadick/nython#nython" title="Nython on GitHub">libraries</a> for <a href="https://medium.com/statch/speeding-up-python-code-with-nim-ec205a8a5d9c" title="Nimpy package benchmark">doing this</a>, both based on an underlying <a href="https://github.com/yglukhov/nimpy" title="Nimpy on GitHub">nimpy</a> library. They both look incredibly easy to use, but notably the support for exporting Python classes in nimpy seems to be experimental. It is also a bit unclear how it deals with Python objects as parameters of procedures rather than basic types.</p>
<p>My only point of comparison is <a href="https://cython.org/" title="Cython project homepage">Cython</a>, which is a really cool project that compiles Python code to C, and includes an optional extended syntax for optimisation, which is essentially writing C code but with a Python-like syntax. As cool as this is I think the breadth of options is confusing, and when you get down to writing optimised routines things start to break in very unhelpful C-like way - i.e. successful compiles and unceremonious runtime segfaults.</p>
<p>I much prefer the idea of writing modules in a language that is its own thing, and with Nim being as easy to write as it is, I’m excited to try it for this purpose.</p>
<h2>Conclusion</h2>
<p>I didn’t perform even rudimentary benchmarks, but I think it’s safe to assume that anything written in Nim will be faster than the equivalent Python code. Luz runs instantaneously, and Sparkle responds to requests almost instantaneously as well. Neither of them are doing anything that I wouldn’t expect Python to do at an acceptable speed under the same circumstances, however.</p>
<p>One thing about Nim benchmarks that I have seen is that they are generally performed with the <code>-d:danger</code> compiler flag, which disables all runtime checks. This is done in the name of “fairness” in comparison with C, but it doesn’t really seem fair to me if the norm for the language in production is <code>-d:release</code>.</p>
<p>I definitely found Nim very natural to develop in. Unlike Rust, which I also tried (<em>failed</em>) to learn recently, most of the concepts were already familiar to me from other languages, and the syntax was also very familiar. I often found myself writing correct Nim code first time, and where I made mistakes they were flagged during compilation in a way that was easy to understand. Runtime errors are also handled relatively gracefully - no segfaults even though Nim compiles to C, like Cython does.</p>
<p>Overall, a very interesting language that I look forward to doing more with.</p>
<p><a href="https://github.com/khoulihan/sparkle" title="Sparkle on GitHub"><img alt="Sparkle in action" src="https://blog.hyperlinkyourheart.com/images/nimpressions/sparkle_screenshot.png" title="It's called Sparkle because it's barely there..."></a></p>Embedding SVGs in Pelican2020-04-04T00:10:00+02:002020-04-04T00:10:00+02:00Kevin Houlihantag:blog.hyperlinkyourheart.com,2020-04-04:/embedding-svgs.html<p>A static alternative to Font Awesome’s dynamic icon embedding.</p><p>In my <a href="https://blog.hyperlinkyourheart.com/remember-blogs.html">inaugural post</a> I mentioned that one problem I had encountered while designing this blog was styling the <span class="caps">SVG</span> icons. I had grabbed a bunch of the individual icon files from <a href="https://fontawesome.com/">Font Awesome</a>, but because of the way SVGs, <span class="caps">CSS</span> and <span class="caps">HTML</span> interact, I wasn’t able to colour them directly using <span class="caps">CSS</span> <code>color</code> or <code>fill</code> properties, and instead had to use <code>filter</code> properties (which I calculated using <a href="https://codepen.io/sosuke/pen/Pjoqqp">this tool</a>, so it wasn’t too much of a hardship).</p>
<p>I also didn’t particularly like that retrieving the icons involved numerous separate requests, nor the visible “pop-in” in Firefox that resulted from having them referenced as external files. The files are tiny, with the request overhead often as large or larger than the files themselves.</p>
<p>A further advantage that I was missing out on by not using Font Awesome as intended was that I couldn’t use their handy <code><i></code> tag shortcuts for specifying the icons to use.</p>
<p>Now, I have taken steps towards solving all of these many problems!</p>
<h2>Just use Font Awesome normally you weirdo</h2>
<p>Let’s back up a sec and talk about why I didn’t just use Font Awesome as intended in the first place (yes tldr; it is probably because I’m a weirdo).</p>
<p>Font Awesome has two ways that it can work: Web Fonts + <span class="caps">CSS</span>, or <span class="caps">SVG</span> + JavaScript. The former would involve retrieving an additional <span class="caps">CSS</span> file or two, as well as a couple of web fonts. The web font for the solid collection alone is 79.<span class="caps">4KB</span> - larger than anything else on this website. The JavaScript that would be required for the other method would likely be approaching <span class="caps">1MB</span> in size - larger than this <em>entire</em> website so far! I want a lean, fast-loading, low-power website, and these approaches seem entirely at odds with those goals.</p>
<p>It also struck me as odd to be statically generating a site, yet also having the client browser swapping in <span class="caps">SVG</span> images. I’ve nothing against JavaScript, but clearly this is work that can be done in advance!</p>
<h2>Doesn’t caching solve this problem?</h2>
<p>Well… maybe? In same cases? But not necessarily.</p>
<p>The average size of an icon in Font Awesome’s “solid” collection is 660B. A visitor would have to encounter over 1500 such embedded icons before downloading the JavaScript and caching it would be cheaper. The Web Fonts are much better, with caching the separate files becoming worthwhile after only 214 icons. That’s about 5 views of this blog’s index page, or 15 individual posts.</p>
<p>As such, if somebody reads 16 posts on this blog, they will have transferred more data than they would have if I’d used the Font Awesome web fonts. However, if 15 people read one post each and never visit again, the embedded approach comes out way ahead. So it very much depends on the traffic profile of the site, and I don’t think this site is one that people will be checking in on daily.</p>
<p>Embedding also offers other advantages, such as reducing initial load times.</p>
<h2>Solutions</h2>
<p>My solution is a <a href="https://github.com/khoulihan/pelican-embed-svg">pelican plugin</a> that post-processes the generated <span class="caps">HTML</span> files and embeds any SVGs it finds, whether specified as <code><img></code> tags or <code><i></code> tags.</p>
<p>It also, crucially, sets the <code>fill</code> attribute of any <span class="caps">SVG</span> paths to <code>currentColor</code>, which causes the fill colour to be taken from the current <span class="caps">CSS</span> text colour.</p>
<p>Taking the plugin beyond being merely a static implementation of Font Awesome, it also supports embedding of arbitrary <span class="caps">SVG</span> files. This can be achieved either by using <code><i></code> tags with the class <code>pi</code> to search a custom icon set, or through <code><img></code> tags where the <span class="caps">SVG</span> file is referenced by <span class="caps">URL</span>.</p>
<h2>Future</h2>
<p>The plugin probably has loads of rough edges at the moment. I haven’t at all tested if it supports Font Awesome’s more advanced behaviour, or even investigated how those features work, so there is a lot to be done there.</p>
<p>I may explore an approach that would combine the advantages of static generation with the advantages of a separate, cacheable <span class="caps">SVG</span> file. My initial thoughts on how to approach this plugin were to combine any referenced SVGs into a single file, and then reference them in the <span class="caps">HTML</span> using an <span class="caps">SVG</span> <code><use></code> tag. I need to learn a lot more about SVGs to know if that’s even feasible.</p>
<p>I also want to try to support other icon frameworks that support a similar <code><i></code> tag shortcut, such as <a href="https://forkaweso.me/Fork-Awesome/">Fork Awesome</a> and <a href="https://friconix.com">Friconix</a>.</p>
<p>In the meantime, it’s serving my purposes already on this site.</p>
<p><i class="fas fa-thumbs-up body-icon"></i> <i class="fas fa-bomb body-icon"></i> <i class="fas fa-cat body-icon"></i> <i class="fas fa-leaf body-icon"></i> <i class="fas fa-file body-icon"></i> <i class="fas fa-fist-raised body-icon"></i></p>Runtime Class Modification2020-03-25T17:51:00+01:002020-03-25T23:12:00+01:00Kevin Houlihantag:blog.hyperlinkyourheart.com,2020-03-25:/runtime-class-modification.html<p>Batteries-included package development in Python requires an unorthodox approach when targeting microcontrollers.</p><p>Python is probably my favourite language, so I was excited some years ago when a project appeared on Kickstarter to develop a <a href="http://micropython.org/" title="MicroPython">Python runtime for microcontrollers</a>, and an associated microcontroller board.</p>
<p>However, writing Python for a microcontroller does have some constraints that aren’t really a factor when writing Python for other environments. Having maybe only <span class="caps">100KB</span> of <span class="caps">RAM</span> to work with, keeping code size as low as possible is essential.</p>
<p>When I wrote a <a href="https://github.com/khoulihan/micropython-tmp102" title="micropython-tmp102 repository">package to support the <span class="caps">TI</span> tmp102 temperature sensor</a>, I initially included all the required functionality in a single importable class. It used <span class="caps">15KB</span> of <span class="caps">RAM</span> after import, which does leave space for other code, but since some of the functionality is mutually exclusive I knew I could probably do better.</p>
<p>This post is about what I ended up with and how it works.</p>
<h2>Importable Features</h2>
<p>The core functionality of the package can be leveraged by importing the <code>Tmp102</code> class and creating an instance. This leaves the sensor in its default configuration, in which it performs a reading 4 times per second and makes the most recent available to your code on request. The details of initialising the object are explained in the <a href="https://github.com/khoulihan/micropython-tmp102/blob/master/README.md">documentation</a> if you actually want to use the module, so I won’t go into them again here.</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">machine</span> <span class="kn">import</span> <span class="n">I2C</span>
<span class="kn">from</span> <span class="nn">tmp102</span> <span class="kn">import</span> <span class="n">Tmp102</span>
<span class="n">bus</span> <span class="o">=</span> <span class="n">I2C</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="n">sensor</span> <span class="o">=</span> <span class="n">Tmp102</span><span class="p">(</span><span class="n">bus</span><span class="p">,</span> <span class="mh">0x48</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">sensor</span><span class="o">.</span><span class="n">temperature</span><span class="p">)</span>
</code></pre></div>
<p>That’s all well and good, but what if you want to make use of some of the more advanced features of the sensor, such as controlling the rate at which it takes readings (the “conversion rate”)? Such features are structured as importable modules which add the required functionality into the <code>Tmp102</code> class. The <code>CONVERSION_RATE_1HZ</code> constant in the example below, as well as other relevant code, are added to the class when the <code>conversionrate</code> module is imported.</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">tmp102</span> <span class="kn">import</span> <span class="n">Tmp102</span>
<span class="kn">import</span> <span class="nn">tmp102.conversionrate</span>
<span class="n">sensor</span> <span class="o">=</span> <span class="n">Tmp102</span><span class="p">(</span>
<span class="n">bus</span><span class="p">,</span>
<span class="mh">0x48</span><span class="p">,</span>
<span class="n">conversion_rate</span><span class="o">=</span><span class="n">Tmp102</span><span class="o">.</span><span class="n">CONVERSION_RATE_1HZ</span>
<span class="p">)</span>
</code></pre></div>
<p>If you don’t need to change the conversion rate in your project then the code to do so is never loaded. If you do need this or other features, all the functionality is still exposed through a single easy to use class.</p>
<h2>How?</h2>
<p>The package is structured like this:</p>
<div class="highlight"><pre><span></span><code>tmp102
+-- __init__.py
+-- _tmp102.py
+-- alert.py
+-- conversionrate.py
+-- convertors.py
+-- extendedmode.py
+-- oneshot.py
+-- shutdown.py
</code></pre></div>
<p>The base <code>Tmp102</code> class is defined in <code>_tmp102.py</code>, along with some private functions and constants.</p>
<div class="highlight"><pre><span></span><code><span class="n">REGISTER_TEMP</span> <span class="o">=</span> <span class="mi">0</span>
<span class="n">REGISTER_CONFIG</span> <span class="o">=</span> <span class="mi">1</span>
<span class="n">EXTENDED_MODE_BIT</span> <span class="o">=</span> <span class="mh">0x10</span>
<span class="k">def</span> <span class="nf">_set_bit</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">mask</span><span class="p">):</span>
<span class="k">return</span> <span class="n">b</span> <span class="o">|</span> <span class="n">mask</span>
<span class="k">def</span> <span class="nf">_clear_bit</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">mask</span><span class="p">):</span>
<span class="k">return</span> <span class="n">b</span> <span class="o">&</span> <span class="o">~</span><span class="n">mask</span>
<span class="k">def</span> <span class="nf">_set_bit_for_boolean</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">mask</span><span class="p">,</span> <span class="n">val</span><span class="p">):</span>
<span class="k">if</span> <span class="n">val</span><span class="p">:</span>
<span class="k">return</span> <span class="n">_set_bit</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">mask</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">return</span> <span class="n">_clear_bit</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">mask</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">Tmp102</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">bus</span><span class="p">,</span> <span class="n">address</span><span class="p">,</span> <span class="n">temperature_convertor</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">bus</span> <span class="o">=</span> <span class="n">bus</span>
<span class="bp">self</span><span class="o">.</span><span class="n">address</span> <span class="o">=</span> <span class="n">address</span>
<span class="bp">self</span><span class="o">.</span><span class="n">temperature_convertor</span> <span class="o">=</span> <span class="n">temperature_convertor</span>
<span class="c1"># The register defaults to the temperature.</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_last_write_register</span> <span class="o">=</span> <span class="n">REGISTER_TEMP</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_extended_mode</span> <span class="o">=</span> <span class="kc">False</span>
<span class="o">.</span>
<span class="o">.</span>
<span class="o">.</span>
</code></pre></div>
<p>To hide the private stuff from users of the package, the <code>__init__.py</code> imports the <code>Tmp102</code> class and then removes the <code>_tmp102</code> module from the namespace.</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">tmp102._tmp102</span> <span class="kn">import</span> <span class="n">Tmp102</span>
<span class="k">del</span> <span class="n">_tmp102</span>
</code></pre></div>
<p>The interesting stuff happens in the feature sub-modules. Each feature module defines an <code>_extend_class</code> function which modifies the <code>Tmp102</code> class. Since importing a module runs it, this function can be called and then deleted to keep the namespace nice and clean - the module will actually be empty once imported. This pattern should be familiar to JavaScript developers!</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">_extend_class</span><span class="p">():</span>
<span class="c1"># Modify Tmp102 here - Check the next code block!</span>
<span class="k">pass</span>
<span class="n">_extend_class</span><span class="p">()</span>
<span class="k">del</span> <span class="n">_extend_class</span>
</code></pre></div>
<p>Let’s take a look at the <code>oneshot</code> module, which adds functionality to the <code>Tmp102</code> class to allow the sensor to be polled as necessary instead of constantly performing readings - very useful if you want to save power.</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">_extend_class</span><span class="p">():</span>
<span class="kn">from</span> <span class="nn">tmp102._tmp102</span> <span class="kn">import</span> <span class="n">Tmp102</span>
<span class="kn">from</span> <span class="nn">tmp102._tmp102</span> <span class="kn">import</span> <span class="n">_set_bit_for_boolean</span>
<span class="kn">import</span> <span class="nn">tmp102.shutdown</span>
<span class="n">SHUTDOWN_BIT</span> <span class="o">=</span> <span class="mh">0x01</span>
<span class="n">ONE_SHOT_BIT</span> <span class="o">=</span> <span class="mh">0x80</span>
<span class="k">def</span> <span class="nf">initiate_conversion</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="w"> </span><span class="sd">"""</span>
<span class="sd"> Initiate a one-shot conversion.</span>
<span class="sd"> """</span>
<span class="n">current_config</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_get_config</span><span class="p">()</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">current_config</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">&</span> <span class="n">SHUTDOWN_BIT</span><span class="p">:</span>
<span class="k">raise</span> <span class="ne">RuntimeError</span><span class="p">(</span><span class="s2">"Device must be shut down to initiate one-shot conversion"</span><span class="p">)</span>
<span class="n">new_config</span> <span class="o">=</span> <span class="nb">bytearray</span><span class="p">(</span><span class="n">current_config</span><span class="p">)</span>
<span class="n">new_config</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="n">_set_bit_for_boolean</span><span class="p">(</span>
<span class="n">new_config</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span>
<span class="n">ONE_SHOT_BIT</span><span class="p">,</span>
<span class="kc">True</span>
<span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_set_config</span><span class="p">(</span><span class="n">new_config</span><span class="p">)</span>
<span class="n">Tmp102</span><span class="o">.</span><span class="n">initiate_conversion</span> <span class="o">=</span> <span class="n">initiate_conversion</span>
<span class="k">def</span> <span class="nf">_conversion_ready</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">current_config</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_get_config</span><span class="p">()</span>
<span class="k">return</span> <span class="p">(</span><span class="n">current_config</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">&</span> <span class="n">ONE_SHOT_BIT</span><span class="p">)</span> <span class="o">==</span> <span class="n">ONE_SHOT_BIT</span>
<span class="n">Tmp102</span><span class="o">.</span><span class="n">conversion_ready</span> <span class="o">=</span> <span class="nb">property</span><span class="p">(</span><span class="n">_conversion_ready</span><span class="p">)</span>
</code></pre></div>
<p>So what’s going on here? First, the <code>Tmp102</code> class and any required functions are imported. Since it was imported in the package’s <code>__init__</code> the class is already defined. Importing the private functions and constants in a function like this keeps them out of the global namespace.</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">tmp102._tmp102</span> <span class="kn">import</span> <span class="n">Tmp102</span>
<span class="kn">from</span> <span class="nn">tmp102._tmp102</span> <span class="kn">import</span> <span class="n">_set_bit_for_boolean</span>
</code></pre></div>
<p>The <code>oneshot</code> module depends on the functionality from the <code>shutdown</code> module, so it is imported next.</p>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">tmp102.shutdown</span>
</code></pre></div>
<p>Next, a couple of constants are defined. Through the magic of closure, these will only be available to the methods defined in this module.</p>
<div class="highlight"><pre><span></span><code><span class="n">SHUTDOWN_BIT</span> <span class="o">=</span> <span class="mh">0x01</span>
<span class="n">ONE_SHOT_BIT</span> <span class="o">=</span> <span class="mh">0x80</span>
</code></pre></div>
<p>The rest of the function defines a method and a property which are added to the class by simply assigning them to attributes. These will be available to any instances of the class, exactly as if they were included in the class definition.</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">initiate_conversion</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="w"> </span><span class="sd">"""</span>
<span class="sd"> Initiate a one-shot conversion.</span>
<span class="sd"> """</span>
<span class="n">current_config</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_get_config</span><span class="p">()</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">current_config</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">&</span> <span class="n">SHUTDOWN_BIT</span><span class="p">:</span>
<span class="k">raise</span> <span class="ne">RuntimeError</span><span class="p">(</span><span class="s2">"Device must be shut down to initiate one-shot conversion"</span><span class="p">)</span>
<span class="n">new_config</span> <span class="o">=</span> <span class="nb">bytearray</span><span class="p">(</span><span class="n">current_config</span><span class="p">)</span>
<span class="n">new_config</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="n">_set_bit_for_boolean</span><span class="p">(</span>
<span class="n">new_config</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span>
<span class="n">ONE_SHOT_BIT</span><span class="p">,</span>
<span class="kc">True</span>
<span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_set_config</span><span class="p">(</span><span class="n">new_config</span><span class="p">)</span>
<span class="n">Tmp102</span><span class="o">.</span><span class="n">initiate_conversion</span> <span class="o">=</span> <span class="n">initiate_conversion</span>
<span class="k">def</span> <span class="nf">_conversion_ready</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">current_config</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_get_config</span><span class="p">()</span>
<span class="k">return</span> <span class="p">(</span><span class="n">current_config</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">&</span> <span class="n">ONE_SHOT_BIT</span><span class="p">)</span> <span class="o">==</span> <span class="n">ONE_SHOT_BIT</span>
<span class="n">Tmp102</span><span class="o">.</span><span class="n">conversion_ready</span> <span class="o">=</span> <span class="nb">property</span><span class="p">(</span><span class="n">_conversion_ready</span><span class="p">)</span>
</code></pre></div>
<p>The other feature modules follow the same pattern.</p>
<h2>Savings</h2>
<p>Importing the base <code>Tmp102</code> class uses about 3.<span class="caps">53KB</span> of <span class="caps">RAM</span> - quite a saving if that is all you need. The feature modules vary between 0.<span class="caps">8KB</span> and <span class="caps">4KB</span>, or thereabouts. Importing them all uses 13.<span class="caps">44KB</span>, but it is unlikely that they would all be required in any given application.</p>
<h2>Conclusion</h2>
<p>I thought of this approach as “monkey-patching” for a long time - the last refuge of the desperate and the damned - but I’m not sure that it is really, because the modifications are all being made internally to the package. It is definitely outside the norm for Python, but it achieved the goal of reducing <span class="caps">RAM</span> usage while maintaining a clean <span class="caps">API</span>.</p>Bad Idioms2020-01-10T14:11:00+01:002020-03-25T23:20:00+01:00Kevin Houlihantag:blog.hyperlinkyourheart.com,2020-01-10:/bad-idioms.html<p>A brief investigation of Python’s <span class="caps">EAFP</span> idiom applied to C#.</p><p>Human languages are full to the brim with idioms - figurative ways of saying things that native speakers trot out without even thinking about them. Often, when translated literally into another language, the result is utter nonsense. For example, the phrase “tomar el pelo” in Spanish translates literally to English as “to take the hair”, but the idiomatic way to say the same thing in English would be “to pull (someone’s) leg”. The same thing is roughly true of programming languages, with different languages having their own idiomatic or expected ways of achieving the same ends.</p>
<p>I recently made the mistake, after a period of writing Python code, of applying one of Python’s idioms to C#. The task at hand was to check if a dictionary of lists already contained a particular key, and if not, add a new list for that key. The C# way to do this would probably be to check for the existence of the key first, then decide what to do - or even better, use the <code>TryGetValue</code> method of the dictionary to assign the value to a variable. This is known as “Look Before You Leap”.</p>
<div class="highlight"><pre><span></span><code><span class="n">List</span><span class="o"><</span><span class="kt">object</span><span class="o">></span><span class="w"> </span><span class="n">l</span><span class="p">;</span>
<span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">dict</span><span class="p">.</span><span class="n">ContainsKey</span><span class="p">(</span><span class="n">objectType</span><span class="p">))</span>
<span class="p">{</span>
<span class="w"> </span><span class="n">l</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">dict</span><span class="p">[</span><span class="n">objectType</span><span class="p">];</span>
<span class="p">}</span>
<span class="k">else</span>
<span class="p">{</span>
<span class="w"> </span><span class="n">l</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">List</span><span class="o"><</span><span class="kt">object</span><span class="o">></span><span class="p">();</span>
<span class="w"> </span><span class="n">dict</span><span class="p">.</span><span class="n">Add</span><span class="p">(</span><span class="n">objectType</span><span class="p">,</span><span class="w"> </span><span class="n">l</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div>
<div class="highlight"><pre><span></span><code><span class="n">List</span><span class="o"><</span><span class="kt">object</span><span class="o">></span><span class="w"> </span><span class="n">l</span><span class="p">;</span>
<span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="o">!</span><span class="n">dict</span><span class="p">.</span><span class="n">TryGetValue</span><span class="p">(</span><span class="n">objectType</span><span class="p">,</span><span class="w"> </span><span class="k">out</span><span class="w"> </span><span class="n">l</span><span class="p">))</span>
<span class="p">{</span>
<span class="w"> </span><span class="n">l</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">List</span><span class="o"><</span><span class="kt">object</span><span class="o">></span><span class="p">();</span>
<span class="w"> </span><span class="n">dict</span><span class="p">.</span><span class="n">Add</span><span class="p">(</span><span class="n">objectType</span><span class="p">,</span><span class="w"> </span><span class="n">l</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div>
<p>But instead of doing either of those things, I applied a more pythonic idiom - that of “Easier to Ask Permission than Forgiveness” - and just tried retrieving the value, and catching the <code>KeyNotFoundException</code>:</p>
<div class="highlight"><pre><span></span><code><span class="n">List</span><span class="o"><</span><span class="kt">object</span><span class="o">></span><span class="w"> </span><span class="n">l</span><span class="p">;</span>
<span class="k">try</span>
<span class="p">{</span>
<span class="w"> </span><span class="n">l</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">dict</span><span class="p">[</span><span class="n">objectType</span><span class="p">];</span>
<span class="p">}</span>
<span class="k">catch</span><span class="w"> </span><span class="p">(</span><span class="n">KeyNotFoundException</span><span class="w"> </span><span class="n">ex</span><span class="p">)</span>
<span class="p">{</span>
<span class="w"> </span><span class="n">l</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">List</span><span class="o"><</span><span class="kt">object</span><span class="o">></span><span class="p">();</span>
<span class="w"> </span><span class="n">dict</span><span class="p">.</span><span class="n">Add</span><span class="p">(</span><span class="n">objectType</span><span class="p">,</span><span class="w"> </span><span class="n">l</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div>
<p>This turned an operation that should have taken milliseconds into one that was taking seconds, introducing a perceptible delay into my application.</p>
<p>Curious to know to exactly what extent performance differed between the above choices, and whether <span class="caps">EAFP</span> really would have been the better choice in Python, I decided to throw together some benchmark tests.</p>
<h2>Python</h2>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">timeit</span>
<span class="n">setup</span> <span class="o">=</span> <span class="s2">"""</span>
<span class="s2">d = {</span>
<span class="s2"> 'a': [1, 2, 3,],</span>
<span class="s2"> 'b': [4, 5, 6,],</span>
<span class="s2"> 'c': [7, 8, 9,],</span>
<span class="s2">}</span>
<span class="s2">"""</span>
<span class="n">test_except</span> <span class="o">=</span> <span class="s2">"""</span>
<span class="s2">try:</span>
<span class="s2"> v = d['d']</span>
<span class="s2">except KeyError:</span>
<span class="s2"> v = []</span>
<span class="s2"> d['d'] = v</span>
<span class="s2">del d['d']</span>
<span class="s2">"""</span>
<span class="n">test_check</span> <span class="o">=</span> <span class="s2">"""</span>
<span class="s2">if 'd' in d:</span>
<span class="s2"> v = d['d']</span>
<span class="s2">else:</span>
<span class="s2"> v = []</span>
<span class="s2"> d['d'] = v</span>
<span class="s2">del d['d']</span>
<span class="s2">"""</span>
<span class="nb">print</span><span class="p">(</span><span class="n">timeit</span><span class="o">.</span><span class="n">timeit</span><span class="p">(</span><span class="n">setup</span><span class="o">=</span><span class="n">setup</span><span class="p">,</span> <span class="n">stmt</span><span class="o">=</span><span class="n">test_except</span><span class="p">,</span> <span class="n">number</span><span class="o">=</span><span class="mi">1000000</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="n">timeit</span><span class="o">.</span><span class="n">timeit</span><span class="p">(</span><span class="n">setup</span><span class="o">=</span><span class="n">setup</span><span class="p">,</span> <span class="n">stmt</span><span class="o">=</span><span class="n">test_check</span><span class="p">,</span> <span class="n">number</span><span class="o">=</span><span class="mi">1000000</span><span class="p">))</span>
</code></pre></div>
<p>This gave results of 0.46 seconds for a million <span class="caps">EAFP</span> operations, and about 0.08 seconds for a million <span class="caps">LBYL</span> operations, with everything else, I hope, being equal between the two tests. If the new key is not deleted every time (so that only the first check fails), the <span class="caps">EAFP</span> operation becomes marginally faster than the alternative (0.026 vs 0.037 seconds) on most runs.</p>
<h2>C#</h2>
<div class="highlight"><pre><span></span><code><span class="n">Dictionary</span><span class="o"><</span><span class="kt">string</span><span class="p">,</span><span class="w"> </span><span class="n">List</span><span class="o"><</span><span class="kt">string</span><span class="o">>></span><span class="w"> </span><span class="n">dict</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">Dictionary</span><span class="o"><</span><span class="kt">string</span><span class="p">,</span><span class="w"> </span><span class="n">List</span><span class="o"><</span><span class="kt">string</span><span class="o">>></span><span class="p">()</span>
<span class="p">{</span>
<span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="s">"a"</span><span class="p">,</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">List</span><span class="o"><</span><span class="kt">string</span><span class="o">></span><span class="p">()</span><span class="w"> </span><span class="p">},</span>
<span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="s">"b"</span><span class="p">,</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">List</span><span class="o"><</span><span class="kt">string</span><span class="o">></span><span class="p">()</span><span class="w"> </span><span class="p">},</span>
<span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="s">"c"</span><span class="p">,</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">List</span><span class="o"><</span><span class="kt">string</span><span class="o">></span><span class="p">()</span><span class="w"> </span><span class="p">}</span>
<span class="p">};</span>
<span class="n">DateTime</span><span class="w"> </span><span class="n">exceptStart</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">DateTime</span><span class="p">.</span><span class="n">UtcNow</span><span class="p">;</span>
<span class="k">for</span><span class="w"> </span><span class="p">(</span><span class="kt">int</span><span class="w"> </span><span class="n">i</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span><span class="p">;</span><span class="w"> </span><span class="n">i</span><span class="w"> </span><span class="o"><</span><span class="w"> </span><span class="m">1000</span><span class="p">;</span><span class="w"> </span><span class="n">i</span><span class="o">++</span><span class="p">)</span>
<span class="p">{</span>
<span class="w"> </span><span class="n">List</span><span class="o"><</span><span class="kt">string</span><span class="o">></span><span class="w"> </span><span class="n">v</span><span class="p">;</span>
<span class="w"> </span><span class="k">try</span>
<span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">v</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">dict</span><span class="p">[</span><span class="s">"d"</span><span class="p">];</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="k">catch</span><span class="w"> </span><span class="p">(</span><span class="n">KeyNotFoundException</span><span class="w"> </span><span class="n">ex</span><span class="p">)</span>
<span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">v</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">List</span><span class="o"><</span><span class="kt">string</span><span class="o">></span><span class="p">();</span>
<span class="w"> </span><span class="n">dict</span><span class="p">.</span><span class="n">Add</span><span class="p">(</span><span class="s">"d"</span><span class="p">,</span><span class="w"> </span><span class="n">v</span><span class="p">);</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="n">dict</span><span class="p">.</span><span class="n">Remove</span><span class="p">(</span><span class="s">"d"</span><span class="p">);</span>
<span class="p">}</span>
<span class="n">TimeSpan</span><span class="w"> </span><span class="n">exceptResult</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">DateTime</span><span class="p">.</span><span class="n">UtcNow</span><span class="w"> </span><span class="o">-</span><span class="w"> </span><span class="n">exceptStart</span><span class="p">;</span>
<span class="n">DateTime</span><span class="w"> </span><span class="n">tryGetStart</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">DateTime</span><span class="p">.</span><span class="n">UtcNow</span><span class="p">;</span>
<span class="k">for</span><span class="w"> </span><span class="p">(</span><span class="kt">int</span><span class="w"> </span><span class="n">i</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span><span class="p">;</span><span class="w"> </span><span class="n">i</span><span class="w"> </span><span class="o"><</span><span class="w"> </span><span class="m">1000000</span><span class="p">;</span><span class="w"> </span><span class="n">i</span><span class="o">++</span><span class="p">)</span>
<span class="p">{</span>
<span class="w"> </span><span class="n">List</span><span class="o"><</span><span class="kt">string</span><span class="o">></span><span class="w"> </span><span class="n">v</span><span class="p">;</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="o">!</span><span class="n">dict</span><span class="p">.</span><span class="n">TryGetValue</span><span class="p">(</span><span class="s">"d"</span><span class="p">,</span><span class="w"> </span><span class="k">out</span><span class="w"> </span><span class="n">v</span><span class="p">))</span>
<span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">v</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">List</span><span class="o"><</span><span class="kt">string</span><span class="o">></span><span class="p">();</span>
<span class="w"> </span><span class="n">dict</span><span class="p">.</span><span class="n">Add</span><span class="p">(</span><span class="s">"d"</span><span class="p">,</span><span class="w"> </span><span class="n">v</span><span class="p">);</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="n">dict</span><span class="p">.</span><span class="n">Remove</span><span class="p">(</span><span class="s">"d"</span><span class="p">);</span>
<span class="p">}</span>
<span class="n">TimeSpan</span><span class="w"> </span><span class="n">tryGetResult</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">DateTime</span><span class="p">.</span><span class="n">UtcNow</span><span class="w"> </span><span class="o">-</span><span class="w"> </span><span class="n">tryGetStart</span><span class="p">;</span>
<span class="n">DateTime</span><span class="w"> </span><span class="n">checkStart</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">DateTime</span><span class="p">.</span><span class="n">UtcNow</span><span class="p">;</span>
<span class="k">for</span><span class="w"> </span><span class="p">(</span><span class="kt">int</span><span class="w"> </span><span class="n">i</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0</span><span class="p">;</span><span class="w"> </span><span class="n">i</span><span class="w"> </span><span class="o"><</span><span class="w"> </span><span class="m">1000000</span><span class="p">;</span><span class="w"> </span><span class="n">i</span><span class="o">++</span><span class="p">)</span>
<span class="p">{</span>
<span class="w"> </span><span class="n">List</span><span class="o"><</span><span class="kt">string</span><span class="o">></span><span class="w"> </span><span class="n">v</span><span class="p">;</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="o">!</span><span class="n">dict</span><span class="p">.</span><span class="n">ContainsKey</span><span class="p">(</span><span class="s">"d"</span><span class="p">))</span>
<span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">v</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">new</span><span class="w"> </span><span class="n">List</span><span class="o"><</span><span class="kt">string</span><span class="o">></span><span class="p">();</span>
<span class="w"> </span><span class="n">dict</span><span class="p">.</span><span class="n">Add</span><span class="p">(</span><span class="s">"d"</span><span class="p">,</span><span class="w"> </span><span class="n">v</span><span class="p">);</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="k">else</span>
<span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">v</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">dict</span><span class="p">[</span><span class="s">"d"</span><span class="p">];</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="n">dict</span><span class="p">.</span><span class="n">Remove</span><span class="p">(</span><span class="s">"d"</span><span class="p">);</span>
<span class="p">}</span>
<span class="n">TimeSpan</span><span class="w"> </span><span class="n">checkResult</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">DateTime</span><span class="p">.</span><span class="n">UtcNow</span><span class="w"> </span><span class="o">-</span><span class="w"> </span><span class="n">checkStart</span><span class="p">;</span>
<span class="n">Console</span><span class="p">.</span><span class="n">WriteLine</span><span class="p">(</span><span class="s">"Except: {0}"</span><span class="p">,</span><span class="w"> </span><span class="n">exceptResult</span><span class="p">.</span><span class="n">TotalSeconds</span><span class="p">);</span>
<span class="n">Console</span><span class="p">.</span><span class="n">WriteLine</span><span class="p">(</span><span class="s">"TryGet: {0:f10}"</span><span class="p">,</span><span class="w"> </span><span class="n">tryGetResult</span><span class="p">.</span><span class="n">TotalSeconds</span><span class="p">);</span>
<span class="n">Console</span><span class="p">.</span><span class="n">WriteLine</span><span class="p">(</span><span class="s">"Check: {0:f10}"</span><span class="p">,</span><span class="w"> </span><span class="n">checkResult</span><span class="p">.</span><span class="n">TotalSeconds</span><span class="p">);</span>
<span class="n">Console</span><span class="p">.</span><span class="n">ReadKey</span><span class="p">(</span><span class="k">true</span><span class="p">);</span>
</code></pre></div>
<p>Note that the <span class="caps">EAFP</span> test here is only performed a thousand times - because even running it that many times takes around 15 <em>entire</em> seconds! The two <span class="caps">LBYL</span> tests are nothing in comparison, executing a million times in around 0.05 seconds. This is a much bigger difference than I would have expected.</p>
<h2>Conclusion</h2>
<p>The performance of a single operation like this doesn’t necessarily say a lot about the real-world performance of any given application, but I think it is probably best to stick to the idioms of the language you’re working in - and in C#, that means only throwing exceptions in exceptional circumstances. In Python, there may be circumstances where it would be better to “Look Before You Leap” as well, but the difference in performance is probably not large enough to matter in most cases.</p>