Sneak peek — tray, citrus chess board, serving board.

“Court of Ice and Roses” chessboard in shop!

Sneak peek — Christmas jewelry in progress.

New in the shop — Starry Woods and Tardigrades! scarves.

Native wine-cups (aka purple poppy mallow) bookends. This is one of my favorite native flower, so particularly pleased with how nicely it shows up in bookends.

Christmas serving board in progress.

Got a light for the little rose house. So cute.

Learned something new today— resin fidget spinner. ????

We just made the decision to ship the remaining copies of Feast to us. There are about 800 left, out of the original 2000 copy print run. Which is a lot of copies, but given that we were originally scheduled to launch March 2020, and had to cancel an entire summer’s worth of scheduled bookstore […]

The great bear himself finally turns up... bloody late as usual.
After waiting a while for the second bear to arrive...the cutting and hacking could begin.
The plastic was very tough to get through with my craft saw. After what seems an age, the great heavy head fell off and the Orcs cheered. They soon stopped when they saw the angry head being wired into place.




I had to try a couple of different fur techniques, for this chunky, thick fur, I went for the classic GW wolf pelt look. This is the one where you cut lots of little triangles into the putty and push them up.
Beorn laughs at his new coat... Hopefully that's a good sign he likes it.
I did take the opportunity to bulk up his hump while applying the putty. I will also add some to his forelegs and make them a bit more shaggy.

The new Deception medic 'Wrench' gets his first taste of combat.

 I love this little cassette bot, he's so cute. I must make some more.

Skyfire brings the rain.

I've been messing around adding to the bear to days and trying not to rush myself. I'm so desperate to get some paint on it that I almost started today. However, I stopped myself and just added a bit more fur. This time I also ad buyded some to the bears face and now I'm finally happy with it.




Cheeky!
Some more help turns up to bolster Lake town and just in the nick of time.


Two of these bases are really shoddy militia types. Pitch forks are a great short hand for 'rabble'. I can't imagine they will hold for long against the goblin horde.

Another hero base .

I think he looks angry, ragged and damn right mean now...let's get him painted up.


I've added hints of blue so the whole force will tie together.


 The men start to suffer from the fierce wolf packs.

The men with their long swords and the tough veterans from the Iron hills.

 This is me working out my warband for a Dark age campaign. I can't really get going on it until I have finished my Hobbit armies for an up coming show. So In the meantime time this sketch of the warband will have to do. Out of the hat I got the Black shield Irish.

The rules will be One hour wargames, and it's interesting to see the small size of the warband. Then again it is a skirmish game.

 

I have an idea of making a few stands of Draugr, the Viking dead. I have bought some nice Colin Pattern sculpts but thought I'd have a go at making some.
I have loads of Viking sprues left over from the Lake town project, these I mixed with some Oathmark undead sprues. I will add some greenstuff to these to make them a little more Viking.


 

A Dwarf king and his elite guard take to the field.

 Amongst the Eagle helms of the shield wall is the kings champion himself. Nothing must get to the king and the champions blue axe, will see that nothing does.

I could use this base for King Balin when he tries to retake Moria. A future project of mine.
The King points out floors in the enemies formation to the chief engineer.
The Dragon standard flies over head echoing the Kings mighty dragon helm.
This base has been created for the upcoming Hobbit battle, replacing Dain's base in the ranks. This allows Dain to roam freely as a character on the field.

 The figures are beautiful old sculpts from Asgard and Citadel. There has been a bit of tweaking on a couple of them, like the axe and the standard.

Here is Beorn in his full fury. He has been given a Matt varnish that really helped. This proved even more effective once his mouth, eyes and nose was given a coat of gloss varnish. This brought him to life and gave him a sparkle in his eye.

I ended up putting more and more greenstuff on him until he was completely covered. It's good to have him covered in thick, shaggy fur as he looks more wild and rugged.


The Eagles are coming! Eagles and White wolves get their various layers of flock.
A lovely new Reaper miniatures Dwarf joins the latest group of Dwarves. Reaper make great 'hero' figures to adorn any warband. They can be quite large sculpts so it can be a risky business ordering them, this one is perfect though.
Beorn's base has to be 24cm wide for the game I'm playing, so two side bases were made. These bring the base up to the right measurement and allow for more bear carnage.
Beorn with his slobbering maw but cute adorable shiny eyes.
More goblin wreckage. Wargs too have not escaped the Bears fury and lie amongst the fallen. A few goblins still cling to life.
It was fun to sculpt great claw wounds in the orcs clothes and equipment. Everything has been torn and crushed to pieces.
Beorn's base with some Bodyguard of Bolg behind him. I'm not sure I'll have time to finish this big orcs before the weekend.

Bard of Lake town with his new fixed banner. This banner was a bit too tall for the storage box and the spearpoint snapped off, twice!. So using my flattened brush bristle technique, he's got a new much tougher one. The plastic used for brush bristles make them fantastic for super glue!

As Bard needs to be free to roam around, another replacement base was needed. Here is Bard with a fellow command stand. Bard is another Reaper Miniatures figure and is slightly bigger than most. This is fine as he is a hero and it suits his manly character.

More carnage, this poor Warg has been disemboweled. This is pretty grim but I did have fun sculpting it. Facing a giant werebear was never going to end well. Again, a coat of gloss helps bring it to life 
Another shot of the basing production line.
Bard of Esgaroth.

 The blue theme really helped to tie these militia type troops together. I tested the new banner tip and it comes below the storage box height, hopefully it won't get crushed again.

The last Dwarf base done just in time for the big weekend. It's the Other Partizan show this weekend and I'll be there with my armies. So if you fancy seeing a massive Hobbit battle...pop along.

The battle will be fought using the new Midguard rules by James Morris. It proves to be a great Sunday especially if you're a Tolkien fan.

Today I have collect everything together and pack it as best I can to stop any travel casualties. I have always painted my figures to play at home so have never magnetised them onto metal sheets.Finally the two meet, Bolg and bigger Beorn.
My old Beorn figure is dwarfed by the new addition. The new bear has a lot more presence to him and should look the part on Sunday.

well, we made it safe and sound. James Morris has done a fantastic job on the Lonely mountain.
My figures are on the board and will be joined by many others tomorrow from other collections.

 The 'really useful boxes' certainly earned their name today.

well...what a bloody marvellous couple of days I've just had. Taking all my figures to a show for the first time was a little nerve racking, but the effort really paid off. I had an excellent time and met some great new friends too in the process.
The board and mountain were excellent too, crafted by the talented James Morris. I had never deployed my whole army in one go before, so what better time to do it than on a set up like this. Everything seemed to come together perfectly.
I started my battle with the orcs at the bottom of the hill advancing across the mountain river. However, due to my terrible dice rolling and the luck of the Elves, I was cut and shot to pieces. It wasn't long before my tattered ranks were tumbling back through the freezing water. The Elves just had time to redress their lines before another dark wave of Goblin folk arrived.
I love the walled up gate that James made, the little gaps in the stonework were perfect. It really reminded me of one of Alan Lees illustrations. Also a very clever use of silver fabric for the river...very effective!

The Eagles arrived later in the game and swept the Goblins from the mountain sides. The game was masterfully run by David Hunter, who made sure the flo was just right.
My Eagles looked very realistic on the slopes of the mountain too, better than I had hoped for. The extra poses were well worth collecting over the last ten years. The flying ones on stands were a very recent purchase and they really stood out.
The game in full flo, it drew a big crowd and it was great to talk figures with people. A lot of people wanted to know where I got my Orcs from and of course the answer was...everywhere. Also the amount of plastic model kits used was very hard to calculate. Just about every Dark age and Fantasy box set all mashed up and mixed together.



I love my Wood elves after lovingly creating each one individually and got a huge buzz from seeing them in action. I didn't really care they were cutting my forces to ribbons, just seeing them used in anger was enough for me.
Hordes of Goblins still plagued the mountainside, their numbers were too much even for the Eagles.
Thorin was cut down by Bolg's bodyguard as he tried to cut through their great shields. Bard of Laketown too, fell to the hacking Goblin blades. Dain of the Iron hills was holding his own but was badly wounded. At last it seemed like the Free people's luck had run out. 
Then with a roar like a thunderclap Beorn burst onto the scene. He tore through the Goblin ranks and made his way to Bolg. Approaching the rear of Bolg's bodyguard he shattered their formation.

It was at this point the Orc's morale was lost and the game was called. The rules were James' new Midguard ones and were perfect for this Dark age epic clash of men and monsters.

I had a great couple of days and it was a real treat to get out and do something exciting like this. To my amazement I didn't suffer any spear snaps of breakages and everything survived intacted. I was expecting a few casualties and had even taken a little repair kit but didn't need it.

Dain's Iron hills Dwarves push the Goblins from the ridge, supported by a group of Lake men.
An earlier shot of the swirl of battle.
The nine Black riders with their Dark lord 
My beloved Silvan elves holding their own against ravaging wolf packs.

I said it once, I'll say it again, what a bloody marvellous couple of days!

 I'm sure I'll have some more pictures soon to post on here of the day...
 

 Here's a shot of James Morris' Lonely mountain with the only missing element...thousands of blood drinking bats. 

 I couldn't believe my luck at the Other Partizan show last Sunday, I actually got to meet Kevin Calder, the creator of the Bonnie Blue Flag rules.

I first saw the Iron Brigade banner and followed it down to reveal an actual BBF game being played. I recognised Kevin instantly and shook his hand. I gushed about his rules and told him how much I like them. I also explained about play testing it a month ago and he said he had seen the post, which was nice. It was only a quick hello as I had to race back to my own game.

I would have liked to have talked for longer and ask him lots of nerdy questions, like how to adapt them for AWI etc. still, I'm sure I'll bump into him again.

 A Fomorian checks out the smell of new resin on the table.

 

It's time to dig out the old stones and compare them. A lot of these previous models had a greenstuff carving in them.

The two weapon shrines. One is a place of worship, the other, sacrifices.


The two stones of the dead. These could be used for any undead project.

I have to say after the mad month of painting for the Partizan show, my painting mojo is well and truly fried. So I thought I would ease back into things with some gentle building and sculpting.

These figures are a One Hour Wargames Rules campaign set in the age of Arthur. I was picked for the Black shield Irish and so bought a couple of plastic boxes.
 

Skirmishers.

I thought a few Celtic blankets and cloaks wouldn't go a miss. The kits come with some cloaks, but I thought I'd have a go at making my own.
The leader and hero figure from Crusader miniatures. The Druid was a purchase from eBay.
These are a mixture of plastic kits from Gripping Beast and Wargames Atlantic. It's always nice to have more heads etc to create variety in the units.

 Here are the unit sizes in Muskets and Tomahawks that are good to see as a guide. All just early days still just throwing ideas around to see if something sticks.

Drilled Vs irregular base.I decided to go with 60x60mm bases to give a more spread out look. Also these Galloping Major figures are quite large for 28mm so it suits them better.

yesterday I popped to Hobbycraft to see if they had any pieces I could use for the new Midgard rules. I couldn't find any goblets but I did track down the beads needed for heroic deeds.

While I was travelling through the bead tray, I saw one that I thought would make a great Celtic shield.

It has a nice wrap around curve to it but more importantly, sports a nice Celtic swirly pattern.

Ancient Irish warriors fight with the sea devils.
It has a good level of detail to make it look well crafted and ornate. Once I sculpt a boss for the centre I think it will look just the ticket. Nice deep grooves that will catch ink and weathering very well. The swirl too is a very Celtic looking design.

 This tub was £4 which is unbelievable for an armies worth of shields. I might end up using these beads as heroic deeds markers too as they won't roll unlike the rounder ones.

You can get a next size up in container to fill with your beads for £8, again for hundreds of shields this isn't bad. The small tub should be enough though.

Here are some rock pools to add to a coastal game mat. I've added some kelp seaweed to make them a bit more North sea.






There are also a couple of other prints that I plan to use as Fomorian shrines. Places where they honour their dead and stack weapons and armour in tribute.
To save on the greenstuff, I switched to Milliput as it's cheaper for bigger pieces. 

I'll add some more skulls and weapons etc to these bases.Update: These models were sprayed black then dry brushed with grey and white. Contrast colours were then painted over the highlighted pools.



To make them look a bit more aquatic and fantasy, I included colours like blue and purple into the palette.



Here we can see a highlighted model before the Contrast paint washes.








Update: started to pick out the barnacles and shields etc.

Here is a small warband of Dark age Irishmen.  They are meant to be the Black shield Irish from the Winter King books. I struggled with just painting black shields as I wanted to paint some designs. I got around this by painting some fancy shields, then painting them black then rubbing off the paint.

This represents the warband painting over their war shields with black. After some heavy campaigning and harsh weather, not to mention dips in the sea, some of the black paint has started to wear off.

There's enough Celtic design underneath to show through.
Not many more to do now, just the druid and four skirmishers.
These are a mix of Crusader, Gripping beast plastic and Wargames Atlantic plastics. They have been very enjoyable to paint too.

 The new rock pools are a perfect setting for Raiding Black shields.

I thought I'd give this a go as I had heard good things about it. I thought the scenic rock pools would be the perfect test subject, as if they dried glossy it wouldn't matter.
The results were good. I sprayed the models after I had added some flock, I thought the lacquer would help seal it.
After the scenic pieces I thought I'd try it on the new Irish figures. I consider this a tough, hard coat. Another layer of super mat varnish will be added with a brush when this first coat is dry.
The rock pools got a layer of flock really nail the realistic look.

 The flock and paint work is all sprayed with the new lacquer, seems to be a winner.

 

The men of Lord Hastings' Retinue struggle through the mud of Tewkesbury.

Vallejo thick mud was the perfect solution for the grim battle conditions of the war. Adding snow to this layer would look fantastic but would rather limit the battles. I think a generic muddy field is a good all rounder for this brutal conflict.

I broke my usual 69x60mm basing after seeing a friend's and decided to copy it. Partly because the cheapness of plastics allows for bigger units. I also have quite a few old Perry miniatures from the old days of Foundry. These old lead figures are great for sprinkling amongst the ranks to add character. The above photo shows the effect of these old sculpts. They have to be mounted on plastic bases etc to bring them up to the height of the newer plastics. The mud is great for covering these and making everyone level.
As the Vallejo mud was drying, I cut up some thin brush bristles and pushed them into the mixture. These make for great arrows and really helps to give the bases a War of the Roses look and feel.
The mud is also great for splashing up the legs and clothes of the soldiers. It's quite subtle but helps to  set them in the scene.


The mud isn't quite dry yet and there are a couple more things to do before they are finished. Layers of 'Rutted field' from Luke's APS should look good over the mud, as well as patches of static grass. Also the arrows will need some white goose fletching on them.


 These new bases are 80x60mm and give a more realistic look to a unit. I got a bit carried away with these bases and they grew to 10 men per base.

The figures In these units are a mix of old Foundry, Perry's plastics and Forlorn Hope metal figures. They all mix together well and make for characterful formations.

I'm a sucker for sage advice much as anyone else, and Kernighan is certainly right on money in the epigraph. Alas there comes a time in programmer's career when you just end up there despite the warning. It could be that you were indeed too clever for your own good, or maybe the code isn't quite yours anymore after each of your colleague's take on it over the years. Or just sometimes, the problem is indeed so hard that it strains your capacity as a coder.

It would usually start with a reasonable idea made into first iteration code. The solution looks fundamentally sound but then as you explore the problem space further it begins to seep nuance, either as manifestation of some real world complexity or your lack of foresight. When I run into this my first instinct is to instrument the code. If the problem is formidable you got to respect it: flailing around blindly modifying things or ugh, doing a rewrite at this stage is almost guaranteed to be a waste of time. It helps to find a promising spot, chisel it, gain a foothold in the problem, and repeat until you crack it. Comfortable debugging tools here can really help to erode the original Kernighan coefficient from 2 to maybe 1.6 or 1.4 where you can still have a chance.

Lisp users are fortunate with the options of interactive debugging, and one facility I reach often for is the plain BREAK. It's easy enough to wrap it into a conditional for particular matches you want to debug. However sometimes you want it to trigger after a particular sequence of events across different positions in code has taken place. While still doable it quickly becomes cumbersome and this state machine starts to occupy too much mental space which is already scarce. So one day, partly as a displacement activity from being intimidated by a Really Hard Problem I wrote down my debugging patterns as a handful of macros.

Enter BRAKE. Its features reflect my personal preferences so are not necessarily your cup of tea but it could be a starting point to explore in this direction. Things it can do:

• act as a simple BREAK with no arguments (duh)
• wrap an s-expression, passing through its values upon continuing
• trigger sequentially based on the specified position for a common tag
• allow for marks that don't trigger the break but mark the position as reached
• provide conditional versions for the expressions above
• print traces of tagged breakpoints/marks

If you compile functions with debug on you hopefully should be able to see the wrapped sexpr's result values.

(use-package '(brake))

(defun fizzbuzz ()
  (loop for n from 100 downto 0
	for fizz = (zerop (mod n 3))
	for buzz = (zerop (mod n 5)) do
	(format t "~a "
		(if (not (or fizz buzz))
		    (format nil "~d" n)
		  (brake-when (= n 0)
			      (concatenate 'string
					   (if fizz "Fizz" "")
					   (if buzz "Buzz" "")))))))

These macros try to detect common cases for tagged sequences being either aborted via break or completed to the last step, resetting them after to the initial state. However it is possible for a sequence to end up "abandoned", which can be cleaned up by a manual command.

Say in the example below we want to break when the two first branches were triggered in a specific order. The sequence of 1, 3, 4 will reinitialize once the state 4 is reached, allowing to trigger continuously. At the same time if we blow our stack it should reset to initial when aborting.

(defun ack (m n)
  (cond ((zerop m) (mark :ack 3 (1+ n)))
        ((zerop n) (mark :ack 1 (ack (1- m) 1)))
        (t (brake :ack 4 (ack (1- m) (ack m (1- n)))))))

In addition there are a few utility functions to report on the state of brakepoints, enable or disable brakes based on tags and turn tracing on or off. Tracing isn't meant to replace the semantics of TRACE but to provide a souped up version of debug by print statements everyone loves.

CL-USER> (report-brakes)
Tag :M is DISABLED, traced, with 3 defined steps, current state is initial
Tag :F is DISABLED with 2 defined steps, current state is 0
Tag :ACK is ENABLED with 3 defined steps, current state is initial

Disabling breakpoints without recompilation is really handy and something I find using all the time. The ability to wrap a sexpr was often sorely missed when using BREAK in constructs without implicit body.

Sequencing across threads is sketchy as the code isn't guarded but in many cases it can work, and the appeal of it in debugging races is clear. One of those days I hope to make it more robust while avoiding potential deadlocks but it isn't there yet. Where it already shines tho is in debugging complex iterations, mutually recursive functions and state machines.

Earlier this year, I started working through the online book Ray Tracing In One Weekend (Book 1). I have been following along with it in Common Lisp, and I have been extending it all from 3-dimensional to n-dimensional.

I reproduced 4-dimensional versions of all of the book images which you can see on my weekend-raytracer github page.

Here is the final image. This is a 250-samples-per-pixel, 640x360x10 image plane of three large hyperspheres (one mirrored, one diffuse, one glass) atop a very large, diffuse hypersphere. Also atop this very large hypersphere are a bunch of smaller hyperspheres of varying colors and materials. The image is rendered with some defocus-blur.

Final image of 4-dimensional scene

Caveat: This depends on a patched version of the policy-cond library that is not in the current Quicklisp distribution but should be in the next.

 New projects: 

• adp-github — ADP extension to generate github markdown files. — MIT
• adp-plain — Add Documentation, Please... using plain text. An extension of ADP to generate files with barely additional features. — MIT
• allioli — Alliolification — MIT
• alternate-asdf-system-connections — Allows for ASDF system to be connected so that auto-loading may occur. This is a fork of asdf-system-connections and incorporates a load-system-driven mechanism for loading dependencies and also loads the dependencies of the connections. — MIT
• cbor — CBOR encoder/decoder — MIT
• charje.documentation — Documentation is an opinionated yet customizable docstring parsing library. — AGPL V3 or any later version
• chipi — House automation bus in Common Lisp — Apache-2
• cl-aseprite — Aseprite file format parser — GPLv3
• cl-astar — A heavily optimized yet flexible A* pathfinding algorithm implementation — MIT
• cl-ceigen-lite — A Common Lisp wrapper around CEIGEN-LITE - which is itself a C wrapper around the C++ Eigen library. — MIT
• cl-cf — Computations using continued fractions — GPL-3
• cl-concord — CONCORD implementation based on Common Lisp — LGPL
• cl-duckdb — CFFI wrapper around the DuckDB C API — MIT License
• cl-fastcgi — FastCGI wrapper for Common Lisp — BSD License
• cl-flx — Rewrite emacs-flx in Common Lisp — MIT
• cl-frugal-uuid — Common Lisp UUID library with zero dependencies — MIT License
• cl-gog-galaxy — A wrapper for the GOG Galaxy SDK — zlib
• cl-lc — List comprehensions — MIT
• cl-naive-ptrees — Functions to make it easier to work with plist(s) and plist trees. Works with plist(s) pairs as units and not as individual list items. — MIT
• cl-qoa — An implementation of the Quite Okay Audio format. — zlib
• cl-reddit — Reddit client api library — BSD
• cl-resvg — An up-to-date bindings library for the resvg SVG rendering library — zlib
• cl-trivial-clock — Common Lisp library to get accurate wall-clock times on multiple platforms — MIT License
• clack-cors — A Clack middleware to set CORS related HTTP headers. — Unlicense
• clack-prometheus — Clack middleware to serve stats in Prometheus format. — Unlicense
• clith — Common Lisp wITH macro. A general WITH macro. — MIT
• clj-arrows — Implements Clojure-styled threading/transformation macros. — MIT
• clos-encounters — A collection of OOP patterns benefiting from the CLOS MOP. — Unlicense
• coalton — An efficient, statically typed functional programming language that supercharges Common Lisp. — MIT
• cocoas — A toolkit library to help deal with CoreFoundation, Cocoa, and objc — zlib
• com.danielkeogh.graph — A fast an reliable graph library. — MIT
• fast-mpsc-queue — Multi-Producer Single-Consumer queue implementation. — MIT
• file-finder — File finder. Enable rapid file search, inspection and manipulation. — GPL3+
• golden-utils — A utility library. — MIT
• hiccl — HTML generator for Common Lisp — MIT
• hsx — Hypertext S-expression — MIT
• hunchentoot-stuck-connection-monitor — Monitors hunchentoot connections and logs the connections stuck in the same state for a long time (due to slow or inactive clients and network stream timeouts that hunchentoot tries to utilize not working properly). Offers an option to shutdown the stuck connections sockets manually or automatically, thus unblocking the connection threads and preventing thread and socket leak. See https://github.com/edicl/hunchentoot/issues/189 — BSD-2-Clause
• incless — A portable and extensible Common Lisp printer implementation (core) — BSD
• inravina — A portable and extensible Common Lisp pretty printer. — MIT
• invistra — A portable and extensible Common Lisp FORMAT implementation — BSD
• knx-conn — KNXnet/IP implementation in Common Lisp — GNU GPL, version 3
• machine-state — Retrieve machine state information about CPU time, memory usage, etc. — zlib
• myweb — simple web server written in common lisp for educational reasons — LGPLv3
• noisy — Perlin noise for arbitrary numbers of dimensions. — MIT
• nontrivial-gray-streams — A compatibility layer for Gray streams including extensions — MIT
• open-with — Open a file in a suitable external program — zlib
• openai-openapi-client — Openai API client — AGPLv3+
• openrpc — CI for Common Lisp OpenRPC library. — BSD
• parse-number-range — Parses LOOP's convenient "for-as-arithmetic" syntax into 5 simple values: from, to, limit-kind (:inclusive, :exclusive or nil if unbounded), by (step) and direction (+ or -)). Further related utilities are provided. Intended for easy implementation of analogous functionality in other constructs. — Public Domain
• precise-time — Precise time measurements — zlib
• pregexp — Portable regular expressions for Common Lisp — MIT-like
• progressons — Display a progress bar on one line. — MIT
• quaviver — A portable and extensible floating point string library — MIT
• quilc — A CLI front-end for the Quil compiler — Apache License 2.0 (See LICENSE.txt)
• qvm — An implementation of the Quantum Abstract Machine. — Apache License 2.0 (See LICENSE.txt)
• random-sampling — Functions to generate random samples with various distributions — zlib
• rs-dlx — Knuth's Algorithm X with dancing links. — Modified BSD License
• scrapycl — The web scraping framework for writing crawlers in Common Lisp. — Unlicense
• smoothers — Statistical methods to create approximating functions that attempt to capture important patterns in the data, while leaving out noise or other fine-scale structures/rapid phenomena. — MS-PL
• trivial-adjust-simple-array — A tiny utility to change array size ensuring it is simple. — MIT
• trivial-system-loader — A system installation/loading abstraction for Common Lisp — MIT
• trivial-toplevel-commands — Trivial Toplevel Commands allows to define toplevel commands available on most implementations in a portable fashion. — BSD-3 Clause
• trivial-toplevel-prompt — Portability library to customize REPL prompts. — BSD-3 Clause
• utf8-input-stream — A UTF-8 string input stream over a binary stream for Common Lisp — MIT
• whereiseveryone.command-line-args — Automatically create a command-line-argument parser for a given Common Lisp function definition. — AGPL v3 or any later version

Updated projects: 3b-bmfont, 3bgl-shader, 3bmd, 3d-math, 3d-spaces, 40ants-asdf-system, 40ants-slynk, access, acclimation, action-list, adhoc, adopt, adp, agnostic-lizard, alexandria, alexandria-plus, anatevka, anypool, april, arc-compat, architecture.builder-protocol, array-utils, arrow-macros, assoc-utils, async-process, atomics, auto-restart, aws-sdk-lisp, babel, bdef, bike, binary-structures, binding-arrows, birch, blackbird, bordeaux-threads, calm, carrier, caveman, ccldoc, cephes.cl, cepl, cerberus, cffi, cffi-object, cffi-ops, chanl, chunga, ci, ci-utils, ciao, cl-6502, cl-algebraic-data-type, cl-all, cl-ansi-term, cl-async, cl-atelier, cl-autowrap, cl-base32, cl-bmas, cl-bmp, cl-bnf, cl-brewer, cl-buchberger, cl-cmark, cl-collider, cl-colors2, cl-confidence, cl-containers, cl-cookie, cl-csv, cl-custom-hash-table, cl-cxx-jit, cl-data-structures, cl-dbi, cl-digraph, cl-dot, cl-enchant, cl-environments, cl-fast-ecs, cl-fbx, cl-fluent-logger, cl-form-types, cl-forms, cl-freetype2, cl-gamepad, cl-github-v3, cl-gltf, cl-gobject-introspection, cl-graph, cl-grip, cl-gserver, cl-hamcrest, cl-hash-util, cl-html-readme, cl-i18n, cl-info, cl-ini, cl-ipfs-api2, cl-kanren, cl-lib-helper, cl-liballegro, cl-liballegro-nuklear, cl-log, cl-markless, cl-marshal, cl-migratum, cl-mixed, cl-modio, cl-mount-info, cl-mpg123, cl-mssql, cl-mustache, cl-mysql, cl-neovim, cl-netpbm, cl-oju, cl-opengl, cl-opensearch-query-builder, cl-opus, cl-patterns, cl-plus-ssl-osx-fix, cl-ppcre, cl-project, cl-protobufs, cl-pslib, cl-pslib-barcode, cl-rashell, cl-readline, cl-sat.minisat, cl-sdl2-image, cl-sdl2-mixer, cl-sdl2-ttf, cl-sendgrid, cl-sentry-client, cl-skkserv, cl-smtp, cl-ssh-keys, cl-steamworks, cl-str, cl-svg, cl-telegram-bot, cl-threadpool, cl-tiled, cl-torrents, cl-tqdm, cl-transducers, cl-transit, cl-unicode, cl-unification, cl-unix-sockets, cl-utils, cl-vectors, cl-vorbis, cl-wavefront, cl-webdriver-client, cl-webkit, cl-webmachine, cl-who, clack, clack-pretend, clad, classimp, clast, clath, clavier, clazy, clerk, clgplot, climacs, clingon, clip, clj-con, clj-re, clobber, clog, clog-ace, clog-collection, clog-plotly, clog-terminal, clohost, closer-mop, clss, cluffer, clunit2, clx, cmd, codata-recommended-values, codex, coleslaw, collectors, colored, com-on, common-lisp-jupyter, commondoc-markdown, compiler-macro-notes, conduit-packages, consfigurator, contextl, croatoan, ctype, cytoscape-clj, damn-fast-priority-queue, dartscluuid, data-frame, data-lens, datafly, dbus, decompress, defenum, definer, definitions, deflate, defmain, deploy, depot, deptree, dexador, dissect, djula, dns-client, doc, docs-builder, dsm, dufy, easter-gauss, easy-audio, easy-macros, easy-routes, eclector, equals, erjoalgo-webutil, erudite, esrap, event-emitter, external-program, external-symbol-not-found, fare-csv, fare-scripts, fast-http, fast-websocket, file-attributes, file-notify, file-select, filesystem-utils, fiveam, fiveam-matchers, flexi-streams, float-features, flow, fn, fset, functional-trees, fuzzy-dates, gadgets, generic-cl, github-api-cl, glfw, glsl-toolkit, harmony, hashtrie, helambdap, http2, hunchentoot, imago, in-nomine, inferior-shell, introspect-environment, ironclad, jose, js, json-mop, jsonrpc, jzon, khazern, lack, lass, lemmy-api, letv, lichat-protocol, lichat-tcp-client, linear-programming, lisp-binary, lisp-chat, lisp-critic, lisp-pay, lisp-stat, lispcord, lla, local-time, log4cl-extras, logging, lru-cache, magicl, maiden, maidenhead, manifolds, math, mcclim, memory-regions, messagebox, method-combination-utilities, mgl-pax, misc-extensions, mito, mk-defsystem, mmap, mnas-package, mnas-string, moira, multiposter, mutility, mutils, named-closure, ndebug, neural-classifier, new-op, nibbles, nibbles-streams, ningle, nodgui, north, numerical-utilities, nytpu.lisp-utils, omglib, ook, open-location-code, openapi-generator, orizuru-orm, overlord, papyrus, parachute, parse-number, pathname-utils, petalisp, phos, picl, plot, plump, plump-sexp, pngload, policy-cond, polymorphic-functions, postmodern, ppath, prometheus-gc, psychiq, purgatory, py4cl, py4cl2, py4cl2-cffi, qlot, qoi, query-fs, quick-patch, quickhull, quri, random-state, reblocks, reblocks-auth, reblocks-file-server, reblocks-lass, reblocks-navigation-widget, reblocks-parenscript, reblocks-prometheus, reblocks-typeahead, reblocks-ui, reblocks-websocket, rove, s-dot2, sandalphon.lambda-list, sb-fastcgi, sc-extensions, sel, select, serapeum, shasht, shop3, si-kanren, sketch, slime, slite, sly, snooze, spinneret, staple, static-vectors, statistics, stepster, stmx, stripe, swank-crew, swank-protocol, sxql, symath, system-locale, taglib, teddy, ten, testiere, tfeb-lisp-hax, tfm, tiny-routes, tooter, trivia, trivial-arguments, trivial-clipboard, trivial-file-size, trivial-gray-streams, trivial-main-thread, trivial-octet-streams, trivial-package-locks, trivial-package-manager, trivial-sanitize, trivial-shell, type-templates, typo, uax-15, uiop, usocket, vellum, vellum-binary, vellum-csv, vellum-postmodern, verbose, vernacular, vom, websocket-driver, winhttp, with-branching, with-contexts, woo, xhtmlambda, xml-emitter, yason, zippy, zpb-ttf.

Removed projects: abstract-arrays, ahungry-fleece, cl-cheshire-cat, cl-darksky, cl-epoch, cl-naive-store, convolution-kernel, dense-arrays, extensible-compound-types, extensible-optimizing-coerce, fast-generic-functions, flac-metadata, freebsd-ffi, listoflist, luckless, one-more-re-nightmare, postmodern-localtime, stumpwm-dynamic-float, stumpwm-sndioctl, unicly.

To get this update, use:

 (ql:update-dist "quicklisp")

Sorry this update took so long. My goal is to resume monthly releases.

Enjoy!

It's no secret that I'm an aficionado of Lisp. It's my go to language, especially when I don't know what I'm doing. I call it research and prototyping, but it's really just playing around until something works.

We had a need for some auditing of some of our databases at work. They ought to agree with each other and with what GitHub and CircleCI think. It took a couple of weeks part time to prototype a solution in Common Lisp. It showed that the databases were in 99% agreement and found the few points of disagreement and anomalies that we ought to fix or look out for.

I want to integrate this information into a dashboard on one of our tools. I prototyped this by spinning up a Common Lisp microservice that returns the information in JSON format.

But management prefers that new services are written in golang. It would be easier for me to rewrite the service in golang than to try to persuade others to use Common Lisp. It also gives me the opportunity to compare the two languages head to head on a real world problem.

No, this is not a fair comparison. When I wrote the Lisp code I was exploring the problem space and prototyping. I'm much more experienced with Lisp than with golang. The golang version has the advantage that I know what I want to do and how to do it. In theory, I can just translate the Common Lisp code into golang. But then again, this is a “second system” which is not a prototype and has slightly larger scope and fuller requirements. So this cannot be a true head to head comparison.

The first point of comparison is macros (or lack thereof). I generally don't use a lot of macros in Common Lisp, but they come in handy when I do use them. One macro I wrote is called audit-step, which you can wrap around any expresion and it prints out a message before and after the expression is evaluated. The steps are numbered in sequence, and nested steps get nested numbers (like step 2.3.1). If you wrap the major function bodies with this macro, you get a nice trace of the call sequence in the log.

Golang doesn't have macros, but it has first class functions. It's easy enough to write a function that takes a function as an argument and wraps it to output the trace messages. In fact, the macro version in Common Lisp just rewrites the form into such a function call. But the macro version hides a level of indentation and a lambda. In golang, my major functions all start with

func MajorFunction (args) int {
        return AuditStep("MajorFunction", "aux message", func() int {
                // body of MajorFunction
                // Actual code goes here.
        })    
}

The bodies of all my major functions are indented by 16 spaces, which is a little much.

I like higher order functions. I can write one higher order function and parameterize it with functions that handle the specific cases. In my auditing code, one such workhorse function is called collate. It takes a list of objects and creates a table that maps values to all objects in the list that contain that value. To give an example, imaging you have a list of objects that all have a field called foo. The foo field is a string. The collate function can return a table that maps strings to all objects that have that string in the foo field.

collate is very general. It takes a list of objects and four keyword arguments. The :key argument is a function that extracts the value to collate on. The :test argument is a function that compares two keys (it defaults to eql if not specified). The :merger argument is a function to add the mapped object to its appropriate collection in the table (it defaults to adjoin). The :default argument specifies the initial value of a collection in the table (it defaults to nil).

The :merger function is the most interesting. It takes the key and the object and the current value of the table at that key. It returns the new value of the table at that key. The default merger function is adjoin, which adds the object to the collection at the key if it is not already there. But you can specify a different merger function. For example, if you want to count the number of objects at each key, you can specify a merger function that increments a counter.

The functional arguments to the collate function are often the results of other higher order functions. For example, the :key argument is often the result of composing selector functions. The :merger argument is often the result of composing a binary merge function with a unary transformer function. The transformer function is often the result of composing a number of primitive selectors and transformers.

In Common Lisp, it is quite easy to write these higher order functions. We can compose two unary functions with the compose2 function:

(defun compose2 (f g)
  (lambda (x) (funcall f (funcall g x)))

and then compose as many functions as we like by fold-left of compose2 starting with the identity function:

(defun compose (&rest fs)
  (fold-left #'compose2 #'identity fs))

We can compose a binary function with a unary function in three ways: we can pipe the output of the binary function into the unary function, or we can pipe the output of the unary function into one or the other of the inputs of the binary function.

(defun binary-compose-output (f g)
  (lambda (x y) (funcall f (funcall g x y))))

(defun binary-compose-left (f g)
  (lambda (x y) (funcall f (funcall g x) y)))

(defun binary-compose-right (f g)
  (lambda (x y) (funcall f x (funcall g y))))

The collate function can now assume that a lot of the work is done by the :key and :merger functions that are passed in. It simply builds a hash table and fills it:

(defun collate (item &key (key #'identity) (test #'eql) (merger (merge-adjoin #'eql)) (default nil))
  (let ((table (make-hash-table :test test)))
    (dolist (item items table)
      (let ((k (funcall key item)))
        (setf (gethash k table) (funcall merger (gethash k table default) item))))))

(defun merge-adjoin (test)
  (lambda (collection item)
    (adjoin item collection :test test)))

So suppose, for example, that we have a list of records. Each record is a three element list. The third element is a struct that contains a string. We want a table mapping strings to the two element lists you get when you strip out the struct. This is easily done with collate:

(collate records
  :key (compose #'get-string #'third)
  :test #'equal      ; or #'string= if you prefer
  :merger (binary-compose-right (merge-adjoin #'equal) #'butlast))

The audit code reads lists of records from the database and from GitHub and from CircleCI and uses collate to build hash tables we can use to quickly walk and validate the data.

Translating this into golang isn't quite so easy. Golang has first class function, true, but golang is a statically typed language. This causes two problems. First, the signature of the higher order functions includes the types of the arguments and the return value. This means you cannot just slap on the lambda symbol, you have to annotate each argument and the return value. This is far more verbose. Second, higher order functions map onto parameterized (generic) types. Generic type systems come with their own little constraint language so that the computer can figure out what concrete types can correctly match the generic types. This makes higher order functions fairly unweildy.

Consider compose2. The functions f and g each have an input and output type, but the output type of g is the input type of f so only three types are involved

func Compose2[T any, U any, V any](f func(U) V, g func(T) U) func(T) V {
	return func(x T) V {
		return f(g(x))
	}
}

If want to compose three functions, we can write this:

func Compose3[T any, U any, V any, W any](f func(V) W, g func(U) V, h func(T) U) func(T) W {
	return func(x T) W {
		return f(g(h(x)))
	}
}

The generic type specifiers take up as much space as the code itself.

I don't see a way to write an n-ary compose function. It would have to be dynamically parameterized by the intermediate types of all the functions it was composing.

For the collate function, we can write this:

func Collate[R any, K comparable, V any](
	list *Cons[R],
	keyfunc func(R) K,
	merger func(V, R) V,
	defaultValue V) map[K]V {
	answer := make(map[K]V)
	for list != nil {
		key := keyfunc(list.Car)
		probe, ok := answer[key]
		if !ok {
			probe = defaultValue
		}
		answer[key] = merger(probe, list.Car)
		list = list.Cdr
	}
	return answer
}

We have three types to parameterize over: the type of the list elements (i.e. the record type) R, the type of the key K, and the type of the value V. The key type is needs to be constrained to be a valid key in a map, so we use the comparable constraint. Now that we have the types, we can annotate the arguments and return value. The list we are collating is a list of R elements. The key function takes an R and returns a K. The merger takes an existing value of type V and the record of type R and returns a new value of type V.

The magic of type inference means that I do not have to annotate all the variables in the body of the function, but the compiler cannot read my mind and infer the types of the arguments and return value. Golang forces you to think about the types of arguments and return values at every step of the way. Yes, one should be aware of what types are being passed around, but it is a burden to have to formally specify them at every step. I could write the Common Lisp code without worrying too much about types. Of couse the types would have to be consistent at runtime, but I could write the code just by considering what was connected to what. In golang, the types are in your face at every function definition. You not only have to think about what is connected to what, you have to think about what sort of thing is passed through the connection.

I'm sure that many would argue that type safety is worth the trouble of annotation. I don't want to argue that it isn't. But the type system is cumbersome, awkward, and unweildy, especially when you are trying to write higher order functions.

It is taking me longer to write the golang version of the audit service than it did to write the Common Lisp version. There are several reasons. First, I am more experienced with Common Lisp than golang, so the right Common Lisp idioms just come to mind. I have to look up many of the golang idioms. Second, the golang code is trying to do more than the Common Lisp code. But third, golang itself introduces more friction than Common Lisp. Programs have to do more than express the algorithm, they have to satisfy the type system.

There are more points of comparison between the two languages. When I get frustrated enough, I'll probably write another post.