in

Austin Ironman 70.3! (Race Weekend!)

Okay, it's been a while, but I thought I'd finally post about the 2017 Austin Ironman 70.3 race weekend. :-). Like I mentioned in my last post, I felt fairly optimistic -- if anything I was a bit burned out on training.

The big kicker, though, was that the weather was forecast to be 38 degrees race morning, which resulted in a bit of last-minute anxiety, mostly due to the mantra, "Don't do anything different on race day." That is, it is unwise in the extreme to test out new equipment or clothing on race day. Unfortunately, faced with the prospect of a 38 degree swim and bike ride (and the fact that it would warm up quickly), we had to make some last minute adjustments.

The week before the race, I picked up a triathlon jersey that had sleeves, and borrowed some arm warmers from one of my training partners.  Saturday morning, I went for a five mile test ride and realized I needed gloves, which necessitated a trip to Austin Tri-Cyclist, where I was not the only one making such a purchase :-). 

I was still a bit concerned about the swim, but I had a wetsuit, so I figured I'd done all I could do. Still, I was coveting one with sleeves...
I next headed over to the Travis County Expo Center to pick up my swag bag, drop off my bicycle and gear, and check out the transition areas.

Our happy faces before we get out of the car into the cold...

 Next morning, three of us drove out together and arrived in the cold dark of the Travis County Expo. Center at around 5:30. Did I mention that it was cold? Eventually, we got on the bus to take us to the staging area at Walter Long (Decker) Lake. There, we put on wetsuits, met up with our other training partners, and shivered a lot.

Eventually, though, as the sun started to peak above the horizon, we lined up according to our projected swim times and prepared for the start of the race. I was still worried about the swim and the cold -- even though I was wearing socks (to be discarded just before start), my feet were starting to go numb and my arms were not happy either.

But then it was time! I ran into the water and dived in as soon as I could. To my surprise, it was actually pleasant -- the water itself was around 68-72 degrees, so compared to the air temperature, it was balmy.  The only real problem was that fog on the water and the glare of the sun were making it hard to see the buoys. About halfway through the swim I began thinking that the temperature really wasn't bad -- if anything, it was a bit too warm. 

Emerging from the water...

But then I finished the swim and stood and was confronted by the reality of confronting an air temp of around 40 while being soaking wet.  I grabbed my glasses from the special needs table and a nice man helped me get the wetsuit off. (This basically involved lying back on the wet ground, sticking my feet in the air and having him pull. Thank you, sir.). My time was a little slower than I would've liked, but I was fairly happy with it.

I made it to transition, took a big swig of water, swallowed the contents of an energy gel pack, and put on my winter cycling garb. Then I was off!

And it was frickin' cold.

Contemplating that wind chill...
 It was this weird Catch-22 where you want to go as fast as possible (for the race, of course, but also so you warm up), but also kind of are thinking that if you slowed down a touch the wind chill wouldn't be quite so bad.  I ended up spending the next hour shivering until the ambient temperature and I warmed up.
Now, I actually kind of like the bike route -- it's mostly country roads with little traffic, and I rode the route about a half-dozen times in training. The problem with it is that a number of the roads are not exactly well=paved. Bumps and potholes and patches proliferate, especially on Monkey Road. In fact, the dip where it crosses a creek is so bumpy that by the time I got there, there were at least a dozen water bottle scattered on the ground.

There were way too many hills, however :-).

Beyond that, the ride felt fine, although my back began to hurt about halfway through -- I wasn't used to spending that much time in the aero position, so most of the second half of my ride was with hands on the brake hoods. I made sure to stick with my hydration and nutrition plan, so I felt pretty good by the end of it.

Again, my time wasn't quite what I wanted it to be, but I was not displeased.

By the time I finished the ride, it was around noon and fairly warm, so I took the time to change from my sleeved singlet to a sleeveless one (Ironman rules require that you wear a shirt). 

Starting to feel the legs...
The run wasn't as bad as I thought it would be, although there were again too many hills :-).  I was pleased at the number and size of the aid stations -- water, electrolyte drink (Gatorade, iirc), Clif energy gels, Coca-Cola, and Red Bull were all available.  

I was definitely feeling my legs, but my quads didn't feel like they were going to seize up like they had when I did triathlons in the 90s -- all that training paid off, I guess :-). I managed to make it through without slowing to a walk (other than at aid stations, because I can't drink and run at the same time) and ended up with a run time that was comparable to my stand-alone half-marathon times.

At the finish line!
My final time was 5:50:36, which I'm pretty happy about. My goal had been 6:00:00, although I did think that 5:45:00 was not out of the question. :-).

Finisher photo! And medal!
The gang...
All in all, it was a great experience. I got out of my comfort zone, made some terrific friends, learned how to most efficiently change a bike tube, and never once thought, "I can't believe I'm paying to do this." (Okay, maybe once...).

After the race!
Celebrating the next day with Coach Peri!























in

Why does Jupiter spin so fast?

The gas giant is the Solar System's largest planet. Here's why it's also the fastest-spinning planet.




in

Serendipity, a super-Jupiter, and saving VIPER

This was a big week in space, from Curiosity stumbling upon sulfur crystals to an exoplanet discovery and a major advocacy effort.




in

A big find on Mars

A big rover makes a big find on Mars. Little rovers have their place in exploration, too.




in

Eureka? Scientists’ first hints of life on other planets may not be so obvious

Knowing that you've found signs of life beyond Earth may not be as clear-cut and simple as one might think.




in

Seeing the unseeable

From X-ray imaging to slithering beneath Enceladus’ crust, space technology is always expanding what we can see for ourselves.




in

Life in other worlds

New research suggests liquid water might be hiding under the surface of Mars. Could life be there too?




in

Ramses: A new mission racing to land on asteroid Apophis

When a skyscraper-sized asteroid narrowly misses Earth in 2029, three spacecraft may be along for the ride.




in

Explore the Cosmos with The Planetary Society and Lerner Publishing

The Planetary Society and Lerner Publishing Group have teamed up to bring young readers an engaging series of books that make space science fun and accessible.




in

The Tianlin Space Telescope

China is in the early stages of planning a huge space observatory to help answer the matter of whether we are alone in the galaxy.




in

Inside, underneath, backward, upside-down

From holes on Mars to a spun-around moon and a flipped reflection, space science involves looking at things from all different angles.




in

Your impact: September equinox 2024

Exploring Europa and defending Earth.




in

Earthlings as aliens

Looking at life on Earth from another perspective.




in

Connecting ancient life to other worlds

Looking to the past to guide the search for life.




in

New insights into asteroid properties: A STEP Grant update

A Planetary Society-funded project to understand asteroids achieved its main goals and scientific objectives this year.




in

Could Europa Clipper find life?

For a mission that doesn’t aim to find alien life, Europa Clipper may come surprisingly close.




in

How to spot Comet Tsuchinshan-Atlas

Catch this once-in-a-lifetime comet over the next few days.




in

Cloudy skies, smooth sailing

A Martian cloud atlas, LightSail wins big, and multiple missions coast toward launch.




in

Twinsies!

Asteroids that come in pairs, matching volcanic features on Mars and Earth, and the potential space policies of two administrations.




in

Space in the 2024 elections

An overview of what U.S. space policy might look like under a Trump or a Harris administration.




in

Journeys worth making

Perseverance faces a hard climb, but New Horizons proves it’s worth going the distance.




in

Why Taylor-Serrano deserves top billing over Tyson-Paul carnival

How the inclusion of Katie Taylor v Amanda Serrano on the bill legitimises the carnival of Mike Tyson v Jake Paul in Texas




in

Barbour Nimbus Wellington Boots




in

Schmidt Bros. Carbon Carving Set




in

Glenmorangie A Tale of Ice Cream Single Malt Scotch Whisky





in

Christmas jewelry in progress.

Sneak peek — Christmas jewelry in progress.




in

Christmas serving board in progress.

Christmas serving board in progress.




in

Learned something new

Learned something new today— resin fidget spinner. ????




in

Operation Beorn Again.

The great bear himself finally turns up... bloody late as usual.
After waiting a while for the second bear to arrive...the cutting and hacking could begin.
The plastic was very tough to get through with my craft saw. After what seems an age, the great heavy head fell off and the Orcs cheered. They soon stopped when they saw the angry head being wired into place.




I had to try a couple of different fur techniques, for this chunky, thick fur, I went for the classic GW wolf pelt look. This is the one where you cut lots of little triangles into the putty and push them up.
Beorn laughs at his new coat... Hopefully that's a good sign he likes it.
I did take the opportunity to bulk up his hump while applying the putty. I will also add some to his forelegs and make them a bit more shaggy.
 It's all about patience with greenstuff, don't rush it and give the first stuff time to harden.

By the time he realized it was too late.

The funniest thing I've seen in weeks, this bought a tear to my eye.


Sorry, this never gets old.

The model I selected for Beorn looked great, he was big and furry, but didn't look fierce enough. I soon came up with a cunning plan to buy another toy I had seen and splice them together. Luckily the toys were the same scale and operation Beorn again was on.
 
Battle of the beasts.

Update: Beorn is now has thick fur to stop any orc blade. Hopefully when painted, he should look a bit more ragged and crazy.






He is now looking a look more the part...






  • lord Of The Rings
  • lord Of The Rings.

in

Creating Draugr

I have an idea of making a few stands of Draugr, the Viking dead. I have bought some nice Colin Pattern sculpts but thought I'd have a go at making some.
I have loads of Viking sprues left over from the Lake town project, these I mixed with some Oathmark undead sprues. I will add some greenstuff to these to make them a little more Viking.


 




in

Dwarf King in the shield wall

A Dwarf king and his elite guard take to the field.

 Amongst the Eagle helms of the shield wall is the kings champion himself. Nothing must get to the king and the champions blue axe, will see that nothing does.

I could use this base for King Balin when he tries to retake Moria. A future project of mine.
The King points out floors in the enemies formation to the chief engineer.
The Dragon standard flies over head echoing the Kings mighty dragon helm.
This base has been created for the upcoming Hobbit battle, replacing Dain's base in the ranks. This allows Dain to roam freely as a character on the field.

 The figures are beautiful old sculpts from Asgard and Citadel. There has been a bit of tweaking on a couple of them, like the axe and the standard.



  • lord Of The Rings
  • lord Of The Rings.

in

Hobbit Production Line

Here is Beorn in his full fury. He has been given a Matt varnish that really helped. This proved even more effective once his mouth, eyes and nose was given a coat of gloss varnish. This brought him to life and gave him a sparkle in his eye.

I ended up putting more and more greenstuff on him until he was completely covered. It's good to have him covered in thick, shaggy fur as he looks more wild and rugged.


The Eagles are coming! Eagles and White wolves get their various layers of flock.
A lovely new Reaper miniatures Dwarf joins the latest group of Dwarves. Reaper make great 'hero' figures to adorn any warband. They can be quite large sculpts so it can be a risky business ordering them, this one is perfect though.
Beorn's base has to be 24cm wide for the game I'm playing, so two side bases were made. These bring the base up to the right measurement and allow for more bear carnage.
Beorn with his slobbering maw but cute adorable shiny eyes.
More goblin wreckage. Wargs too have not escaped the Bears fury and lie amongst the fallen. A few goblins still cling to life.
It was fun to sculpt great claw wounds in the orcs clothes and equipment. Everything has been torn and crushed to pieces.
Beorn's base with some Bodyguard of Bolg behind him. I'm not sure I'll have time to finish this big orcs before the weekend.

Bard of Lake town with his new fixed banner. This banner was a bit too tall for the storage box and the spearpoint snapped off, twice!. So using my flattened brush bristle technique, he's got a new much tougher one. The plastic used for brush bristles make them fantastic for super glue!

As Bard needs to be free to roam around, another replacement base was needed. Here is Bard with a fellow command stand. Bard is another Reaper Miniatures figure and is slightly bigger than most. This is fine as he is a hero and it suits his manly character.

More carnage, this poor Warg has been disemboweled. This is pretty grim but I did have fun sculpting it. Facing a giant werebear was never going to end well. Again, a coat of gloss helps bring it to life 
Another shot of the basing production line.
Bard of Esgaroth.

 The blue theme really helped to tie these militia type troops together. I tested the new banner tip and it comes below the storage box height, hopefully it won't get crushed again.



  • lord Of The Rings
  • lord Of The Rings.

in

Ancient Erin

I saw these new ancient Irish scenary pieces from Alternative Armies last week. I thought they looked good so bought a few. These will be added to with more skulls and blades once the glue is dry.


It'll be nice to cover these in grass to give them an ancient look and feel.
This one will look great with some extra swords laying across the new rocks. Fantasy stuff really gets the creativity flowing and is so rewarding.
This is a pebble I found and thought it had an interesting shape. Taken back through the mists of time, it is now a huge fertility alter. I can't tell you what happens here, you have to use your imagination.
The great fertility stone looks very different in the moonlight. Some have said to see it move...and take on the shape of a giant woman.

 A Fomorian checks out the smell of new resin on the table.

Update: I've added some greenstuff furs on some of the alters. 



A few extra swords from the bits box help to finish off this weapon shrine.



Update: The next step is to dry brush them with three tones of grey. The last one is almost a white, to really bring out the details.


After the greys are dry I started playing around with watered down Contrast paint from GW. I leave the Contrast paint darker around the base of the rocks, this helps to weather them.

This is the fun part where you can add loads of subtle colours to the stones.
For this sort of scenery work I find a small make up brush is perfect. Just make sure you put it back when you're finished.





in

Stones, Shrines and Alters of Erin


Here are the finished standing stones from Alternative Armies amongst others.
I've added a few extra skulls to the base once flocked.
A bear hide has been left as an offering too.

Battle shine, again extra weapons have been added to the base. The rocks form a sacred circle in which to knee and offer your items.

Paper ferns were added to this base. I'm not sure how they will last as they are fragile. Being hard up against the stone will help them.
This place must be strong in fey magic because flowers have sprung up around the stones. Flowers are a nice way to show the affinity of a place. Flowers equal a nice holy area, where dead and dying grass says the opposite.


The great fertility rock with it's comfy furs to lie on. I wanted to give this the impression of a giant lady in a fur cloak. The cloak is made from moss and ivy.
Ancient pillar, I added some ivy to this to weather it in. Also putting flock to act as moss up one side helped to age it. Moss normally grows on the north side of trees, so I added this trick to a lot of the stones.



This is a miniature from Reaper Bones, I think it's called 'Evil tower'. I thought it would make a good Fomorian piece or maybe something older and darker. What ever it is, it has been thrust up from the ground and killed the surrounding grass. It must be cursed...


Ivy again helps weather these small stones. This isn't a bad place as flowers grow around the mound, new life from old.

I've rediscovered Celtos as a figure range that might be useful for this growing project.


 


It's time to dig out the old stones and compare them. A lot of these previous models had a greenstuff carving in them.







The two weapon shrines. One is a place of worship, the other, sacrifices.


The two stones of the dead. These could be used for any undead project.






Recently I thought about making more of an effort to photograph my miniatures. I have seen other people use backdrops and so thought I'd play around with the idea. By using a photo from a book as a background, the results can be quite different. Rather than a cluttered work table full of detail taking away from an image, the background actual enhances and compliments it. This simple book set up has really improved these stone pictures and it's something I'll do again for sure.




in

FIW using Bonnie Blue Flag Rules


Here are the craziest scribblings of a madman. Having selected BBF for my black powder games, I'm now just trying to down scale it for a skirmish type game. Smaller units but still using multi bases for figures. Units of four seem to work as a base number.
 
Here are the unit sizes in Muskets and Tomahawks that are good to see as a guide. All just early days still just throwing ideas around to see if something sticks.

Drilled Vs irregular base.

Update: Right! I've bit the bullet and revised my Indians for the game.
I decided to go with 60x60mm bases to give a more spread out look. Also these Galloping Major figures are quite large for 28mm so it suits them better.






I imagine four bases of Indians to a unit with a base of skirmishers. I just want one rule set for my black powder games and I think BBF is the one, with a few tweaks of course.
My latest thinking is make Indians 'green', so they won't stick around long once the casualties start to mount up. A +10 melee modifier makes them dangerous close up so it's worth trying to get them into hand to hand. They are just too wise to stick around when things go bad. An experienced unit of warriors would be quite imposing with their combat bonus, something to be feared.
I had thought of giving Rangers the same combat bonus but as they will be veterans, a plus 10 would be too much. I think let's class them as Elites with a +5.



  • Bonnie Blue Flag

in

Finished Rock pools


The last stage was to gloss varnish the water in the pools. This really brought them to life and gave them a sparkle.

The kelp and some of the green rocks were also given a lick with the gloss brush too. Not too much, just enough to give them a damp look.






The bottom of the pools were painted with loads of people's of different colours. This was then given a green wash, then a brown one to add depth to the pool.

 




in

Canadian Basing

Here are my test bases for the French and Indian wars, using the Bonnie Blue Flag rules.
I quite like these bases, they have more character than the single ones for me. Hopefully, these few test bases will inspire me to do a few more and finish the unit.

Galloping Major figures are so nice that you want to do them justice.


These have just been flocked and need a bit of a dusting off.
Mixed herbs make great leaf litter that is a classic look for the ancient Canadian forests.
When painting these figures I used a few coloured fine tip pens to detail the belts and pouches. I also gave them a coat of Dark tone dip from the tin. I was trying to think of a way of speeding up the painting process.
The Huron, allied to the French.

I love the long muskets and rifles.

 Twigs from the garden make for great moss covered logs.




  • French and Indian War

in

War of the Roses Basing

The men of Lord Hastings' Retinue struggle through the mud of Tewkesbury.

Vallejo thick mud was the perfect solution for the grim battle conditions of the war. Adding snow to this layer would look fantastic but would rather limit the battles. I think a generic muddy field is a good all rounder for this brutal conflict.



I broke my usual 69x60mm basing after seeing a friend's and decided to copy it. Partly because the cheapness of plastics allows for bigger units. I also have quite a few old Perry miniatures from the old days of Foundry. These old lead figures are great for sprinkling amongst the ranks to add character. The above photo shows the effect of these old sculpts. They have to be mounted on plastic bases etc to bring them up to the height of the newer plastics. The mud is great for covering these and making everyone level.
As the Vallejo mud was drying, I cut up some thin brush bristles and pushed them into the mixture. These make for great arrows and really helps to give the bases a War of the Roses look and feel.
The mud is also great for splashing up the legs and clothes of the soldiers. It's quite subtle but helps to  set them in the scene.


The mud isn't quite dry yet and there are a couple more things to do before they are finished. Layers of 'Rutted field' from Luke's APS should look good over the mud, as well as patches of static grass. Also the arrows will need some white goose fletching on them.


 These new bases are 80x60mm and give a more realistic look to a unit. I got a bit carried away with these bases and they grew to 10 men per base.

The figures In these units are a mix of old Foundry, Perry's plastics and Forlorn Hope metal figures. They all mix together well and make for characterful formations.



  • War of the Roses

in

Eugene Zaikonnikov: Breaking the Kernighan's Law

"Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.." — Brian W. Kernighan.

I'm a sucker for sage advice much as anyone else, and Kernighan is certainly right on money in the epigraph. Alas there comes a time in programmer's career when you just end up there despite the warning. It could be that you were indeed too clever for your own good, or maybe the code isn't quite yours anymore after each of your colleague's take on it over the years. Or just sometimes, the problem is indeed so hard that it strains your capacity as a coder.

It would usually start with a reasonable idea made into first iteration code. The solution looks fundamentally sound but then as you explore the problem space further it begins to seep nuance, either as manifestation of some real world complexity or your lack of foresight. When I run into this my first instinct is to instrument the code. If the problem is formidable you got to respect it: flailing around blindly modifying things or ugh, doing a rewrite at this stage is almost guaranteed to be a waste of time. It helps to find a promising spot, chisel it, gain a foothold in the problem, and repeat until you crack it. Comfortable debugging tools here can really help to erode the original Kernighan coefficient from 2 to maybe 1.6 or 1.4 where you can still have a chance.

Lisp users are fortunate with the options of interactive debugging, and one facility I reach often for is the plain BREAK. It's easy enough to wrap it into a conditional for particular matches you want to debug. However sometimes you want it to trigger after a particular sequence of events across different positions in code has taken place. While still doable it quickly becomes cumbersome and this state machine starts to occupy too much mental space which is already scarce. So one day, partly as a displacement activity from being intimidated by a Really Hard Problem I wrote down my debugging patterns as a handful of macros.

Enter BRAKE. Its features reflect my personal preferences so are not necessarily your cup of tea but it could be a starting point to explore in this direction. Things it can do:

  • act as a simple BREAK with no arguments (duh)
  • wrap an s-expression, passing through its values upon continuing
  • trigger sequentially based on the specified position for a common tag
  • allow for marks that don't trigger the break but mark the position as reached
  • provide conditional versions for the expressions above
  • print traces of tagged breakpoints/marks

If you compile functions with debug on you hopefully should be able to see the wrapped sexpr's result values.

(use-package '(brake))

(defun fizzbuzz ()
  (loop for n from 100 downto 0
	for fizz = (zerop (mod n 3))
	for buzz = (zerop (mod n 5)) do
	(format t "~a "
		(if (not (or fizz buzz))
		    (format nil "~d" n)
		  (brake-when (= n 0)
			      (concatenate 'string
					   (if fizz "Fizz" "")
					   (if buzz "Buzz" "")))))))

These macros try to detect common cases for tagged sequences being either aborted via break or completed to the last step, resetting them after to the initial state. However it is possible for a sequence to end up "abandoned", which can be cleaned up by a manual command.

Say in the example below we want to break when the two first branches were triggered in a specific order. The sequence of 1, 3, 4 will reinitialize once the state 4 is reached, allowing to trigger continuously. At the same time if we blow our stack it should reset to initial when aborting.

(defun ack (m n)
  (cond ((zerop m) (mark :ack 3 (1+ n)))
        ((zerop n) (mark :ack 1 (ack (1- m) 1)))
        (t (brake :ack 4 (ack (1- m) (ack m (1- n)))))))

In addition there are a few utility functions to report on the state of brakepoints, enable or disable brakes based on tags and turn tracing on or off. Tracing isn't meant to replace the semantics of TRACE but to provide a souped up version of debug by print statements everyone loves.

CL-USER> (report-brakes)
Tag :M is DISABLED, traced, with 3 defined steps, current state is initial
Tag :F is DISABLED with 2 defined steps, current state is 0
Tag :ACK is ENABLED with 3 defined steps, current state is initial

Disabling breakpoints without recompilation is really handy and something I find using all the time. The ability to wrap a sexpr was often sorely missed when using BREAK in constructs without implicit body.

Sequencing across threads is sketchy as the code isn't guarded but in many cases it can work, and the appeal of it in debugging races is clear. One of those days I hope to make it more robust while avoiding potential deadlocks but it isn't there yet. Where it already shines tho is in debugging complex iterations, mutually recursive functions and state machines.




in

Patrick Stein: Ray Tracing In One Weekend (in Lisp, and n-dimenions)

Earlier this year, I started working through the online book Ray Tracing In One Weekend (Book 1). I have been following along with it in Common Lisp, and I have been extending it all from 3-dimensional to n-dimensional.

I reproduced 4-dimensional versions of all of the book images which you can see on my weekend-raytracer github page.

Here is the final image. This is a 250-samples-per-pixel, 640x360x10 image plane of three large hyperspheres (one mirrored, one diffuse, one glass) atop a very large, diffuse hypersphere. Also atop this very large hypersphere are a bunch of smaller hyperspheres of varying colors and materials. The image is rendered with some defocus-blur.

Final image of 4-dimensional scene

Caveat: This depends on a patched version of the policy-cond library that is not in the current Quicklisp distribution but should be in the next.




in

vindarel: Running my 4th Common Lisp script in production© - you can do it too

Last week I finished a new service written in Common Lisp. It now runs in production© every mornings, and it expands the set of services I offer to clients.

It’s the 4th service of this kind that I developed: - they are not big - but have to be done nonetheless, and the quicker the better (they each amount to 1k to 2k lines of Lisp code), - they are not part of a super advanced domain that requires Common Lisp superpowers - I am the one who benefits from CL during development, - I could have written them in Python - and conversely nothing prevented me from writing them in Common Lisp.

So here lies the goal of this post: illustrate that you don’t need to need a super difficult problem to use Common Lisp. This has been asked many times, directly to me or on social media :)

At the same time, I want to encourage you to write a little something about how you use Common Lisp in the real world. Sharing creates emulation. Do it! If you don’t have a blog you can simply write in a new GitHub repository or in a Gist and come share on /r/lisp. We don’t care. Thanks <3

We’ll briefly see what my scripts do, what libraries I use, how I deploy them, what I did along the way.

Needless to say that I dogfooded my CIEL (beta) meta-library and scripting tool for all those projects.

Table of Contents

Scripts n°4 and 2 - shaping and sending data - when you can write Lisp on the side

My latest script needs to read data from a DB, format what’s necessary according to specifications, and send the result by SFTP.

In this case I read a DB that I own, created by a software that I develop and host. So I could have developed this script in the software itself, right? I could have, but I would have been tied to the main project’s versioning scheme, quirks, and deployment. I rather had to write this script on the side. And since it can be done on the side, it can be done in Common Lisp.

I have to extract products and their data (price, VAT...), aggregate the numbers for each day, write this to a file, according to a specification.

To read the DB, I used cl-dbi. I didn’t format the SQL with SxQL this time like in my web apps (where I use the Mito light ORM), but I wrote SQL directly. I’m spoiled by the Django ORM (which has its idiosyncrasies and shortcomings), so I double checked the different kinds of JOINs and all went well.

I had to group rows by some properties, so it was a great time to use serapeum:assort. I left you an example here: https://dev.to/vindarel/common-lisps-group-by-is-serapeumassort-32ma

Dates have to be handled in different formats. I used local-time of course, and I still greatly appreciate its lispy formatter syntax:

(defun date-yymmddhhnnss (&optional date stream)
  (local-time:format-timestring stream
                                (or date (local-time:now))
                                :format
                                '((:year 4)
                                  (:month 2)
                                  (:day 2)
                                  (:hour 2)
                                  (:min 2)
                                  (:sec 2)
                                  )))

the 2 in (:month 2) is to ensure the month is written with 2 digits.

Once the file is written, I have to send it to a SFTP server, with the client’s codes.

I wrote a profile class to encapsulate the client’s data as well as some functions to read the credentials from either environment variables, the file system, or a lisp variable. I had a top-level profile object for ease of testing, but I made sure that my functions formatting or sending data required a profile parameter.

(defun send-stock (profile &key date) ...)
(defun write-stock (profile filename) ...)

Still nothing surprising, but it’s tempting to only use global parameters for a one-off script. Except the program grows and you pay the mess later.

SFTP

To send the result through SFTP, I had to make a choice. The SFTP command line doesn’t make it possible to give a password as argument (or via an environment variable, etc). So I use lftp (in Debian repositories) that allows to do that. In the end, we format a command like this:

lftp sftp://user:****@host  -e "CD I/; put local-file.name; bye"

You can format the command string and run it with uiop:run-program: no problem, but I took the opportunity to release another utility:

First, you create a profile object. This one-liner reads the credentials from a lispy file:

(defvar profile (make-profile-from-plist (uiop:read-file-form "CREDS.lisp-expr"))

then you define the commands you’ll want to run:

(defvar command (put :cd "I/" :local-filename "data.csv"))
;; #<PUT cd: "I/", filename: "data.csv" {1007153883}>

and finally you call the run method on a profile and a command. Tada.

Deploying

Build a binary the classic way (it’s all on the Cookbook), send it to your server, run it.

(during a testing phase I have deployed “as a script”, from sources, which is a bit quicker to pull changes and try again on the server)

Set up a CRON job.

No Python virtual env to activate in the CRON environment...

Add command line arguments the easy way or with the library of your choice (I like Clingon).

Script n°2 and simple FTP

My script #2 at the time was similar and simpler. I extract the same products but only take their quantities, and I assemble lines like

EXTRACTION STOCK DU 11/04/2008
....978202019116600010000001387
....978270730656200040000000991

For this service, we have to send the file to a simple FTP server.

We have a pure Lisp library for FTP (and not SFTP) which works very well, cl-ftp.

It’s a typical example of an old library that didn’t receive any update in years and so that looks abandoned, that has seldom documentation but whose usage is easy to infer, and that does its job as requested.

For example we do this to send a file:

(ftp:with-ftp-connection (conn :hostname hostname
                                   :username username
                                   :password password
                                   :passive-ftp-p t)
      (ftp:store-file conn local-filename filename))

I left you notes about cl-ftp and my SFTP wrapper here:

Scripts n°3 and n°1 - specialized web apps

A recent web app that I’m testing with a couple clients extends an existing stock management system.

This one also was done in order to avoid a Python monolith. I still needed additions in the Python main software, but this little app can be independent and grow on its own. The app maintains its state and communicates it with a REST API.

 

It gives a web interface to their clients (so my clients’ clients, but not all of them, only the institutional) so that they can:

  • search for products
  • add them in shopping carts
  • validate the cart, which sends the data to the main software and notifies the owner, who will work on them.

The peculiarities of this app are that:

  • there is no user login, we use unique URLs with UUIDs in the form: http://command.client.com/admin-E9DFOO82-R2D2-007/list?id=1
  • I need a bit of file persistence but I didn’t want the rigidity of a database so I am using the clache library. Here also, not a great activity, but it works©. I persist lists and hash-tables. Now that the needs grow and the original scope doesn’t cut it any more, I wonder how long I’ll survive without a DB. Only for its short SQL queries VS lisp code to filter data.

I deploy a self-contained binary: code + html templates in the same binary (+ the implementation, the web server, the debugger...), with Systemd.

I wrote more on how to ship a standalone binary with templates and static assets with Djula templates here:

I can connect to the running app with a Swank server to check and set parameters, which is super helpful and harmless.

It is possible to reload the whole app from within itself and I did it with no hiccups for a couple years, but it isn’t necessary the most reliable, easiest to set up and fastest method. You can do it, but nobody forces you to do this because you are running CL in production. You can use the industry’s boring and best practices too. Common Lisp doesn’t inforce a “big ball of mud” approach. Develop locally, use Git, use a CI, deploy a binary...

Every thing that I learned I documented it along the way in the Cookbook ;)

Another app that I’ll mention but about which I also wrote earlier is my first web app. This one is open-source. It still runs :)

 

In this project I had my friend and colleague contribute five lines of Lisp code to add a theme switcher in the backend that would help him do the frontend. He had never written a line of Lisp before. Of course, he did so by looking at my existing code to learn the existing functions at hand, and he could do it because the project was easy to install and run.

(defun get-template(template &optional (theme *theme*))
  "Loads template from the base templates directory or from the given theme templates directory if it exists."
  (if (and (str:non-blank-string-p theme)
           (probe-file (asdf:system-relative-pathname "abstock" (str:concat "src/templates/themes/" theme "/" template))))
      ;; then
      (str:concat "themes/" theme "/" template)
      ;; else :D
      template))

He had to annotate the if branches :] This passed the code review.

Lasting words

The 5th script/app is already on the way, and the next ones are awaiting that I open their .docx specification files. This one was a bit harder but the Lisp side was done sucessfully with the efficient collaboration of another freelance lisper (Kevin to not name him).

All those tasks (read a DB, transform data...) are very mundane.

They are everywhere. They don’t always need supercharged web framework or integrations.

You have plenty of opportunities to make yourself a favor, and use Common Lisp in the wild. Not counting the super-advanced domains where Lisp excels at ;)


Links

I have done some preliminary Common Lisp exploration prior to this course but had a lot of questions regarding practical use and development workflows. This course was amazing for this! I learned a lot of useful techniques for actually writing the code in Emacs, as well as conversational explanations of concepts that had previously confused me in text-heavy resources. Please keep up the good work and continue with this line of topics, it is well worth the price! [Preston, October of 2024]




in

Joe Marshall: Don't Try to Program in Lisp

A comment on my previous post said,

The most difficult thing when coming to a different language is to leave the other language behind. The kind of friction experienced here is common when transliterating ideas from one language to another. Go (in this case) is telling you it just doesn't like to work like this.
Try writing simple Go, instead of reaching for Lisp idioms. Then find the ways that work for Go to express the concepts you find.

That's not at all how I approach programming.

A friend of mine once paid me a high compliment. He said, “Even your C code looks like Lisp.”

When I write code, I don't think in terms of the language I'm using, I think in terms of the problem I'm solving. I'm a mostly functional programmer, so I like to think in terms of functions and abstractions. I mostly reason about my code informally, but I draw upon the formal framework of Lambda Calculus. Lambda Calculus is a simple, but powerful (and universal) model of computation.

Programming therefore becomes a matter of expressing the solution to a problem with the syntax and idioms of the language I'm using. Lisp was inspired by Lambda Calculus, so there is little friction in expressing computations in Lisp. Lisp is extensible and customizable, so I can add new syntax and idioms as desired.

Other languages are less accommodating. Some computations are not easily expressable in the syntax of the language, or the semantics of the language are quirky and inconsistent. Essentially, every general purpose fourth generation programming language can be viewed as a poorly-specified, half-assed, incomplete, bug-ridden implementation of half of Common Lisp. The friction comes from working around the limitations of the language.




in

"Dragons of Paris" and the Role of Time in the Mongolian Wizard Series

 .

The kind people at Reactor Magazine have posted my two latest Mongolian Wizard stories, one yesterday and the other today. Thursday's "Halcyon Afternoon" took place during a rare moment of peace for Franz-Karl Ritter. But in today's "Dragons of Paris," it's warfare as usual. 

Time has always been a little tricky in this series. The first story was clearly set in the Nineteenth Century but, though only a few years have passed, the series has now reached what is recognizably World War I. Mostly this occurred for reasons explained in "The Phantom in the Maze" and "Murder in the Spook House." (And which I anticipate giving me increasing difficulties in writing the next ten stories.) But also, in a more literary background sense, I wanted to cover the transition from a way of life now alien to us to something more modern, if not contemporary. 

So time may get a bit more slippery in the future. That's if, of course, the stories go in the direction I intend. Sometimes the fiction has its own ideas where it wants to go and the author can only follow along meekly in its wake.

You can read the story here. Or just go to the ezine and poke around. It's a good place to poke around.


Above: The illustration is by Dave Palumbo. I'm grateful for that.


*




in

My Halloween Season Story, "Unquiet Graves," in CLARKESWORLD

 .


 

I am always happiest when a story of mine comes into print. Today, I have the joy of introducing you to "Unquiet Graves," a seasonal tale of graveyard misbehavior and betrayal. Oh, and there's nothing supernatural about it at all.

You can read the story here. But if you're like me, you'll just go to Clarkesworld, look over the table of contents, and decide which story you want to read first. Mine by preference, but follow your whim.

 

And for those who like trivia . . .

I came up with the handheld's app many long years ago and it took forever to come up with a story for it. You'll notice that it is left unnamed in the story. That's because its secret name was "The Graveyard Reader." Which is the title of a well-known story by Theodore Sturgeon.  While I was writing the story, I thought of it as "The New Graveyard Reader." But Sturgeon's story and mine go off in totally different directions, and giving mine (or even the app) a title suggesting there was some implicit connection between the two would only cause confusion.

The title I finally came up with was derived from "The Unquiet Grave" by that most prolific of all poets, Anonymous. If you look it up, I suggest you do so after reading my story. It gives away some of the plot.


*

 




in

ALL SOULS NIGHT Complete! In One Easy-to-Read Location!

.


This year's Halloween story, written on leaves and serialized daily on my blog, one sentence at a time, is done. Every day in October, I added to it, it reached its last words on Halloween.

Funny thing, though. In conversations with two different friends, I learned that neither of them had realized it was a story. They each thought I was just posting random sentences written on leaves. One of them is an artist, and thinks primarily in visual terms, so I thought at first that was a misunderstanding curious to her. The other, however, is a well-known writer and, what's more important, quite a good one. I have no idea what's going on there.

Long story short, at my behest, my son Sean, put all the photos up on Imgur, subtitled. So, if you didn't realize that they told a story... Or if, somehow, you weren't able to hold all the sentences in your head until the story was complete... Now you can find out what was going on. (The stone angels mark the ends of paragraphs.)

You can find it by clicking on the link here.


*




in

Popular destinations in Sweden

Popular destinations in Sweden



View Comic!









in

Why Virat Kohli, Jasprit Bumrah were missing from Perth nets; India ramp up privacy amid Manchester United-like security - Hindustan Times

  1. Why Virat Kohli, Jasprit Bumrah were missing from Perth nets; India ramp up privacy amid Manchester United-like security  Hindustan Times
  2. Virat Kohli in focus: Intense net session begins for upcoming Test series against Australia  The Times of India
  3. Virat Kohli in Australia for BGT: A timeline  India Today
  4. Black veil of secrecy: India begin training in privacy in Perth  ESPNcricinfo
  5. India to play intra-squad warm-up match at WACA on Friday ahead of Australia Tests but BCCI denies public viewing  Hindustan Times